正在加载视频...

视频加载失败

返回首页

More work on STARSPEED's procedural spaceship generator 🚀 Three.js/spark web app is a visualizer & parametric GUI for Blender 🔶 API scripting + export Using World Labs Marble 1.1 environments w/ 3DGS env maps to preview glTF content w/ various scene lighting options

jamesckane's profile picture

jamesckane

1,329 subscribers

10,968 次观看 • 2 个月前 •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

I made ‘Shadow of the Czar’ in ~6 weeks for World Labs launch, including screenplay, environments, audio, VFX, green screen shoots, gameplay, engineering & more using Marble + spark Sneak a peek behind-the-scenes now… 🔎🕵️
1:55

I made ‘Shadow of the Czar’ in ~6 weeks for World Labs launch, including screenplay, environments, audio, VFX, green screen shoots, gameplay, engineering & more using Marble + spark Sneak a peek behind-the-scenes now… 🔎🕵️

jamesckane

40,995 次观看 • 7 个月前

STARSPEED: Zero-G Aerial Combat is out now! My Descent/DOOM-inspired multiplayer spaceship shooter w/ a 10-song synthwave OST Battle in km-scale sci-fi environs of 100m+ gaussian splats on 💻 & 📱 via the web w/ World Labs & spark's .rad format
0:46

STARSPEED: Zero-G Aerial Combat is out now! My Descent/DOOM-inspired multiplayer spaceship shooter w/ a 10-song synthwave OST Battle in km-scale sci-fi environs of 100m+ gaussian splats on 💻 & 📱 via the web w/ World Labs & spark's .rad format

jamesckane

37,342 次观看 • 3 个月前

This might be the craziest workflow you'll see this week. From #VR greyboxing to 3D worlds with the help of #AI. The 2 tools I used are ShapesXR and Marble from World Labs. Let me tell you why and how. - ShapesXR is the most powerful and fastest way to assemble 3D scenes at scale. - The library of assets + ability to sketch from procedural primitives gives the perfect balance between speed and freedom. - ShapesXR allows you to export your creation as a glTF that can be used as a "3D prompt" into Marble - Marble now allows you to create a panorama using a 3D scene as input increasing DRAMATICALLY the control you have during the creation process - Through the workflow you can adjust textures and objects in the scene using AI inpainting - There are tons of options on how to use the final output (create a #GaussianSplatting, export collision mesh and even a detailed mesh 🤯) This is the perfect examples of how VR creative tools improve and accelerate the design process... so make sure to follow me if that's your jam 😉.
1:01

This might be the craziest workflow you'll see this week. From #VR greyboxing to 3D worlds with the help of #AI. The 2 tools I used are ShapesXR and Marble from World Labs. Let me tell you why and how. - ShapesXR is the most powerful and fastest way to assemble 3D scenes at scale. - The library of assets + ability to sketch from procedural primitives gives the perfect balance between speed and freedom. - ShapesXR allows you to export your creation as a glTF that can be used as a "3D prompt" into Marble - Marble now allows you to create a panorama using a 3D scene as input increasing DRAMATICALLY the control you have during the creation process - Through the workflow you can adjust textures and objects in the scene using AI inpainting - There are tons of options on how to use the final output (create a #GaussianSplatting, export collision mesh and even a detailed mesh 🤯) This is the perfect examples of how VR creative tools improve and accelerate the design process... so make sure to follow me if that's your jam 😉.

Gabriele Romagnoli

20,126 次观看 • 8 个月前

📻 Brazen brute bashes bank! Czar’s zealots embezzle zillions! Cops can't quell criminal caper! is at large for World Labs launch! ‘Shadow’ is the jazz-infused, film noir-style narrative game made for Ben Mildenhall & Fei-Fei Li's AI research group 🕵️
0:57

📻 Brazen brute bashes bank! Czar’s zealots embezzle zillions! Cops can't quell criminal caper! is at large for World Labs launch! ‘Shadow’ is the jazz-infused, film noir-style narrative game made for Ben Mildenhall & Fei-Fei Li's AI research group 🕵️

jamesckane

55,373 次观看 • 8 个月前

Robot simulation is getting a big upgrade! 🔥 Normally, building a simulation environment, a kitchen, warehouse, office takes days or weeks. You have to model the geometry, add textures, set lighting, generate collision meshes… all before a robot can even interact with it. Marble by World Labs removes most of that work. You type a text prompt or give it an image, and it generates a full 3D scene with depth, lighting, and a collider mesh you can export into engines like MuJoCo. That means researchers can quickly create lots of different environments to train robots on, which is exactly what domain randomization needs. If we want robots to learn faster, we need to create worlds faster. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →
0:59

Robot simulation is getting a big upgrade! 🔥 Normally, building a simulation environment, a kitchen, warehouse, office takes days or weeks. You have to model the geometry, add textures, set lighting, generate collision meshes… all before a robot can even interact with it. Marble by World Labs removes most of that work. You type a text prompt or give it an image, and it generates a full 3D scene with depth, lighting, and a collider mesh you can export into engines like MuJoCo. That means researchers can quickly create lots of different environments to train robots on, which is exactly what domain randomization needs. If we want robots to learn faster, we need to create worlds faster. ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

15,498 次观看 • 7 个月前

Fast Company just published a great piece on World Labs , Fei-Fei Li , Marble, and the idea that spatial intelligence / world models may be one of the next big shifts in AI. I was happy to be quoted in the article, but I also wanted to share more context about my own experience with World Labs and Marble, and why this direction is especially interesting to me. My starting point: volumetric capture — For the past few years I’ve been exploring and using volumetric capture and reconstruction (photogrammetry, NeRFs, 3D Gaussian Splats) mostly capturing locations around Montreal. Alleys, museums, urban interiors. I love every step of it: the capture itself, the pipeline, and what can be done with the output. Turning real spaces into real-time explorable systems. I do this personally, sharing explorations here, and professionally as chief technologist, and co-founder of Dpt. Physical reality + generative manipulation — In my work I’m especially drawn to mixing physical reality with generative and digital manipulation: using physical interfaces (light, clay, ink, ... ) to drive generative AI pipelines, building mixed reality prototypes that reshape your surroundings, or starting from real captured spaces and transforming them using tools like Marble. Like many people, I saw the World Labs announcement on Twitter in September 2024, and Marble when it surfaced in early December. But by then, I already had a sense something was coming. The first conversation — As someone deep into volumetric capture and radiance fields, I obviously knew about Ben Mildenhall and his pioneering work on NeRF. To my surprise, Ben reached out to me in late June 2024. He’d been following some of my experiments and wanted to chat about my process and workflows and how I was using this “stuff” creatively. At that point he didn’t share what he was building, but we had a genuinely great conversation about radiance fields, AI, and my work. He was curious about the creative perspective, not just the technical one. When the World Labs announcement dropped a few months later, it all made sense. I understood what Ben had been working on, and why the creative angle mattered to them. Then in August 2025, he invited me to try the Marble beta, and I’ve been experimenting with it since. Experimenting with Marble — The first thing I used Marble for was materializing scene and world concepts during ideation at the studio, and seeing if and how it could fit into our production pipeline. In parallel, I dove into a series of experiments focused on world manipulation: starting from real captured spaces and transforming them using Marble. I’d already been exploring that idea using img2img diffusion with ControlNet on NeRF renders, real-time video streams, and even mixed reality using headset camera feeds. But Marble brings something different. It generates persistent, spatially cohesive 3D worlds that can be rendered in real time across a wide range of devices. That’s a real shift. Experiment 01: Parallel Realities — The first experiment, Parallel Realities, starts from a volumetric capture of a real location, reconstructed as 3D Gaussian Splats. Using Marble, I generate an alternate version of that same space, something informed by the original architecture: abandoned, nature-reclaimed, alternate era. Then, using Spark (World Labs’ 3D Gaussian Splatting renderer for THREE.js) I make both realities coexist in the same spatial coordinate system. From there, I use a portal UX mechanic to let the user step between the real reconstruction and the Marble-generated version. Experiment 02: Hidden Depth The second experiment, Hidden Depth, does not transform a space as much as expand it. A captured location has a visual boundary (a mural, a doorway, a dark corridor) and Marble generates what exists beyond it. For example: a Montreal alley has a painted mural; step through it and you’re inside a world informed by what is actually depicted there. World Labs showcased part of this work here: And in their Spark 2.0 post: The project page is here: Why this matters to me — Being able to start from a real 3D Gaussian Splat scene and manipulate it with Marble opens up a lot of ideas. The 3DGS pipeline is becoming an increasingly compelling foundation for exploration, experimentation, and storytelling. What matters most to me right now is more control. The more I can steer the generated scene or world, the more useful the tool becomes. I want more features like the already existing multiple input images and Chisel, the blockout-based approach. I would like better local control, the ability to expand a generated world more and more while preserving coherence, and the ability to directly import 3D Gaussian Splat scenes to be used as a starting point. I want more ways to shape the result, not just a “prompt and hope” approach. — It is exciting to see this field moving from research and demos toward actual creative workflows.
1:04

Fast Company just published a great piece on World Labs , Fei-Fei Li , Marble, and the idea that spatial intelligence / world models may be one of the next big shifts in AI. I was happy to be quoted in the article, but I also wanted to share more context about my own experience with World Labs and Marble, and why this direction is especially interesting to me. My starting point: volumetric capture — For the past few years I’ve been exploring and using volumetric capture and reconstruction (photogrammetry, NeRFs, 3D Gaussian Splats) mostly capturing locations around Montreal. Alleys, museums, urban interiors. I love every step of it: the capture itself, the pipeline, and what can be done with the output. Turning real spaces into real-time explorable systems. I do this personally, sharing explorations here, and professionally as chief technologist, and co-founder of Dpt. Physical reality + generative manipulation — In my work I’m especially drawn to mixing physical reality with generative and digital manipulation: using physical interfaces (light, clay, ink, ... ) to drive generative AI pipelines, building mixed reality prototypes that reshape your surroundings, or starting from real captured spaces and transforming them using tools like Marble. Like many people, I saw the World Labs announcement on Twitter in September 2024, and Marble when it surfaced in early December. But by then, I already had a sense something was coming. The first conversation — As someone deep into volumetric capture and radiance fields, I obviously knew about Ben Mildenhall and his pioneering work on NeRF. To my surprise, Ben reached out to me in late June 2024. He’d been following some of my experiments and wanted to chat about my process and workflows and how I was using this “stuff” creatively. At that point he didn’t share what he was building, but we had a genuinely great conversation about radiance fields, AI, and my work. He was curious about the creative perspective, not just the technical one. When the World Labs announcement dropped a few months later, it all made sense. I understood what Ben had been working on, and why the creative angle mattered to them. Then in August 2025, he invited me to try the Marble beta, and I’ve been experimenting with it since. Experimenting with Marble — The first thing I used Marble for was materializing scene and world concepts during ideation at the studio, and seeing if and how it could fit into our production pipeline. In parallel, I dove into a series of experiments focused on world manipulation: starting from real captured spaces and transforming them using Marble. I’d already been exploring that idea using img2img diffusion with ControlNet on NeRF renders, real-time video streams, and even mixed reality using headset camera feeds. But Marble brings something different. It generates persistent, spatially cohesive 3D worlds that can be rendered in real time across a wide range of devices. That’s a real shift. Experiment 01: Parallel Realities — The first experiment, Parallel Realities, starts from a volumetric capture of a real location, reconstructed as 3D Gaussian Splats. Using Marble, I generate an alternate version of that same space, something informed by the original architecture: abandoned, nature-reclaimed, alternate era. Then, using Spark (World Labs’ 3D Gaussian Splatting renderer for THREE.js) I make both realities coexist in the same spatial coordinate system. From there, I use a portal UX mechanic to let the user step between the real reconstruction and the Marble-generated version. Experiment 02: Hidden Depth The second experiment, Hidden Depth, does not transform a space as much as expand it. A captured location has a visual boundary (a mural, a doorway, a dark corridor) and Marble generates what exists beyond it. For example: a Montreal alley has a painted mural; step through it and you’re inside a world informed by what is actually depicted there. World Labs showcased part of this work here: And in their Spark 2.0 post: The project page is here: Why this matters to me — Being able to start from a real 3D Gaussian Splat scene and manipulate it with Marble opens up a lot of ideas. The 3DGS pipeline is becoming an increasingly compelling foundation for exploration, experimentation, and storytelling. What matters most to me right now is more control. The more I can steer the generated scene or world, the more useful the tool becomes. I want more features like the already existing multiple input images and Chisel, the blockout-based approach. I would like better local control, the ability to expand a generated world more and more while preserving coherence, and the ability to directly import 3D Gaussian Splat scenes to be used as a starting point. I want more ways to shape the result, not just a “prompt and hope” approach. — It is exciting to see this field moving from research and demos toward actual creative workflows.

Hugues Bruyère

65,393 次观看 • 1 个月前

Dormant Memories — a gallery regrouping my recent experiments built from real-world captures, reconstructed as 3D Gaussian Splats, and in many cases manipulated using World Labs' Marble. Everything is built with Spark, the 3D Gaussian splat renderer released by World Labs. It runs in the browser and can be experienced across devices: desktop, mobile, and headsets like Apple Vision Pro and Quest 3. This work comes out of something I've been exploring for a few years now: capturing fragments of the world (places, objects, moments...) and turning them into spaces that can then be revisited, manipulated, and explored in real time. What particularly interests me here is the process of creating alternate versions of the same place; starting from something that physically exists and using it as a base to explore variations, instead of generating from scratch. These explorations are part of what we do at Dpt.... same tools, same curiosity, same collaborative energy that shape the studio's work on interactive and immersive experiences. Some of these experiments were featured as part of the Spark 2.0 release by World Labs. Their technical blog post is a must-read; link in the thread ↓ These are experiments. They aren't fully optimized, they aren't bug-free, and the UX has rough edges. The goal is to explore, test, and learn. #GaussianSplatting #3DGS #pointcloud #WorldModels #sparkjs #threejs
1:20

Dormant Memories — a gallery regrouping my recent experiments built from real-world captures, reconstructed as 3D Gaussian Splats, and in many cases manipulated using World Labs' Marble. Everything is built with Spark, the 3D Gaussian splat renderer released by World Labs. It runs in the browser and can be experienced across devices: desktop, mobile, and headsets like Apple Vision Pro and Quest 3. This work comes out of something I've been exploring for a few years now: capturing fragments of the world (places, objects, moments...) and turning them into spaces that can then be revisited, manipulated, and explored in real time. What particularly interests me here is the process of creating alternate versions of the same place; starting from something that physically exists and using it as a base to explore variations, instead of generating from scratch. These explorations are part of what we do at Dpt.... same tools, same curiosity, same collaborative energy that shape the studio's work on interactive and immersive experiences. Some of these experiments were featured as part of the Spark 2.0 release by World Labs. Their technical blog post is a must-read; link in the thread ↓ These are experiments. They aren't fully optimized, they aren't bug-free, and the UX has rough edges. The goal is to explore, test, and learn. #GaussianSplatting #3DGS #pointcloud #WorldModels #sparkjs #threejs

Hugues Bruyère

12,408 次观看 • 3 个月前

meta muse spark 1.1 vs gpt 5.6 sol vs fable 5 vs grok 4.5 meta recently dropped muse spark 1.1 – a multimodal reasoning model from meta superintelligence labs built for agentic tasks. key facts: • 1m token context with active self-management – the model compacts its own history and keeps only the steps needed for later work • trained to orchestrate multi-agent systems: as main agent it plans and delegates to parallel subagents, as subagent it sticks to its job and knows when to escalate back • computer use trained to pick between scripting and clicking – writes automation when it's faster, clicks when it's simpler, batches actions per step • first public api from meta: the meta model api is now in preview • benchmarks: sweeps the agent column – mcp atlas 88.1 (opus 4.8: 82.2), jobbench 54.7 (opus: 48.4), humanity's last exam 62.1 (1st). loses coding – deepswe 1.1 53.3 vs gpt 5.5's 67.0, swe bench pro 61.5 vs opus's 69.2 our test – 3 prompts, single-file html, three.js, fully procedural, no assets: 1. norwegian house cantilevered over a fjord in a snowstorm – transmissive glass wall, fully modelled interior 2. beijing siheyuan courtyard house in dawn fog – instanced roof tiles, dougong brackets, glowing paper windows 3. new mexico adobe pueblo in an approaching dust storm – deep window reveals, windward grit accumulation we ran the test on AI/ML API platform results: - cost #1 muse spark 1.1 – $0.20 #2 grok 4.5 – $0.51 #3 gpt 5.6 sol – $1.93 #4 fable 5 – ~$5.20 - output tokens #1 muse spark 1.1 – 41,868 #2 gpt 5.6 sol – 49,139 #3 grok 4.5 – 64,954 #4 fable 5 – 81,849 - lines of code #1 muse spark 1.1 – 1,799 #2 gpt 5.6 sol – 2,377 #3 fable 5 – 3,088 #4 grok 4.5 – 4,216 observations: • muse spark is the cheapest of the four by a wide margin – 2.5x under grok, ~26x under fable per run. output quality tracks the price • only 7.4% of its output tokens are reasoning (3,104 of 41,868) – the model barely thinks before writing. economic, not pedantic: it commits to the first plan and ships it • the low loc is not compression, it's omission – all three prompts demanded instancing, muse spark delivered it in one muse spark's code quality – reviewed by fable 5: upsides: 1. all three files run 2. the adobe grit effect is legit – shader injection via onbeforecompile, windward faces detect storm direction through a normal-dot-wind term and darken procedurally 3. the fjord glass is real meshphysicalmaterial with transmission and ior, not a transparent quad 4. the siheyuan properly instances barrel tiles, dougong blocks and courtyard pavers downsides: 1. in the fjord file the strafe vector is negated – press a, you move right; press d, you move left. exactly the key mix-up we kept hitting with this model 2. all three files ship the model's self-doubt as comments: "// actually yaw orientation: need correct" sits above a direction vector that gets computed, abandoned and recomputed – dead vectors allocated every frame, 60 times a second 3. the siheyuan registers two separate keydown listeners, one containing an empty if-block 4. snow "accumulation" on the norway roof is a sine wobble on a scale value, not accumulation 5. "instanced snow" became 3,500 plain points. zero dispose calls anywhere pattern: minimal reasoning, minimal code, minimal price. it nails the flashy requirements – shaders, transmissive glass – and quietly drops the boring ones: instancing, controls, cleanup. you get a demo that mostly runs and a control scheme you can't trust follow thehype. for 24/7 ai news, analysis and breakdowns
1:01

meta muse spark 1.1 vs gpt 5.6 sol vs fable 5 vs grok 4.5 meta recently dropped muse spark 1.1 – a multimodal reasoning model from meta superintelligence labs built for agentic tasks. key facts: • 1m token context with active self-management – the model compacts its own history and keeps only the steps needed for later work • trained to orchestrate multi-agent systems: as main agent it plans and delegates to parallel subagents, as subagent it sticks to its job and knows when to escalate back • computer use trained to pick between scripting and clicking – writes automation when it's faster, clicks when it's simpler, batches actions per step • first public api from meta: the meta model api is now in preview • benchmarks: sweeps the agent column – mcp atlas 88.1 (opus 4.8: 82.2), jobbench 54.7 (opus: 48.4), humanity's last exam 62.1 (1st). loses coding – deepswe 1.1 53.3 vs gpt 5.5's 67.0, swe bench pro 61.5 vs opus's 69.2 our test – 3 prompts, single-file html, three.js, fully procedural, no assets: 1. norwegian house cantilevered over a fjord in a snowstorm – transmissive glass wall, fully modelled interior 2. beijing siheyuan courtyard house in dawn fog – instanced roof tiles, dougong brackets, glowing paper windows 3. new mexico adobe pueblo in an approaching dust storm – deep window reveals, windward grit accumulation we ran the test on AI/ML API platform results: - cost #1 muse spark 1.1 – $0.20 #2 grok 4.5 – $0.51 #3 gpt 5.6 sol – $1.93 #4 fable 5 – ~$5.20 - output tokens #1 muse spark 1.1 – 41,868 #2 gpt 5.6 sol – 49,139 #3 grok 4.5 – 64,954 #4 fable 5 – 81,849 - lines of code #1 muse spark 1.1 – 1,799 #2 gpt 5.6 sol – 2,377 #3 fable 5 – 3,088 #4 grok 4.5 – 4,216 observations: • muse spark is the cheapest of the four by a wide margin – 2.5x under grok, ~26x under fable per run. output quality tracks the price • only 7.4% of its output tokens are reasoning (3,104 of 41,868) – the model barely thinks before writing. economic, not pedantic: it commits to the first plan and ships it • the low loc is not compression, it's omission – all three prompts demanded instancing, muse spark delivered it in one muse spark's code quality – reviewed by fable 5: upsides: 1. all three files run 2. the adobe grit effect is legit – shader injection via onbeforecompile, windward faces detect storm direction through a normal-dot-wind term and darken procedurally 3. the fjord glass is real meshphysicalmaterial with transmission and ior, not a transparent quad 4. the siheyuan properly instances barrel tiles, dougong blocks and courtyard pavers downsides: 1. in the fjord file the strafe vector is negated – press a, you move right; press d, you move left. exactly the key mix-up we kept hitting with this model 2. all three files ship the model's self-doubt as comments: "// actually yaw orientation: need correct" sits above a direction vector that gets computed, abandoned and recomputed – dead vectors allocated every frame, 60 times a second 3. the siheyuan registers two separate keydown listeners, one containing an empty if-block 4. snow "accumulation" on the norway roof is a sine wobble on a scale value, not accumulation 5. "instanced snow" became 3,500 plain points. zero dispose calls anywhere pattern: minimal reasoning, minimal code, minimal price. it nails the flashy requirements – shaders, transmissive glass – and quietly drops the boring ones: instancing, controls, cleanup. you get a demo that mostly runs and a control scheme you can't trust follow thehype. for 24/7 ai news, analysis and breakdowns

thehype.

133,801 次观看 • 15 天前

🚀 Announcing Echo — our new frontier model for 3D world generation. Echo turns a simple text prompt or image into a fully explorable, 3D-consistent world. Instead of disconnected views, the result is a single, coherent spatial representation you can move through freely. This is part of a bigger shift in AI: from generating pixels and tokens to generating spaces. Echo predicts a geometry-grounded 3D scene at metric scale, meaning every novel view, depth map, and interaction comes from the same underlying world — not independent hallucinations. Once generated, the world is interactive in real time. You control the camera, explore from any angle, and render instantly — even on low-end hardware, directly in the browser. High-quality 3D world exploration is no longer gated by expensive equipment. Under the hood, Echo infers a physically grounded 3D representation and converts it into a renderable format. For our web demo, we use 3D Gaussian Splatting (3DGS) for fast, GPU-friendly rendering — but the representation itself is flexible and can be easily adapted. Why this matters: consistent 3D worlds unlock real workflows — digital twins, 3D design, game environments, robotics simulation, and more. From a single photo or a line of text, Echo builds worlds that are reliable, editable, and spatially faithful. Echo also enables scene editing and restyling. Change materials, remove or add objects, explore design variations — all while preserving global 3D consistency. Editing no longer breaks the world. This is only the beginning. Echo is the foundation for future world models with dynamics, physical reasoning, and richer interaction — environments that don’t just look right, but behave right. Explore the generated worlds on our website and sign up for the closed beta. The era of spatial intelligence starts here. 🌍 #Echo #WorldModels #SpatialAI #3DFoundationModels Check it out:
1:07

🚀 Announcing Echo — our new frontier model for 3D world generation. Echo turns a simple text prompt or image into a fully explorable, 3D-consistent world. Instead of disconnected views, the result is a single, coherent spatial representation you can move through freely. This is part of a bigger shift in AI: from generating pixels and tokens to generating spaces. Echo predicts a geometry-grounded 3D scene at metric scale, meaning every novel view, depth map, and interaction comes from the same underlying world — not independent hallucinations. Once generated, the world is interactive in real time. You control the camera, explore from any angle, and render instantly — even on low-end hardware, directly in the browser. High-quality 3D world exploration is no longer gated by expensive equipment. Under the hood, Echo infers a physically grounded 3D representation and converts it into a renderable format. For our web demo, we use 3D Gaussian Splatting (3DGS) for fast, GPU-friendly rendering — but the representation itself is flexible and can be easily adapted. Why this matters: consistent 3D worlds unlock real workflows — digital twins, 3D design, game environments, robotics simulation, and more. From a single photo or a line of text, Echo builds worlds that are reliable, editable, and spatially faithful. Echo also enables scene editing and restyling. Change materials, remove or add objects, explore design variations — all while preserving global 3D consistency. Editing no longer breaks the world. This is only the beginning. Echo is the foundation for future world models with dynamics, physical reasoning, and richer interaction — environments that don’t just look right, but behave right. Explore the generated worlds on our website and sign up for the closed beta. The era of spatial intelligence starts here. 🌍 #Echo #WorldModels #SpatialAI #3DFoundationModels Check it out:

SpAItial AI

176,105 次观看 • 7 个月前

ALRIGHT! Introducing SEEDTHREE! A Three.js webgpu procedural plant generator. I'm releasing this fully open source. I built this with Claude Fable and GPT-5.5 Codex + gptimage2 over just the past few days. The best part is: given the code, your agents can add new plants to it all you want! They just need an understanding of the code and an image generator capable of creating material maps. I highly encourage you to do PRs for any extra plants you add! I'll be adding more myself in future versions. The system is fully robust: it provides complete LOD and billboards/imposters generation for the plants and you can adjust the detail levels all you want for different platform targets like mobile and/or use lower LODs as your main for extreme optimization. As this is the first release alpha version, some bugs may have been missed, I encourage you to report any bugs. The repo includes all the assets for the scenery world too, even the ambient sounds of birds and wind! This is targeted at high end laptop and PC use, it requires baking generated geometry, so you won't be using this on a phone (I mean, feel free to try, but I doubt it'll run well, lol). This is fundamentally an asset generation tool for 3d dev. Have fun! Github repo link and live demo (you don't even have to download anything for basic non-editing/contributing use!) below ⤵️
0:56

ALRIGHT! Introducing SEEDTHREE! A Three.js webgpu procedural plant generator. I'm releasing this fully open source. I built this with Claude Fable and GPT-5.5 Codex + gptimage2 over just the past few days. The best part is: given the code, your agents can add new plants to it all you want! They just need an understanding of the code and an image generator capable of creating material maps. I highly encourage you to do PRs for any extra plants you add! I'll be adding more myself in future versions. The system is fully robust: it provides complete LOD and billboards/imposters generation for the plants and you can adjust the detail levels all you want for different platform targets like mobile and/or use lower LODs as your main for extreme optimization. As this is the first release alpha version, some bugs may have been missed, I encourage you to report any bugs. The repo includes all the assets for the scenery world too, even the ambient sounds of birds and wind! This is targeted at high end laptop and PC use, it requires baking generated geometry, so you won't be using this on a phone (I mean, feel free to try, but I doubt it'll run well, lol). This is fundamentally an asset generation tool for 3d dev. Have fun! Github repo link and live demo (you don't even have to download anything for basic non-editing/contributing use!) below ⤵️

Utah teapot 🫖

29,124 次观看 • 22 天前

GeoLibre v1.3.0 is here! GeoLibre is a free and open-source, lightweight, cloud-native GIS platform for visualizing, exploring, and analyzing geospatial data. One application that runs everywhere: in your web browser, as a native desktop app, on your phone, and inside a Jupyter notebook. No account, no server, no cost. Everything runs locally and your data stays private. This release packs in 50+ pull requests of new capabilities. A few highlights: - GIS in your pocket. A native Android build with offline tile caching and download-a-region support, so you can take your maps into the field with no signal. - AI, built in. A natural-language GIS assistant that turns plain-English requests into real geoprocessing, plus an AI segmentation toolbox powered by SamGeo and SAM 3 for extracting features from imagery. - Automate everything with Python. A full scripting API and an in-app Python Console, with new helpers for local rasters, choropleths, marker clusters, split-map comparisons, legends, and colorbars. - Map together, live. Real-time multi-user collaboration so you can open a project and edit the map with others at the same time. - Tell stories with maps. A scroll-driven story map builder and presenter that exports interactive narrative maps to standalone HTML. - A much bigger analysis toolbox. Reproject, explode, and aggregate tools, IDW and kriging interpolation, zonal statistics, a raster calculator, a Spatial Statistics toolbox, and network analysis with isochrones, service areas, and OD cost matrices, plus batch runs and model/pipeline chaining. - Smarter raster and SQL. Single-band pseudocolor classification, RGB band combinations, a no-backend client-side raster fallback, Apache Sedona as a SQL Workspace engine, and transparent S3, GCS, and Azure URL support in queries. - More ways to add, view, and share. New Shapefile and GeoPackage export, glTF/GLB 3D model layers, multi-provider batch and reverse geocoding, collapsible layer groups, and a macOS Homebrew cask. Try the live demo: Star it on GitHub: Docs and roadmap: Release notes: #GIS #OpenSource #Geospatial #MapLibre #WebGIS #Android #GeoLibre
1:45

GeoLibre v1.3.0 is here! GeoLibre is a free and open-source, lightweight, cloud-native GIS platform for visualizing, exploring, and analyzing geospatial data. One application that runs everywhere: in your web browser, as a native desktop app, on your phone, and inside a Jupyter notebook. No account, no server, no cost. Everything runs locally and your data stays private. This release packs in 50+ pull requests of new capabilities. A few highlights: - GIS in your pocket. A native Android build with offline tile caching and download-a-region support, so you can take your maps into the field with no signal. - AI, built in. A natural-language GIS assistant that turns plain-English requests into real geoprocessing, plus an AI segmentation toolbox powered by SamGeo and SAM 3 for extracting features from imagery. - Automate everything with Python. A full scripting API and an in-app Python Console, with new helpers for local rasters, choropleths, marker clusters, split-map comparisons, legends, and colorbars. - Map together, live. Real-time multi-user collaboration so you can open a project and edit the map with others at the same time. - Tell stories with maps. A scroll-driven story map builder and presenter that exports interactive narrative maps to standalone HTML. - A much bigger analysis toolbox. Reproject, explode, and aggregate tools, IDW and kriging interpolation, zonal statistics, a raster calculator, a Spatial Statistics toolbox, and network analysis with isochrones, service areas, and OD cost matrices, plus batch runs and model/pipeline chaining. - Smarter raster and SQL. Single-band pseudocolor classification, RGB band combinations, a no-backend client-side raster fallback, Apache Sedona as a SQL Workspace engine, and transparent S3, GCS, and Azure URL support in queries. - More ways to add, view, and share. New Shapefile and GeoPackage export, glTF/GLB 3D model layers, multi-provider batch and reverse geocoding, collapsible layer groups, and a macOS Homebrew cask. Try the live demo: Star it on GitHub: Docs and roadmap: Release notes: #GIS #OpenSource #Geospatial #MapLibre #WebGIS #Android #GeoLibre

Qiusheng Wu

18,075 次观看 • 1 个月前

I MADE THE BEST 2D SPRITE AI PIPELINE EVER! After testing so much, doing tons of math, trying various different methods I finally got truly the cleanest sprite result I could ask for! You should bookmark this post! ✅ I think this is the happiest 'aha' moment I've had since starting this journey. Day 48 of making a custom webGL game engine using Cursor 🔽🔽🔽 I've been trying to solve sprites since I started this project. I came close a couple of times but it was always something - either dynamic lights wouldn't work, or the genAI results would make them grainy, or I would have to have incorrect alpha etc. After so much struggling, wanting to give up, thinking it's impossible - it was finally done! Perfect matte interpolation of black & white backgrounds resulting in a clean outline of the sprite, half transparent effects keeping their full quality. But this was not enough for me. I wanted more. I wanted those dynamic lights for the most badass anime effects that I'll be creating for the next many weeks! And somehow, I managed to come up with the perfect balance that is nice to the eye with almost no visible imperfections or artifacts when using any kind of dynamic lights. See for yourself! 🔽🔽🔽 How does this magic happen? 1⃣ First, the animation is created using Kling AI. I found it the best to keep the quality anime 2D style I wanted, but I guess others may work as well. 2⃣ Then, the frames are carefully extracted at lossless quality and processed via automation through Nano Banana pass to create an opposite background (white->black or black->white). 3⃣ Once we have a proper spritesheet, I then used MiDaS to compute the depth maps, and then algorithmically create PBR maps for lighting. The remaining depth maps can also be used for various effects, although haven't explored it much yet. 4⃣ Finally, all elements are cut into single frames and overlapped with their respective normal maps. If done right, they're pretty much ready for runtime. I like my heroes to be 512 to truly stand out like fighting game characters, but all of this works with far smaller or bigger sprites. Mind you, it's important to note that Nano Banana has a max output of 4k. Therefore you don't want to make your atlases larger than 4k - which is still quite a lot since it allows you 8x8 512 frames. 💡💡💡 I believe the output quality is beyond anything currently available, and I'll be looking into packaging the whole thing into a web app so people can use it. If you find this useful, please repost it! I really wanted to share this method to the world. Thank you for reading! Look forward to day 49 update!
0:58

I MADE THE BEST 2D SPRITE AI PIPELINE EVER! After testing so much, doing tons of math, trying various different methods I finally got truly the cleanest sprite result I could ask for! You should bookmark this post! ✅ I think this is the happiest 'aha' moment I've had since starting this journey. Day 48 of making a custom webGL game engine using Cursor 🔽🔽🔽 I've been trying to solve sprites since I started this project. I came close a couple of times but it was always something - either dynamic lights wouldn't work, or the genAI results would make them grainy, or I would have to have incorrect alpha etc. After so much struggling, wanting to give up, thinking it's impossible - it was finally done! Perfect matte interpolation of black & white backgrounds resulting in a clean outline of the sprite, half transparent effects keeping their full quality. But this was not enough for me. I wanted more. I wanted those dynamic lights for the most badass anime effects that I'll be creating for the next many weeks! And somehow, I managed to come up with the perfect balance that is nice to the eye with almost no visible imperfections or artifacts when using any kind of dynamic lights. See for yourself! 🔽🔽🔽 How does this magic happen? 1⃣ First, the animation is created using Kling AI. I found it the best to keep the quality anime 2D style I wanted, but I guess others may work as well. 2⃣ Then, the frames are carefully extracted at lossless quality and processed via automation through Nano Banana pass to create an opposite background (white->black or black->white). 3⃣ Once we have a proper spritesheet, I then used MiDaS to compute the depth maps, and then algorithmically create PBR maps for lighting. The remaining depth maps can also be used for various effects, although haven't explored it much yet. 4⃣ Finally, all elements are cut into single frames and overlapped with their respective normal maps. If done right, they're pretty much ready for runtime. I like my heroes to be 512 to truly stand out like fighting game characters, but all of this works with far smaller or bigger sprites. Mind you, it's important to note that Nano Banana has a max output of 4k. Therefore you don't want to make your atlases larger than 4k - which is still quite a lot since it allows you 8x8 512 frames. 💡💡💡 I believe the output quality is beyond anything currently available, and I'll be looking into packaging the whole thing into a web app so people can use it. If you find this useful, please repost it! I really wanted to share this method to the world. Thank you for reading! Look forward to day 49 update!

Startracker 🔺

145,925 次观看 • 5 个月前

Vibe Coding a AAA game with threejs Day 10: Deep dive into world authoring What started as a "i need terrain to build a racing game" slowly evolved to an open world Take a last look at the buildings, because they are getting kicked out next for some really good ones (i don't know how yet, but i'm confident gpt will tell me, but i feel like blender will be involved, ughhh) Anyways, fully reworked the road system, it is now a graph and junctions, bridges, elevations etc. are first class. Previous demos were filled with me carefully placing the race tracks, deleting them, placing them again, and again, until they were okay to show for a demo. Not so fun behind the camera! It's now basically using a very similar system to cities skylines 2, which has really nailed the perfect UX to build road systems. It's also much more flexible now and i believe the world editor can be used for much more than racing games: Rail networks (train game?) or maybe a 3rd person open world game? or actually another cities skylines? who knows, feels like i got a moat on the web now We'll see, after yesterday i've received lots of requests to open source it Which i may actually do at any time if i feel like it, but not for now. There's so many interesting problems with world modeling, i feel like i finally found a rabbit hole worth diving into for a long time! Might have to delay the human designer app until the point where i need humans in the city, we'll see, it'll all eventually come together - it always does Anyways, not gatekeeping, if you have ANY questions on how i solve things, feel free to ask! Also i will open source the procedural buildings soon, i really don't need them anymore
0:41

Vibe Coding a AAA game with threejs Day 10: Deep dive into world authoring What started as a "i need terrain to build a racing game" slowly evolved to an open world Take a last look at the buildings, because they are getting kicked out next for some really good ones (i don't know how yet, but i'm confident gpt will tell me, but i feel like blender will be involved, ughhh) Anyways, fully reworked the road system, it is now a graph and junctions, bridges, elevations etc. are first class. Previous demos were filled with me carefully placing the race tracks, deleting them, placing them again, and again, until they were okay to show for a demo. Not so fun behind the camera! It's now basically using a very similar system to cities skylines 2, which has really nailed the perfect UX to build road systems. It's also much more flexible now and i believe the world editor can be used for much more than racing games: Rail networks (train game?) or maybe a 3rd person open world game? or actually another cities skylines? who knows, feels like i got a moat on the web now We'll see, after yesterday i've received lots of requests to open source it Which i may actually do at any time if i feel like it, but not for now. There's so many interesting problems with world modeling, i feel like i finally found a rabbit hole worth diving into for a long time! Might have to delay the human designer app until the point where i need humans in the city, we'll see, it'll all eventually come together - it always does Anyways, not gatekeeping, if you have ANY questions on how i solve things, feel free to ask! Also i will open source the procedural buildings soon, i really don't need them anymore

robot 2.0

43,872 次观看 • 1 个月前

Today, we’re excited to announce our $50M Series B, led by Greenfield Partners, with participation from Lightspeed and Notable Capital. 🚀 At Patronus AI, we develop simulations and evals to train and improve AI. The first phase of AI was built on static benchmarks, but that era is over. As agents are used to solve longer and longer tasks, they need to practice in dynamic, living worlds to get better. Simulations are the critical infrastructure powering this next phase. As a company, we’re behind the most influential research and products in AI evaluation, like FinanceBench, Lynx, and Percival. And things have moved at the speed of light since.⚡ We partner with the world's leading frontier AI labs and enterprises, and our revenue has grown more than 15x over the past year. Additionally, today, we’re introducing a preview of the first Digital World Model for AI agent training and simulation: Patronus-DWM. Digital World Models are language diffusion world models that predict realistic environment behaviors and steer agent actions across digital workflows. Just as physical world models predict how objects move through space, we’re developing the equivalent for the digital world: predicting how agents act in digital workflows, then using that to scale the creation of high-quality training data for LLMs. Digital World Models help us push the frontier of ultra long horizon workflows, and unlock a new class of self-improving RL environments. This is our scalable approach to simulating all of the world’s intelligence. The round was also joined by Datadog, Inc., Samsung Ventures, Gokul Rajaram, Factorial Capital, and a large cohort of amazing AI leaders across Anthropic, OpenAI, Google DeepMind, NVIDIA, Recursive, and more.✨ It has been the ride of a lifetime. But we’re just getting started. The best is yet to come. "Do not go gentle into that good night, Rage, rage against the dying of the light" - Dylan Thomas (1954)
1:50

Today, we’re excited to announce our $50M Series B, led by Greenfield Partners, with participation from Lightspeed and Notable Capital. 🚀 At Patronus AI, we develop simulations and evals to train and improve AI. The first phase of AI was built on static benchmarks, but that era is over. As agents are used to solve longer and longer tasks, they need to practice in dynamic, living worlds to get better. Simulations are the critical infrastructure powering this next phase. As a company, we’re behind the most influential research and products in AI evaluation, like FinanceBench, Lynx, and Percival. And things have moved at the speed of light since.⚡ We partner with the world's leading frontier AI labs and enterprises, and our revenue has grown more than 15x over the past year. Additionally, today, we’re introducing a preview of the first Digital World Model for AI agent training and simulation: Patronus-DWM. Digital World Models are language diffusion world models that predict realistic environment behaviors and steer agent actions across digital workflows. Just as physical world models predict how objects move through space, we’re developing the equivalent for the digital world: predicting how agents act in digital workflows, then using that to scale the creation of high-quality training data for LLMs. Digital World Models help us push the frontier of ultra long horizon workflows, and unlock a new class of self-improving RL environments. This is our scalable approach to simulating all of the world’s intelligence. The round was also joined by Datadog, Inc., Samsung Ventures, Gokul Rajaram, Factorial Capital, and a large cohort of amazing AI leaders across Anthropic, OpenAI, Google DeepMind, NVIDIA, Recursive, and more.✨ It has been the ride of a lifetime. But we’re just getting started. The best is yet to come. "Do not go gentle into that good night, Rage, rage against the dying of the light" - Dylan Thomas (1954)

PatronusAI

94,014 次观看 • 1 个月前

Today, we’re excited to announce our $50M Series B, led by Greenfield Partners (formerly TPG Capital), with participation from Lightspeed and Notable Capital. 🚀 At PatronusAI, we develop simulations and evals to train and improve AI. The first phase of AI was built on static benchmarks, but that era is over now. As agents are used to solve longer and longer tasks, they need to practice in dynamic, living worlds to get better. Simulations are the critical infrastructure powering this next phase. As a company, we’re behind the most influential research and products in AI evaluation, like FinanceBench, Lynx, and Percival. And things have moved at the speed of light since. ⚡ We partner with the world's leading frontier AI labs and enterprises, and our revenue has grown more than 15x over the past year. Additionally, today, we’re introducing a preview of the first Digital World Model for AI agent training and simulation: Patronus-DWM. Digital World Models are language diffusion world models that predict realistic environment behaviors and steer agent actions across digital workflows. Just as physical world models predict how objects move through space, we’re developing the equivalent for the digital world: predicting how agents act in digital workflows, then using that to scale the creation of high-quality training data for LLMs. Digital World Models help us push the frontier of ultra long horizon workflows, and unlock a new class of self-improving RL environments. This is our scalable approach to simulating all of the world’s intelligence. The round was also joined by Datadog, Inc., Samsung Ventures, Gokul Rajaram, Factorial Capital, and a large cohort of amazing AI leaders and researchers across Anthropic, OpenAI, Google DeepMind, NVIDIA, Recursive, and more. ✨ It has been the ride of a lifetime. But we’re just getting started. The best is yet to come. "Do not go gentle into that good night, Rage, rage against the dying of the light" - Dylan Thomas (1954)
1:50

Today, we’re excited to announce our $50M Series B, led by Greenfield Partners (formerly TPG Capital), with participation from Lightspeed and Notable Capital. 🚀 At PatronusAI, we develop simulations and evals to train and improve AI. The first phase of AI was built on static benchmarks, but that era is over now. As agents are used to solve longer and longer tasks, they need to practice in dynamic, living worlds to get better. Simulations are the critical infrastructure powering this next phase. As a company, we’re behind the most influential research and products in AI evaluation, like FinanceBench, Lynx, and Percival. And things have moved at the speed of light since. ⚡ We partner with the world's leading frontier AI labs and enterprises, and our revenue has grown more than 15x over the past year. Additionally, today, we’re introducing a preview of the first Digital World Model for AI agent training and simulation: Patronus-DWM. Digital World Models are language diffusion world models that predict realistic environment behaviors and steer agent actions across digital workflows. Just as physical world models predict how objects move through space, we’re developing the equivalent for the digital world: predicting how agents act in digital workflows, then using that to scale the creation of high-quality training data for LLMs. Digital World Models help us push the frontier of ultra long horizon workflows, and unlock a new class of self-improving RL environments. This is our scalable approach to simulating all of the world’s intelligence. The round was also joined by Datadog, Inc., Samsung Ventures, Gokul Rajaram, Factorial Capital, and a large cohort of amazing AI leaders and researchers across Anthropic, OpenAI, Google DeepMind, NVIDIA, Recursive, and more. ✨ It has been the ride of a lifetime. But we’re just getting started. The best is yet to come. "Do not go gentle into that good night, Rage, rage against the dying of the light" - Dylan Thomas (1954)

Anand Kannappan

38,973 次观看 • 1 个月前

🎉 Tailkit 4.0 is here, and I couldn't be more excited! 🙌🥳 But first - Giveaway Alert! Want to get a free Tailkit Developer license? Just drop a reply and give a like or repost (totally optional, but super appreciated). The lucky winner will be announced next Tuesday (October 8th) – good luck! Can you believe it’s already been 4 years since Tailkit’s journey started? ❤️ It feels like just yesterday I launched Tailkit 1.0 as an offline standalone web app back on October 1st, 2020. I'm really attached to this project because it was launched just a few days before I became a dad 👶😍 Fast forward to today, and Tailkit has grown into a fully customizable, feature-packed online app that gives you access to: - 550+ Tailwind CSS components (fully responsive + dark mode support) - 1,750+ Code snippets for HTML, React, Vue.js, and Alpine.js - 7 Marketing & Application Templates for HTML, React, Vue.js, and Laravel (fully responsive + dark mode support) - 10 Starter Kits for HTML, HTML with Vite, React, Vue.js, Laravel, Next.js, Nuxt, Astro, Svelte and Angular - 30 Days of Unlimited Design Service with every new Team license - Exclusive deals on third-party tools and projects - Handy helper tools like Button Builder, Color Palette, and Icon Finder - 3,700+ SVG Icons from Bootstrap and Heroicons - Free lifetime updates - Email support whenever you need it And the 4.0 update is packed with even more awesome features and upgrades: - 6 new UI components (Notifications) were added in Application UI package - 20 new UI components (Image/Content Sliders++) were added in Marketing package - React version (uses Vite) is introduced for all Templates - React code snippets were improved in all packages - Astro Starter Kit was added - Remove Dark Classes (from code snippets) option was added in App - Universal Dark Mode (preview pane can default to global dark mode) option was added in App - New preview colors (fuchsia, gray) were added in App - Heroicons v2 icons were updated to v2.1.5 adding 28+ brand new icons in App - 3 new exclusive deals are now available - UI design improvements in App - Various improvements and fixes in App - All dependencies were updated in Starter Kits - All dependencies were updated in Templates - Marketing website redesigned - Color Palette tool was made available to all I’ve put over 200 hours into this update alone, and I’m beyond excited to finally share it with you. Your continued support has made this journey possible, and I’m so grateful to have you along for the ride. 🙏 Wishing you an amazing day – remember, YOU ARE AWESOME! Go build something incredible! 🚀
1:17

🎉 Tailkit 4.0 is here, and I couldn't be more excited! 🙌🥳 But first - Giveaway Alert! Want to get a free Tailkit Developer license? Just drop a reply and give a like or repost (totally optional, but super appreciated). The lucky winner will be announced next Tuesday (October 8th) – good luck! Can you believe it’s already been 4 years since Tailkit’s journey started? ❤️ It feels like just yesterday I launched Tailkit 1.0 as an offline standalone web app back on October 1st, 2020. I'm really attached to this project because it was launched just a few days before I became a dad 👶😍 Fast forward to today, and Tailkit has grown into a fully customizable, feature-packed online app that gives you access to: - 550+ Tailwind CSS components (fully responsive + dark mode support) - 1,750+ Code snippets for HTML, React, Vue.js, and Alpine.js - 7 Marketing & Application Templates for HTML, React, Vue.js, and Laravel (fully responsive + dark mode support) - 10 Starter Kits for HTML, HTML with Vite, React, Vue.js, Laravel, Next.js, Nuxt, Astro, Svelte and Angular - 30 Days of Unlimited Design Service with every new Team license - Exclusive deals on third-party tools and projects - Handy helper tools like Button Builder, Color Palette, and Icon Finder - 3,700+ SVG Icons from Bootstrap and Heroicons - Free lifetime updates - Email support whenever you need it And the 4.0 update is packed with even more awesome features and upgrades: - 6 new UI components (Notifications) were added in Application UI package - 20 new UI components (Image/Content Sliders++) were added in Marketing package - React version (uses Vite) is introduced for all Templates - React code snippets were improved in all packages - Astro Starter Kit was added - Remove Dark Classes (from code snippets) option was added in App - Universal Dark Mode (preview pane can default to global dark mode) option was added in App - New preview colors (fuchsia, gray) were added in App - Heroicons v2 icons were updated to v2.1.5 adding 28+ brand new icons in App - 3 new exclusive deals are now available - UI design improvements in App - Various improvements and fixes in App - All dependencies were updated in Starter Kits - All dependencies were updated in Templates - Marketing website redesigned - Color Palette tool was made available to all I’ve put over 200 hours into this update alone, and I’m beyond excited to finally share it with you. Your continued support has made this journey possible, and I’m so grateful to have you along for the ride. 🙏 Wishing you an amazing day – remember, YOU ARE AWESOME! Go build something incredible! 🚀

John Champ

10,940 次观看 • 1 年前

Everyone is talking about Vibe Coding (Using AI to Create Apps Only using AI) This is the most Comprehensive Guide for Vibe Coding with Cursor (By Far) 250 Minutes, All the vibe code basics of cursor, plus 4 Projects in one video! This is how I, as someone who has never written a line of code, approach building apps (every day). Part 1A Intro to Cursor, Composer, and some basics --------------------- 00:00 Intro 03:41 Downloading Cursor 06:09 What the hell is Composer? 10:47 A Note on Context and Keeping Composer Threads Small 11:38 Simple Desings with Cursor Composer From Blank Project 14:04 Editing a Simple Animation With Cursor Composer 16:35 Setting Up The Voice to Talk to Cursor Composer Whispr Flow 17:54 Lets an Early 2000's Landing Page Part 1B AI Image Generator --------------------- 23:59 Using the GitHub Template to Create a NextJS App 26:43 Template is Open, Let's Edit it 28:55 Drawing Out My Idea With Whimsical 30:11 First Prompt Using Place Holders For Image Generation 32:10 Accept All Vs Save All and Restoring in Composer (Saving your work) 33:54 Adding AI Feature (Brief Teaser, Deep Dive Later) 35:15 What is an API 37:22 Perplexity the best place to learn about API's 40:21 Api keys and running prompt for first AI Feature 42:48 Debugging, Woohoo! Learn to love this :) 43:20 Inspect - Console, In Browser Debugging Hack 48:02 AI Image Generation Works! Lets add more Part 2: Landing Page ---------------------- 51:03 Pause and Reflect, What have we done so far? 53:41 Plan for rest of video 54:34 Ok Let's Talk about (1) Designs 56:19 GitHub is like --sref for those who do image gen 58:20 Starting Cursor project from a GitHub Repo we found on Perplexity 01:00:48 Yolo Mode... Wtf is that? 01:02:38 Inspecting GitHub Repo's Examples, to use in our landing page 01:02:58 The Project We're making - A landing page 01:03:56 Landing Page from Screenshot 01:06:17 Making Changes to Landing Page 01:11:42 Making a more epic section 01:13:42 The Essence of Vibe Coding 01:15:17 Creating Cool Testimonials Section From Screenshot 01:18:18 Deploy to Vercel! But First New Repo on GitHub 01:20:45 Ok it's on GitHub... Now lets do vercel 01:21:17 Untechnical Explanation of what Vercel is Lol 01:24:18 Connecting Custom Domain (Bought on Name Cheap) To Vercel Deployment Part 3: App With Database and Authentication ---------------------- 01:27:59 Recap and Prep For The Bigger Project! 01:35:13 Getting Started from Template (Again) 01:38:52 Setting Up Database and Authentication (Firebase) 01:44:01 Back To Cursor, Let's Set up The Auth in the app 01:48:35 Switching to mermaid because compatibility issues 01:51:13 Using AI (Claude) to Generate Mermaid Diagrams 01:52:19 Adding Docs to Cursor to use AI Features over and over again 01:54:38 Let's Troubleshoot 01:56:10 Adding View Button and EDIT WITH AI 02:01:45 AI Diagram Edit Feature is DOPE 02:03:17 Using Search Feature on Cursor to find text in Codebase 02:05:55 Lets add ability to save these to Database 02:09:33 What does saved to Google Firebase even mean? 02:13:00 We can Export as PDF! 02:15:48 GitHub and Vercel Again! 02:17:27 Vercel with CLI From Cursor 02:20:52 Setting Vercel Domain as an Authorized Domain 02:27:34 How To Learn More
2:28:16

Everyone is talking about Vibe Coding (Using AI to Create Apps Only using AI) This is the most Comprehensive Guide for Vibe Coding with Cursor (By Far) 250 Minutes, All the vibe code basics of cursor, plus 4 Projects in one video! This is how I, as someone who has never written a line of code, approach building apps (every day). Part 1A Intro to Cursor, Composer, and some basics --------------------- 00:00 Intro 03:41 Downloading Cursor 06:09 What the hell is Composer? 10:47 A Note on Context and Keeping Composer Threads Small 11:38 Simple Desings with Cursor Composer From Blank Project 14:04 Editing a Simple Animation With Cursor Composer 16:35 Setting Up The Voice to Talk to Cursor Composer Whispr Flow 17:54 Lets an Early 2000's Landing Page Part 1B AI Image Generator --------------------- 23:59 Using the GitHub Template to Create a NextJS App 26:43 Template is Open, Let's Edit it 28:55 Drawing Out My Idea With Whimsical 30:11 First Prompt Using Place Holders For Image Generation 32:10 Accept All Vs Save All and Restoring in Composer (Saving your work) 33:54 Adding AI Feature (Brief Teaser, Deep Dive Later) 35:15 What is an API 37:22 Perplexity the best place to learn about API's 40:21 Api keys and running prompt for first AI Feature 42:48 Debugging, Woohoo! Learn to love this :) 43:20 Inspect - Console, In Browser Debugging Hack 48:02 AI Image Generation Works! Lets add more Part 2: Landing Page ---------------------- 51:03 Pause and Reflect, What have we done so far? 53:41 Plan for rest of video 54:34 Ok Let's Talk about (1) Designs 56:19 GitHub is like --sref for those who do image gen 58:20 Starting Cursor project from a GitHub Repo we found on Perplexity 01:00:48 Yolo Mode... Wtf is that? 01:02:38 Inspecting GitHub Repo's Examples, to use in our landing page 01:02:58 The Project We're making - A landing page 01:03:56 Landing Page from Screenshot 01:06:17 Making Changes to Landing Page 01:11:42 Making a more epic section 01:13:42 The Essence of Vibe Coding 01:15:17 Creating Cool Testimonials Section From Screenshot 01:18:18 Deploy to Vercel! But First New Repo on GitHub 01:20:45 Ok it's on GitHub... Now lets do vercel 01:21:17 Untechnical Explanation of what Vercel is Lol 01:24:18 Connecting Custom Domain (Bought on Name Cheap) To Vercel Deployment Part 3: App With Database and Authentication ---------------------- 01:27:59 Recap and Prep For The Bigger Project! 01:35:13 Getting Started from Template (Again) 01:38:52 Setting Up Database and Authentication (Firebase) 01:44:01 Back To Cursor, Let's Set up The Auth in the app 01:48:35 Switching to mermaid because compatibility issues 01:51:13 Using AI (Claude) to Generate Mermaid Diagrams 01:52:19 Adding Docs to Cursor to use AI Features over and over again 01:54:38 Let's Troubleshoot 01:56:10 Adding View Button and EDIT WITH AI 02:01:45 AI Diagram Edit Feature is DOPE 02:03:17 Using Search Feature on Cursor to find text in Codebase 02:05:55 Lets add ability to save these to Database 02:09:33 What does saved to Google Firebase even mean? 02:13:00 We can Export as PDF! 02:15:48 GitHub and Vercel Again! 02:17:27 Vercel with CLI From Cursor 02:20:52 Setting Vercel Domain as an Authorized Domain 02:27:34 How To Learn More

Riley Brown

368,141 次观看 • 1 年前

Use this prompt in OpenClaw to create your own AI agent command center that syncs up your life like Tony Stark's Jarvis in Iron Man. Adapt the specifics (agent names, data sources, branding) below to your own setup. Prompt: Build me a mission control dashboard for my OpenClaw AI agent system. Stack: Next.js 15 (App Router) + Convex (real-time backend) + Tailwind CSS v4 + Framer Motion + ShadCN UI + Lucide icons. TypeScript throughout. This is the command center where I monitor and control my autonomous AI agent(s) running on OpenClaw. The agent operates 24/7 on a Mac Mini, connected to Telegram/Discord, running cron jobs, spawning sub-agents, and reading/writing to a filesystem-based memory and state system. Dark mode only. Ultra-premium aesthetic, think Iron Man's JARVIS HUD meets a Bloomberg terminal. Subtle glass effects (backdrop-blur-xl, bg-white/[0.03]), no heavy gradients or glow. Rounded corners (16-20px on cards). Framer Motion for page transitions, stagger animations on card grids, spring physics on interactions. Mobile-first responsive. Never cookie-cutter. ## Architecture The dashboard reads live data from TWO sources: 1. **Convex**: real-time database for structured data (tasks, contacts, content drafts, calendar events, activity logs) 2. **Local API routes** (`/api/*`): read files from the agent's workspace filesystem at `~/.openclaw/workspace/` and return JSON. This is how live system state flows into the dashboard. ## Pages & Views (8 nav items, some with tab sub-views) ### 1. HOME (`/`) Dashboard overview. Grid of live status cards: - **System Health**: read from `/api/system-state` (parses `state/servers.json`). Show each service with UP/DOWN indicator, port, last check time. - **Agent Status**: read from `/api/agents` (parses `agents/registry.json` + agent workspace files). Show active agent count, healthy/unhealthy ratio, active sub-agent count from OpenClaw sessions API. - **Cron Health**: read from `/api/cron-health` (parses `state/crons.json`). Table of all scheduled jobs with name, schedule, last status (green/red dot), consecutive errors. - **Revenue Tracker**: read from `/api/revenue` (parses `state/revenue.json`). Current revenue, monthly burn, net. - **Content Pipeline**: read from `/api/content-pipeline` (parses `content/queue.md`). Kanban-style: Draft | Review | Approved | Published counts. - **Quick Stats**: total tasks, pending approvals, active sessions, uptime. All panels auto-refresh every 15 seconds. Live indicator dot + "AUTO 15S" badge in header. ### 2. OPS (`/ops`) with 3 tabs: Operations | Tasks | Calendar **Operations tab:** Full operational view. Server health table, branch status (from `state/branch-check.json`), observations feed (from `state/observations.md`), system priorities (from `shared-context/priorities.md`). **Tasks tab:** Strategic task suggestion system. API route `/api/suggested-tasks` reads/writes `state/suggested-tasks.json`. Cards grouped by category (Revenue, Product, Community, Content, Operations, Clients, Trading, Brand) with emoji headers. Each card shows title, reasoning, next action, priority badge, effort badge, approve/reject buttons. Filter bar by status and category. **Calendar tab:** Weekly calendar view from Convex `calendarEvents` table. Drag-to-create, color-coded by type, time slots. ### 3. AGENTS (`/agents`) with 2 tabs: Agents | Models **Agents tab:** Card grid of all registered agents from `/api/agents`. Each card shows name, role, model, level (L1-L4), status. Cards are CLICKABLE: expanding into a detail panel showing: - Agent personality (reads their SOUL .md) - Capabilities and rules (reads their RULES .md) - Sub-agents they can spawn - Recent outputs (reads from `shared-context/agent-outputs/`) **Models tab:** Model inventory table showing all available models, their routing (which tasks go to which model), costs, and failover chains. ### 4. CHAT (`/chat`): 2 tabs: Chat | Command **Chat tab:** Chat interface to communicate with the agent. Left sidebar shows session list (from `/api/chat-history` reading .jsonl transcript files). Main area shows messages with role-aligned bubbles (user right, assistant left), date separators, channel badges (telegram/discord/webchat). Input bar with send button + voice input (Web Speech API with SpeechRecognition). Messages sent via `/api/chat-send` which queues to a file the agent reads. **Command tab:** Quick command interface for common operations. ### 5. CONTENT (`/content`) Content pipeline management. Read from Convex `contentDrafts` table AND `/api/content-pipeline`. Show drafts in kanban columns. Each card shows title, platform target, draft text preview, status, created date. Edit/approve/reject actions. ### 6. COMMS (`/comms`) with 2 tabs: Comms | CRM **Comms tab:** Communication hub showing recent Discord digest, Telegram messages, notification history. **CRM tab:** Client pipeline kanban (Prospect → Contacted → Meeting → Proposal → Active). API route `/api/clients` reads markdown files from `clients/` directory. Each card shows client name, status, contacts, last interaction, next action. ### 7. KNOWLEDGE (`/knowledge`) with 2 tabs: Knowledge | Ecosystem **Knowledge tab:** Searchable knowledge base. Global search across all workspace files using `/api/knowledge` endpoint. **Ecosystem tab:** Product grid showing all products/apps in the ecosystem. Each card shows product name, status (Active/Development/Concept), health indicator, key metrics. Cards link to `/ecosystem/[slug]` detail pages with tabbed views (Overview, Brand, Community, Content, Legal, Product, Website, Actions). Detail pages read from `/api/ecosystem/[slug]` which parses workspace memory files. ### 8. CODE (`/code`) Code pipeline view. Shows repositories from `/api/repos` (scans ~/Desktop/Projects/ for git repos). Each repo card shows name, branch, last commit, dirty file count, language breakdown. Detail view at `/api/repos/detail` shows recent commits, file tree, open PRs. ## Navigation Top horizontal nav bar, NOT sidebar. All 8 items visible at all viewport widths. Use `flex` layout with `flex-1` items. Text size uses `clamp(0.45rem, 0.75vw, 0.6875rem)` for fluid scaling. Active item gets `text-primary bg-primary/[0.06]` static highlight (no sliding animation). Agent/app name visible at md+ breakpoints (`hidden md:inline`). Tab sub-views use a reusable `TabBar` component with pill/glass styling and Framer Motion `layoutId` transitions. Tab state stored in URL via `?tab=` search params. ## API Routes (all under `src/app/api/`) Each API route reads from the agent's workspace filesystem and returns JSON: - `/api/system-state` → reads `state/servers.json`, `state/branch-check.json` - `/api/agents` → reads `agents/registry.json`, agent SOUL .md files - `/api/agents/[id]` → reads specific agent's SOUL .md, RULES .md, outputs - `/api/cron-health` → reads `state/crons.json` - `/api/revenue` → reads `state/revenue.json` - `/api/content-pipeline` → parses `content/queue.md` (markdown with status markers) - `/api/suggested-tasks` → GET (read) / POST (approve/reject) on `state/suggested-tasks.json` - `/api/observations` → reads `state/observations.md` - `/api/priorities` → reads `shared-context/priorities.md` - `/api/chat-history` → reads .jsonl transcript files with pagination/search/channel filter - `/api/chat-send` → writes to queue file - `/api/clients` → reads markdown files from `clients/` directory - `/api/ecosystem/[slug]` → reads memory files for specific ecosystem - `/api/repos` → scans project directories for git repos - `/api/health` → returns status, uptime, memory usage, Convex connectivity All filesystem paths should be configurable via environment variable (default: `~/.openclaw/workspace/`). ## Convex Schema Define tables for: activities, calendarEvents, tasks, contacts, contentDrafts, ecosystemProducts. Include seed scripts (`convex/seed.ts`) to populate initial data. ## Key Design Rules - Mobile-first, test at 320px minimum - Font sizes 10-14px for body text, everything must fit naturally at small viewports - Cards use consistent border radius (16-20px) - Glass cards: `bg-white/[0.03] backdrop-blur-xl border border-white/[0.06]` - No heavy blur blobs or grain overlays - Stagger animations on card grids (0.05s delay per item) - Skeleton loading states for all async data - Custom scrollbar styling - Empty states with helpful messaging - All text must use Inter or system font stack - Never mix sharp and rounded corners in the same view - Premium = lighter feel, more whitespace, less visual noise ## File Structure ``` src/ app/ page.tsx, layout.tsx, providers.tsx agents/page.tsx calendar/page.tsx chat/page.tsx code/page.tsx comms/page.tsx content/page.tsx ecosystem/page.tsx, ecosystem/[slug]/page.tsx knowledge/page.tsx ops/page.tsx api/[...all routes above] components/ nav.tsx tab-bar.tsx dashboard-overview.tsx ops-view.tsx, suggested-tasks-view.tsx agents-view.tsx, models-view.tsx chat-center-view.tsx, voice-input.tsx content-view.tsx comms-view.tsx, crm-view.tsx knowledge-base.tsx, ecosystem-view.tsx code-pipeline.tsx activity-feed.tsx, calendar-view.tsx ui/ (ShadCN primitives) hooks/ lib/ convex/ schema.ts functions for each table seed.ts ``` Build the complete application. Every component, every API route, every Convex function. Production-quality code and premium design, not stubs. Dark mode only. Make it look incredibly beautiful and premium, no cookie cutter UI / AI slop.
1:14

Use this prompt in OpenClaw to create your own AI agent command center that syncs up your life like Tony Stark's Jarvis in Iron Man. Adapt the specifics (agent names, data sources, branding) below to your own setup. Prompt: Build me a mission control dashboard for my OpenClaw AI agent system. Stack: Next.js 15 (App Router) + Convex (real-time backend) + Tailwind CSS v4 + Framer Motion + ShadCN UI + Lucide icons. TypeScript throughout. This is the command center where I monitor and control my autonomous AI agent(s) running on OpenClaw. The agent operates 24/7 on a Mac Mini, connected to Telegram/Discord, running cron jobs, spawning sub-agents, and reading/writing to a filesystem-based memory and state system. Dark mode only. Ultra-premium aesthetic, think Iron Man's JARVIS HUD meets a Bloomberg terminal. Subtle glass effects (backdrop-blur-xl, bg-white/[0.03]), no heavy gradients or glow. Rounded corners (16-20px on cards). Framer Motion for page transitions, stagger animations on card grids, spring physics on interactions. Mobile-first responsive. Never cookie-cutter. ## Architecture The dashboard reads live data from TWO sources: 1. **Convex**: real-time database for structured data (tasks, contacts, content drafts, calendar events, activity logs) 2. **Local API routes** (`/api/*`): read files from the agent's workspace filesystem at `~/.openclaw/workspace/` and return JSON. This is how live system state flows into the dashboard. ## Pages & Views (8 nav items, some with tab sub-views) ### 1. HOME (`/`) Dashboard overview. Grid of live status cards: - **System Health**: read from `/api/system-state` (parses `state/servers.json`). Show each service with UP/DOWN indicator, port, last check time. - **Agent Status**: read from `/api/agents` (parses `agents/registry.json` + agent workspace files). Show active agent count, healthy/unhealthy ratio, active sub-agent count from OpenClaw sessions API. - **Cron Health**: read from `/api/cron-health` (parses `state/crons.json`). Table of all scheduled jobs with name, schedule, last status (green/red dot), consecutive errors. - **Revenue Tracker**: read from `/api/revenue` (parses `state/revenue.json`). Current revenue, monthly burn, net. - **Content Pipeline**: read from `/api/content-pipeline` (parses `content/queue.md`). Kanban-style: Draft | Review | Approved | Published counts. - **Quick Stats**: total tasks, pending approvals, active sessions, uptime. All panels auto-refresh every 15 seconds. Live indicator dot + "AUTO 15S" badge in header. ### 2. OPS (`/ops`) with 3 tabs: Operations | Tasks | Calendar **Operations tab:** Full operational view. Server health table, branch status (from `state/branch-check.json`), observations feed (from `state/observations.md`), system priorities (from `shared-context/priorities.md`). **Tasks tab:** Strategic task suggestion system. API route `/api/suggested-tasks` reads/writes `state/suggested-tasks.json`. Cards grouped by category (Revenue, Product, Community, Content, Operations, Clients, Trading, Brand) with emoji headers. Each card shows title, reasoning, next action, priority badge, effort badge, approve/reject buttons. Filter bar by status and category. **Calendar tab:** Weekly calendar view from Convex `calendarEvents` table. Drag-to-create, color-coded by type, time slots. ### 3. AGENTS (`/agents`) with 2 tabs: Agents | Models **Agents tab:** Card grid of all registered agents from `/api/agents`. Each card shows name, role, model, level (L1-L4), status. Cards are CLICKABLE: expanding into a detail panel showing: - Agent personality (reads their SOUL .md) - Capabilities and rules (reads their RULES .md) - Sub-agents they can spawn - Recent outputs (reads from `shared-context/agent-outputs/`) **Models tab:** Model inventory table showing all available models, their routing (which tasks go to which model), costs, and failover chains. ### 4. CHAT (`/chat`): 2 tabs: Chat | Command **Chat tab:** Chat interface to communicate with the agent. Left sidebar shows session list (from `/api/chat-history` reading .jsonl transcript files). Main area shows messages with role-aligned bubbles (user right, assistant left), date separators, channel badges (telegram/discord/webchat). Input bar with send button + voice input (Web Speech API with SpeechRecognition). Messages sent via `/api/chat-send` which queues to a file the agent reads. **Command tab:** Quick command interface for common operations. ### 5. CONTENT (`/content`) Content pipeline management. Read from Convex `contentDrafts` table AND `/api/content-pipeline`. Show drafts in kanban columns. Each card shows title, platform target, draft text preview, status, created date. Edit/approve/reject actions. ### 6. COMMS (`/comms`) with 2 tabs: Comms | CRM **Comms tab:** Communication hub showing recent Discord digest, Telegram messages, notification history. **CRM tab:** Client pipeline kanban (Prospect → Contacted → Meeting → Proposal → Active). API route `/api/clients` reads markdown files from `clients/` directory. Each card shows client name, status, contacts, last interaction, next action. ### 7. KNOWLEDGE (`/knowledge`) with 2 tabs: Knowledge | Ecosystem **Knowledge tab:** Searchable knowledge base. Global search across all workspace files using `/api/knowledge` endpoint. **Ecosystem tab:** Product grid showing all products/apps in the ecosystem. Each card shows product name, status (Active/Development/Concept), health indicator, key metrics. Cards link to `/ecosystem/[slug]` detail pages with tabbed views (Overview, Brand, Community, Content, Legal, Product, Website, Actions). Detail pages read from `/api/ecosystem/[slug]` which parses workspace memory files. ### 8. CODE (`/code`) Code pipeline view. Shows repositories from `/api/repos` (scans ~/Desktop/Projects/ for git repos). Each repo card shows name, branch, last commit, dirty file count, language breakdown. Detail view at `/api/repos/detail` shows recent commits, file tree, open PRs. ## Navigation Top horizontal nav bar, NOT sidebar. All 8 items visible at all viewport widths. Use `flex` layout with `flex-1` items. Text size uses `clamp(0.45rem, 0.75vw, 0.6875rem)` for fluid scaling. Active item gets `text-primary bg-primary/[0.06]` static highlight (no sliding animation). Agent/app name visible at md+ breakpoints (`hidden md:inline`). Tab sub-views use a reusable `TabBar` component with pill/glass styling and Framer Motion `layoutId` transitions. Tab state stored in URL via `?tab=` search params. ## API Routes (all under `src/app/api/`) Each API route reads from the agent's workspace filesystem and returns JSON: - `/api/system-state` → reads `state/servers.json`, `state/branch-check.json` - `/api/agents` → reads `agents/registry.json`, agent SOUL .md files - `/api/agents/[id]` → reads specific agent's SOUL .md, RULES .md, outputs - `/api/cron-health` → reads `state/crons.json` - `/api/revenue` → reads `state/revenue.json` - `/api/content-pipeline` → parses `content/queue.md` (markdown with status markers) - `/api/suggested-tasks` → GET (read) / POST (approve/reject) on `state/suggested-tasks.json` - `/api/observations` → reads `state/observations.md` - `/api/priorities` → reads `shared-context/priorities.md` - `/api/chat-history` → reads .jsonl transcript files with pagination/search/channel filter - `/api/chat-send` → writes to queue file - `/api/clients` → reads markdown files from `clients/` directory - `/api/ecosystem/[slug]` → reads memory files for specific ecosystem - `/api/repos` → scans project directories for git repos - `/api/health` → returns status, uptime, memory usage, Convex connectivity All filesystem paths should be configurable via environment variable (default: `~/.openclaw/workspace/`). ## Convex Schema Define tables for: activities, calendarEvents, tasks, contacts, contentDrafts, ecosystemProducts. Include seed scripts (`convex/seed.ts`) to populate initial data. ## Key Design Rules - Mobile-first, test at 320px minimum - Font sizes 10-14px for body text, everything must fit naturally at small viewports - Cards use consistent border radius (16-20px) - Glass cards: `bg-white/[0.03] backdrop-blur-xl border border-white/[0.06]` - No heavy blur blobs or grain overlays - Stagger animations on card grids (0.05s delay per item) - Skeleton loading states for all async data - Custom scrollbar styling - Empty states with helpful messaging - All text must use Inter or system font stack - Never mix sharp and rounded corners in the same view - Premium = lighter feel, more whitespace, less visual noise ## File Structure ``` src/ app/ page.tsx, layout.tsx, providers.tsx agents/page.tsx calendar/page.tsx chat/page.tsx code/page.tsx comms/page.tsx content/page.tsx ecosystem/page.tsx, ecosystem/[slug]/page.tsx knowledge/page.tsx ops/page.tsx api/[...all routes above] components/ nav.tsx tab-bar.tsx dashboard-overview.tsx ops-view.tsx, suggested-tasks-view.tsx agents-view.tsx, models-view.tsx chat-center-view.tsx, voice-input.tsx content-view.tsx comms-view.tsx, crm-view.tsx knowledge-base.tsx, ecosystem-view.tsx code-pipeline.tsx activity-feed.tsx, calendar-view.tsx ui/ (ShadCN primitives) hooks/ lib/ convex/ schema.ts functions for each table seed.ts ``` Build the complete application. Every component, every API route, every Convex function. Production-quality code and premium design, not stubs. Dark mode only. Make it look incredibly beautiful and premium, no cookie cutter UI / AI slop.

klöss

201,167 次观看 • 5 个月前

AGI? One day, but not yet. The only AI that works well right now is the one behind the screen [12-17]. But passing the Turing Test [9] behind a screen is easy compared to Real AI for real robots in the real world. No current AI-driven robot could be certified as a plumber [13-17]. Hence, the Turing Test isn't a good measure of intelligence (and neither is IQ). And AGI without mastery of the physical world is no AGI. That’s why I created the TUM CogBotLab for learning robots in 2004 [5], co-founded a company for AI in the physical world in 2014 [6], and had teams at TUM, IDSIA, and now KAUST work towards baby robots [4,10-11,18]. Such soft robots don't just slavishly imitate humans and they don't work by just downloading the web like LLMs/VLMs. No. Instead, they exploit the principles of Artificial Curiosity to improve their neural World Models (two terms I used back in 1990 [1-4]). These robots work with lots of sensors, but only weak actuators, such that they cannot easily harm themselves [18] when they collect useful data by devising and running their own self-invented experiments. Remarkably, since the 1970s, many have made fun of my old goal to build a self-improving AGI smarter than myself and then retire. Recently, however, many have finally started to take this seriously, and now some of them are suddenly TOO optimistic. These people are often blissfully unaware of the remaining challenges we have to solve to achieve Real AI. My 2024 TED talk [15] summarises some of that. REFERENCES (easy to find on the web): [1] J. Schmidhuber. Making the world differentiable: On using fully recurrent self-supervised neural networks (NNs) for dynamic reinforcement learning and planning in non-stationary environments. TR FKI-126-90, TUM, Feb 1990, revised Nov 1990. This paper also introduced artificial curiosity and intrinsic motivation through generative adversarial networks where a generator NN is fighting a predictor NN in a minimax game. [2] J. S. A possibility for implementing curiosity and boredom in model-building neural controllers. In J. A. Meyer and S. W. Wilson, editors, Proc. of the International Conference on Simulation of Adaptive Behavior: From Animals to Animats, pages 222-227. MIT Press/Bradford Books, 1991. Based on [1]. [3] J.S. AI Blog (2020). 1990: Planning & Reinforcement Learning with Recurrent World Models and Artificial Curiosity. Summarising aspects of [1][2] and lots of later papers including [7][8]. [4] J.S. AI Blog (2021): Artificial Curiosity & Creativity Since 1990. Summarising aspects of [1][2] and lots of later papers including [7][8]. [5] J.S. TU Munich CogBotLab for learning robots (2004-2009) [6] NNAISENSE, founded in 2014, for AI in the physical world [7] J.S. (2015). On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning (RL) Controllers and Recurrent Neural World Models. arXiv 1210.0118. Sec. 5.3 describes an RL prompt engineer which learns to query its model for abstract reasoning and planning and decision making. Today this is called "chain of thought." [8] J.S. (2018). One Big Net For Everything. arXiv 1802.08864. See also patent US11853886B2 and my DeepSeek tweet: DeepSeek uses elements of the 2015 reinforcement learning prompt engineer [7] and its 2018 refinement [8] which collapses the RL machine and world model of [7] into a single net. This uses my neural net distillation procedure of 1991: a distilled chain of thought system. [9] J.S. Turing Oversold. It's not Turing's fault, though. AI Blog (2021, was #1 on Hacker News) [10] J.S. Intelligente Roboter werden vom Leben fasziniert sein. (Intelligent robots will be fascinated by life.) F.A.Z., 2015 [11] J.S. at Falling Walls: The Past, Present and Future of Artificial Intelligence. Scientific American, Observations, 2017. [12] J.S. KI ist eine Riesenchance für Deutschland. (AI is a huge chance for Germany.) F.A.Z., 2018 [13] H. Jones. J.S. Says His Life's Work Won't Lead To Dystopia. Forbes Magazine, 2023. [14] Interview with J.S. Jazzyear, Shanghai, 2024. [15] J.S. TED talk at TED AI Vienna (2024): Why 2042 will be a big year for AI. See the attached video clip. [16] J.S. Baut den KI-gesteuerten Allzweckroboter! (Build the AI-controlled all-purpose robot!) F.A.Z., 2024 [17] J.S. 1995-2025: The Decline of Germany & Japan vs US & China. Can All-Purpose Robots Fuel a Comeback? AI Blog, Jan 2025, based on [16]. [18] M. Alhakami, D. R. Ashley, J. Dunham, Y. Dai, F. Faccio, E. Feron, J. Schmidhuber. Towards an Extremely Robust Baby Robot With Rich Interaction Ability for Advanced Machine Learning Algorithms. Preprint arxiv 2404.08093, 2024.
1:41

AGI? One day, but not yet. The only AI that works well right now is the one behind the screen [12-17]. But passing the Turing Test [9] behind a screen is easy compared to Real AI for real robots in the real world. No current AI-driven robot could be certified as a plumber [13-17]. Hence, the Turing Test isn't a good measure of intelligence (and neither is IQ). And AGI without mastery of the physical world is no AGI. That’s why I created the TUM CogBotLab for learning robots in 2004 [5], co-founded a company for AI in the physical world in 2014 [6], and had teams at TUM, IDSIA, and now KAUST work towards baby robots [4,10-11,18]. Such soft robots don't just slavishly imitate humans and they don't work by just downloading the web like LLMs/VLMs. No. Instead, they exploit the principles of Artificial Curiosity to improve their neural World Models (two terms I used back in 1990 [1-4]). These robots work with lots of sensors, but only weak actuators, such that they cannot easily harm themselves [18] when they collect useful data by devising and running their own self-invented experiments. Remarkably, since the 1970s, many have made fun of my old goal to build a self-improving AGI smarter than myself and then retire. Recently, however, many have finally started to take this seriously, and now some of them are suddenly TOO optimistic. These people are often blissfully unaware of the remaining challenges we have to solve to achieve Real AI. My 2024 TED talk [15] summarises some of that. REFERENCES (easy to find on the web): [1] J. Schmidhuber. Making the world differentiable: On using fully recurrent self-supervised neural networks (NNs) for dynamic reinforcement learning and planning in non-stationary environments. TR FKI-126-90, TUM, Feb 1990, revised Nov 1990. This paper also introduced artificial curiosity and intrinsic motivation through generative adversarial networks where a generator NN is fighting a predictor NN in a minimax game. [2] J. S. A possibility for implementing curiosity and boredom in model-building neural controllers. In J. A. Meyer and S. W. Wilson, editors, Proc. of the International Conference on Simulation of Adaptive Behavior: From Animals to Animats, pages 222-227. MIT Press/Bradford Books, 1991. Based on [1]. [3] J.S. AI Blog (2020). 1990: Planning & Reinforcement Learning with Recurrent World Models and Artificial Curiosity. Summarising aspects of [1][2] and lots of later papers including [7][8]. [4] J.S. AI Blog (2021): Artificial Curiosity & Creativity Since 1990. Summarising aspects of [1][2] and lots of later papers including [7][8]. [5] J.S. TU Munich CogBotLab for learning robots (2004-2009) [6] NNAISENSE, founded in 2014, for AI in the physical world [7] J.S. (2015). On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning (RL) Controllers and Recurrent Neural World Models. arXiv 1210.0118. Sec. 5.3 describes an RL prompt engineer which learns to query its model for abstract reasoning and planning and decision making. Today this is called "chain of thought." [8] J.S. (2018). One Big Net For Everything. arXiv 1802.08864. See also patent US11853886B2 and my DeepSeek tweet: DeepSeek uses elements of the 2015 reinforcement learning prompt engineer [7] and its 2018 refinement [8] which collapses the RL machine and world model of [7] into a single net. This uses my neural net distillation procedure of 1991: a distilled chain of thought system. [9] J.S. Turing Oversold. It's not Turing's fault, though. AI Blog (2021, was #1 on Hacker News) [10] J.S. Intelligente Roboter werden vom Leben fasziniert sein. (Intelligent robots will be fascinated by life.) F.A.Z., 2015 [11] J.S. at Falling Walls: The Past, Present and Future of Artificial Intelligence. Scientific American, Observations, 2017. [12] J.S. KI ist eine Riesenchance für Deutschland. (AI is a huge chance for Germany.) F.A.Z., 2018 [13] H. Jones. J.S. Says His Life's Work Won't Lead To Dystopia. Forbes Magazine, 2023. [14] Interview with J.S. Jazzyear, Shanghai, 2024. [15] J.S. TED talk at TED AI Vienna (2024): Why 2042 will be a big year for AI. See the attached video clip. [16] J.S. Baut den KI-gesteuerten Allzweckroboter! (Build the AI-controlled all-purpose robot!) F.A.Z., 2024 [17] J.S. 1995-2025: The Decline of Germany & Japan vs US & China. Can All-Purpose Robots Fuel a Comeback? AI Blog, Jan 2025, based on [16]. [18] M. Alhakami, D. R. Ashley, J. Dunham, Y. Dai, F. Faccio, E. Feron, J. Schmidhuber. Towards an Extremely Robust Baby Robot With Rich Interaction Ability for Advanced Machine Learning Algorithms. Preprint arxiv 2404.08093, 2024.

Jürgen Schmidhuber

72,331 次观看 • 1 年前