Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

AI interfaces should go beyond a chatbot: generative UI is next. Here's a demo of a library for LLMs that I've been working on. It prompts models with a vocabulary of rich components described using natural language. Presented at Causal Islands Conference last week.

rupertmanfredi's profile picture

ruperts.world

1,736 subscribers

186,060 görüntüleme • 3 yıl önce •via X (Twitter)

EğitimBilim & TeknolojiHaberler & Politika
LIVE

Anya Rossi• Live Now

Private livecam show

10 Yorum

Rupert profil fotoğrafı
Rupert3 yıl önce

This has come out of some of our innovation work at Mozilla, for more: May open source this officially if there's community interest.

Rupert profil fotoğrafı
Rupert3 yıl önce

Open sourced here!

Jesse Ezell profil fotoğrafı
Jesse Ezell3 yıl önce

@rupert_parry @causalislands I think a lot of people miss that the reason the chatbot is so successful is that the power is in non one shot scenarios. If you provide a google style single shot prompt, you are throwing away the best part.

Rupert profil fotoğrafı
Rupert3 yıl önce

@causalislands No reason it needs to be either/or

max drake (⨍) profil fotoğrafı
max drake (⨍)3 yıl önce

@rupert_parry @causalislands this is very refreshing! are you using the ReAct pattern and giving it tools that correspond to each UI element?

Rupert profil fotoğrafı
Rupert3 yıl önce

@causalislands Exactly – the model is prompted with a simple schema of components, as you would prompt with actions in ReAct. The output is runtime-agnostic – in this case I’ve implemented the output schema with web components (but it could be React, Swift UI, etc).

Luca Matteis profil fotoğrafı
Luca Matteis3 yıl önce

@rupert_parry @hwk73 @causalislands And yet it got the carbonara wrong. It's with pecorino, not parmigiano

Jason Yuan profil fotoğrafı
Jason Yuan3 yıl önce

@rupert_parry @causalislands Love it!

Alex Bowe profil fotoğrafı
Alex Bowe3 yıl önce

@rupert_parry @causalislands That's cool :) Would be neat to pre-emptively generate UI for likely subqueries (e.g. a dropdown menu + slider for cost constraints, or allowing users to click "Rome" and add other nearby destinations).

Slavo profil fotoğrafı
Slavo3 yıl önce

@rupert_parry @Scobleizer @causalislands This is already available in @weweb_io for some time now.

Benzer Videolar

AI + robotics research is starting to pick up steam. We can now instruct Spot using natural language using Language-guided Skill Coordination (LSC). A user provides a natural language instruction: "Bring me the chocolates box, cereal box, and pill bottle, and put them on the bedroom table" and the robot navigates to the location of the target objects and places them on the room table. Current versions of Spot rely on a fixed-set vocabulary that cannot generalize to diverse instructions. In this demo, the researchers presented a method that uses large language models to receive a free-form natural language instruction for object rearrangement, which it then executes. It combines: -A voice-to-text model that processes the instructions into text -An LLM that converts natural language instruction to call a library of skills -A perception module that provides ground truth locations and visual object detection And what your left with is a glimpse into what our future will hold: robots executing tasks from human voice prompts.
0:30

AI + robotics research is starting to pick up steam. We can now instruct Spot using natural language using Language-guided Skill Coordination (LSC). A user provides a natural language instruction: "Bring me the chocolates box, cereal box, and pill bottle, and put them on the bedroom table" and the robot navigates to the location of the target objects and places them on the room table. Current versions of Spot rely on a fixed-set vocabulary that cannot generalize to diverse instructions. In this demo, the researchers presented a method that uses large language models to receive a free-form natural language instruction for object rearrangement, which it then executes. It combines: -A voice-to-text model that processes the instructions into text -An LLM that converts natural language instruction to call a library of skills -A perception module that provides ground truth locations and visual object detection And what your left with is a glimpse into what our future will hold: robots executing tasks from human voice prompts.

AI Breakfast

33,725 görüntüleme • 3 yıl önce

I've created a monster 👀 UI design is dead ☠️ I'm working on a library that will allow you to reliably build components or full systems with ANY design aesthetic. Check it out: Brutaltech, Earlycyber and Swiss ↓
0:38

I've created a monster 👀 UI design is dead ☠️ I'm working on a library that will allow you to reliably build components or full systems with ANY design aesthetic. Check it out: Brutaltech, Earlycyber and Swiss ↓

Josh Millgate

181,388 görüntüleme • 4 ay önce

"Maybe the right UI for AI is a generative UI." Here's my new interview with Stephen Chau featuring a live demo of Cove's ground-breaking visual AI interface that breaks free from chatbots. We had a great chat about: - Real world use cases for visual AI interfaces - Generative UI and just-in-time AI apps - How Stephen scaled Uber Eats to $25B ARR Some quotes from Stephen Chau: "In computing, we moved from the command line to graphic user interfaces. Chatbots are like the command line era of AI. What's next is this..." "Imagine the entire layout of the workspace and all its contents is something made just in time, based on what you're trying to do..." "Our goal with Uber Eats was to build a business that's the same size as the Rides business. Here's what mattered..." 📌 Watch now:
1:09

"Maybe the right UI for AI is a generative UI." Here's my new interview with Stephen Chau featuring a live demo of Cove's ground-breaking visual AI interface that breaks free from chatbots. We had a great chat about: - Real world use cases for visual AI interfaces - Generative UI and just-in-time AI apps - How Stephen scaled Uber Eats to $25B ARR Some quotes from Stephen Chau: "In computing, we moved from the command line to graphic user interfaces. Chatbots are like the command line era of AI. What's next is this..." "Imagine the entire layout of the workspace and all its contents is something made just in time, based on what you're trying to do..." "Our goal with Uber Eats was to build a business that's the same size as the Rides business. Here's what mattered..." 📌 Watch now:

Peter Yang

74,721 görüntüleme • 1 yıl önce

There's an Israeli influence operation called "Generative AI for Good" that creates AI "victims" of Iran to agitate for regime change. They just held a conference last week in NYC. Here's a short clip of one of the deepfake propaganda videos they released a few weeks ago.
0:36

There's an Israeli influence operation called "Generative AI for Good" that creates AI "victims" of Iran to agitate for regime change. They just held a conference last week in NYC. Here's a short clip of one of the deepfake propaganda videos they released a few weeks ago.

Chris Menahan 🇺🇸

1,632,220 görüntüleme • 4 ay önce

demo of a The Browser Company AI feature for Arc manipulate websites (via code) using natural language longer video with context on Browser Co YouTube page
1:22

demo of a The Browser Company AI feature for Arc manipulate websites (via code) using natural language longer video with context on Browser Co YouTube page

Josh Miller

223,977 görüntüleme • 3 yıl önce

I made a guide for prompting impressive UI code After creating 120 components in a week while working on Aura, listed all the techniques I used: frameworks, styles, framing, html-to-code, image-to-code, etc. AI is getting so good at creating user interfaces.
0:37

I made a guide for prompting impressive UI code After creating 120 components in a week while working on Aura, listed all the techniques I used: frameworks, styles, framing, html-to-code, image-to-code, etc. AI is getting so good at creating user interfaces.

Meng To

117,197 görüntüleme • 1 yıl önce

ok so I've been working on an AI tool that scans your website for conversion issues and generates a deep CRO audit report that agencies charge $1-$5k for. here's a quick demo:
2:39

ok so I've been working on an AI tool that scans your website for conversion issues and generates a deep CRO audit report that agencies charge $1-$5k for. here's a quick demo:

Yash Bhardwaj

12,599 görüntüleme • 1 yıl önce

Here's a thing I've been passionately working away on: What if I told you that one of these characters is using a Motion Matching setup, and the other is using a simple State Machine (sort of)? Would you believe me? Take a guess at which is which.
0:38

Here's a thing I've been passionately working away on: What if I told you that one of these characters is using a Motion Matching setup, and the other is using a simple State Machine (sort of)? Would you believe me? Take a guess at which is which.

Caleb Longmire

169,225 görüntüleme • 1 yıl önce

ChatGPT but for maps.💡Here's a first look at our work on a Smart Mapping Assistant prototype that combines the power of GIS with generative AI. #AI #GIS
1:50

ChatGPT but for maps.💡Here's a first look at our work on a Smart Mapping Assistant prototype that combines the power of GIS with generative AI. #AI #GIS

Esri

29,839 görüntüleme • 1 yıl önce

Gemini 2.5 Flash-Lite for instant UI! 🚀 Just vibe coded a chat app that only responds in dynamically generated UI to your prompts. No generation time was sped up. Enjoy five quick components in a matter of seconds, fun to see what fast models will enable. Demo + code below!
1:06

Gemini 2.5 Flash-Lite for instant UI! 🚀 Just vibe coded a chat app that only responds in dynamically generated UI to your prompts. No generation time was sped up. Enjoy five quick components in a matter of seconds, fun to see what fast models will enable. Demo + code below!

Ammaar Reshi

122,736 görüntüleme • 1 yıl önce

Very cool demo of Cosmos Transfer 2.5 at the NVIDIA Robotics booth during CES 2026. Developers can take a single recorded robot movement and generate hundreds of variations using natural language prompts. Thanks Spencer Huang and Edith Llontop for the demo!
3:11

Very cool demo of Cosmos Transfer 2.5 at the NVIDIA Robotics booth during CES 2026. Developers can take a single recorded robot movement and generate hundreds of variations using natural language prompts. Thanks Spencer Huang and Edith Llontop for the demo!

The Humanoid Hub

30,093 görüntüleme • 7 ay önce

Aesthetron AI is now live in Figma community! New users can now get 100 credits for free. Here's a simple demo of using Aesthetron AI and Figma native image generation to create a product card UI. Built with Figma AI SDK Polar
0:21

Aesthetron AI is now live in Figma community! New users can now get 100 credits for free. Here's a simple demo of using Aesthetron AI and Figma native image generation to create a product card UI. Built with Figma AI SDK Polar

Joshua Guo

17,024 görüntüleme • 8 ay önce

For the last couple months, our team has been working hard on a few paradigm-changing features for HTML Canvas. One of them is the ability to have HTML elements in a canvas. Here's a ThreeJS demo with a live HTML texture.
0:43

For the last couple months, our team has been working hard on a few paradigm-changing features for HTML Canvas. One of them is the ability to have HTML elements in a canvas. Here's a ThreeJS demo with a live HTML texture.

Fernando Serboncini

681,719 görüntüleme • 2 yıl önce

Introducing PAN — MBZUAI’s New World Model for Interactive Intelligence Developed by MBZUAI’s Institute of Foundation Models, PAN is built for simulation, prediction, and agentic reasoning. Unlike traditional video generators that only output frames, PAN maintains a persistent internal state that evolves when guided with natural language. Its Generative Latent Prediction architecture combines: • A latent encoder to capture the world state • A dynamics module that evolves that state step-by-step • A video diffusion decoder that visualizes outcomes By decoding at every step using a causal sliding-window diffusion process, PAN stays grounded in real-world physics and maintains long-horizon continuity, a leap beyond single-shot models. Evaluated on action fidelity, long-horizon stability, and simulative planning, PAN delivers state-of-the-art performance compared to open models and rivals leading commercial systems. For robotics, autonomy, and decision support, PAN is a foundation for the next wave of intelligent, foresight-driven AI.
1:06

Introducing PAN — MBZUAI’s New World Model for Interactive Intelligence Developed by MBZUAI’s Institute of Foundation Models, PAN is built for simulation, prediction, and agentic reasoning. Unlike traditional video generators that only output frames, PAN maintains a persistent internal state that evolves when guided with natural language. Its Generative Latent Prediction architecture combines: • A latent encoder to capture the world state • A dynamics module that evolves that state step-by-step • A video diffusion decoder that visualizes outcomes By decoding at every step using a causal sliding-window diffusion process, PAN stays grounded in real-world physics and maintains long-horizon continuity, a leap beyond single-shot models. Evaluated on action fidelity, long-horizon stability, and simulative planning, PAN delivers state-of-the-art performance compared to open models and rivals leading commercial systems. For robotics, autonomy, and decision support, PAN is a foundation for the next wave of intelligent, foresight-driven AI.

MBZUAI

98,734 görüntüleme • 9 ay önce

ChatGPT but for maps.💡Here's a first look at our work on a Smart Mapping Assistant prototype that combines the power of GIS with generative AI. #AIinMapping #GeospatialAI
0:20

ChatGPT but for maps.💡Here's a first look at our work on a Smart Mapping Assistant prototype that combines the power of GIS with generative AI. #AIinMapping #GeospatialAI

Esri

451,976 görüntüleme • 2 yıl önce

here's a little demo of swsh-challenge-pack! a mod I've been working on that's intended to provide a randomizer/IronMon-like experience for Sword & Shield. it's at a point where I feel like it's able to be tested if anyone is interested!
1:52

here's a little demo of swsh-challenge-pack! a mod I've been working on that's intended to provide a randomizer/IronMon-like experience for Sword & Shield. it's at a point where I feel like it's able to be tested if anyone is interested!

lincoln

16,759 görüntüleme • 1 yıl önce

For the last few weekends, I've been building a real-time music visualizer on top of soundxyz API. I built it with Three.js, using almost the full arsenal from Poimandres. Here's a sneak peek of its capabilities with my favorite Mija track... (SOUND ON)
0:24

For the last few weekends, I've been building a real-time music visualizer on top of soundxyz API. I built it with Three.js, using almost the full arsenal from Poimandres. Here's a sneak peek of its capabilities with my favorite Mija track... (SOUND ON)

brain function collapse

75,092 görüntüleme • 3 yıl önce

Tomorrow my game that I've been working on for the last few months is coming out. It's a simple experience with a quite different idle mechanic. I've put a lot of love and polish into it. 🍔 Burgie's cozy kitchen 🍔 Hope you like it!
0:43

Tomorrow my game that I've been working on for the last few months is coming out. It's a simple experience with a quite different idle mechanic. I've put a lot of love and polish into it. 🍔 Burgie's cozy kitchen 🍔 Hope you like it!

HeyNau

25,641 görüntüleme • 1 yıl önce