Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

working towards action-conditioned video diffusion models for now this is just cs2 gameplay, and i parse the keypresses from the .dem game file next steps will be to work on a scalable data loader then coding the model

arnie_hacker's profile picture

Arnie Ramesh

6,141 subscribers

31,135 Aufrufe • vor 5 Monaten •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

New Stage2 PoC payload loader commit fixes the USB Loader and can now goldhen (or any payload/HEN) from payload.bin from USB root or from /data, the USB Loader has now been set to the default loader, this is the end game Offsets file to port to 11.00
0:59

New Stage2 PoC payload loader commit fixes the USB Loader and can now goldhen (or any payload/HEN) from payload.bin from USB root or from /data, the USB Loader has now been set to the default loader, this is the end game Offsets file to port to 11.00

LM

78,913 Aufrufe • vor 2 Jahren

Glad that our work “Inference-Time Enhancement of Generative Robot Policies via Predictive World Modeling”, led by Han Qi, has been accepted to IEEE Robotics and Automation Letters! 🎉 We propose Generative Predictive Control (GPC): sample action proposals from a pretrained diffusion policy (“look back”), roll them out with a diffusion-based action-conditioned video world model (“look forward”), then rank or optimize the actions using either a learned reward model or VLM preferences. Conceptually, this is trajectory optimization / MPC with hybrid sampling + gradient optimization, interpreted through modern diffusion priors and video world models. Interestingly, we first posted the paper on arXiv in Feb 2025, when action-conditioned video world models for planning were still rare—now this direction is rapidly gaining traction. Still many open questions, e.g., • how to avoid local minima in planning • what representations work best for world models • how to balance physics priors vs. data-driven learning Paper:
1:00

Glad that our work “Inference-Time Enhancement of Generative Robot Policies via Predictive World Modeling”, led by Han Qi, has been accepted to IEEE Robotics and Automation Letters! 🎉 We propose Generative Predictive Control (GPC): sample action proposals from a pretrained diffusion policy (“look back”), roll them out with a diffusion-based action-conditioned video world model (“look forward”), then rank or optimize the actions using either a learned reward model or VLM preferences. Conceptually, this is trajectory optimization / MPC with hybrid sampling + gradient optimization, interpreted through modern diffusion priors and video world models. Interestingly, we first posted the paper on arXiv in Feb 2025, when action-conditioned video world models for planning were still rare—now this direction is rapidly gaining traction. Still many open questions, e.g., • how to avoid local minima in planning • what representations work best for world models • how to balance physics priors vs. data-driven learning Paper:

Heng Yang

18,994 Aufrufe • vor 4 Monaten

Wow, diffusion models (used in AI image generation) are also game engines - a type of world simulation. By predicting the next frame of the classic shooter DOOM, you get a playable game at 20 fps without any underlying real game engine. This video is from the diffusion model.
2:30

Wow, diffusion models (used in AI image generation) are also game engines - a type of world simulation. By predicting the next frame of the classic shooter DOOM, you get a playable game at 20 fps without any underlying real game engine. This video is from the diffusion model.

Ethan Mollick

1,768,933 Aufrufe • vor 1 Jahr

Diffusion has shown great promise for generating robot **actions**, can it act as a **world model** to generate the future conditioned on actions? In our work led by han qi Haocheng Yin and in collaboration with Yilun Du, we show a **controllable** action-conditioned video diffusion model can produce photorealistic and (near) physics-accurate future predictions. This ability strengthens the policy via: - ranking different action proposals and selecting the best, or - **visual** trajectory optimization by optimizing the action proposals using gradient ascent. Learn more about Generative Predictive Control (GPC) at:
1:37

Diffusion has shown great promise for generating robot **actions**, can it act as a **world model** to generate the future conditioned on actions? In our work led by han qi Haocheng Yin and in collaboration with Yilun Du, we show a **controllable** action-conditioned video diffusion model can produce photorealistic and (near) physics-accurate future predictions. This ability strengthens the policy via: - ranking different action proposals and selecting the best, or - **visual** trajectory optimization by optimizing the action proposals using gradient ascent. Learn more about Generative Predictive Control (GPC) at:

Heng Yang

38,428 Aufrufe • vor 1 Jahr

I finally released my new video on YouTube about Diffusion Models / Score-Based Generative Models. Literally planned this for a year and put so much work in. I think this approach to diffusion models is so intuitive and highly recommend giving that a go! Video is 38min long, so you will need some time to watch that haha.
0:59

I finally released my new video on YouTube about Diffusion Models / Score-Based Generative Models. Literally planned this for a year and put so much work in. I think this approach to diffusion models is so intuitive and highly recommend giving that a go! Video is 38min long, so you will need some time to watch that haha.

dome | Outlier

54,560 Aufrufe • vor 1 Jahr

The ultimate form of agentic coding will not be the terminal. This is an experiment. An ADE (Agentic Coding Environment), designed from the ground up to work with agentic coding tools. Right now, this only supports Claude Code and runs on a Mac.
5:46

The ultimate form of agentic coding will not be the terminal. This is an experiment. An ADE (Agentic Coding Environment), designed from the ground up to work with agentic coding tools. Right now, this only supports Claude Code and runs on a Mac.

Santiago

40,126 Aufrufe • vor 5 Monaten

Gemini Diffusion will be one of the steps that will define how user interfaces will evolve within the next several years. This is a huge prerequisite for the next level of generative UIs. There won't be a need to build a frontend UIs soon. Insane 🤯
0:31

Gemini Diffusion will be one of the steps that will define how user interfaces will evolve within the next several years. This is a huge prerequisite for the next level of generative UIs. There won't be a need to build a frontend UIs soon. Insane 🤯

TestingCatalog News 🗞

18,924 Aufrufe • vor 1 Jahr

After building out a game of my own and using Retro Diffusion for every piece of art, I'm really curious to see what other people can come up with. So I'm going to host a mini game jam. $500 of Retro Diffusion credits in prizes (1st = $300, 2nd & 3rd = $100 each). Build a game over the next week that uses RD for the artwork. Whatever genre, whatever type of gameplay. On Tuesday March 10th I'll gather up all the games (video of gameplay is fine) and decide which I think is the most interesting. Make sure to repost this and/or @ me so I know what you're making! I'd love to see progress along the way as well :) TLDR: Make a game, $500 in prizes, you've got until the 10th, gotta use Retro Diffusion for the art, it has to be a *new* game, not something you've been working on.
1:18

After building out a game of my own and using Retro Diffusion for every piece of art, I'm really curious to see what other people can come up with. So I'm going to host a mini game jam. $500 of Retro Diffusion credits in prizes (1st = $300, 2nd & 3rd = $100 each). Build a game over the next week that uses RD for the artwork. Whatever genre, whatever type of gameplay. On Tuesday March 10th I'll gather up all the games (video of gameplay is fine) and decide which I think is the most interesting. Make sure to repost this and/or @ me so I know what you're making! I'd love to see progress along the way as well :) TLDR: Make a game, $500 in prizes, you've got until the 10th, gotta use Retro Diffusion for the art, it has to be a *new* game, not something you've been working on.

Astropulse

30,716 Aufrufe • vor 4 Monaten

Check out the Gameplay Video for #AtelierYumia A free public demo will be available from March 17th on PlayStation Store, Nintendo e-shop, Microsoft Store and Steam®. In addition, save data from the demo can be carried over to the full version of the game! #KTfamily
5:47

Check out the Gameplay Video for #AtelierYumia A free public demo will be available from March 17th on PlayStation Store, Nintendo e-shop, Microsoft Store and Steam®. In addition, save data from the demo can be carried over to the full version of the game! #KTfamily

KOEI TECMO AMERICA

34,760 Aufrufe • vor 1 Jahr

Another video! This weeks goal is to get the outfit in game and working, after that ill push an update to the testing discord and we can iron out bugs that will appear from the quest. then its just spell checking, and I will throw this bad boy up for public use.
1:08

Another video! This weeks goal is to get the outfit in game and working, after that ill push an update to the testing discord and we can iron out bugs that will appear from the quest. then its just spell checking, and I will throw this bad boy up for public use.

Der Kernel

12,213 Aufrufe • vor 1 Jahr

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:
0:50

📢 Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation Got only one or a few images and wondering if recovering the 3D environment is a reconstruction or generation problem? Why not do it with a generative reconstruction model! We show that a camera-conditioned video diffusion model can be transformed into a generative reconstruction model that directly outputs a high-quality 3D Gaussian Splatting representation through self-distillation, without requiring real-world training data. Check out our results in the video (wait for dynamic scenes in the second half!) : Project Page: Code and Models: Paper:

Sherwin Bahmani

66,590 Aufrufe • vor 10 Monaten

I give you, 7 minutes of gameplay of Sol's current progress on FOOTMEN FRENZY 5.4 AR, the warcraft 3 custom game. I did absolutely NOTHING besides prompt and give some suggestions on how to solve some problems. Sol on its own went online, downloaded a w3x file of the game, found an npm package to parse the data out of it, and attempted to recreate the game 1:1. It generated its own custom assets using blender and even animated it. Far from perfect, but extremely impressive what it could do all on its own working about 15 hours.
7:47

I give you, 7 minutes of gameplay of Sol's current progress on FOOTMEN FRENZY 5.4 AR, the warcraft 3 custom game. I did absolutely NOTHING besides prompt and give some suggestions on how to solve some problems. Sol on its own went online, downloaded a w3x file of the game, found an npm package to parse the data out of it, and attempted to recreate the game 1:1. It generated its own custom assets using blender and even animated it. Far from perfect, but extremely impressive what it could do all on its own working about 15 hours.

Barbog Da Big Finka

25,933 Aufrufe • vor 13 Tagen

Tencent presents GameGen-O Open-world Video Game Generation We introduce GameGen-O, the first diffusion transformer model tailored for the generation of open-world video games. This model facilitates high-quality, open-domain generation by simulating a wide array of game engine features, such as innovative characters, dynamic environments, complex actions, and diverse events. Additionally, it provides interactive controllability, thus allowing for the gameplay simulation. The development of GameGen-O involves a comprehensive data collection and processing effort from scratch. We collect and build the first Open-World Video Game Dataset (OGameData), amassed extensive data from over a hundred of next-generation open-world games, employing a proprietary data pipeline for efficient sorting, scoring, filtering, and decoupled captioning. This robust and extensive OGameData forms the foundation of our model's training process. GameGen-O undergoes a two-stage training process, consisting of foundation model pretraining and instruction tuning. In the first phase, the model is pre-trained on the OGameData via the text-to-video and video continuation, endowing GameGen-O with the capability for open-domain video game generation. In the second phase, the pre-trained model is frozen, and we fine-tuned using a trainable InstructNet, which enables the production of subsequent frames based on multimodal structural instructions. This whole training process imparts the model with the ability to generate and interactively control content. In summary, GameGen-O represents a notable initial step forward in the realm of open-world video game generation via generative models. It underscores the potential of generative models to serve as an alternative to rendering techniques, which can efficiently combine creative generation with interactive capabilities.
0:51

Tencent presents GameGen-O Open-world Video Game Generation We introduce GameGen-O, the first diffusion transformer model tailored for the generation of open-world video games. This model facilitates high-quality, open-domain generation by simulating a wide array of game engine features, such as innovative characters, dynamic environments, complex actions, and diverse events. Additionally, it provides interactive controllability, thus allowing for the gameplay simulation. The development of GameGen-O involves a comprehensive data collection and processing effort from scratch. We collect and build the first Open-World Video Game Dataset (OGameData), amassed extensive data from over a hundred of next-generation open-world games, employing a proprietary data pipeline for efficient sorting, scoring, filtering, and decoupled captioning. This robust and extensive OGameData forms the foundation of our model's training process. GameGen-O undergoes a two-stage training process, consisting of foundation model pretraining and instruction tuning. In the first phase, the model is pre-trained on the OGameData via the text-to-video and video continuation, endowing GameGen-O with the capability for open-domain video game generation. In the second phase, the pre-trained model is frozen, and we fine-tuned using a trainable InstructNet, which enables the production of subsequent frames based on multimodal structural instructions. This whole training process imparts the model with the ability to generate and interactively control content. In summary, GameGen-O represents a notable initial step forward in the realm of open-world video game generation via generative models. It underscores the potential of generative models to serve as an alternative to rendering techniques, which can efficiently combine creative generation with interactive capabilities.

AK

367,088 Aufrufe • vor 1 Jahr

𝐃𝐞𝐬𝐭𝐫𝐚 𝐎𝐂𝐀𝐈 (𝐎𝐧𝐞-𝐂𝐥𝐢𝐜𝐤 𝐀𝐈) 𝐌𝐨𝐝𝐞𝐥𝐬 - $DSYNC 𝐄𝐚𝐫𝐥𝐲 𝐀𝐜𝐜𝐞𝐬𝐬 𝐏𝐫𝐞𝐯𝐢𝐞𝐰 We are training and building a wide range of AI models for the Destra OCAI platform. Today, we will be showcasing an early preview of one of the initial models we are working on—a bullish/bearish prediction model designed to 𝐟𝐨𝐫𝐞𝐜𝐚𝐬𝐭 𝐭𝐡𝐞 𝐦𝐚𝐫𝐤𝐞𝐭 𝐝𝐢𝐫𝐞𝐜𝐭𝐢𝐨𝐧 𝐨𝐟 𝐬𝐩𝐞𝐜𝐢𝐟𝐢𝐜 𝐭𝐨𝐤𝐞𝐧𝐬. This is just one of the initial models we are working on; as we progress, the models will become more complex and intricate. In this behind-the-scenes video, we demonstrate how the model predicts the future market trend of the BTC-USD pair with an 𝐚𝐜𝐜𝐮𝐫𝐚𝐜𝐲 𝐨𝐟 𝟔𝟓%. This is the barebones structure of the models we are training, which will soon be available to users with a single click through the Destra OCAI platform. The OCAI platform includes a wide array of Destra Exclusive AI models, specifically designed to simplify life for crypto audiences, that are envisioned and built from scratch at Destra Labs.
0:54

𝐃𝐞𝐬𝐭𝐫𝐚 𝐎𝐂𝐀𝐈 (𝐎𝐧𝐞-𝐂𝐥𝐢𝐜𝐤 𝐀𝐈) 𝐌𝐨𝐝𝐞𝐥𝐬 - $DSYNC 𝐄𝐚𝐫𝐥𝐲 𝐀𝐜𝐜𝐞𝐬𝐬 𝐏𝐫𝐞𝐯𝐢𝐞𝐰 We are training and building a wide range of AI models for the Destra OCAI platform. Today, we will be showcasing an early preview of one of the initial models we are working on—a bullish/bearish prediction model designed to 𝐟𝐨𝐫𝐞𝐜𝐚𝐬𝐭 𝐭𝐡𝐞 𝐦𝐚𝐫𝐤𝐞𝐭 𝐝𝐢𝐫𝐞𝐜𝐭𝐢𝐨𝐧 𝐨𝐟 𝐬𝐩𝐞𝐜𝐢𝐟𝐢𝐜 𝐭𝐨𝐤𝐞𝐧𝐬. This is just one of the initial models we are working on; as we progress, the models will become more complex and intricate. In this behind-the-scenes video, we demonstrate how the model predicts the future market trend of the BTC-USD pair with an 𝐚𝐜𝐜𝐮𝐫𝐚𝐜𝐲 𝐨𝐟 𝟔𝟓%. This is the barebones structure of the models we are training, which will soon be available to users with a single click through the Destra OCAI platform. The OCAI platform includes a wide array of Destra Exclusive AI models, specifically designed to simplify life for crypto audiences, that are envisioned and built from scratch at Destra Labs.

Destra Network

37,135 Aufrufe • vor 1 Jahr

“ xAI team is currently working heavily on coding models. Right now, the main focus is training a specialized coding model that will be both fast and smart. I believe we’ll share it with you guys in a few weeks. That’s exciting. Second ,after coding, we all see that the main weakness of Grok 4 is its multimodal capabilities. In fact, it was so bad that Grok was effectively looking at the world while squinting through glass trying to see blurry features and make sense of them. The most immediate improvement we’re stepping on with the next generation pre trained model is huge gains in image understanding, video understanding, and audio. Right now, the model can hear and see the world just like any of you. And with all the tools and other agents it can talk to, we’re going to see a huge unlock for many different application layers once multimodal agents arrive.” — xAI Team
2:01

“ xAI team is currently working heavily on coding models. Right now, the main focus is training a specialized coding model that will be both fast and smart. I believe we’ll share it with you guys in a few weeks. That’s exciting. Second ,after coding, we all see that the main weakness of Grok 4 is its multimodal capabilities. In fact, it was so bad that Grok was effectively looking at the world while squinting through glass trying to see blurry features and make sense of them. The most immediate improvement we’re stepping on with the next generation pre trained model is huge gains in image understanding, video understanding, and audio. Right now, the model can hear and see the world just like any of you. And with all the tools and other agents it can talk to, we’re going to see a huge unlock for many different application layers once multimodal agents arrive.” — xAI Team

Apurv Kochara

88,142 Aufrufe • vor 5 Monaten

I tested GLM 5.2 on three real builds. A horror house game. A 3D stealth game. A Remotion marketing video. The results are exactly what I expect from Chinese open source coding models right now. The horror house has broken game logic. Collect all three keys, complete the objective, and the front door still won't open. The stealth game shipped with lighting so dark you can't see your own character. The Remotion video is rough. GLM 5.2 is faster than 5.1 and 10x cheaper than the frontier. But Opus 4.8 and GPT 5.5 are still in a different league for serious coding work. Chinese models keep missing the attention to detail that matters. I MISS FABLE 5 ALREADY. Full review live now.
17:43

I tested GLM 5.2 on three real builds. A horror house game. A 3D stealth game. A Remotion marketing video. The results are exactly what I expect from Chinese open source coding models right now. The horror house has broken game logic. Collect all three keys, complete the objective, and the front door still won't open. The stealth game shipped with lighting so dark you can't see your own character. The Remotion video is rough. GLM 5.2 is faster than 5.1 and 10x cheaper than the frontier. But Opus 4.8 and GPT 5.5 are still in a different league for serious coding work. Chinese models keep missing the attention to detail that matters. I MISS FABLE 5 ALREADY. Full review live now.

BridgeMind

24,527 Aufrufe • vor 1 Monat

Not the flashiest demos, but what’s under the hood represents a foundational shift for general-purpose robotics. World models are the next-gen foundation of Physical AI, not the VLM backbones found in typical VLAs. DreamZero is a 14B-parameter World Action Model (WAM) by NVIDIA that treats robotics as a joint video-and-action prediction task. Unlike traditional Vision-Language-Action (VLA) models that map images directly to motor commands, DreamZero leverages a pretrained video diffusion backbone to predict future world states and actions simultaneously. - achieves 2× better zero-shot generalization to unseen tasks and environments compared to state-of-the-art VLAs. - learns effectively from heterogeneous, non-repetitive data (500 hours), breaking the need for thousands of repeated demonstrations. - adapts to new robot embodiments with just 30 minutes of play data. - enables 7Hz closed-loop control via system optimizations and "DreamZero-Flash," making high-capacity diffusion models viable for real-time use.
1:01

Not the flashiest demos, but what’s under the hood represents a foundational shift for general-purpose robotics. World models are the next-gen foundation of Physical AI, not the VLM backbones found in typical VLAs. DreamZero is a 14B-parameter World Action Model (WAM) by NVIDIA that treats robotics as a joint video-and-action prediction task. Unlike traditional Vision-Language-Action (VLA) models that map images directly to motor commands, DreamZero leverages a pretrained video diffusion backbone to predict future world states and actions simultaneously. - achieves 2× better zero-shot generalization to unseen tasks and environments compared to state-of-the-art VLAs. - learns effectively from heterogeneous, non-repetitive data (500 hours), breaking the need for thousands of repeated demonstrations. - adapts to new robot embodiments with just 30 minutes of play data. - enables 7Hz closed-loop control via system optimizations and "DreamZero-Flash," making high-capacity diffusion models viable for real-time use.

The Humanoid Hub

35,204 Aufrufe • vor 5 Monaten

🎬This is from a static MJ 7 image + HeyGen 4! The voice over is human performance, but the nuances the diffusion model picks up is pretty next level for the character. I masked in a video of the cars moving from Kling. Pretty impressive.
0:20

🎬This is from a static MJ 7 image + HeyGen 4! The voice over is human performance, but the nuances the diffusion model picks up is pretty next level for the character. I masked in a video of the cars moving from Kling. Pretty impressive.

Dave Clark

11,224 Aufrufe • vor 1 Jahr

Contract is signed. It’s time to take the next steps toward this dream! Over the next 6 weeks we will be exploring what this game could be!
1:17

Contract is signed. It’s time to take the next steps toward this dream! Over the next 6 weeks we will be exploring what this game could be!

aDrive

25,699 Aufrufe • vor 1 Jahr

In the next 2-3 years, the skills you need to build an ETL process will fundamentally differ from anything we've done in the last decade. The EXTRACT and LOAD steps of an ETL pipeline have been solved forever. They are boring, and I suspect they will continue to be boring. But the TRANSFORM step is where the money is at (and where innovation will continue happening!) An ETL pipeline that can use Large Language Models effectively to transform data is a whole new ballgame. I recorded a video to talk about this. And more importantly, I talk about how you can build ETL pipelines to transform UNSTRUCTURED data (the kind of data that's really hard to handle).
5:39

In the next 2-3 years, the skills you need to build an ETL process will fundamentally differ from anything we've done in the last decade. The EXTRACT and LOAD steps of an ETL pipeline have been solved forever. They are boring, and I suspect they will continue to be boring. But the TRANSFORM step is where the money is at (and where innovation will continue happening!) An ETL pipeline that can use Large Language Models effectively to transform data is a whole new ballgame. I recorded a video to talk about this. And more importantly, I talk about how you can build ETL pipelines to transform UNSTRUCTURED data (the kind of data that's really hard to handle).

Santiago

56,412 Aufrufe • vor 1 Jahr