Try this when you play this: Instead of just moving forward, try keep rotating. You'll notice that your environment keeps changing, basically unplayable as a game. This is what I mean by "The Statement 'Every pixel will be generated, not rendered' is JUST A MEME".

Yes theoretically the AI model can be trained to have memory, but what is more likely to happen before that--if it ever happens--is: An AI native game engine made up of not just the AI model but also memory and caching OUTSIDE of the AI model for efficiency and determinism.

More on this here

People responding saying "This is not impossible, can be done", didn't read my linked post above. The whole point here is that regardless of whether it's possible or not, there will always be more efficient ways of doing this instead of trying to generate everything in realtime. Which means when this actually gets productized, it will probably be both generated AND rendered. It's not a zero sum game, is my point here.

Using an analogy, imagine a hypothetical PC that is solely made up of a single CPU. No RAM, No SSD, NO GPU. CPU does everything. How cool would that be? A single seamless CPU that not only stores ephemeral data through registers, but also stores long term memory, effectively replacing RAM and disk drives. And this is definitely possible today, theoretically. But this hypothetical machine while sounding so cool, will never be used by the mainstream because it's too expensive and inefficient. And this is why we have RAM, SSD, and so on in the real world. They don't sound as cool and futuristic as the hypothetical PC made up of "one CPU to rule them all", but it gets job done in the most efficient way, and affordable and efficient enough to be used by every single person on earth, which is all I care about. Everything proposed by the "Everything will be generated not rendered" mantra can be achieved with "Everything will be generated AND rendered", but million times more efficiently. Furthermore, the latter can achieve things that are impossible with the former, monolithic approach, such as generating multiple versions of the reality simultaneously, which is impossible when you're generating everything in realtime, since time is linear.

TLDR

For those of you who want to try running this locally and experiment, I share my experience here so you can get started quickly

maybe this is part of the game

could be, but even then, no need to generate everything in realtime, can pre-generate the vicinity (can even pre-generate multiple versions), cache, and render dynamically. still a generated world but much more efficient

This assumes object permanence is required to have fun.

Can you read the rest of the thread? This is not the point of my post. This thread is not about whether this model is good or bad, but what an AI model based game engine may look like in practice