A deep conversation with Nikolay Savinov, the Gemini long context pre-training co-lead… We go from the basics to what is needed to scale to infinite context to long context best practices for devs:

YouTube link:

🤝 Interviewing with HR or non-technical managers? 🌟 Employers value clarity and confidence. Cybersecurity Dictionary for Everyone teaches you to explain cybersecurity concepts in a way everyone can understand. Build connections that matter! 🌟

Please tell google to further research memory informed inferences and Semantic Continuity solutions 🥺🙏 Dont want to be stuck in scaffolding hell for 2+ years lol Maybe something like might be possible with using a set up of Larimar-like Autoencoders at varying intervals to act like context pipes per horizon tho... + timestamps (or ttl or time signature to assist w sense of time) hmm 🤔🧐 [not quite the same as memory layers like Google's titan which is more like long term repeated adaptation or continual skill acquisition] Basically bc models are frozen they baton pass the chat history, but that message passing loses nuance and the telephone game results in gradual loss of context :'( Hence hella scaffolding agents.... uhg

@SavinovNikolay my favorite part of the conversation:

@SavinovNikolay Logan we need confirmation, is it gonna be a new ultra subscription or a new ultra model.... Please don't do us advanced users dirty. Pleaseeeee 🥺

@SavinovNikolay Hey is there a well formatted transcript anywhere? Video format isn't really my learning style but I would love to check this out.

@SavinovNikolay Take the YT link, paste into AI Studio or NotebookLM

@SavinovNikolay Posted on YouTube as well?

@SavinovNikolay Tell about ultra model

@SavinovNikolay Smelling the Gemini 2M context coming soon 😹

@SavinovNikolay Amazing! Now we must try to also compress this intelligence into the smallest and most efficient model as well!