Загрузка видео...

Не удалось загрузить видео

На главную

Watch LTM-1 generate complex suggestions:

18,226 просмотров • 3 лет назад •via X (Twitter)

Комментарии: 4

Фото профиля Magic
Magic3 лет назад

Meet LTM-1: LLM with *5,000,000 prompt tokens* That's ~500k lines of code or ~5k files, enough to fully cover most repositories. LTM-1 is a prototype of a neural network architecture we designed for giant context windows.

Фото профиля Magic
Magic3 лет назад

Watch LTM-1 reuse and synthesize information across files:

Фото профиля Magic
Magic3 лет назад

How? We tried to scale standard GPT context windows but quickly got stuck. So, we designed a new approach: the Long-term Memory Network (LTM Net). Training and serving LTM Nets required a custom ML stack, from GPU kernels to how we distribute the model across a cluster.

Фото профиля Magic
Magic3 лет назад

What’s next? More compute. LTM Nets see more context than GPTs, but LTM-1 has fewer parameters than today’s frontier models, making it less smart. Knowing how drastically model scale improves the performance of GPTs, we're excited to see how far we can take LTM Nets.

Похожие видео