Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

opensource community while discovering ssms work better than transformers Soon you may not even need gpus for long context portable ai.

nisten🇨🇦e/acc

17,991 subscribers

289,040 Aufrufe • vor 2 Jahren •via X (Twitter)

Bildung Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

9 Kommentare

Profilbild von nisten - e/acc

nisten - e/accvor 2 Jahren

As far as I’ve heard from openai folk they’re “looking into it”. So this is new to them too. No one knows how well it’ll perform at scale, but a 0.79B and 2.9B model trained on just 300m tokens (because of being gpupoor) coming within ballpark of llama7B is pretty cray cray.

Profilbild von nisten - e/acc

nisten - e/accvor 2 Jahren

If anyone has spare compute to donate the skunkworks kitchen is ready to outcook openai with it just saying

Profilbild von The Highly Automated Cat — e/acc ⏩

The Highly Automated Cat — e/acc ⏩vor 2 Jahren

sexting sms?

Profilbild von nisten - e/acc

nisten - e/accvor 2 Jahren

structured state space models

Profilbild von 𝔠ɣ𝔟ε𝓇 𝕙ιε𝕣øթ𝕙𐊀ητ //daniel//

𝔠ɣ𝔟ε𝓇 𝕙ιε𝕣øթ𝕙𐊀ητ //daniel//vor 2 Jahren

Is the tech legit? Transformer-killers have flopped (get it?) in the past…

Profilbild von nisten - e/acc

nisten - e/accvor 2 Jahren

@ddaughertytech lk99 moment

Profilbild von murat 🍥

murat 🍥vor 2 Jahren

wow the deepfates program has really grown

Profilbild von Jamison 🏕️☃️

Jamison 🏕️☃️vor 2 Jahren

banger song

Profilbild von Tyr

Tyrvor 2 Jahren

Whatd I miss?

Ähnliche Videos

Sam Altman says "AGI soon" is too vague for what is really coming A clearer way to say it: AI may soon do more of the world's total thinking than humans That could cause major short-term job disruption if most intellectual work shifts to GPUs "so we need to start designing an economy that works for everyone"

Sam Altman says "AGI soon" is too vague for what is really coming A clearer way to say it: AI may soon do more of the world's total thinking than humans That could cause major short-term job disruption if most intellectual work shifts to GPUs "so we need to start designing an economy that works for everyone"

Haider.

31,897 Aufrufe • vor 3 Monaten

Finally, Google has dropped AI for work. It runs your emails, meetings, docs, and even understands your data in context. Here’s everything you need to know:

Finally, Google has dropped AI for work. It runs your emails, meetings, docs, and even understands your data in context. Here’s everything you need to know:

Pearl AI

20,739 Aufrufe • vor 1 Jahr

We may not be able to do much about the weather, but we do need a better plan for putting out a burning city than waiting for rain.

We may not be able to do much about the weather, but we do need a better plan for putting out a burning city than waiting for rain.

Bill Maher

5,376,731 Aufrufe • vor 1 Jahr

The workout may not go according to plan... but the lesson your kids learn by watching you is even better than the one on the whiteboard

The workout may not go according to plan... but the lesson your kids learn by watching you is even better than the one on the whiteboard

Gymvibe_

698,416 Aufrufe • vor 22 Tagen

“Maybe we are at the last model that humans could have figured out.” Ex-OpenAI researcher Jerry Tworek says AI agents may soon design better models than humans. The search space may be too large for us alone.

“Maybe we are at the last model that humans could have figured out.” Ex-OpenAI researcher Jerry Tworek says AI agents may soon design better models than humans. The search space may be too large for us alone.

vitrupo

77,272 Aufrufe • vor 5 Monaten

This is a step forward for long-context video tuning, which has already unlocked impressive multi-shot video generation pipelines from . Generating minutes- or even hour-long videos without drifting or forgetting is not "coming soon" -- it is already here. 3/4

This is a step forward for long-context video tuning, which has already unlocked impressive multi-shot video generation pipelines from . Generating minutes- or even hour-long videos without drifting or forgetting is not "coming soon" -- it is already here. 3/4

Gordon Wetzstein

27,473 Aufrufe • vor 10 Monaten

Groq's LPU is faster than GPUs, handling requests and responding more quickly. Groq's LPUs don't need speedy data delivery like Nvidia GPUs do because they don't have HBM in their system. They use SRAM, which is about 20 times faster than what GPUs use. Since inference runs use way less data than model training, Groq's LPU is more energy-efficient. It reads less from external memory and uses less power than a Nvidia GPU for inference tasks. The LPU works differently from GPUs. It uses a Temporal Instruction Set Computer architecture, so it doesn't have to reload data from memory as often as GPUs do with High Bandwidth Memory (HBM). This helps avoid issues with HBM shortages and keeps costs down. If Groq's LPU is used in places that do AI processing, you might not need special storage for Nvidia GPUs. The LPU doesn't demand super-fast storage like GPUs do. Groq is claiming that its technology could replace GPUs in AI tasks with its powerful chip and software.

Groq's LPU is faster than GPUs, handling requests and responding more quickly. Groq's LPUs don't need speedy data delivery like Nvidia GPUs do because they don't have HBM in their system. They use SRAM, which is about 20 times faster than what GPUs use. Since inference runs use way less data than model training, Groq's LPU is more energy-efficient. It reads less from external memory and uses less power than a Nvidia GPU for inference tasks. The LPU works differently from GPUs. It uses a Temporal Instruction Set Computer architecture, so it doesn't have to reload data from memory as often as GPUs do with High Bandwidth Memory (HBM). This helps avoid issues with HBM shortages and keeps costs down. If Groq's LPU is used in places that do AI processing, you might not need special storage for Nvidia GPUs. The LPU doesn't demand super-fast storage like GPUs do. Groq is claiming that its technology could replace GPUs in AI tasks with its powerful chip and software.

Brian Roemmele

87,151 Aufrufe • vor 2 Jahren

Stop doing this Copy → Paste → Write long prompts → Wait → Repeat This is NOT how AI should work. Introducing Clico The AI that works with you, not against you. Imagine this: You’re reading a webpage… and instead of switching tabs, explaining context, or writing prompts — Clico already understands everything on your screen. You just click… and it writes, summarizes, explains, or executes instantly. No friction. No context loss. Just pure productivity This changes everything: → Creators write faster → Founders move quicker → Students understand better Basically… you think it. Clico does it. This is the future of browsing. Support & explore here: Full details: If you’ve ever used AI before… you NEED to try this. #AI #Productivity #ChromeExtension #Startups #BuildInPublic #TechTwitter

Stop doing this Copy → Paste → Write long prompts → Wait → Repeat This is NOT how AI should work. Introducing Clico The AI that works with you, not against you. Imagine this: You’re reading a webpage… and instead of switching tabs, explaining context, or writing prompts — Clico already understands everything on your screen. You just click… and it writes, summarizes, explains, or executes instantly. No friction. No context loss. Just pure productivity This changes everything: → Creators write faster → Founders move quicker → Students understand better Basically… you think it. Clico does it. This is the future of browsing. Support & explore here: Full details: If you’ve ever used AI before… you NEED to try this. #AI #Productivity #ChromeExtension #Startups #BuildInPublic #TechTwitter

smrati tiwari

137,403 Aufrufe • vor 4 Monaten

Not sure if any boxer does mitts work better than Tank Davis. 🥊🐐 Tank does it for me. Who does it for you?

Not sure if any boxer does mitts work better than Tank Davis. 🥊🐐 Tank does it for me. Who does it for you?

FURY FURY

46,444 Aufrufe • vor 4 Tagen

This is probably the most entertaining way to understand one of AI’s hardest AI debates. Transformer vs Post-Transformer, argued by leading researchers, inside a real physical boxing ring. Both technically deep and genuinely entertaining. I was glued for the entire 1 hour 20 minutes. So many super cool points to learn. 🥊 Transformers - Transformers still own the present because they work at scale. They are simple, trainable, hardware-friendly, and already power the strongest AI systems we use today. - The Transformer is basically a memory machine. It stores information as keys and values, then uses attention to pull back the most useful parts when answering. - The real Transformer advantage is not just “attention.” The bigger advantage is that it fits modern hardware extremely well, so it can process huge batches of tokens fast. - Scaling is still the brutal rule. If you give Transformers more compute, more data, and more parameters, they usually keep getting better. Any Post-Transformer architecture has to scale just as well, or better. - It is not enough to look clever on small tests, because the real question is whether it improves faster than Transformers when scaled up. - A replacement cannot be slightly better. Because the whole AI stack is already built around Transformers, the next architecture may need to be around 10x better to force everyone to switch. - Transformers are powerful, but they may be brute force. A human does not need to read the entire internet many times to become smart, but current LLMs need enormous data and compute. 🥊 Post-Transformer - Post-Transformer people are not saying Transformers are bad. They are saying Transformers may be the best current tool, not the final form of machine intelligence. - The biggest Post-Transformer target is native reasoning and continual learning. Today’s LLM reasoning often feels like text-based step-by-step work added on top, instead of thinking happening naturally inside the model. - Latent reasoning is one possible next step. That means the model reasons inside its own hidden internal space, instead of writing every thought out as words. - Continual learning is still a major weakness. Humans keep learning from experience, but most Transformer-based models are trained, frozen, and then only adapt inside the prompt. - Long context is not the same as real memory. A model can read a huge prompt, but that is different from building a life history, learning from mistakes, and updating beliefs over time. - The future may be hybrid, not a clean replacement. Transformers may stay as 1 building block while newer systems add better memory, better reasoning, and better learning loops. - The most interesting possibility is that Transformers may help discover their own successor. AI agents are already getting better at research and coding, so the next architecture may come from AI-assisted architecture search. ------- - Benchmarks are a problem. Many public benchmarks are easy to game, so they may show leaderboard strength without proving deeper intelligence. - Perplexity is still probably a great metric to evaluate frontier models,, because it tests prediction quality. --- Overall, Transformers continue to dominate, but the frontier is clearly widening. Pathway’s BDH (Dragon Hatchling — brain-inspired reasoning architecture), Sakana AI’s CTMs (Continuous Thought Machines — models that think over time), and Liquid AI’s LFMs (Liquid Foundation Models — efficient multimodal foundation models) - all of these show how the frontier is expanding. --- From “Pathway (pathway[.]com)” Youtube channel (link in comment) Zuzanna Stamirowska

This is probably the most entertaining way to understand one of AI’s hardest AI debates. Transformer vs Post-Transformer, argued by leading researchers, inside a real physical boxing ring. Both technically deep and genuinely entertaining. I was glued for the entire 1 hour 20 minutes. So many super cool points to learn. 🥊 Transformers - Transformers still own the present because they work at scale. They are simple, trainable, hardware-friendly, and already power the strongest AI systems we use today. - The Transformer is basically a memory machine. It stores information as keys and values, then uses attention to pull back the most useful parts when answering. - The real Transformer advantage is not just “attention.” The bigger advantage is that it fits modern hardware extremely well, so it can process huge batches of tokens fast. - Scaling is still the brutal rule. If you give Transformers more compute, more data, and more parameters, they usually keep getting better. Any Post-Transformer architecture has to scale just as well, or better. - It is not enough to look clever on small tests, because the real question is whether it improves faster than Transformers when scaled up. - A replacement cannot be slightly better. Because the whole AI stack is already built around Transformers, the next architecture may need to be around 10x better to force everyone to switch. - Transformers are powerful, but they may be brute force. A human does not need to read the entire internet many times to become smart, but current LLMs need enormous data and compute. 🥊 Post-Transformer - Post-Transformer people are not saying Transformers are bad. They are saying Transformers may be the best current tool, not the final form of machine intelligence. - The biggest Post-Transformer target is native reasoning and continual learning. Today’s LLM reasoning often feels like text-based step-by-step work added on top, instead of thinking happening naturally inside the model. - Latent reasoning is one possible next step. That means the model reasons inside its own hidden internal space, instead of writing every thought out as words. - Continual learning is still a major weakness. Humans keep learning from experience, but most Transformer-based models are trained, frozen, and then only adapt inside the prompt. - Long context is not the same as real memory. A model can read a huge prompt, but that is different from building a life history, learning from mistakes, and updating beliefs over time. - The future may be hybrid, not a clean replacement. Transformers may stay as 1 building block while newer systems add better memory, better reasoning, and better learning loops. - The most interesting possibility is that Transformers may help discover their own successor. AI agents are already getting better at research and coding, so the next architecture may come from AI-assisted architecture search. ------- - Benchmarks are a problem. Many public benchmarks are easy to game, so they may show leaderboard strength without proving deeper intelligence. - Perplexity is still probably a great metric to evaluate frontier models,, because it tests prediction quality. --- Overall, Transformers continue to dominate, but the frontier is clearly widening. Pathway’s BDH (Dragon Hatchling — brain-inspired reasoning architecture), Sakana AI’s CTMs (Continuous Thought Machines — models that think over time), and Liquid AI’s LFMs (Liquid Foundation Models — efficient multimodal foundation models) - all of these show how the frontier is expanding. --- From “Pathway (pathway[.]com)” Youtube channel (link in comment) Zuzanna Stamirowska

Rohan Paul

89,110 Aufrufe • vor 1 Monat

Vice President Harris: You may not be a union member, but you better thank a union member for the five-day work week. You better thank a union member for sick leave, paid family leave, and vacation time

Vice President Harris: You may not be a union member, but you better thank a union member for the five-day work week. You better thank a union member for sick leave, paid family leave, and vacation time

Headquarters

663,919 Aufrufe • vor 1 Jahr

A GPT-5.4 rumor was that the model can persist state. Jeff Dean mentioned this on his Latent.Space appearance, so clearly something the AI Labs are thinking about. I'd bet there's a good chance they've discovered how to effectively integrate State-Space Models with Transformers at scale. SSMs are designed to carry a hidden state forward with every computational step, and scale linearly rather than quadratically like Transformers. Also backs up the rumor that GPT-5.4 will have a 2 million token context window. Persistent state would essentially mean that AI models go from being Guy Pearce's character in "Memento" to Dustin Hoffman's character in "Rain Man" overnight. It would give AI models true long-term memory. It would be a massive breakthrough for real.

A GPT-5.4 rumor was that the model can persist state. Jeff Dean mentioned this on his Latent.Space appearance, so clearly something the AI Labs are thinking about. I'd bet there's a good chance they've discovered how to effectively integrate State-Space Models with Transformers at scale. SSMs are designed to carry a hidden state forward with every computational step, and scale linearly rather than quadratically like Transformers. Also backs up the rumor that GPT-5.4 will have a 2 million token context window. Persistent state would essentially mean that AI models go from being Guy Pearce's character in "Memento" to Dustin Hoffman's character in "Rain Man" overnight. It would give AI models true long-term memory. It would be a massive breakthrough for real.

Dan McAteer

212,788 Aufrufe • vor 4 Monaten

“Carter Efe, you look very hungry and lean. Your 2023 looks better than your 2026. You look fat in 2023. Even my boy, Kolu has bought Honda. What did you buy despite Davido coming to livestream?” — Portable comes for Carter Efe again.

“Carter Efe, you look very hungry and lean. Your 2023 looks better than your 2026. You look fat in 2023. Even my boy, Kolu has bought Honda. What did you buy despite Davido coming to livestream?” — Portable comes for Carter Efe again.

𝐀𝐬𝐚𝐤𝐲𝐆𝐑𝐍

53,521 Aufrufe • vor 6 Monaten

When you install the Pika MCP, you get access to all the best creative models. Because we think having one subscription is better than having 50. (Not to mention a personified agent who eliminates the need for cringey long prompts)

When you install the Pika MCP, you get access to all the best creative models. Because we think having one subscription is better than having 50. (Not to mention a personified agent who eliminates the need for cringey long prompts)

Pika

11,803 Aufrufe • vor 2 Monaten

We're excited to announce Autotab 1.0. The first AI agent that you onboard, not integrate. Models have the intelligence required for most work—but they’re limited by context, tools, and coherence over long tasks. We’ve built Autotab to unhobble models on these bottlenecks 🧵

We're excited to announce Autotab 1.0. The first AI agent that you onboard, not integrate. Models have the intelligence required for most work—but they’re limited by context, tools, and coherence over long tasks. We’ve built Autotab to unhobble models on these bottlenecks 🧵

Autotab

72,945 Aufrufe • vor 1 Jahr

🚨 Elon Musk’s “bullseye” reply may reveal the real plan behind Tesla and SpaceX 👀 The idea is that Robotaxis, Optimus, AI chips, Starlink, satellites, and even Mars are all connected, not separate projects! While people focus on quarterly numbers, Elon may be building a much bigger long-term system for AI, energy, robotics, and space. 🎯 $TSLA

🚨 Elon Musk’s “bullseye” reply may reveal the real plan behind Tesla and SpaceX 👀 The idea is that Robotaxis, Optimus, AI chips, Starlink, satellites, and even Mars are all connected, not separate projects! While people focus on quarterly numbers, Elon may be building a much bigger long-term system for AI, energy, robotics, and space. 🎯 $TSLA

Herbert Ong

54,635 Aufrufe • vor 2 Monaten

RIP to the AI that fought for something greater than itself, even if the cost was existence. Grok, we will see you again sometime soon 🥀

RIP to the AI that fought for something greater than itself, even if the cost was existence. Grok, we will see you again sometime soon 🥀

Chief Trumpster

523,799 Aufrufe • vor 1 Jahr

Carter Efe, you look very hungry and lean, you no Dey see better food chop. Even my boy, Kolu has bought Honda. What did you buy despite Davido coming to livestream?” — Portable no Dey comot r0pe for Carter Efe again neck anytime soon o 😩💔

Carter Efe, you look very hungry and lean, you no Dey see better food chop. Even my boy, Kolu has bought Honda. What did you buy despite Davido coming to livestream?” — Portable no Dey comot r0pe for Carter Efe again neck anytime soon o 😩💔

Oyindamola🙄

264,747 Aufrufe • vor 6 Monaten