Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Been looking into token optimization and model routing, I think super obvious optimization to tackle both cost + demand on inference Here’s a small post about different techniques and methods

Sean Geng

4,079 subscribers

16,441 Aufrufe • vor 2 Monaten •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

$NBIS cofounder Roman Chernin describes how their recent acquisitions of Eigen AI and Clarifai were all about speed, incredible talent, and acceleration: "The philosophy is very simple. We need to build so many things, and we need to move so fast, that we're always looking for people who can accelerate us. It should be exceptional talent, and/or something that has a great adoption." "Our two recent acquisitions [were] two teams that work on inference optimization. A big part of our business is how efficiently we convert GPUs into tokens. And these two teams — Eigen AI and Clarifai — one is focused on model optimization, the engine of inference. How you run specific models and all the techniques around spec decoding, quantization, and so on." "And the other is system optimization. All the routing, KV caching, and orchestration across the big cluster of compute and so on." "We have a very strong internal team working on inference. But we felt that we needed to move faster, bring more capabilities. Because the market is so fast."

$NBIS cofounder Roman Chernin describes how their recent acquisitions of Eigen AI and Clarifai were all about speed, incredible talent, and acceleration: "The philosophy is very simple. We need to build so many things, and we need to move so fast, that we're always looking for people who can accelerate us. It should be exceptional talent, and/or something that has a great adoption." "Our two recent acquisitions [were] two teams that work on inference optimization. A big part of our business is how efficiently we convert GPUs into tokens. And these two teams — Eigen AI and Clarifai — one is focused on model optimization, the engine of inference. How you run specific models and all the techniques around spec decoding, quantization, and so on." "And the other is system optimization. All the routing, KV caching, and orchestration across the big cluster of compute and so on." "We have a very strong internal team working on inference. But we felt that we needed to move faster, bring more capabilities. Because the market is so fast."

TBPN

30,608 Aufrufe • vor 3 Monaten

The defining differentiator in AI right now isn't model performance, it's cost per token. Because in this market, efficiency is what turns demand into actual growth. Roman Chernin of Nebius outlines why inference economics are becoming the central challenge, as companies operating on thin margins need infrastructure that can scale without eroding profitability. Token Factory reflects that shift, focusing on orchestration, optimization, and full-stack control to make AI systems economically viable.

The defining differentiator in AI right now isn't model performance, it's cost per token. Because in this market, efficiency is what turns demand into actual growth. Roman Chernin of Nebius outlines why inference economics are becoming the central challenge, as companies operating on thin margins need infrastructure that can scale without eroding profitability. Token Factory reflects that shift, focusing on orchestration, optimization, and full-stack control to make AI systems economically viable.

Six Five Media

294,803 Aufrufe • vor 4 Monaten

[ Model Optimization ] Here's a comparison video of the Female Dancer's second optimization (the third one). Her legs are no longer thick as they were in the first optimization, and her chest size has been reduced to match the original model. ※ Source:

[ Model Optimization ] Here's a comparison video of the Female Dancer's second optimization (the third one). Her legs are no longer thick as they were in the first optimization, and her chest size has been reduced to match the original model. ※ Source:

Identity V | News

55,821 Aufrufe • vor 7 Monaten

As AI labs race to train and deploy new frontier models, existing models become more affordable with better tokenomics. ✨ "Everybody's trying to get to the next frontier. And every time they get to the next frontier, the last generation AI tokens, the cost starts to decline about a factor of 10x every year," said NVIDIA CEO Jensen Huang in a recent keynote. Model optimization techniques such as speculative decoding and multi-token prediction, combined with inference serving platforms like NVIDIA Dynamo on NVIDIA Blackwell NVL72 systems, enable AI factories to boost throughput by 10x with one-tenth of the cost per token. Learn more about AI factory tokenomics ➡️

As AI labs race to train and deploy new frontier models, existing models become more affordable with better tokenomics. ✨ "Everybody's trying to get to the next frontier. And every time they get to the next frontier, the last generation AI tokens, the cost starts to decline about a factor of 10x every year," said NVIDIA CEO Jensen Huang in a recent keynote. Model optimization techniques such as speculative decoding and multi-token prediction, combined with inference serving platforms like NVIDIA Dynamo on NVIDIA Blackwell NVL72 systems, enable AI factories to boost throughput by 10x with one-tenth of the cost per token. Learn more about AI factory tokenomics ➡️

NVIDIA AI

16,053 Aufrufe • vor 7 Monaten

Every AI product running in production depends on an inference system someone had to engineer, optimize and more. Scheduling, batching, routing, cost per token. This is a craft. The Inference Frontier Program spotlights the builders behind that work. 💡 Watch the video and nominate a team:

Every AI product running in production depends on an inference system someone had to engineer, optimize and more. Scheduling, batching, routing, cost per token. This is a craft. The Inference Frontier Program spotlights the builders behind that work. 💡 Watch the video and nominate a team:

Nebius

54,701 Aufrufe • vor 5 Monaten

In case you missed it, we recently launched "Post-training of LLMs," a short course where you'll: ✅ Understand when and why to use post-training methods like Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning. ✅ Learn the concepts underlying the three post-training methods of SFT, DPO, and Online RL, their common use-cases, and how to curate high-quality data to effectively train a model using each method. ✅ Download a pre-trained model and implement post-training pipelines to turn a base model into an instruct model, change the identity of a chat assistant, and improve a model’s math capabilities. Learn more and enroll for free:

In case you missed it, we recently launched "Post-training of LLMs," a short course where you'll: ✅ Understand when and why to use post-training methods like Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning. ✅ Learn the concepts underlying the three post-training methods of SFT, DPO, and Online RL, their common use-cases, and how to curate high-quality data to effectively train a model using each method. ✅ Download a pre-trained model and implement post-training pipelines to turn a base model into an instruct model, change the identity of a chat assistant, and improve a model’s math capabilities. Learn more and enroll for free:

DeepLearning.AI

16,779 Aufrufe • vor 1 Jahr

the 3 phases of AI sticker shock: 1. board asks “whats our AI strategy?” 2. company says “everyone use AI for everything” 3. the token bill arrives, the crackdown begins ...and model routing starts to look pretty obvious. Matan Grinberg: "“There are different models that are good at different tasks. They have different trade-offs between cost, quality, and speed"

the 3 phases of AI sticker shock: 1. board asks “whats our AI strategy?” 2. company says “everyone use AI for everything” 3. the token bill arrives, the crackdown begins ...and model routing starts to look pretty obvious. Matan Grinberg: "“There are different models that are good at different tasks. They have different trade-offs between cost, quality, and speed"

Deirdre Bosa

31,903 Aufrufe • vor 2 Monaten

🚨 Forget LIDAR. The Robbyant team just dropped a streaming 3D model that reconstructs scenes live, at ~20 FPS, over long sequences. One single camera. Runs in real time. Open-source. Entirely end-to-end. NO iterative optimization tricks and no post-processing cleanup steps! It outperforms both existing streaming approaches and several offline methods. 100% Free and open-source. Repo, paper and model weights in 🧵↓

🚨 Forget LIDAR. The Robbyant team just dropped a streaming 3D model that reconstructs scenes live, at ~20 FPS, over long sequences. One single camera. Runs in real time. Open-source. Entirely end-to-end. NO iterative optimization tricks and no post-processing cleanup steps! It outperforms both existing streaming approaches and several offline methods. 100% Free and open-source. Repo, paper and model weights in 🧵↓

Charly Wargnier

36,229 Aufrufe • vor 2 Monaten

New Course: Post-training of LLMs Learn to post-train and customize an LLM in this short course, taught by Banghua Zhu, Assistant Professor at the University of Washington University of Washington, and co-founder of @NexusflowX. Training an LLM to follow instructions or answer questions has two key stages: pre-training and post-training. In pre-training, it learns to predict the next word or token from large amounts of unlabeled text. In post-training, it learns useful behaviors such as following instructions, tool use, and reasoning. Post-training transforms a general-purpose token predictor—trained on trillions of unlabeled text tokens—into an assistant that follows instructions and performs specific tasks. Because it is much cheaper than pre-training, it is practical for many more teams to incorporate post-training methods into their workflows than pre-training. In this course, you’ll learn three common post-training methods—Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning (RL)—and how to use each one effectively. With SFT, you train the model on pairs of input and ideal output responses. With DPO, you provide both a preferred (chosen) and a less preferred (rejected) response and train the model to favor the preferred output. With RL, the model generates an output, receives a reward score based on human or automated feedback, and updates the model to improve performance. You’ll learn the basic concepts, common use cases, and principles for curating high-quality data for effective training. Through hands-on labs, you’ll download a pre-trained model from Hugging Face and post-train it using SFT, DPO, and RL to see how each technique shapes model behavior. In detail, you’ll: - Understand what post-training is, when to use it, and how it differs from pre-training. - Build an SFT pipeline to turn a base model into an instruct model. - Explore how DPO reshapes behavior by minimizing contrastive loss—penalizing poor responses and reinforcing preferred ones. - Implement a DPO pipeline to change the identity of a chat assistant. - Learn online RL methods such as Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO), and how to design reward functions. - Train a model with GRPO to improve its math capabilities using a verifiable reward. Post-training is one of the most rapidly developing areas of LLM training. Whether you’re building a high-accuracy context-specific assistant, fine-tuning a model's tone, or improving task-specific accuracy, this course will give you experience with the most important techniques shaping how LLMs are post-trained today. Please sign up here:

New Course: Post-training of LLMs Learn to post-train and customize an LLM in this short course, taught by Banghua Zhu, Assistant Professor at the University of Washington University of Washington, and co-founder of @NexusflowX. Training an LLM to follow instructions or answer questions has two key stages: pre-training and post-training. In pre-training, it learns to predict the next word or token from large amounts of unlabeled text. In post-training, it learns useful behaviors such as following instructions, tool use, and reasoning. Post-training transforms a general-purpose token predictor—trained on trillions of unlabeled text tokens—into an assistant that follows instructions and performs specific tasks. Because it is much cheaper than pre-training, it is practical for many more teams to incorporate post-training methods into their workflows than pre-training. In this course, you’ll learn three common post-training methods—Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning (RL)—and how to use each one effectively. With SFT, you train the model on pairs of input and ideal output responses. With DPO, you provide both a preferred (chosen) and a less preferred (rejected) response and train the model to favor the preferred output. With RL, the model generates an output, receives a reward score based on human or automated feedback, and updates the model to improve performance. You’ll learn the basic concepts, common use cases, and principles for curating high-quality data for effective training. Through hands-on labs, you’ll download a pre-trained model from Hugging Face and post-train it using SFT, DPO, and RL to see how each technique shapes model behavior. In detail, you’ll: - Understand what post-training is, when to use it, and how it differs from pre-training. - Build an SFT pipeline to turn a base model into an instruct model. - Explore how DPO reshapes behavior by minimizing contrastive loss—penalizing poor responses and reinforcing preferred ones. - Implement a DPO pipeline to change the identity of a chat assistant. - Learn online RL methods such as Proximal Policy Optimization (PPO) and Group Relative Policy Optimization (GRPO), and how to design reward functions. - Train a model with GRPO to improve its math capabilities using a verifiable reward. Post-training is one of the most rapidly developing areas of LLM training. Whether you’re building a high-accuracy context-specific assistant, fine-tuning a model's tone, or improving task-specific accuracy, this course will give you experience with the most important techniques shaping how LLMs are post-trained today. Please sign up here:

Andrew Ng

125,146 Aufrufe • vor 1 Jahr

Hot take: COD Mobile should completely remove TPP mode and focus only on FPP... They should try to make CODM BR more like Warzone Mobile... WZM movements and gameplay was insane..just bad optimization COD Mobile can provide these movements with good optimization.. What u think about this ?

Hot take: COD Mobile should completely remove TPP mode and focus only on FPP... They should try to make CODM BR more like Warzone Mobile... WZM movements and gameplay was insane..just bad optimization COD Mobile can provide these movements with good optimization.. What u think about this ?

𝕏𝖎𝖆

33,496 Aufrufe • vor 8 Tagen

Learn to optimize RAG for cost and performance in our new short course, Prompt Compression and Query Optimization, created with MongoDB and taught by Richmond Alake. This course teaches you to combine traditional database capabilities with vector search using MongoDB for RAG. You'll learn these techniques: - Vector search: For semantic matching of user queries - Filtering using metadata: Pre- and post-filtering to narrow search results - Projections: Selecting only necessary fields to minimize data returned - Boosting: Reranking results to improve relevance - Prompt compression: Using a small LLM to compress context, significantly reducing token count and processing costs These methods address scaling, performance, and security challenges in large-scale RAG applications. You can sign up here:

Learn to optimize RAG for cost and performance in our new short course, Prompt Compression and Query Optimization, created with MongoDB and taught by Richmond Alake. This course teaches you to combine traditional database capabilities with vector search using MongoDB for RAG. You'll learn these techniques: - Vector search: For semantic matching of user queries - Filtering using metadata: Pre- and post-filtering to narrow search results - Projections: Selecting only necessary fields to minimize data returned - Boosting: Reranking results to improve relevance - Prompt compression: Using a small LLM to compress context, significantly reducing token count and processing costs These methods address scaling, performance, and security challenges in large-scale RAG applications. You can sign up here:

Andrew Ng

71,722 Aufrufe • vor 2 Jahren

Understanding OpenAI o1: Noam Brown on integrating reasoning into the model. Takeaways: - Avoid MCTS and current paradigm of using processes outside of the model during inference - Think about how to directly integrate reasoning into the model architecture

Understanding OpenAI o1: Noam Brown on integrating reasoning into the model. Takeaways: - Avoid MCTS and current paradigm of using processes outside of the model during inference - Think about how to directly integrate reasoning into the model architecture

Casper Hansen

313,371 Aufrufe • vor 1 Jahr

Jim Cramer on Leopold Aschenbrenner’s $NBIS stake “When this man touches something, it turns to gold.” The next AI bottleneck is efficient inference at scale which is why Nebius Token Factory is built around optimization, orchestration and agentic deployment rather than basic GPU rental.

Jim Cramer on Leopold Aschenbrenner’s $NBIS stake “When this man touches something, it turns to gold.” The next AI bottleneck is efficient inference at scale which is why Nebius Token Factory is built around optimization, orchestration and agentic deployment rather than basic GPU rental.

Shay Boloor

289,153 Aufrufe • vor 3 Monaten

Game menus had a different vibe back then. I miss this style. The modern minimalistic design feels bland, and it's lacking soul. I am sure a lot of research goes into AB testing and optimization, but I will always prefer this...

Game menus had a different vibe back then. I miss this style. The modern minimalistic design feels bland, and it's lacking soul. I am sure a lot of research goes into AB testing and optimization, but I will always prefer this...

exQUIZitely 🕹️

372,553 Aufrufe • vor 5 Monaten

Traditional GPU rentals often fluctuate due to supply bottlenecks and geopolitical factors. Aethir takes a different approach by providing bare-metal, on-demand GPU access, ensuring cost-efficient pricing and availability for AI training, inference, & agentic workflows.

Traditional GPU rentals often fluctuate due to supply bottlenecks and geopolitical factors. Aethir takes a different approach by providing bare-metal, on-demand GPU access, ensuring cost-efficient pricing and availability for AI training, inference, & agentic workflows.

Aethir

13,297 Aufrufe • vor 7 Monaten

Submodular optimization for token/sentence selection from long contexts. Here's an interesting exp: first used jina-embeddings-v4's multi-vector feature to extract token-level embeddings from a passage, then applied submodular optimization to cherry-pick the tokens that provide the best coverage, finally call tokenizer and convert selections back to the strings at their org. positions. Think of it as a form of "compression"—you can adjust the top-k slider to dial in different "compress rates". Can you still make sense of the compressed text?

Submodular optimization for token/sentence selection from long contexts. Here's an interesting exp: first used jina-embeddings-v4's multi-vector feature to extract token-level embeddings from a passage, then applied submodular optimization to cherry-pick the tokens that provide the best coverage, finally call tokenizer and convert selections back to the strings at their org. positions. Think of it as a form of "compression"—you can adjust the top-k slider to dial in different "compress rates". Can you still make sense of the compressed text?

Jina AI

13,177 Aufrufe • vor 1 Jahr

With Intelligent Model Routing, Replit picks the model for every task you send, so you can operate with quality, speed, and efficiency, at no extra cost. Here’s how it works, plus a breakdown on the difference between Free, Power and Max ↓

With Intelligent Model Routing, Replit picks the model for every task you send, so you can operate with quality, speed, and efficiency, at no extra cost. Here’s how it works, plus a breakdown on the difference between Free, Power and Max ↓

Replit ⠕

10,980 Aufrufe • vor 2 Tagen

built my first AI trading agent and it actually works 3 things that made it possible as a total beginner: - brimigs tutorial on youtube - deployer bankrbot skills - mattgansak (on ig) token optimization guide which reduces costs by up to 95% bookmark this if you've been wanting to try

built my first AI trading agent and it actually works 3 things that made it possible as a total beginner: - brimigs tutorial on youtube - deployer bankrbot skills - mattgansak (on ig) token optimization guide which reduces costs by up to 95% bookmark this if you've been wanting to try

Naomi Metzger

29,294 Aufrufe • vor 6 Monaten

"With the On-Demand model, users can choose their gas cost with the lowest latency possible for their transactions" - DoctorBlocks Listen to what DoctorBlocks says about functions and the importance of Switchboard's On-Demand model 👇

"With the On-Demand model, users can choose their gas cost with the lowest latency possible for their transactions" - DoctorBlocks Listen to what DoctorBlocks says about functions and the importance of Switchboard's On-Demand model 👇

Switchboard ⚡️

40,729 Aufrufe • vor 2 Jahren