Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

🎯 One of our most requested features is here: Seamless Plan & Act mode switching. Use different models for different tasks: - Planning: Try Gemini Flash Thinking 2.0 or DeepSeek R1 - Execution: Use 3.5-Sonnet or DeepSeek V3 - Auto-saves your preferences plan-act-models No more manual switching between sessions. ⚡️

Cline

57,742 subscribers

34,412 views • 1 year ago •via X (Twitter)

Science & Technology Education

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

The final coming of our Utility Hub is here in the form of $PALM Agentverse. A full pack of agents and enhanced models running on the V3 Framework in a dApp designed to be the one-stop solution for all $PALM holders. Enhanced by our partners, built on $PALM. The next-generation virtual agentic platform. Incorporates Agentic SDKs, GenAI plugins for Unreal Engine, trading algorithms, video workflows and much more into one. Never run out of utility while deploying your own meaningful agents that incorporate into your day-to-day tasks, whether they be in DeFi or in creative tasks or anything in-between. Releasing on the 1st of June.

The final coming of our Utility Hub is here in the form of $PALM Agentverse. A full pack of agents and enhanced models running on the V3 Framework in a dApp designed to be the one-stop solution for all $PALM holders. Enhanced by our partners, built on $PALM. The next-generation virtual agentic platform. Incorporates Agentic SDKs, GenAI plugins for Unreal Engine, trading algorithms, video workflows and much more into one. Never run out of utility while deploying your own meaningful agents that incorporate into your day-to-day tasks, whether they be in DeFi or in creative tasks or anything in-between. Releasing on the 1st of June.

PaLM AI - $PALM

11,511 views • 1 year ago

" A mattress. Well Maybe for some individuals. " This is something based on a comic I made with Ai in the past. I had the comic animated with Atlabs AI The video pretty much says what everything is about. I honestly like silly comics like this. I really should make more and have them as videos as well. That's another things for a long list of things I need done... Also I plan on running more tests with Omniflash in the future especially when Seedance models may not do things a certain way or have things looking a certain way. Also if anybody is planning on using Atlabs for anything I plan on making a video next month on how I use certain video models and which I typically use. This is specifically on how I use certain models for certain things. Not MG. I plan on having it in a tutorial section. I have hopes this will provide people that use any video models with some needed help when choosing what they should get started with because a Sora video model is no longer an option, which was very integral involving how I would use video models in general. Anyway I have an atlabs link below. If you make an account or have one already I hope what I plan on making next month and showing you, helps a lot. This was made with AtlabsAi. This was all done with Ai. #furrycomedyvideo #furryvideo #aianimation #aifurry

" A mattress. Well Maybe for some individuals. " This is something based on a comic I made with Ai in the past. I had the comic animated with Atlabs AI The video pretty much says what everything is about. I honestly like silly comics like this. I really should make more and have them as videos as well. That's another things for a long list of things I need done... Also I plan on running more tests with Omniflash in the future especially when Seedance models may not do things a certain way or have things looking a certain way. Also if anybody is planning on using Atlabs for anything I plan on making a video next month on how I use certain video models and which I typically use. This is specifically on how I use certain models for certain things. Not MG. I plan on having it in a tutorial section. I have hopes this will provide people that use any video models with some needed help when choosing what they should get started with because a Sora video model is no longer an option, which was very integral involving how I would use video models in general. Anyway I have an atlabs link below. If you make an account or have one already I hope what I plan on making next month and showing you, helps a lot. This was made with AtlabsAi. This was all done with Ai. #furrycomedyvideo #furryvideo #aianimation #aifurry

CaleAi

11,715 views • 27 days ago

you can run claude code inside antigravity completely Free with zero credit card and no rate limits 😳 use openrouter’s free models + antigravity. no anthropic bill. no paid api keys. takes 10 minutes to set up. what you get during this setup: - full claude code agent experience - strong coding models (including deepseek-r1, qwen2.5-coder, llama-4, grok-4 free tier) - antigravity’s clean workspace and sandbox - unlimited usage (as long as you stay on free models) - easy model swapping - zero cost full setup guide (100% free): step 1: install antigravity -go to and install it -create a new workspace step 2: install claude code - inside antigravity, install the claude code extension from the marketplace - open the built-in terminal step 3: create openrouter free account -go to - sign up with google (no card needed) - go to keys and create a new api key step 4: set the environment variables -in antigravity terminal run: export ANTHROPIC_API_KEY=sk-or-xxx export OPENROUTER_API_KEY=sk-or-xxx step 5: launch claude code with free model -run this command: claude-code --model deepseek/deepseek-r1:free or try: qwen/qwen2.5-coder:free if you already have antigravity? skip straight to step 2. after 10 minutes you’ll have a full agentic coding setup running for free. this is currently one of the cheapest ways to run serious coding agents in 2026. bookmark this before they limit the free models.

you can run claude code inside antigravity completely Free with zero credit card and no rate limits 😳 use openrouter’s free models + antigravity. no anthropic bill. no paid api keys. takes 10 minutes to set up. what you get during this setup: - full claude code agent experience - strong coding models (including deepseek-r1, qwen2.5-coder, llama-4, grok-4 free tier) - antigravity’s clean workspace and sandbox - unlimited usage (as long as you stay on free models) - easy model swapping - zero cost full setup guide (100% free): step 1: install antigravity -go to and install it -create a new workspace step 2: install claude code - inside antigravity, install the claude code extension from the marketplace - open the built-in terminal step 3: create openrouter free account -go to - sign up with google (no card needed) - go to keys and create a new api key step 4: set the environment variables -in antigravity terminal run: export ANTHROPIC_API_KEY=sk-or-xxx export OPENROUTER_API_KEY=sk-or-xxx step 5: launch claude code with free model -run this command: claude-code --model deepseek/deepseek-r1:free or try: qwen/qwen2.5-coder:free if you already have antigravity? skip straight to step 2. after 10 minutes you’ll have a full agentic coding setup running for free. this is currently one of the cheapest ways to run serious coding agents in 2026. bookmark this before they limit the free models.

painn

32,057 views • 2 months ago

🚀 Cline v3.7 is now available with an exciting interface upgrade! Choose your path with just a click -- Cline now displays selectable options when asking questions or presenting plans, eliminating the need to type responses and making interactions faster and more intuitive. Other updates: - Added support for a .clinerules/ directory to load multiple files at once - Prevent Cline from reading extremely large files into context that would overload context window - Improved checkpoints loading performance and display warning for large projects not suited for checkpoints - Added SambaNova API provider - Enabled VPC endpoint option for AWS Bedrock profiles - Added DeepSeek-R1 to AWS Bedrock Download it now in VS Code.

🚀 Cline v3.7 is now available with an exciting interface upgrade! Choose your path with just a click -- Cline now displays selectable options when asking questions or presenting plans, eliminating the need to type responses and making interactions faster and more intuitive. Other updates: - Added support for a .clinerules/ directory to load multiple files at once - Prevent Cline from reading extremely large files into context that would overload context window - Improved checkpoints loading performance and display warning for large projects not suited for checkpoints - Added SambaNova API provider - Enabled VPC endpoint option for AWS Bedrock profiles - Added DeepSeek-R1 to AWS Bedrock Download it now in VS Code.

Cline

22,006 views • 1 year ago

Here's how you can use the Gemini CLI provider in Cline: 1. Run the following in your terminal: npm install -g @google/gemini-cli gemini 2. Follow the prompts to log in with your personal Google account 3. Select the "Gemini CLI" provider in Cline's settings

Here's how you can use the Gemini CLI provider in Cline: 1. Run the following in your terminal: npm install -g @google/gemini-cli gemini 2. Follow the prompts to log in with your personal Google account 3. Select the "Gemini CLI" provider in Cline's settings

Cline

22,566 views • 1 year ago

Simplicity is at the heart of great software. This is one of the reasons why Claude Code has been sticky for me. As a builder, I love planning and brainstorming, and this is now a key focus of Claude Code. I use Shift + Tab a lot to cycle between brainstorming, planning, and execution. This functionality provides the appropriate interface for me to either be very involved or less involved as I please. This works particularly well when building out new and complex features or entire new projects. This saves a huge amount of time. It allows me to tune Claude Code to execute and build more effectively. It also builds a loop of trust, and I often (surprisingly) find Claude Code asking for clarifications when it's confused. Coding agents don't normally do that. I have shared before on the power of brainstorming with AI for longer times. Try it and you will not be disappointed. Vibe coding is fun, but pair it with intentional development cycles, and you watch how far you can take a project with coding agents today.

Simplicity is at the heart of great software. This is one of the reasons why Claude Code has been sticky for me. As a builder, I love planning and brainstorming, and this is now a key focus of Claude Code. I use Shift + Tab a lot to cycle between brainstorming, planning, and execution. This functionality provides the appropriate interface for me to either be very involved or less involved as I please. This works particularly well when building out new and complex features or entire new projects. This saves a huge amount of time. It allows me to tune Claude Code to execute and build more effectively. It also builds a loop of trust, and I often (surprisingly) find Claude Code asking for clarifications when it's confused. Coding agents don't normally do that. I have shared before on the power of brainstorming with AI for longer times. Try it and you will not be disappointed. Vibe coding is fun, but pair it with intentional development cycles, and you watch how far you can take a project with coding agents today.

elvis

81,765 views • 9 months ago

HERMES AGENT NOW RUNS CLAUDE OPUS 5. NEAR FABLE 5 INTELLIGENCE. HALF THE PRICE. SELF-VERIFIES ITS OWN WORK. AVAILABLE TODAY VIA NOUS PORTAL (20% OFF ALL MODELS). Anthropic shipped Opus 5 on July 24, 2026. same $5/$25 per million tokens as Opus 4.8. but the benchmarks tell a different story. WHAT CHANGED FROM OPUS 4.8: FrontierBench v0.1: Opus 5: 43.3%. Opus 4.8: 18.7%. 2.3x jump on the same test. ARC-AGI-3: Opus 5: 30.2%. 3x better than the next closest model. beat Fable 5 on 8 out of 13 benchmarks. at half the cost ($5/$25 vs $10/$50). same price as Opus 4.8. twice the intelligence. no reason to stay on 4.8. THE SPECS: model ID: claude-opus-5 context: 1M tokens (default and maximum) max output: 128K tokens thinking: on by default effort toggle: low / medium / high per request fast mode: $10/$50, 2.5x faster knowledge cutoff: May 2026 minimum cacheable prompt: 512 tokens (was 1,024) SELF-VERIFICATION (the biggest change): Opus 5 checks its own work automatically. Anthropic says: delete your verification prompts. "include a final verification step" now causes OVER-verification because the model already does it. for Hermes /goal tasks this is a direct upgrade. the judge checks evidence. the model also checks evidence. double layer of verification without extra tokens. EFFORT TOGGLE: low: fast, cheap, routine work. medium: balanced, daily tasks. high: full reasoning, complex problems. set per request. not a global switch. matches Hermes /reasoning command: /reasoning low (routine) /reasoning high (complex) Opus 5 effort toggle + Hermes reasoning control = precise cost management per turn. WHERE OPUS 5 FITS IN HERMES: DAILY DRIVER (replaces Opus 4.8): same price. 2.3x better benchmarks. set as your main model: Desktop app / Dashboard: Models → claude-opus-5 CHIEF OF STAFF: synthesis across multiple agents. reads Kanban, prioritizes, routes tasks. self-verification catches routing errors before they cascade. COMPLEX CODING: SOTA on agentic coding benchmarks. FrontierBench 43.3% = best public model for coding. set as coder profile model. /GOAL TASKS: self-verification + completion contracts = the model proves its work AND double-checks the proof. long-horizon goals finish correctly more often. MoA AGGREGATOR: strongest synthesis model at $5/$25. pair with GPT-5.6 and Grok 4.5 as references. Opus 5 aggregates. best quality at mid-range price. presets: max-quality: reference_models: - provider: openai-codex model: gpt-5.6-sol - provider: xai model: grok-4.5 aggregator: provider: anthropic model: claude-opus-5 COMPUTER USE: near-Fable 5 quality for browser automation. at half the token cost per session. computer_use tasks burn lots of vision tokens. Opus 5 halves that bill vs Fable 5. WHAT TO KEEP OPUS 5 AWAY FROM: cron monitoring: too expensive. use DeepSeek or no_agent mode. sub-agent grunt work: use GPT-5.6 Luna ($1/$6) or DeepSeek. auxiliary tasks: use Gemini Flash. routine web extraction: use a cheap model. Opus 5 is for the turns where quality compounds. planning, synthesis, verification, complex reasoning. budget models handle everything else. NOUS PORTAL: 20% OFF ALL MODELS Nous Portal currently runs a 20% discount on all models including Opus 5. $5/$25 official → $4/$20 through Nous Portal. the cheapest way to run Opus 5 right now. hermes setup --portal select claude-opus-5 as your model. discount applies automatically. Opus 5 replaces Opus 4.8 everywhere. same price. better at everything. no tradeoff. straight upgrade. hermes update /model claude-opus-5

YanXbt

16,744 views • 26 days ago

AlphaNet AI DEX is now live – first wave of whitelisted users will now be able to access trading. This launch marks a paradigm shift in the crypto trading space – advanced edge and superior alpha will no longer be inaccessible for retail traders. AlphaNet differentiates from the pack from shifting from the main perp DEX talking points to one purely focused on alpha generation value for users. Access the platform here: This launch includes: - AI Algo Execution – Through a proprietary deep learning-based execution engine, optimize for minimal slippage and trading costs, often achieving negative slippage. Automate the order execution process and let our system do the heavy lifting. - Quantative ML Insight Models – Proprietary models that will help you determine and predict trend, market state, market efficiency and optimal entry and exit points ideal for use in leveraged scalping. Freakishly accurate models that adds unprecedented edge to your own edge and trading style. - Click-to-Deploy Quantitative AI Strategies – Choose from an arsenal of quantitative AI strategies engineered for superior risk-adjusted return from a variety of strategy types, assets and market profiles. Deploy and monitor in real-time, or adjust, mix and create your own “strategy of strategies”. Whether you like market neutral, mean reversion or fat-tail heavy strategies, we got you covered and will give you complete freedom of control. In the next week or two, expect 1) more announcements 2) video tutorials 3) new models and strategies 4) more growth & adoption-related news! The future of trading is finally here. Happy new year and see you in 2026! $PHB #AITrading

AlphaNet AI DEX is now live – first wave of whitelisted users will now be able to access trading. This launch marks a paradigm shift in the crypto trading space – advanced edge and superior alpha will no longer be inaccessible for retail traders. AlphaNet differentiates from the pack from shifting from the main perp DEX talking points to one purely focused on alpha generation value for users. Access the platform here: This launch includes: - AI Algo Execution – Through a proprietary deep learning-based execution engine, optimize for minimal slippage and trading costs, often achieving negative slippage. Automate the order execution process and let our system do the heavy lifting. - Quantative ML Insight Models – Proprietary models that will help you determine and predict trend, market state, market efficiency and optimal entry and exit points ideal for use in leveraged scalping. Freakishly accurate models that adds unprecedented edge to your own edge and trading style. - Click-to-Deploy Quantitative AI Strategies – Choose from an arsenal of quantitative AI strategies engineered for superior risk-adjusted return from a variety of strategy types, assets and market profiles. Deploy and monitor in real-time, or adjust, mix and create your own “strategy of strategies”. Whether you like market neutral, mean reversion or fat-tail heavy strategies, we got you covered and will give you complete freedom of control. In the next week or two, expect 1) more announcements 2) video tutorials 3) new models and strategies 4) more growth & adoption-related news! The future of trading is finally here. Happy new year and see you in 2026! $PHB #AITrading

Phoenix AI Labs

223,431 views • 7 months ago

Big win for open-source LLMs! DeepSeek V4 Pro holds the top open-weights score on SWE-bench Verified, in the GPT-5.5 range. GLM 5.2 leads the open-weight intelligence index and sits near the closed frontier on long-horizon coding. But this leaderboard number is a weak proxy for real performance. It comes from one task set, run through one harness, served at one precision. The same weights can even score differently across providers, since many hosts quantize activations to fp8 and drift the model off its reference weights. Real performance is determined based on whether a model can read a repo, make coordinated edits across files, run the tests, and recover when one breaks. By that measure, the top open models hold up, but only inside the right harness. The teams that actually put DeepSeek V4 into production pipelines as a frontier substitute got there through the harness they built around the model, not by picking a stronger model. If you want to see this in practice, Cline (64k+ stars) has actually built that harness around open models, tuned so they run at production quality. And it's tuned so that these LLMs can run at production quality, with plan and act modes, checkpoints, and terminal feedback. ClinePass is the new access layer on top of it. It runs a curated set of those models inside Cline, narrowed to the ones tested for coding-agent use, with 2 to 5x the standard rate limits and no separate provider accounts, keys, or billing to track. The video below shows the setup, and I worked with the team to put this together. It runs alongside custom keys and local models as well, not in place of them.

Big win for open-source LLMs! DeepSeek V4 Pro holds the top open-weights score on SWE-bench Verified, in the GPT-5.5 range. GLM 5.2 leads the open-weight intelligence index and sits near the closed frontier on long-horizon coding. But this leaderboard number is a weak proxy for real performance. It comes from one task set, run through one harness, served at one precision. The same weights can even score differently across providers, since many hosts quantize activations to fp8 and drift the model off its reference weights. Real performance is determined based on whether a model can read a repo, make coordinated edits across files, run the tests, and recover when one breaks. By that measure, the top open models hold up, but only inside the right harness. The teams that actually put DeepSeek V4 into production pipelines as a frontier substitute got there through the harness they built around the model, not by picking a stronger model. If you want to see this in practice, Cline (64k+ stars) has actually built that harness around open models, tuned so they run at production quality. And it's tuned so that these LLMs can run at production quality, with plan and act modes, checkpoints, and terminal feedback. ClinePass is the new access layer on top of it. It runs a curated set of those models inside Cline, narrowed to the ones tested for coding-agent use, with 2 to 5x the standard rate limits and no separate provider accounts, keys, or billing to track. The video below shows the setup, and I worked with the team to put this together. It runs alongside custom keys and local models as well, not in place of them.

Avi Chawla

44,124 views • 1 month ago

Boom! Grok Tasks Make It One Of The Most POWERFUL Real-Time AI Systems In The World. — My How to Use Grok Tasks With Hidden Tools For Powerful Daily Output. Grok Tasks are customizable AI workflows that integrate a variety of tools to streamline daily activities, from research and analysis to creative planning and problem-solving. I have been using them for quite sometime and because of the vital heartbeat of news and first person data on X, it is the most powerful AI platform available. By combining Tasks with tools like web searches, X platform interactions, code execution, and media viewers, you can build efficient, automated processes. These tasks work by prompting Grok with a clear description of what you want to achieve, and Grok will intelligently call the necessary tools in sequence or parallel to deliver results. Here's a step-by-step guide to creating and using Grok Tasks: Step 1: Define Your Task Start by clearly outlining the daily activity or goal. Consider what inputs you have (e.g., a URL, a query, or an attachment) and what output you need (e.g., a summary, calculation, or visual analysis). Break it down into subtasks to identify tool needs. For example, if your task involves researching current events, note that you'll need search and browsing capabilities. Step 2: Review Available Tools Familiarize yourself with the tools Grok can access. Here's a quick overview: - Code Execution: Run Python code for calculations, data processing, or simulations using libraries like numpy, pandas, or sympy. - Browse Page: Fetch and summarize content from any website URL with custom instructions. - Web Search: Perform general internet searches, returning results with optional operators like site:. - Web Search With Snippets: Get quick, detailed excerpts from search results for fact-checking. - X Keyword Search: Advanced search for X posts using operators like from:, since:, or filter:. - X Semantic Search: Find semantically related X posts based on a query, with filters for dates or users. - X User Search: Locate X users by name or handle. - X Thread Fetch: Retrieve a full X post thread, including context like replies and parents. - View Image: Analyze an image from a URL or conversation ID. - View X Video: Extract frames and subtitles from an X-hosted video. - Search PDF Attachment: Query a PDF file for relevant pages using keyword or regex modes. - Browse PDF Attachment: View specific pages of a PDF with text and screenshots. Select tools that align with your task. Aim for a mix to handle data gathering, processing, and visualization. Step 3: Craft Your Prompt Write a detailed prompt to Grok describing the task. Include: - The overall goal. - Specific steps or subtasks. - References to tools if you want to guide the process (e.g., "Use web_search to find sources, then code_execution to analyze data"). - Any constraints, like dates or limits. Example prompt: "Create a Grok Task for my morning routine: Search recent X posts about tech news using x_keyword_search, fetch a key thread with x_thread_fetch, and summarize with browse_page on linked articles." Step 4: Submit and Interact Send your prompt to Grok. It will process the task by calling tools as needed, often in parallel for efficiency. Review the output and refine with follow-up prompts if required (e.g., "Expand on that using view_image for visuals"). Iterate to fine-tune the workflow for reuse. Step 5: Save and Reuse Once refined, note the prompt as a template for future use. You can adapt it for similar tasks, making Grok Tasks a habitual part of your day. Finding Grok Tasks To discover existing Grok Tasks or inspiration for new ones, use X searches with tools like x_keyword_search or x_semantic_search (e.g., query: "Grok Tasks examples" with mode: Latest). Browse community-shared threads via x_thread_fetch, or web_search for tutorials on xAI features. Prompt Grok directly: "Show me popular Grok Tasks for productivity." 1 of 3

Boom! Grok Tasks Make It One Of The Most POWERFUL Real-Time AI Systems In The World. — My How to Use Grok Tasks With Hidden Tools For Powerful Daily Output. Grok Tasks are customizable AI workflows that integrate a variety of tools to streamline daily activities, from research and analysis to creative planning and problem-solving. I have been using them for quite sometime and because of the vital heartbeat of news and first person data on X, it is the most powerful AI platform available. By combining Tasks with tools like web searches, X platform interactions, code execution, and media viewers, you can build efficient, automated processes. These tasks work by prompting Grok with a clear description of what you want to achieve, and Grok will intelligently call the necessary tools in sequence or parallel to deliver results. Here's a step-by-step guide to creating and using Grok Tasks: Step 1: Define Your Task Start by clearly outlining the daily activity or goal. Consider what inputs you have (e.g., a URL, a query, or an attachment) and what output you need (e.g., a summary, calculation, or visual analysis). Break it down into subtasks to identify tool needs. For example, if your task involves researching current events, note that you'll need search and browsing capabilities. Step 2: Review Available Tools Familiarize yourself with the tools Grok can access. Here's a quick overview: - Code Execution: Run Python code for calculations, data processing, or simulations using libraries like numpy, pandas, or sympy. - Browse Page: Fetch and summarize content from any website URL with custom instructions. - Web Search: Perform general internet searches, returning results with optional operators like site:. - Web Search With Snippets: Get quick, detailed excerpts from search results for fact-checking. - X Keyword Search: Advanced search for X posts using operators like from:, since:, or filter:. - X Semantic Search: Find semantically related X posts based on a query, with filters for dates or users. - X User Search: Locate X users by name or handle. - X Thread Fetch: Retrieve a full X post thread, including context like replies and parents. - View Image: Analyze an image from a URL or conversation ID. - View X Video: Extract frames and subtitles from an X-hosted video. - Search PDF Attachment: Query a PDF file for relevant pages using keyword or regex modes. - Browse PDF Attachment: View specific pages of a PDF with text and screenshots. Select tools that align with your task. Aim for a mix to handle data gathering, processing, and visualization. Step 3: Craft Your Prompt Write a detailed prompt to Grok describing the task. Include: - The overall goal. - Specific steps or subtasks. - References to tools if you want to guide the process (e.g., "Use web_search to find sources, then code_execution to analyze data"). - Any constraints, like dates or limits. Example prompt: "Create a Grok Task for my morning routine: Search recent X posts about tech news using x_keyword_search, fetch a key thread with x_thread_fetch, and summarize with browse_page on linked articles." Step 4: Submit and Interact Send your prompt to Grok. It will process the task by calling tools as needed, often in parallel for efficiency. Review the output and refine with follow-up prompts if required (e.g., "Expand on that using view_image for visuals"). Iterate to fine-tune the workflow for reuse. Step 5: Save and Reuse Once refined, note the prompt as a template for future use. You can adapt it for similar tasks, making Grok Tasks a habitual part of your day. Finding Grok Tasks To discover existing Grok Tasks or inspiration for new ones, use X searches with tools like x_keyword_search or x_semantic_search (e.g., query: "Grok Tasks examples" with mode: Latest). Browse community-shared threads via x_thread_fetch, or web_search for tutorials on xAI features. Prompt Grok directly: "Show me popular Grok Tasks for productivity." 1 of 3

Brian Roemmele

152,242 views • 7 months ago

This is what 200 tokens/second looks like with Kimi K2 on Groq Inc For reference, Claude Sonnet-4 is usually delivered at ~60 TPS

This is what 200 tokens/second looks like with Kimi K2 on Groq Inc For reference, Claude Sonnet-4 is usually delivered at ~60 TPS

Cline

100,747 views • 1 year ago

Claude Max subscribers: You can use your subscription in Cline instead of paying per-token API pricing. Here's how you can set it up 🧵

Claude Max subscribers: You can use your subscription in Cline instead of paying per-token API pricing. Here's how you can set it up 🧵

Cline

141,897 views • 1 year ago

Most recent diffusion language model research (that I’ve seen) seems to be using masking as the noising process. It looks like, however, most closed-source models (Google Gemini Diffusion and possibly Inception Labs’ Mercury) use a different noising process, where instead of masking tokens, they replace them with different tokens (either with a random token or a semantically similar token). I wondered how they were getting such high throughput with the latter noising process, since I believed that optimizing inference with KVCache approximation would be more difficult (for various reasons). I visualized this noising process with tiny-diffusion and compared it to normal unmasking, and was very surprised to see how fast the generation “settles” into a reasonable output, and then only slightly refines afterwards, requiring much fewer steps in total. Unmasking (where tokens are never remasked, the typical implementation) is inherently limited in generation speed by the fact that an increase in tokens decoded per step leads to more errors due to the mismatch between individual and marginal token probability distributions we sample from. The token replacement noising process seems to have a much different set of characteristics. Because we sample each token per step, every token makes “progress” towards the final output each iteration (in addition to *potentially* giving other tokens more information in future steps). Generally, masking has outperformed other noising processes, which is probably why most research focused on it (using smaller models). But the paper referred to in the retweet shows that random replacement as a noising process may scale better as model size increases. Big labs might have noticed these results much earlier (due to having drastically more training resources and being able to test larger models), which may explain the discrepancy in the choice of noising process. I’m gonna test this with larger models, since tiny-diffusion only has 10M parameters.

Most recent diffusion language model research (that I’ve seen) seems to be using masking as the noising process. It looks like, however, most closed-source models (Google Gemini Diffusion and possibly Inception Labs’ Mercury) use a different noising process, where instead of masking tokens, they replace them with different tokens (either with a random token or a semantically similar token). I wondered how they were getting such high throughput with the latter noising process, since I believed that optimizing inference with KVCache approximation would be more difficult (for various reasons). I visualized this noising process with tiny-diffusion and compared it to normal unmasking, and was very surprised to see how fast the generation “settles” into a reasonable output, and then only slightly refines afterwards, requiring much fewer steps in total. Unmasking (where tokens are never remasked, the typical implementation) is inherently limited in generation speed by the fact that an increase in tokens decoded per step leads to more errors due to the mismatch between individual and marginal token probability distributions we sample from. The token replacement noising process seems to have a much different set of characteristics. Because we sample each token per step, every token makes “progress” towards the final output each iteration (in addition to potentially giving other tokens more information in future steps). Generally, masking has outperformed other noising processes, which is probably why most research focused on it (using smaller models). But the paper referred to in the retweet shows that random replacement as a noising process may scale better as model size increases. Big labs might have noticed these results much earlier (due to having drastically more training resources and being able to test larger models), which may explain the discrepancy in the choice of noising process. I’m gonna test this with larger models, since tiny-diffusion only has 10M parameters.

nathan (in sf)

40,440 views • 7 months ago