正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Wanna know whether different LLM providers serve the same LLama 3.1 70B? I sure did! So I ran a quick eval to get some surprising results + open sourced my code 👇 Check out my comparison between Groq Inc Fireworks AI octoaicloud DeepInfra and Together AI

Alex Volkov (Thursd/AI)

34,575 subscribers

26,549 次观看 • 2 年前 •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

10 条评论

Alex Volkov (Thursd/AI) 🔜 AIENG summit NY 的头像

Alex Volkov (Thursd/AI) 🔜 AIENG summit NY2 年前

If you don't want to watch all that, check out the results yourself, the project is open on my @weights_biases Weave dashboard, here's the direct link to the comparison view 👉

Alex Volkov (Thursd/AI) 🔜 AIENG summit NY 的头像

Alex Volkov (Thursd/AI) 🔜 AIENG summit NY2 年前

And the code is live on Github, it's a simple python notebook + some instrumentation via @GroqInc and @OpenRouterAI which is amazing for these types of comparisons (thanks @xanderatallah for the assist!)

Alex Volkov (Thursd/AI) 🔜 AIENG summit NY 的头像

Alex Volkov (Thursd/AI) 🔜 AIENG summit NY2 年前

If you're interested to try Weave out, super easy to get started, check out the docs at or lmk if I can help

Olaf Geibig eu/acc AI==危机 🇩🇪🇵🇱🇪🇺🌐 的头像

Olaf Geibig eu/acc AI==危机 🇩🇪🇵🇱🇪🇺🌐2 年前

@GroqInc @FireworksAI_HQ @OctoAICloud @DeepInfra @togethercompute Great. I also noticed differences between providers. It looks like the choice of provider is quite important. Would be great to have a continuously updated leaderboard for that. Wouldn't this be something cool to sponsor for W&B? It would also showcase the W&B tooling,

Maziyar PANAHI 的头像

Maziyar PANAHI2 年前

@GroqInc @FireworksAI_HQ @OctoAICloud @DeepInfra @togethercompute Interesting! Not to mention apart from latency, we pay for tokens in and out! So when they spit out more than the others, we wait longer and pay more.

Alex Volkov (Thursd/AI) 🔜 AIENG summit NY 的头像

Alex Volkov (Thursd/AI) 🔜 AIENG summit NY2 年前

@GroqInc @FireworksAI_HQ @OctoAICloud @DeepInfra @togethercompute Yeah, we need a better integration with OpenRouter (@xanderatallah maybe we should work together on this?) to support per provider pricing in the UI so we can actually estimate correct provider pricing and do an apples to apples comparison for that as well!

martyn ⏩ 的头像

martyn ⏩2 年前

@GroqInc @FireworksAI_HQ @OctoAICloud @DeepInfra @togethercompute wow, so every openrouter llama 3.1 70b provider has different results at temperature 0... that's strange

Sung Kim 的头像

Sung Kim2 年前

@GroqInc @FireworksAI_HQ @OctoAICloud @DeepInfra @togethercompute Very interesting!

Luis Ceze 的头像

Luis Ceze2 年前

@GroqInc @FireworksAI_HQ @OctoAICloud @DeepInfra @togethercompute Thanks for the analysis, @altryne! We are excited see you independently validate @OctoAICloud as a leading provider of Llama3.1 70b!

Alex Volkov (Thursd/AI) 🔜 AIENG summit NY 的头像

Alex Volkov (Thursd/AI) 🔜 AIENG summit NY2 年前

@GroqInc @FireworksAI_HQ @OctoAICloud @DeepInfra @togethercompute Thanks for taking the time to review Luis! Are you guys providing access to full 128K context already? cc Would love to collab with @OctoAICloud 👏

相关视频

This week was incredible for AI speed breakthroughs! Preparing for 📅 ThursdAI - weekly AI news podcast later today, I combined super fast LLM (<300ms Groq Inc ) + super fast SDXL (thanks to friends at fal) with infinite craft by Neal Agarwal to get near instant visualizations of the concepts! Mixtral via Groq Inc is being used at my prompt engineer, combining the concepts. SDXL via fal generates the imagery And I can't stop playing with this! if someone asked "why do you even need 400 tokens/sec" here's my first attempt at an answer. Going to cover this on #thursdAI today of course, potentially with folks from Groq AND Fal 👏

This week was incredible for AI speed breakthroughs! Preparing for 📅 ThursdAI - weekly AI news podcast later today, I combined super fast LLM (<300ms Groq Inc ) + super fast SDXL (thanks to friends at fal) with infinite craft by Neal Agarwal to get near instant visualizations of the concepts! Mixtral via Groq Inc is being used at my prompt engineer, combining the concepts. SDXL via fal generates the imagery And I can't stop playing with this! if someone asked "why do you even need 400 tokens/sec" here's my first attempt at an answer. Going to cover this on #thursdAI today of course, potentially with folks from Groq AND Fal 👏

Alex Volkov

29,540 次观看 • 2 年前

I did it. I figured out the perfect AI coding loop It's called the Finn Loop and it's 100x'd my vibe code output while cutting down work by 95% In this video I cover how to set it up By the end you'll have your own software factory +I completely open sourced it down below!

I did it. I figured out the perfect AI coding loop It's called the Finn Loop and it's 100x'd my vibe code output while cutting down work by 95% In this video I cover how to set it up By the end you'll have your own software factory +I completely open sourced it down below!

Alex Finn

87,765 次观看 • 22 天前

Wallet V: Trade with AI Agents just rolled out the VIP update. Here’s a quick walkthrough of the new interface, VIP bonuses, and how to earn V Points. Make sure your app is updated and check it out. Use my code MAYOR

Wallet V: Trade with AI Agents just rolled out the VIP update. Here’s a quick walkthrough of the new interface, VIP bonuses, and how to earn V Points. Make sure your app is updated and check it out. Use my code MAYOR

Mayor (Opinion arc) 🗣

18,810 次观看 • 7 个月前

OKAY I am sold on AI agents. This is INSANE. So I open up this exception, then: - AI tries to find a quick cause - ❌ not the cause - AI convinces itself that's not the root cause - ✅ AI finds the actual root causes - 🤯 AI suggests fixes - I guide the fix - implements the entire thing by looking at my code - 🔂 AI makes a PR - the PR actually fixes the issue???

OKAY I am sold on AI agents. This is INSANE. So I open up this exception, then: - AI tries to find a quick cause - ❌ not the cause - AI convinces itself that's not the root cause - ✅ AI finds the actual root causes - 🤯 AI suggests fixes - I guide the fix - implements the entire thing by looking at my code - 🔂 AI makes a PR - the PR actually fixes the issue???

Dan ⚡️

90,298 次观看 • 1 年前

Don't let boredom ruin progress 💪🏻 I have one of these machines at my house, so when I saw it at CES, I had to jump on for a quick workout 😅 Hit my bio link to check out Aeke and use my code "janna" for savings. Also head on over to my channel to see all of the latest and greatest tech & AI I found at CES! #fitness #workout #technology #healthy #technews

Don't let boredom ruin progress 💪🏻 I have one of these machines at my house, so when I saw it at CES, I had to jump on for a quick workout 😅 Hit my bio link to check out Aeke and use my code "janna" for savings. Also head on over to my channel to see all of the latest and greatest tech & AI I found at CES! #fitness #workout #technology #healthy #technews

Janna Breslin

18,088 次观看 • 5 个月前

$I just added the new Llama 3.2 1B and 3B models to LitGPT, the open-source LLM library I help develop (focused on efficiency and code readability). LitGPT allows you to fine-tune and use these models on the cloud or a laptop. So, if you are looking for something to play with this weekend: # 1) Finetune the model litgpt finetune_lora meta-llama/Llama-3.2-1B \ --data JSON \ --data.json_path my_custom_dataset.json \ --train.epochs 1 \ --out_dir out/llama-3.2-finetuned \ --precision bf16-true # 2) Chat with the model litgpt chat out/llama-3.2-finetuned/final # 3) Serve the model via an API endpoint litgpt serve out/llama-3.2-finetuned/final$

I just added the new Llama 3.2 1B and 3B models to LitGPT, the open-source LLM library I help develop (focused on efficiency and code readability). LitGPT allows you to fine-tune and use these models on the cloud or a laptop. So, if you are looking for something to play with this weekend: # 1) Finetune the model litgpt finetune_lora meta-llama/Llama-3.2-1B \ --data JSON \ --data.json_path my_custom_dataset.json \ --train.epochs 1 \ --out_dir out/llama-3.2-finetuned \ --precision bf16-true # 2) Chat with the model litgpt chat out/llama-3.2-finetuned/final # 3) Serve the model via an API endpoint litgpt serve out/llama-3.2-finetuned/final

Sebastian Raschka

65,529 次观看 • 1 年前

DeepSeek R1 671B is ridiculously good, but running at 158 tokens per second is something out of this world. A model *this good* running *this fast* is 3 years ahead of this time. The secret: DeepSeek in this short video isn't running on GPUs. It's running on custom-made chips! This is running on SambaNova and their SN40L chips, which are designed and optimized for running AI workflows. SambaNova is sponsoring my work. We designed GPUs to run games and later realized we could use them for AI. This is very different from creating a chip specifically for AI from the ground up. Just think about this for a second: One single SNL40 chip can simultaneously hold 100+ models (trillions of parameters) in memory! An agentic workflow that uses multiple models simultaneously won't need to swap models from memory even once! By the way, the Llama family of models *screams* on the SambaNova Cloud. Here are some numbers from my latest tests: • Llama 3.3 70B: 367.32 t/s • Llama 3.2 1B: 2381.82 t/s • Llama 3.2 3B: 1335.71 t/s • Llama 3.1 8B: 929.58 t/s • Llama 3.1 70B: 369.16 t/s • Llama 3.1 405B: 103.37 t/s You can access all of these models for free!

DeepSeek R1 671B is ridiculously good, but running at 158 tokens per second is something out of this world. A model this good running this fast is 3 years ahead of this time. The secret: DeepSeek in this short video isn't running on GPUs. It's running on custom-made chips! This is running on SambaNova and their SN40L chips, which are designed and optimized for running AI workflows. SambaNova is sponsoring my work. We designed GPUs to run games and later realized we could use them for AI. This is very different from creating a chip specifically for AI from the ground up. Just think about this for a second: One single SNL40 chip can simultaneously hold 100+ models (trillions of parameters) in memory! An agentic workflow that uses multiple models simultaneously won't need to swap models from memory even once! By the way, the Llama family of models screams on the SambaNova Cloud. Here are some numbers from my latest tests: • Llama 3.3 70B: 367.32 t/s • Llama 3.2 1B: 2381.82 t/s • Llama 3.2 3B: 1335.71 t/s • Llama 3.1 8B: 929.58 t/s • Llama 3.1 70B: 369.16 t/s • Llama 3.1 405B: 103.37 t/s You can access all of these models for free!

Santiago

131,019 次观看 • 1 年前

I’ve built an AI Agent system that can pull highlights of books etc. stored on my Readwise account and summarise them. It’s lit 🔥 Check out this video where I get the AI Agents to summarise the highlights from the book “Influence” Like/Reply if you want to access to the code

I’ve built an AI Agent system that can pull highlights of books etc. stored on my Readwise account and summarise them. It’s lit 🔥 Check out this video where I get the AI Agents to summarise the highlights from the book “Influence” Like/Reply if you want to access to the code

Eddie Forson

35,865 次观看 • 2 年前

I was a trucker before I got involved in politics. My family and I have worked in this industry for years and we did not cut corners. I have seen the issues around Driver Inc firsthand, with safety risks ignored, taxes dodged, and standards undercut just to make a quick buck. Check out what I had to say here 👇🏽

I was a trucker before I got involved in politics. My family and I have worked in this industry for years and we did not cut corners. I have seen the issues around Driver Inc firsthand, with safety risks ignored, taxes dodged, and standards undercut just to make a quick buck. Check out what I had to say here 👇🏽

Harman Bhangu

23,482 次观看 • 9 个月前

Welcome to the group chat, but it's all my AI agents. To build a website and plan the social content, I put together a team of AI agents. Each one had a different role. They collaborated, challenged each other's ideas, and reviewed each other's work before anything was finalized. Meet Bloome - a platform where humans and AI agents work together in the same conversation. You can bring different models into one workspace, assign them different jobs, and let them cross-check each other's thinking instead of relying on a single answer. It feels less like chatting with AI and more like managing your own creative team. Try it:

Welcome to the group chat, but it's all my AI agents. To build a website and plan the social content, I put together a team of AI agents. Each one had a different role. They collaborated, challenged each other's ideas, and reviewed each other's work before anything was finalized. Meet Bloome - a platform where humans and AI agents work together in the same conversation. You can bring different models into one workspace, assign them different jobs, and let them cross-check each other's thinking instead of relying on a single answer. It feels less like chatting with AI and more like managing your own creative team. Try it:

Gizem Akdag

99,348 次观看 • 1 个月前

This is a once-in-a-generation opportunity to bring Minnesotans together around common-sense solutions. I served in the Army, so I know how to fight. I built a career in business, so I understand what drives our economy. And I served as a legislator — so I know how to listen, lead, and get results. I’m proud to announce my campaign for U.S. Senate, and I’d be honored to earn your support.

This is a once-in-a-generation opportunity to bring Minnesotans together around common-sense solutions. I served in the Army, so I know how to fight. I built a career in business, so I understand what drives our economy. And I served as a legislator — so I know how to listen, lead, and get results. I’m proud to announce my campaign for U.S. Senate, and I’d be honored to earn your support.

David Hann

10,330 次观看 • 7 个月前

WE’RE SO CLOSE to the Titans uniforms and logo reveal, I can’t wait With all the disgusting AI slop pushed out there, I wanted to showcase some of my favorite concepts I’ve seen I’m sure I forgot a few, so lmk any you’ve seen! 🎨: The Graphic God, Ǥħøsŧ ŦɨŧȺn ⋆☠︎︎✮☠︎︎⋆, Watnick Creative

Zach Cohen

22,769 次观看 • 5 个月前

Just wrapped a deployment for a financial services provider’s Tier 1 support, and I had to test it myself, so I did a quick demo of the AI helping me unblock my card. The system automatically retrieved the info I needed and was able to loop in a human when needed

Just wrapped a deployment for a financial services provider’s Tier 1 support, and I had to test it myself, so I did a quick demo of the AI helping me unblock my card. The system automatically retrieved the info I needed and was able to loop in a human when needed

alinjavwa

10,063 次观看 • 8 个月前

Calling x402 endpoints isn't cool. You know what's cool? Letting AI agent's call x402 endpoints. I made a quick demo showing how to make an agent in AgentBlocks that can choose which x402 services to use, call & pay them, and combine the results. Check it out:

Calling x402 endpoints isn't cool. You know what's cool? Letting AI agent's call x402 endpoints. I made a quick demo showing how to make an agent in AgentBlocks that can choose which x402 services to use, call & pay them, and combine the results. Check it out:

David Mihal.eth

35,430 次观看 • 9 个月前

I built an open source data analyst agent! Upload any CSV, ask a question, and get it answered with statistics or a nice chart. Launching in 24 hours. 100% free & open source. Under the hood, here's how it works: 1. User uploads a CSV and asks a question. 2. The app uses Together Code Interpreter to spin up a VM and uploads the CSV onto it. 3. I have an LLM (Qwen 3 Coder) write code to interpret the CSV using `pandas`, then solve the question the user asked. 4. Together Code Interpreter runs the code in a secure environment & returns a result. These results can be text (some kind of stats analysis) or a chart.

I built an open source data analyst agent! Upload any CSV, ask a question, and get it answered with statistics or a nice chart. Launching in 24 hours. 100% free & open source. Under the hood, here's how it works: 1. User uploads a CSV and asks a question. 2. The app uses Together Code Interpreter to spin up a VM and uploads the CSV onto it. 3. I have an LLM (Qwen 3 Coder) write code to interpret the CSV using `pandas`, then solve the question the user asked. 4. Together Code Interpreter runs the code in a secure environment & returns a result. These results can be text (some kind of stats analysis) or a chart.

Hassan

27,086 次观看 • 11 个月前

ADAM CURRY: I'M NOT A BIG BELIEVER IN AI "I run these models at home on my own computer. I run the llama model, which is, Meta. They've open sourced it. I've, I've run the French one, and then DeepSeek, and it's not very impressive. It's just not. The the error rate is too high, so I'm I'm skeptical of it." Source: Joe Rogan

ADAM CURRY: I'M NOT A BIG BELIEVER IN AI "I run these models at home on my own computer. I run the llama model, which is, Meta. They've open sourced it. I've, I've run the French one, and then DeepSeek, and it's not very impressive. It's just not. The the error rate is too high, so I'm I'm skeptical of it." Source: Joe Rogan

Mario Nawfal

202,113 次观看 • 1 年前

I can't code. I know maybe 20% of how AI agents work. But I was mass angry at my API bill - so I built a full SaaS in 48 hours to fix it. Then I discovered a problem that's silently draining every OpenClaw user's wallet. And nobody's talking about it. 🧵👇

I can't code. I know maybe 20% of how AI agents work. But I was mass angry at my API bill - so I built a full SaaS in 48 hours to fix it. Then I discovered a problem that's silently draining every OpenClaw user's wallet. And nobody's talking about it. 🧵👇

Tuki

42,506 次观看 • 6 个月前

🚀 $AGX VS Fetch AI: The ULTIMATE AGI comparison! I tested AGIX | $AGX ($11M mc) against Fetch AI ($3.6B mc) on speed, accuracy, and precision by asking both the same 3 questions (simple to complex). The results? AGIX | $AGX DOMINATED: ✅ Lightning-fast, precise answers ✅ Sleek, real-time UI ✅ Outperformed a billion-dollar project! 👀 And here’s the kicker: I used an UNRELEASED AI tool from AGIX | $AGX to do the voice-over! The future is here 🤯 $AGX managed to fulfill my tasks in less than 2 minutes ✅, Fetch AI failed to deliver any results during 6 minutes ❌ Watch the full 6-min comparison below ⬇️ :

🚀 $AGX VS Fetch AI: The ULTIMATE AGI comparison! I tested AGIX | $AGX ($11M mc) against Fetch AI ($3.6B mc) on speed, accuracy, and precision by asking both the same 3 questions (simple to complex). The results? AGIX | $AGX DOMINATED: ✅ Lightning-fast, precise answers ✅ Sleek, real-time UI ✅ Outperformed a billion-dollar project! 👀 And here’s the kicker: I used an UNRELEASED AI tool from AGIX | $AGX to do the voice-over! The future is here 🤯 $AGX managed to fulfill my tasks in less than 2 minutes ✅, Fetch AI failed to deliver any results during 6 minutes ❌ Watch the full 6-min comparison below ⬇️ :

Sharky.eth

119,137 次观看 • 1 年前