🚀Introducing The LLM Inference Provider Leaderboard - a live-updated,... unbiased eval of API Inference products. Featuring: Abacus.AI, Anyscale, DeepInfra, Decart, Fireworks AI, Lepton AI, Together AI, Perplexity, Replicate, as well as OpenAI and Anthropic models For each provider's Mixtral-8x7B and Llama-2-70B-Chat public endpoint, we benchmark cost, rate limit, P50 & P90 of throughput & TTFT, and average daily collections overtime for long term tracking. At Martian, we route each API request to the best LLM to reduce cost, reduce latency, and get the best performance. So finding the best providers is an important problem for us. We found that there's a > 5x cost difference, > 6x throughput variation, and even larger rate limit discrepancies among providers! Choosing between different LLMs is only part of the equation -- the selection of different inference endpoints is also crucial to get the best performance for your use case. See highlights of provider performance in🧵👇show more

Access Mixtral with the fastest inference performance anywhere! Up... to 100 token/s for $0.0006/1K tokens — to our knowledge the fastest performance at the lowest price! Mixtral-8x7b-32kseqlen Mistral AI & DiscoLM-mixtral-8x7b-v2 are live on Together API!show more

You can now route each request to the "fastest"... ⚡ or "cheapest" 💸 provider with Hugging Face Inference Providers in Copilot Chat on Visual Studio Code!🤗 For each model, you can explicitly choose "cheapest" when price matters - or "fastest" when throughput is the priority. thanks to Simon Brandeis for shipping this!show more

"The best organizations won't manage token budgets manually. Instead,... they'll rely on orchestration layers that automatically route workloads to the right models based on performance, cost, and use case. Keeping track of which model is best for coding, finance, research, or support will become too complex for humans to manage directly, so intelligent routing will become a core part of enterprise AI infrastructure." Aravind Srinivas How will the best orgs of the future manage token budgeting Michael Mignano Nikesh Arora Ryan Petersen mmurphshow more

Yesterday, we announced the launch of DeepProve: verifiable AI... for the future of humanity. Already, we’re lining up with partners across web3 to build a future where we can prove that AI models are operating as expected. Inference Labs - already pioneers of trustless AI - is collaborating with Lagrange to build the next gen of DeAI. 🧵show more

$ZKHIVE X $NMT 🤝 We are excited to announce... our major partnership with one of the leading AI projects in the space - NetMind.AI ! $NMT is an award winning AI powerhouse that has quickly become one of the most recognized decentralized AI platforms in the space, thanks to their wide range of offerings, from cost effective compute farms to elite AI consultancy services and AI chatbots. As part of our long term partnership, $ZKHIVE and NMT are going to build security AI models together 🤖🐝 #zkHive will also receive an access to $NMT’s compute farm as well as AI consultancy from $NMT’s team of experts, and $NMT will get access to all of #zkHive’s products and APIs! This partnership will improve zkHive’s AI capabilities and introduce thousands of new users to $ZKHIVE ecosystem!show more

Something NVIDIA & Google do better than anyone else... is software-hardware-system co-design, and not just optimizing hardware for current model architectures, but predicting future ones. Back in early 2022, when NVIDIA started the design process for NVL72, MoE (Mixture of Experts) models were not yet the standard, and dense models were still dominant for frontier models. However, NVIDIA's strong software-hardware co-design culture enabled them to make a calculated bet that MoEs were the future, and they built NVL72 specifically for best MoE performance per TCO (Total Cost of Ownership). Furthermore, back in 2022, disaggregated prefill and wide expert parallelism (wideEP) MoE inference optimizations hadn't been invented yet, but it turns out that these MoE inference optimizations work best on large-scale systems like NVL72. While most other AI chip companies' in-house AI labs focus on training small 5B models that mainly use data parallelism, NVIDIA and Google's in-house AI labs continuously push the boundaries of model architecture and training recipes, such as NVFP4 training. Just like Super Idol & IShowSpeed, there must be a strong partnership between software engineers and hardware engineers to deliver the best systems that maximize performance per TCO.show more

Today, we’re excited to announce a partnership with Mind... Network. Both Nesa and Mind Network specialize in decentralized security, and we share a mission to bring this tech to crypto AI. Together we will explore a close technological collaboration, sharing components of our stacks with one another. Mind Network will also be using Nesa for its AI inference. Mind Network is the first FHE Restaking Layer for AI, focused on enhancing security at the consensus and validator levels of AI networks. Nesa is the Layer-1 blockchain for AI, specializing in private inference and building infrastructure to make it easy for any application, protocol, and smart contract to fuse with AI. Look out for our AMA together and other activations soon.show more

Today, Nesa is excited to announce cross-chain support with... Sei. As part of this new development, Nesa will be featuring a dedicated ramp for Sei’s rich ecosystem of applications to seamlessly integrate with AI. Now any Sei dapp can run AI inference on hundreds of models supported by Nesa, including the largest LLMs and most popular models across Vision, Language, and Generative AI. For developers, we’ll have information soon on how to access AI inference on Nesa via smart contracts from the Sei blockchain. Stay tuned for more details on this ground breaking development for decentralized AI.show more

At Dreamforce, the exploration of the possibilities of AI... is incredible. At the same time, it's important that we understand that policies never move at the same speed as technology. So having a grasp over what is happening and how the negative aspects of AI don't proliferate in a big way, to minimize collateral damage is most important. I hope the companies and governments across the world wake up to this and ensure the best of AI possibilities will reach the people, as this is a phenomenal enabler for humanity like never before. -Sg Marc Benioff Dreamforce Salesforce #DF24show more

Johnson: We are going to continue to reduce the... cost of healthcare. We’ll continue reducing prices and the cost of livingshow more

Seedance 2.0 is best tool for AI yappers and... as we all know, yapper videos are RIPPING right now and the best part? they can be used for literally anything... - Cost per video: ~$1 - Production time: under a minute - Scale: instant Comment “AI” and I’ll DM the full workflow. (must be connected)show more

Partnership Announcement! We are thrilled to partner with Optopia,... the AI-focused L2 Blockchain. 🤝 This collaboration with Optopia | 1st AI Agent Layer2 will bring a new AI application standard to the ICB Network, providing our ecosystem with a high-performance, low-cost, and secure platform for the next generation of AI dApps. Together, we are building the future of AI. #ICBNetwork #Optopia #AI #Web3 #L2show more

DeepSeek DeepSeek is now live on Heurist. Open source... AI just reached new heights. Access DeepSeek-R1 and DeepSeek-V3 for smarter inference at a fraction of cost. Try it via Heurist API or explore on Pondera.show more

We believe the future of AI must be open.... While other providers want to own your models, your data, and your agents, we offer you the freedom to: 🔵 choose the world’s best models 🔴 run AI wherever your data lives 🟡 control your own destinyshow more

Season 2 of Fableborne went live this morning and... we are now trending in gaming! Hundreds of thousands of raids already complete! For 2 years we waited and waited, patiently building, restraining ourselves from all the noise and short term distractions focsuing just on building the best game to use as the base of what is to come. As a founder, working with this team each day, its easy for me to be unrealistically bullish. Its so satisfying to see the rest of web3 now seeing why.show more

We raised $7.1M at Alien led by Initialized and... Finality to give unique identities to humans and AI agents The world is going wild and the trust online is breaking as we see it due to the rate of AI acceleration and the society’s inability to keep upshow more

Kling AI 3.0 is the Nano Banana Pro moment... for video models. Highlight: Multi cut with up to 15s per run and enhanced lip sync. The performance of characters is the best I’ve seen so far! And you can literally use it like a reference model. This is the image I used:show more

One of the coolest uses of LM Studio and... AI at AMD is using LM Link to have a Framework Desktop be a remote inference server for a laptop, including something little like Framework Laptop 12. It feels native on the laptop, but gets the desktop’s performance.show more