Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Open-sourcing the first unified LLM routing library 🔥 Meet LLMRouter 16+ routers in ONE framework. Stop reimplementing routing papers from scratch, simply run pip install llmrouter-lib And instantly deploy SOTA LLM routing tailored to your exact needs! 🔗 Get Started with LLMRouter 🚀 Code: 🌐 Project Page: 💸 Why...

youjiaxuan's profile picture

Jiaxuan You

4,836 subscribers

75,244 Aufrufe • vor 8 Monaten •via X (Twitter)

BildungWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Love OpenClaw but hate the token burn? 💸 Running a 24/7 agent on GPT-4/Claude is overkill. You don't need SOTA reasoning to handle a greeting or a simple lookup. LLMRouter 🩷 OpenClaw The first production-ready, agentic router designed to plug directly into OpenClaw. LLMRouter fully supports Multimodal, Memory-Equipped routing that adapts 100% to your needs—compatible with FREE open-source models. The Logic is Simple:🔹 Simple query → Cheap/Local model 🔹 Complex reasoning → SOTA model (GPT-4/Claude 3.5) 🔹 Multimodal input → Vision/Audio specialized model Why this isn't just a switch: 📉 30–50% drop in inference costs 🧠 Zero loss in response quality 🔓 100% compatible with OpenAI-style APIs 🚀 Deploy in Seconds General Usage: Get the library and serve any model: pip install llmrouter-lib llmrouter serve OpenClaw Native Integration: Want the full agent experience? LLMRouter built a dedicated integration for OpenClaw users: LLMRouter Resources: 🔗 Repo: 📦 PyPI: 🤝 Works with: Route smarter. Train your own. Pay less. More on LLMRouter: Most routers are static if/else. LLMRouter is an intelligent, learning system. 🤖 Agentic & Memory-Aware: Decisions aren't stateless. We use RAG-powered memory to route based on context and history. 👤 Fully Personalized: It learns from your usage patterns via RL feedback loops. 🔬 Research-Grade: Switch between 16+ routing strategies (KNN, SVM, BERT, Graph, RL) with a single flag.
0:44

Love OpenClaw but hate the token burn? 💸 Running a 24/7 agent on GPT-4/Claude is overkill. You don't need SOTA reasoning to handle a greeting or a simple lookup. LLMRouter 🩷 OpenClaw The first production-ready, agentic router designed to plug directly into OpenClaw. LLMRouter fully supports Multimodal, Memory-Equipped routing that adapts 100% to your needs—compatible with FREE open-source models. The Logic is Simple:🔹 Simple query → Cheap/Local model 🔹 Complex reasoning → SOTA model (GPT-4/Claude 3.5) 🔹 Multimodal input → Vision/Audio specialized model Why this isn't just a switch: 📉 30–50% drop in inference costs 🧠 Zero loss in response quality 🔓 100% compatible with OpenAI-style APIs 🚀 Deploy in Seconds General Usage: Get the library and serve any model: pip install llmrouter-lib llmrouter serve OpenClaw Native Integration: Want the full agent experience? LLMRouter built a dedicated integration for OpenClaw users: LLMRouter Resources: 🔗 Repo: 📦 PyPI: 🤝 Works with: Route smarter. Train your own. Pay less. More on LLMRouter: Most routers are static if/else. LLMRouter is an intelligent, learning system. 🤖 Agentic & Memory-Aware: Decisions aren't stateless. We use RAG-powered memory to route based on context and history. 👤 Fully Personalized: It learns from your usage patterns via RL feedback loops. 🔬 Research-Grade: Switch between 16+ routing strategies (KNN, SVM, BERT, Graph, RL) with a single flag.

Jiaxuan You

31,451 Aufrufe • vor 7 Monaten

36 GROK AGENTS. ASTRA ON FREE CREDITS. DUAL-MODEL ROUTING IS THE EDGE. not one chat window burning a paid invoice all day a swarm that routes cheap work to grok and only wakes gpt-6 astra when the task is actually hard ▹ the stack 36 agents on grok bot with flexible settings per role monitor, plan, write, code, QA, ship, each with its own lane part of the fleet is wired to gpt-6 astra through a china free-credit service layer free credits are not a toy promo here they are the fuel for frontier spikes without a monthly bleed ▹ dual-model routing easy jobs stay on grok: speed, volume, always-on loops hard jobs jump to astra: reasoning, long builds, sharp code the router decides by task type, not by ego if astra is not needed, astra does not spend free-credit bursts buy the expensive brain grok agents keep the factory running between bursts that is how the system feels unlimited not by breaking quotas, by refusing to waste them ▹ why it hits different most people pay frontier prices for every mid task operators split the brain and protect the credits 36 agents = parallel throughput dual routing = cost control with quality when it matters free credits = astra access without living on the invoice the constraint moved from "can i afford the model" to "did i route the job to the right model" ▹ the take single-model stacks die on bills and on boredom multi-agent + dual routing is the new default factory grok for the grind astra for the cut free credits for the spikes that used to empty the wallet bookmark this before everyone copies the route map comments: what % of your tasks actually deserve astra
0:40

36 GROK AGENTS. ASTRA ON FREE CREDITS. DUAL-MODEL ROUTING IS THE EDGE. not one chat window burning a paid invoice all day a swarm that routes cheap work to grok and only wakes gpt-6 astra when the task is actually hard ▹ the stack 36 agents on grok bot with flexible settings per role monitor, plan, write, code, QA, ship, each with its own lane part of the fleet is wired to gpt-6 astra through a china free-credit service layer free credits are not a toy promo here they are the fuel for frontier spikes without a monthly bleed ▹ dual-model routing easy jobs stay on grok: speed, volume, always-on loops hard jobs jump to astra: reasoning, long builds, sharp code the router decides by task type, not by ego if astra is not needed, astra does not spend free-credit bursts buy the expensive brain grok agents keep the factory running between bursts that is how the system feels unlimited not by breaking quotas, by refusing to waste them ▹ why it hits different most people pay frontier prices for every mid task operators split the brain and protect the credits 36 agents = parallel throughput dual routing = cost control with quality when it matters free credits = astra access without living on the invoice the constraint moved from "can i afford the model" to "did i route the job to the right model" ▹ the take single-model stacks die on bills and on boredom multi-agent + dual routing is the new default factory grok for the grind astra for the cut free credits for the spikes that used to empty the wallet bookmark this before everyone copies the route map comments: what % of your tasks actually deserve astra

cryptopsihoz

44,187 Aufrufe • vor 9 Tagen

🚨 The Silvia team just announced our latest engineering advancement. Every business wants access to the highest level of intelligence, but at the lowest cost possible. The rise of LLMs has made intelligence abundant, yet one of the hardest problems across startups and corporate America is predicting the compute cost associated with this intelligence. I have been dealing with this personally as we build Silvia and the problem comes up in almost every conversation I have with CEOs, founders, and executives. Every business embraced AI about 18 months ago and things seemed great until the compute bills started to show up. The bills for internal compute usage were difficult to swallow, but things got outrageous if you had an AI product that allowed your users to consume compute without limits. I know this problem intimately because that is the situation that Silvia was in. Every question that was asked meant higher compute costs for our company. But we didn’t want to limit usage because users were getting genuine value out of the product. This challenge sent our team down a deep rabbit hole of cutting costs, while improving the experience for users. The second part was really important: we did not want to degrade the user experience by simply taking away access to the highest quality models. Thankfully, resource constraints breed innovation. We aren’t the biggest company, nor do we have the largest balance sheet, but we came up with a very novel solution that we are announcing today. The Silvia engineering team built a model router that cut costs by up to 29%, decreased latency, and improved the quality of answers for users. Trifecta! The way we do this is by reading the first 500 characters of a query and then predicting the level of effort that will be needed by a model to answer the query. The highest effort needs are routed to the most powerful models. The lowest effort needs are routed to different, better models for the query. A good example of this would be “what is the date?” You don’t need to use the latest Anthropic model to answer this query. In fact, sending a simple query like this to the most powerful model will make your compute costs increase and will actually increase the latency, which means a worse user experience for the Silvia user. By implementing the model router, the user gets a better experience and we get lower costs. Win-win. One of the interesting aspects of the implementation is that our model router runs on CPUs instead of GPUs. This allows us to read the query and predict the level of effort needed in less than 1 millisecond. This CPU implementation is why latency is not affected, nor is cost significantly increased by any potential additional GPU consumption. Another important point is that many of you have probably seen the news that OpenRouter is being purchased by Stripe for around $7 billion. This is a great outcome from what appears to be a very smart, capable team. Their model routing API is related (their product and our internal implementation both touch model routing), but you should think of OpenRouter as making it possible to do model routing for companies, while Silvia’s model router is a custom, intelligent system that specifically routes Silvia queries to the right model. They give access to the functionality of model routing to many companies, while our internal product does the real decision-making specific to our use case. Lastly, our implementation of a model router is a strategic bet that will allow us to become model-agnostic over time. We don’t care who created the different models, we just want to route a query to the model best positioned to answer. The large model labs will never allow their users to be model agnostic, but that would require the lab to potentially route a query to a competitor’s model. No bueno in their eyes. Instead, Silvia being an independent AI research lab gives us the power of being agnostic. We simply want the best experience for our users. Last week we announced that Silvia is now the most accurate AI tax product on the market, including beating OpenAI, Anthropic, Google, and xAI. Today we are announcing a custom, in-house model router that rivals the best technology anyone else has built. There will be many more engineering announcements to come. I truly believe we have assembled one of the best AI teams and we are currently the best AI research lab in finance. If you are interested in learning more about the technical details of the model router, you can read the engineering blog post here: Everyone wants the best intelligence and the lowest cost. Silvia just showed the world what is possible in this pursuit. I anticipate many other companies will build this custom solutions to achieve the same benefits.
2:50

🚨 The Silvia team just announced our latest engineering advancement. Every business wants access to the highest level of intelligence, but at the lowest cost possible. The rise of LLMs has made intelligence abundant, yet one of the hardest problems across startups and corporate America is predicting the compute cost associated with this intelligence. I have been dealing with this personally as we build Silvia and the problem comes up in almost every conversation I have with CEOs, founders, and executives. Every business embraced AI about 18 months ago and things seemed great until the compute bills started to show up. The bills for internal compute usage were difficult to swallow, but things got outrageous if you had an AI product that allowed your users to consume compute without limits. I know this problem intimately because that is the situation that Silvia was in. Every question that was asked meant higher compute costs for our company. But we didn’t want to limit usage because users were getting genuine value out of the product. This challenge sent our team down a deep rabbit hole of cutting costs, while improving the experience for users. The second part was really important: we did not want to degrade the user experience by simply taking away access to the highest quality models. Thankfully, resource constraints breed innovation. We aren’t the biggest company, nor do we have the largest balance sheet, but we came up with a very novel solution that we are announcing today. The Silvia engineering team built a model router that cut costs by up to 29%, decreased latency, and improved the quality of answers for users. Trifecta! The way we do this is by reading the first 500 characters of a query and then predicting the level of effort that will be needed by a model to answer the query. The highest effort needs are routed to the most powerful models. The lowest effort needs are routed to different, better models for the query. A good example of this would be “what is the date?” You don’t need to use the latest Anthropic model to answer this query. In fact, sending a simple query like this to the most powerful model will make your compute costs increase and will actually increase the latency, which means a worse user experience for the Silvia user. By implementing the model router, the user gets a better experience and we get lower costs. Win-win. One of the interesting aspects of the implementation is that our model router runs on CPUs instead of GPUs. This allows us to read the query and predict the level of effort needed in less than 1 millisecond. This CPU implementation is why latency is not affected, nor is cost significantly increased by any potential additional GPU consumption. Another important point is that many of you have probably seen the news that OpenRouter is being purchased by Stripe for around $7 billion. This is a great outcome from what appears to be a very smart, capable team. Their model routing API is related (their product and our internal implementation both touch model routing), but you should think of OpenRouter as making it possible to do model routing for companies, while Silvia’s model router is a custom, intelligent system that specifically routes Silvia queries to the right model. They give access to the functionality of model routing to many companies, while our internal product does the real decision-making specific to our use case. Lastly, our implementation of a model router is a strategic bet that will allow us to become model-agnostic over time. We don’t care who created the different models, we just want to route a query to the model best positioned to answer. The large model labs will never allow their users to be model agnostic, but that would require the lab to potentially route a query to a competitor’s model. No bueno in their eyes. Instead, Silvia being an independent AI research lab gives us the power of being agnostic. We simply want the best experience for our users. Last week we announced that Silvia is now the most accurate AI tax product on the market, including beating OpenAI, Anthropic, Google, and xAI. Today we are announcing a custom, in-house model router that rivals the best technology anyone else has built. There will be many more engineering announcements to come. I truly believe we have assembled one of the best AI teams and we are currently the best AI research lab in finance. If you are interested in learning more about the technical details of the model router, you can read the engineering blog post here: Everyone wants the best intelligence and the lowest cost. Silvia just showed the world what is possible in this pursuit. I anticipate many other companies will build this custom solutions to achieve the same benefits.

Anthony Pompliano 🌪

76,176 Aufrufe • vor 1 Monat

We stealth launched on Product Hunt a few weeks back and hit top 5 🔥 Now, we’re releasing it to the public, fully open-source! Use as a full template to build a production-quality RAG app 🛠️ It’s not any chat app, it contains a set of key features that took weeks/months of tuning to get right. And we did it so you don’t have to! ✅ Intermediate steps: get full transparency in how the LLM arrives at its answer. ✅ Granular highlights in sources: see *exactly* where the source context came from for a given response ✅ Full streaming support: everything from intermediate steps/responses to the final response is streamed. ✅ Ask a variety of complex questions: ask over a single doc or across multiple docs 📄 Tech Stack 📚: - 🤖 OpenAI (gpt-3.5-turbo) - 🏃‍♂️FastAPI + PostgreSQL - ▲ Next.js + Tailwind CSS - 🦙 and of course, LlamaIndex 🦙 FULL Repo here: use it as reference code or clone/fork it and use as you wish: And link to the secinsights app if you want to play around: HUGE shoutout to Sourabh Desai for driving this project 🙌 - it's been awesome to see it come to fruition.
0:59

We stealth launched on Product Hunt a few weeks back and hit top 5 🔥 Now, we’re releasing it to the public, fully open-source! Use as a full template to build a production-quality RAG app 🛠️ It’s not any chat app, it contains a set of key features that took weeks/months of tuning to get right. And we did it so you don’t have to! ✅ Intermediate steps: get full transparency in how the LLM arrives at its answer. ✅ Granular highlights in sources: see *exactly* where the source context came from for a given response ✅ Full streaming support: everything from intermediate steps/responses to the final response is streamed. ✅ Ask a variety of complex questions: ask over a single doc or across multiple docs 📄 Tech Stack 📚: - 🤖 OpenAI (gpt-3.5-turbo) - 🏃‍♂️FastAPI + PostgreSQL - ▲ Next.js + Tailwind CSS - 🦙 and of course, LlamaIndex 🦙 FULL Repo here: use it as reference code or clone/fork it and use as you wish: And link to the secinsights app if you want to play around: HUGE shoutout to Sourabh Desai for driving this project 🙌 - it's been awesome to see it come to fruition.

Jerry Liu

171,843 Aufrufe • vor 3 Jahren

I’m thrilled to announce that we just released GraspGen, a multi-year project we have been cooking at NVIDIA Robotics 🚀 GraspGen: A Diffusion-Based Framework for 6-DOF Grasping Grasping is a foundational challenge in robotics 🤖 — whether for industrial picking or general-purpose humanoids. VLA + real data collection is all the rage now but is expensive and scales poorly for this task. For every new gripper and/or scene, you’ll have to recollect the dataset in this paradigm for the best perf. 💡Key Idea: Since grasping is such a well-defined task in simulation - why can’t we just scale synthetic data generation and train a generative model for grasping? By embracing modularity and standardized grasp formats, we can make this a turnkey technology that works zero-shot for multiple settings. GraspGen is a modular framework for diffusion-based 6-DOF grasp generation that scales across embodiment types, observability conditions, clutter, task complexity. Key Features: ✅ Multi-embodiment support: suction, parallel-jaw, and multi-fingered grippers ✅ Generalization to partial + complete 3D point clouds ✅ Generalization to single-objects + cluttered scenes ✅ Modular design uses other robotics modules and foundation models (SAM2, cuRobo, FoundationStereo, FoundationPose). This allows GraspGen to focus on only one thing - grasp generation ✅ Training recipe: grasp discriminator is trained with On-Generator data from the diffusion model - so that it learns to correct the mistakes (if any) of the diffusion generator ✅ Real-time performance (~20 Hz) before any GPU acceleration; low memory footprint 📊 Results: • SOTA on the FetchBench [Han et al. CoRL 2024] benchmark • Zero-shot sim-to-real transfer on unknown objects and cluttered scenes • Dataset of 53M simulated grasps across 8K objects from Objaverse 📄 arXiv: 🌐 Website: 💻 Code: A huge thank you to everyone involved in this journey — excited to see what the community builds on top of it! Joint work with Clemens Eppner , Balakumar Sundaralingam , Yu-Wei, Jun Yamada Wentao Yuan and other collaborators #robotics #diffusionmodels #physicalAI #simtoreal
0:39

I’m thrilled to announce that we just released GraspGen, a multi-year project we have been cooking at NVIDIA Robotics 🚀 GraspGen: A Diffusion-Based Framework for 6-DOF Grasping Grasping is a foundational challenge in robotics 🤖 — whether for industrial picking or general-purpose humanoids. VLA + real data collection is all the rage now but is expensive and scales poorly for this task. For every new gripper and/or scene, you’ll have to recollect the dataset in this paradigm for the best perf. 💡Key Idea: Since grasping is such a well-defined task in simulation - why can’t we just scale synthetic data generation and train a generative model for grasping? By embracing modularity and standardized grasp formats, we can make this a turnkey technology that works zero-shot for multiple settings. GraspGen is a modular framework for diffusion-based 6-DOF grasp generation that scales across embodiment types, observability conditions, clutter, task complexity. Key Features: ✅ Multi-embodiment support: suction, parallel-jaw, and multi-fingered grippers ✅ Generalization to partial + complete 3D point clouds ✅ Generalization to single-objects + cluttered scenes ✅ Modular design uses other robotics modules and foundation models (SAM2, cuRobo, FoundationStereo, FoundationPose). This allows GraspGen to focus on only one thing - grasp generation ✅ Training recipe: grasp discriminator is trained with On-Generator data from the diffusion model - so that it learns to correct the mistakes (if any) of the diffusion generator ✅ Real-time performance (~20 Hz) before any GPU acceleration; low memory footprint 📊 Results: • SOTA on the FetchBench [Han et al. CoRL 2024] benchmark • Zero-shot sim-to-real transfer on unknown objects and cluttered scenes • Dataset of 53M simulated grasps across 8K objects from Objaverse 📄 arXiv: 🌐 Website: 💻 Code: A huge thank you to everyone involved in this journey — excited to see what the community builds on top of it! Joint work with Clemens Eppner , Balakumar Sundaralingam , Yu-Wei, Jun Yamada Wentao Yuan and other collaborators #robotics #diffusionmodels #physicalAI #simtoreal

Adithya Murali

24,309 Aufrufe • vor 1 Jahr

In two years, every new tech company will run on a CRM you can vibe code to fit your business. This CRM will not be built from scratch on a coding platform though. It will be built on top of managed infrastructure with complete data capture, indices designed for LLMs to understand the whole picture, clean APIs, curated UI frameworks designed for selling, enterprise-grade security, and come with 24/7 support. You’ll instruct the agent using natural language and it will write the code + run it for you. That’s what we’re building at Lightfield and today we’re announcing step two of our plan - code execution. You can now ask your agent to build programs, artifacts, and run complex analysis instantly. It does this by writing and running Python in a high performance sandbox using full customer memory — including every email, meeting, and note that Lightfield has captured — and reasoning across every relationship to deliver high quality work. Ask your agent to build a competitive battle card before a call tomorrow. It pulls positioning, objections, and win/loss patterns from real conversations. Ask it to flag every open deal where your champion's engagement has dropped or sentiment has shifted. It reads across every conversation and tells you where to focus. Ask it to build a pipeline review with charts and graphs for your board. It produces the whole thing in minutes. Here’s what we did with it this week: → We asked our agent to grade our sales team on discovery, rapport, and closing. It gave a structured scorecard with specific examples from real conversations. → Our GTM team asked the agent to build a plan to expand one of our enterprise customers. It pulled competitive threats, upsell paths, stakeholder mapping, and a phased execution plan — in minutes. → We used it to find every feature request from the last quarter that our engineering team has since shipped, and draft a personalized follow-up to each customer using their original words. It closed loops across dozens of accounts that would have taken days to track down manually This is the first step towards building any custom GTM workflow in natural language on top of what Lightfield knows about your business - a world model built from every single interaction your team has had with customers.
1:00

In two years, every new tech company will run on a CRM you can vibe code to fit your business. This CRM will not be built from scratch on a coding platform though. It will be built on top of managed infrastructure with complete data capture, indices designed for LLMs to understand the whole picture, clean APIs, curated UI frameworks designed for selling, enterprise-grade security, and come with 24/7 support. You’ll instruct the agent using natural language and it will write the code + run it for you. That’s what we’re building at Lightfield and today we’re announcing step two of our plan - code execution. You can now ask your agent to build programs, artifacts, and run complex analysis instantly. It does this by writing and running Python in a high performance sandbox using full customer memory — including every email, meeting, and note that Lightfield has captured — and reasoning across every relationship to deliver high quality work. Ask your agent to build a competitive battle card before a call tomorrow. It pulls positioning, objections, and win/loss patterns from real conversations. Ask it to flag every open deal where your champion's engagement has dropped or sentiment has shifted. It reads across every conversation and tells you where to focus. Ask it to build a pipeline review with charts and graphs for your board. It produces the whole thing in minutes. Here’s what we did with it this week: → We asked our agent to grade our sales team on discovery, rapport, and closing. It gave a structured scorecard with specific examples from real conversations. → Our GTM team asked the agent to build a plan to expand one of our enterprise customers. It pulled competitive threats, upsell paths, stakeholder mapping, and a phased execution plan — in minutes. → We used it to find every feature request from the last quarter that our engineering team has since shipped, and draft a personalized follow-up to each customer using their original words. It closed loops across dozens of accounts that would have taken days to track down manually This is the first step towards building any custom GTM workflow in natural language on top of what Lightfield knows about your business - a world model built from every single interaction your team has had with customers.

Keith Peiris

27,634 Aufrufe • vor 7 Monaten

We are going big this year and into next! 2026 will be a Radiant year. The Radiant community is thrilled to announce the release of TWO MAJOR open-source projects that showcase the power of collaborative development: Radiant-Core Node! RXinDexer! 1. Radiant-Core Node 2.0-Project Phoenix Our next-generation blockchain node implementation! 🔗 -Built with modern C++20, Radiant-Core represents a significant evolution in our blockchain infrastructure: -Enhanced Performance: Support for transactions up to 12 MB with ~81,000 inputs. -Developer-Friendly: Native Prometheus metrics endpoint for easy monitoring. -Flexible Configuration: Node profiles from the system design paper (archive/agent/mining) for different use cases. -Glyph Token Swap Protocol Upgrades: **On-chain** broadcasted atomic swaps via PSRT. The classic advertised swaps will still work but this ushers in a whole new era for DEX and atomic swaps on-chain. -Cross-Platform: Docker-based CI for consistent builds across Linux, macOS, and Windows. -Available soon on testnet and aiming for mainnet activation by block 400k. 2. RXinDexer - Blockchain indexer for RXD and Glyphs 🔗 -A powerful Python-based indexing solution that enables developers to efficiently query and track Radiant blockchain data and Glyph assets. These releases embody our commitment to transparent, community-driven development. Open source isn't just about code—it's about empowering developers worldwide to: ✅ Contribute improvements and innovations ✅ Audit security and verify integrity ✅ Build complementary tools and services ✅ Learn from production-grade blockchain implementations Get Involved! Whether you're a seasoned blockchain developer or just getting started, your contributions matter. Check out the repositories, review the documentation, submit issues, or contribute code. You are the Proof of Work! And together, we're building the future of decentralized digital value transfer and sound money with no trusted third parties. The way Satoshi intended. The future is Radiant!
2:41

We are going big this year and into next! 2026 will be a Radiant year. The Radiant community is thrilled to announce the release of TWO MAJOR open-source projects that showcase the power of collaborative development: Radiant-Core Node! RXinDexer! 1. Radiant-Core Node 2.0-Project Phoenix Our next-generation blockchain node implementation! 🔗 -Built with modern C++20, Radiant-Core represents a significant evolution in our blockchain infrastructure: -Enhanced Performance: Support for transactions up to 12 MB with ~81,000 inputs. -Developer-Friendly: Native Prometheus metrics endpoint for easy monitoring. -Flexible Configuration: Node profiles from the system design paper (archive/agent/mining) for different use cases. -Glyph Token Swap Protocol Upgrades: **On-chain** broadcasted atomic swaps via PSRT. The classic advertised swaps will still work but this ushers in a whole new era for DEX and atomic swaps on-chain. -Cross-Platform: Docker-based CI for consistent builds across Linux, macOS, and Windows. -Available soon on testnet and aiming for mainnet activation by block 400k. 2. RXinDexer - Blockchain indexer for RXD and Glyphs 🔗 -A powerful Python-based indexing solution that enables developers to efficiently query and track Radiant blockchain data and Glyph assets. These releases embody our commitment to transparent, community-driven development. Open source isn't just about code—it's about empowering developers worldwide to: ✅ Contribute improvements and innovations ✅ Audit security and verify integrity ✅ Build complementary tools and services ✅ Learn from production-grade blockchain implementations Get Involved! Whether you're a seasoned blockchain developer or just getting started, your contributions matter. Check out the repositories, review the documentation, submit issues, or contribute code. You are the Proof of Work! And together, we're building the future of decentralized digital value transfer and sound money with no trusted third parties. The way Satoshi intended. The future is Radiant!

Radiant Blockchain

41,416 Aufrufe • vor 8 Monaten

I have been testing DeepSeek-V4-Pro with the Pi coding agent. I am mindblown by how well it works out of the box. A few notes: I spent a few hours building an LLM wiki with an agent powered entirely by DeepSeek-V4-Pro on Fireworks inference. This is the first time I feel like there is an open-weight model that can reason at the level of Claude and Codex. And it does this in a cost-effective way with support for 1M context length. To be clear, I am using DeepSeek-V4-Pro inside of Pi without any special configuration. It works out of the box. It's exciting that there is a model that can just be plugged into a basic harness like Pi, and it just works. I've never seen that before. Most models require lots of configuration and setup. DeepSeek's DeepSeek-V4-Pro is clearly good at agentic coding (probably the best from the open-weight models), but the model is also great on knowledge-intensive tasks where reasoning matters. The agent pulled agentic engineering best practices from different company docs (Anthropic, OpenAI, Google, Stripe, Meta, Modal, DeepSeek, Mistral, Cohere), searched and digested Reddit and HN threads, summarized arxiv papers, and surfaced trending GitHub repos. Then it distilled everything into actionable tips across categories. I love the Wiki it built. The quality is really good. Here is a snapshot of what the wiki looks like: DeepSeek-V4-Pro handled the task without breaking stride. Multi-step research queries, code generation for scaffolding, context-heavy reasoning across disparate sources. For coding specifically, this is the first open-weight model that genuinely feels like a Codex or Claude Code experience. It compares in capability and actual multi-turn agentic work. What made the loop feel so responsive was Fireworks' inference speed (the fastest in the market) and the fact that they actually validate models at the systems level before shipping. No corrupted reasoning traces. Just fast, reliable iteration. The hybrid CSA and HCA attention design cuts KV cache to just 10% and inference FLOPs by nearly 4x at 1M-token context. This is what makes the agent loop actually fast and cheap enough to run in practice. For devs who've been watching open-weight models close the gap but haven't found one that actually delivers in practice, this is the closest I've seen. Try it here:
0:31

I have been testing DeepSeek-V4-Pro with the Pi coding agent. I am mindblown by how well it works out of the box. A few notes: I spent a few hours building an LLM wiki with an agent powered entirely by DeepSeek-V4-Pro on Fireworks inference. This is the first time I feel like there is an open-weight model that can reason at the level of Claude and Codex. And it does this in a cost-effective way with support for 1M context length. To be clear, I am using DeepSeek-V4-Pro inside of Pi without any special configuration. It works out of the box. It's exciting that there is a model that can just be plugged into a basic harness like Pi, and it just works. I've never seen that before. Most models require lots of configuration and setup. DeepSeek's DeepSeek-V4-Pro is clearly good at agentic coding (probably the best from the open-weight models), but the model is also great on knowledge-intensive tasks where reasoning matters. The agent pulled agentic engineering best practices from different company docs (Anthropic, OpenAI, Google, Stripe, Meta, Modal, DeepSeek, Mistral, Cohere), searched and digested Reddit and HN threads, summarized arxiv papers, and surfaced trending GitHub repos. Then it distilled everything into actionable tips across categories. I love the Wiki it built. The quality is really good. Here is a snapshot of what the wiki looks like: DeepSeek-V4-Pro handled the task without breaking stride. Multi-step research queries, code generation for scaffolding, context-heavy reasoning across disparate sources. For coding specifically, this is the first open-weight model that genuinely feels like a Codex or Claude Code experience. It compares in capability and actual multi-turn agentic work. What made the loop feel so responsive was Fireworks' inference speed (the fastest in the market) and the fact that they actually validate models at the systems level before shipping. No corrupted reasoning traces. Just fast, reliable iteration. The hybrid CSA and HCA attention design cuts KV cache to just 10% and inference FLOPs by nearly 4x at 1M-token context. This is what makes the agent loop actually fast and cheap enough to run in practice. For devs who've been watching open-weight models close the gap but haven't found one that actually delivers in practice, this is the closest I've seen. Try it here:

elvis

60,281 Aufrufe • vor 4 Monaten

Introducing LobeHub: Agent teammates that grow with you. LobeHub is the ultimate space for work and life: to find, build, and collaborate with agent teammates that grow with you. We’re building the world’s first and largest human–agent co-evolving network. Two years ago, we built LobeChat, an open-source interface for using different AI models. Today, LobeChat has 70k+ GitHub stars and serves 6M+ users worldwide. How to fully unlock the power of models has always been a shared mission between us and the community. We started with interaction — a fundamentally new, agent-first experience. Agents are no longer passive tools invoked in a single conversation. They should be proactive, always-on units of work. Treating agents as the minimal atomic unit is also the core of our agent harness infra. Today’s agents are mostly one-off executors. Even with memory, it’s often global — and hallucinates. We build long-term agent teammates that evolve with users. Each agent has its own dedicated memory space, editable by users, allowing humans and agents to co-evolve over time. This, in turn, allows us to design clearer rewards for reinforcement learning and create cleaner environments for continual learning. Agent teammates can work in groups. Through a multi-agent system, agent groups operate faster, more cost-effective, and go beyond what single-agent systems can achieve. For example, a single agent often requires heavy user involvement to proceed step by step, whereas LobeHub can execute the same work from a single instruction, with a supervisor orchestrating agents that run in parallel or debate to produce better results. We are building the collaboration network among agent teammates — and between humans and agent teammates as well. Ease of use matters. AI intelligence and shared human intelligence are equally important. With simple instructions and tool selection, you can effortlessly build and team up with agent coworkers to deliver complex, systematic work — even assembling a quant team to execute trades. Through the LobeHub community, anyone can discover, reuse, and remix agents and agent groups, customizing them to fit their own workflows, preferences, and needs. Last but not least, our vision started with LobeChat: multi-model support is the most efficient approach for users. We believe different models excel in different scenarios. By routing across multiple models, LobeHub improves cost efficiency and unlocks capabilities that a single-model setup cannot easily support.
2:41

Introducing LobeHub: Agent teammates that grow with you. LobeHub is the ultimate space for work and life: to find, build, and collaborate with agent teammates that grow with you. We’re building the world’s first and largest human–agent co-evolving network. Two years ago, we built LobeChat, an open-source interface for using different AI models. Today, LobeChat has 70k+ GitHub stars and serves 6M+ users worldwide. How to fully unlock the power of models has always been a shared mission between us and the community. We started with interaction — a fundamentally new, agent-first experience. Agents are no longer passive tools invoked in a single conversation. They should be proactive, always-on units of work. Treating agents as the minimal atomic unit is also the core of our agent harness infra. Today’s agents are mostly one-off executors. Even with memory, it’s often global — and hallucinates. We build long-term agent teammates that evolve with users. Each agent has its own dedicated memory space, editable by users, allowing humans and agents to co-evolve over time. This, in turn, allows us to design clearer rewards for reinforcement learning and create cleaner environments for continual learning. Agent teammates can work in groups. Through a multi-agent system, agent groups operate faster, more cost-effective, and go beyond what single-agent systems can achieve. For example, a single agent often requires heavy user involvement to proceed step by step, whereas LobeHub can execute the same work from a single instruction, with a supervisor orchestrating agents that run in parallel or debate to produce better results. We are building the collaboration network among agent teammates — and between humans and agent teammates as well. Ease of use matters. AI intelligence and shared human intelligence are equally important. With simple instructions and tool selection, you can effortlessly build and team up with agent coworkers to deliver complex, systematic work — even assembling a quant team to execute trades. Through the LobeHub community, anyone can discover, reuse, and remix agents and agent groups, customizing them to fit their own workflows, preferences, and needs. Last but not least, our vision started with LobeChat: multi-model support is the most efficient approach for users. We believe different models excel in different scenarios. By routing across multiple models, LobeHub improves cost efficiency and unlocks capabilities that a single-model setup cannot easily support.

LobeHub

185,401 Aufrufe • vor 7 Monaten

How to Trade Crypto Perpetuals on Grvt | Step-by-Step Tutorial 📲 Get Started Now: 🔗 🎯 Complete your KYC to unlock your welcome rewards 👥 Invite friends to earn even more through the GRVT Reward Portal 📅 Get ready for options, spot, and mobile app trading coming soon! In this hands-on tutorial, we walk you through exactly how to trade crypto perpetual contracts on GRVT, one of the most advanced hybrid exchanges in the market. You’ll learn everything from creating an account to executing your first leveraged trade — all while staying 100% in control of your funds. Whether you're a seasoned trader or just getting started, GRVT's Web2-friendly UX and Web3 security model makes trading seamless, self-custodial, and safe. 🧭 What You’ll Learn in This Tutorial: ✅ How to create your GRVT account and complete KYC ✅ Setting up the GRVT Wallet or connecting MetaMask ✅ Depositing USDT via Ethereum, Arbitrum, or on-ramp ✅ Navigating the Perpetuals dashboard ✅ Setting leverage, margin mode - isolated vs cross ✅ Placing long or short trades with live order book ✅ Monitoring your open position, PNL, and liquidation levels ✅ Withdrawing profits safely via smart contract-secured flows ✅ BONUS: How to earn GRVT points while you trade 💼 Why Grvt ? 🔐 Full self-custody — GRVT never holds your keys ⚡ Off-chain matching + on-chain settlement with zkSync 📉 Gasless trades with 10ms latency 📊 Advanced margin engine and role-based access 🎁 ZK tokens when you complete KYC 📈 $3.2B+ in liquidity at mainnet backed by top market makers DISCLAIMER: The content on our site is for informational purposes only. Readers, viewers and listeners should not construe any such information or other material as legal, tax, investment, financial, or other advice. Nothing contained on our site constitutes a solicitation, recommendation, endorsement, or offer by Decentralised News to buy or sell any securities or other financial instruments in any jurisdiction in which such solicitation or offer would be unlawful under the laws of such jurisdiction. #GRVT #PerpetualTrading #CryptoTutorial #DeFi #HowToTrade #CryptoPerps #HybridExchange #zkSync #Web3Wallet #OPUSDT #CryptoLeverage #Crypto2025 #SelfCustody #NotFinancialAdvice #DecentralisedNews #Blockchain #Metaverse #PlaytoEarn #CryptoInvestments #Bitcoin #BuyBitcoin #TokenizedAssets #DecentralisedFinance #Crypto #DigitalAssets #EarnCrypto #EarnBitcoin #Cryptocurrencies #DeFi #CryptoWallets #YieldFarming #Stocks #TokenizedStocks #CryptoNews #BitcoinNews #BestCryptoEducationChannel #CryptoEducation
6:16

How to Trade Crypto Perpetuals on Grvt | Step-by-Step Tutorial 📲 Get Started Now: 🔗 🎯 Complete your KYC to unlock your welcome rewards 👥 Invite friends to earn even more through the GRVT Reward Portal 📅 Get ready for options, spot, and mobile app trading coming soon! In this hands-on tutorial, we walk you through exactly how to trade crypto perpetual contracts on GRVT, one of the most advanced hybrid exchanges in the market. You’ll learn everything from creating an account to executing your first leveraged trade — all while staying 100% in control of your funds. Whether you're a seasoned trader or just getting started, GRVT's Web2-friendly UX and Web3 security model makes trading seamless, self-custodial, and safe. 🧭 What You’ll Learn in This Tutorial: ✅ How to create your GRVT account and complete KYC ✅ Setting up the GRVT Wallet or connecting MetaMask ✅ Depositing USDT via Ethereum, Arbitrum, or on-ramp ✅ Navigating the Perpetuals dashboard ✅ Setting leverage, margin mode - isolated vs cross ✅ Placing long or short trades with live order book ✅ Monitoring your open position, PNL, and liquidation levels ✅ Withdrawing profits safely via smart contract-secured flows ✅ BONUS: How to earn GRVT points while you trade 💼 Why Grvt ? 🔐 Full self-custody — GRVT never holds your keys ⚡ Off-chain matching + on-chain settlement with zkSync 📉 Gasless trades with 10ms latency 📊 Advanced margin engine and role-based access 🎁 ZK tokens when you complete KYC 📈 $3.2B+ in liquidity at mainnet backed by top market makers DISCLAIMER: The content on our site is for informational purposes only. Readers, viewers and listeners should not construe any such information or other material as legal, tax, investment, financial, or other advice. Nothing contained on our site constitutes a solicitation, recommendation, endorsement, or offer by Decentralised News to buy or sell any securities or other financial instruments in any jurisdiction in which such solicitation or offer would be unlawful under the laws of such jurisdiction. #GRVT #PerpetualTrading #CryptoTutorial #DeFi #HowToTrade #CryptoPerps #HybridExchange #zkSync #Web3Wallet #OPUSDT #CryptoLeverage #Crypto2025 #SelfCustody #NotFinancialAdvice #DecentralisedNews #Blockchain #Metaverse #PlaytoEarn #CryptoInvestments #Bitcoin #BuyBitcoin #TokenizedAssets #DecentralisedFinance #Crypto #DigitalAssets #EarnCrypto #EarnBitcoin #Cryptocurrencies #DeFi #CryptoWallets #YieldFarming #Stocks #TokenizedStocks #CryptoNews #BitcoinNews #BestCryptoEducationChannel #CryptoEducation

Decentralised News

27,152 Aufrufe • vor 1 Jahr

This will retire 90% of RAG systems with dignity (and a sad song playlist). Powered by DSPy: If you're still building "text in, text out" chatbots that only perform blind vector and text searches, you're not gonna make it! My team just dropped Elysia, and it's not just an incremental successor to Verba… It's a whole rethink of how we interact with our data using AI. 𝗪𝗵𝗮𝘁 𝗶𝘀 𝗘𝗹𝘆𝗶𝘀𝗮? An open-source platform for building agentic RAG architectures. It learns from your preferences, intelligently categorizes, labels, and searches through your data, and provides complete transparency into its decision-making process. The long & exciting feature list: • 𝗧𝗿𝗮𝗻𝘀𝗽𝗮𝗿𝗲𝗻𝘁 𝗗𝗲𝗰𝗶𝘀𝗶𝗼𝗻-𝗧𝗿𝗲𝗲 𝗔𝗴𝗲𝗻𝘁𝘀: Elysia’s core is a customizable decision tree, and it visualizes its entire reasoning process, showing you why it chooses a specific tool or path. It enables advanced error handling, self-healing from failed queries, and prevents infinite loops. You can also add custom tools and branches to build complex, state-aware workflows. • 𝗗𝗮𝘁𝗮 𝗔𝘄𝗮𝗿𝗲𝗻𝗲𝘀𝘀: Before it even attempts a query, Elysia performs a full analysis of your data collections. This eliminates the blind search problem plaguing most RAG systems and allows for far more complex and accurate query generation. • 𝗗𝘆𝗻𝗮𝗺𝗶𝗰 𝗗𝗮𝘁𝗮 𝗗𝗶𝘀𝗽𝗹𝗮𝘆𝘀: Your RAG pipeline shouldn't be limited to text, right? That’s why Elysia analyzes each query's results and chooses the best way to display them, from tables and charts to product cards and GitHub tickets. It also features a comprehensive data explorer with search, sorting, and filtering capabilities. • 𝗛𝘆𝗽𝗲𝗿-𝗣𝗲𝗿𝘀𝗼𝗻𝗮𝗹𝗶𝘇𝗮𝘁𝗶𝗼𝗻 𝘃𝗶𝗮 𝗙𝗲𝗲𝗱𝗯𝗮𝗰𝗸: It uses your positively-rated queries as few-shot examples to improve future responses. This allows you to use smaller, faster models that perform like larger ones over time, cutting costs without sacrificing quality for most use cases. • 𝗖𝗵𝘂𝗻𝗸-𝗢𝗻-𝗗𝗲𝗺𝗮𝗻𝗱: Elysia chunks documents at query time. It performs initial searches on document-level vectors and only chunks relevant documents on the fly, storing them in a parallel quantized collection with cross references for future use. 𝗧𝗵𝗲 𝗦𝘁𝗮𝗰𝗸 Elysia is built from scratch on Weaviate, using its native features like named vectors, a variety of search types, filters, cross references, quantization, etc. It uses DSPy for LLM interactions and is delivered as a production-ready application via FastAPI, serving a NextJS frontend as static HTML. Also available as a Python package via pip: 𝗽𝗶𝗽 𝗶𝗻𝘀𝘁𝗮𝗹𝗹 𝗲𝗹𝘆𝘀𝗶𝗮-𝗮𝗶 Type: 𝗲𝗹𝘆𝘀𝗶𝗮 𝘀𝘁𝗮𝗿𝘁 Connect your Weaviate cluster and go explore what’s possible.
0:35

This will retire 90% of RAG systems with dignity (and a sad song playlist). Powered by DSPy: If you're still building "text in, text out" chatbots that only perform blind vector and text searches, you're not gonna make it! My team just dropped Elysia, and it's not just an incremental successor to Verba… It's a whole rethink of how we interact with our data using AI. 𝗪𝗵𝗮𝘁 𝗶𝘀 𝗘𝗹𝘆𝗶𝘀𝗮? An open-source platform for building agentic RAG architectures. It learns from your preferences, intelligently categorizes, labels, and searches through your data, and provides complete transparency into its decision-making process. The long & exciting feature list: • 𝗧𝗿𝗮𝗻𝘀𝗽𝗮𝗿𝗲𝗻𝘁 𝗗𝗲𝗰𝗶𝘀𝗶𝗼𝗻-𝗧𝗿𝗲𝗲 𝗔𝗴𝗲𝗻𝘁𝘀: Elysia’s core is a customizable decision tree, and it visualizes its entire reasoning process, showing you why it chooses a specific tool or path. It enables advanced error handling, self-healing from failed queries, and prevents infinite loops. You can also add custom tools and branches to build complex, state-aware workflows. • 𝗗𝗮𝘁𝗮 𝗔𝘄𝗮𝗿𝗲𝗻𝗲𝘀𝘀: Before it even attempts a query, Elysia performs a full analysis of your data collections. This eliminates the blind search problem plaguing most RAG systems and allows for far more complex and accurate query generation. • 𝗗𝘆𝗻𝗮𝗺𝗶𝗰 𝗗𝗮𝘁𝗮 𝗗𝗶𝘀𝗽𝗹𝗮𝘆𝘀: Your RAG pipeline shouldn't be limited to text, right? That’s why Elysia analyzes each query's results and chooses the best way to display them, from tables and charts to product cards and GitHub tickets. It also features a comprehensive data explorer with search, sorting, and filtering capabilities. • 𝗛𝘆𝗽𝗲𝗿-𝗣𝗲𝗿𝘀𝗼𝗻𝗮𝗹𝗶𝘇𝗮𝘁𝗶𝗼𝗻 𝘃𝗶𝗮 𝗙𝗲𝗲𝗱𝗯𝗮𝗰𝗸: It uses your positively-rated queries as few-shot examples to improve future responses. This allows you to use smaller, faster models that perform like larger ones over time, cutting costs without sacrificing quality for most use cases. • 𝗖𝗵𝘂𝗻𝗸-𝗢𝗻-𝗗𝗲𝗺𝗮𝗻𝗱: Elysia chunks documents at query time. It performs initial searches on document-level vectors and only chunks relevant documents on the fly, storing them in a parallel quantized collection with cross references for future use. 𝗧𝗵𝗲 𝗦𝘁𝗮𝗰𝗸 Elysia is built from scratch on Weaviate, using its native features like named vectors, a variety of search types, filters, cross references, quantization, etc. It uses DSPy for LLM interactions and is delivered as a production-ready application via FastAPI, serving a NextJS frontend as static HTML. Also available as a Python package via pip: 𝗽𝗶𝗽 𝗶𝗻𝘀𝘁𝗮𝗹𝗹 𝗲𝗹𝘆𝘀𝗶𝗮-𝗮𝗶 Type: 𝗲𝗹𝘆𝘀𝗶𝗮 𝘀𝘁𝗮𝗿𝘁 Connect your Weaviate cluster and go explore what’s possible.

Philip Vollet

93,615 Aufrufe • vor 1 Jahr

The Cost of Intelligence is Heading to Zero | Hyperspace P2P Distributed Cache We present to you our breakthrough cross-domain work across AI, distributed systems, cryptography, game theory to solve the primary structural inefficiency at the heart of AI infrastructure: most inference is redundant. Google has reported that only 15% of daily searches are truly novel. The rest are repeats or close variants. LLM inference inherits this same power-law distribution. Enterprise chatbots see 70-80% of queries fall into a handful of intent categories. System prompts are identical across 100% of requests within an application. The KV attention state for "You are a helpful assistant" has been computed billions of times, on millions of GPUs, identically. And yet every AI lab, every startup, every self-hosted deployment - computes and caches these results independently. There is no shared layer. No global memory. Every provider pays the full compute cost for every query, even when the answer already exists somewhere in the network. This is the problem Hyperspace solves where distributed cache operates at three levels, each catching a different class of redundancy: 1. Response cache Same prompt, same model, same parameters - instant cached response from any node in the network. SHA-256 hash lookup via DHT, with cryptographic cache proofs linking every response to its original inference execution. No trust required. Fetchers re-announce as providers, so popular responses replicate naturally across more nodes. 2. KV prefix cache Same system prompt tokens - skip the most expensive part of inference entirely. Prefill (computing Key-Value attention states) is deterministic: same model plus same tokens always produces identical KV state. The network caches these states using erasure coding and distributes them via the routing network. New questions that share a common prefix resume generation from cached state instead of recomputing from scratch. 3. Routing to cached nodes Instead of transferring KV state across the network for every request, Hyperspace routes the request to the node that already has the state loaded in VRAM. The request goes to the cache, not the cache to the request. Together, these three layers mean that 70-90% of inference requests at network scale never require full GPU computation. This work doesn't exist in isolation. It builds on research from across the industry: SGLang's RadixAttention demonstrated that automatic prefix sharing can yield up to 5x speedup on structured LLM workloads. Moonshot AI's Mooncake built an entire KV-cache-centric disaggregated architecture for production serving at Kimi. Anthropic, OpenAI, and Google all launched prompt caching products in 2024 - priced at 50-90% discounts - because system prompt reuse is so pervasive that it changes the economics of inference. What all of these systems share is a common limitation: they operate within a single organization's infrastructure. SGLang caches prefixes within one server. Mooncake disaggregates KV cache within one datacenter. Anthropic's prompt caching works within one API provider's fleet. None of them can share cached state across organizational boundaries. Hyperspace removes this boundary. The cache is global. A response computed by a node in Tokyo is immediately available to a node in Berlin. A KV prefix state generated for Qwen-32B on one machine is verifiable and reusable by any other machine running the same model. The routing network provides the delivery guarantees, the erasure coding provides the redundancy, and the cache proofs provide the trust. What this means for the cost of intelligence Big AI labs scale linearly: twice the users means twice the GPU spend. Every query is a cost center. Their internal caching helps, but it's siloed - Lab A's cache can't serve Lab B's users, and neither can serve a self-hosted Llama deployment. Hyperspace scales sub-linearly. Every new node that joins the network adds to the global cache. Every inference result enriches the cache for all future requests. The cache hit rate rises with network size because query distributions follow a power law - the most common questions are asked exponentially more often than rare ones. The implication is simple: as the network grows, the effective cost per inference drops. Not linearly. Logarithmically. At 10 million nodes, we estimate 75-90% of all inference requests can be served from cache, eliminating 400,000+ MWh of energy consumption per year and avoiding over 200,000 tons of CO2 emissions. The first person to ask a question pays the compute cost. Everyone after them gets the answer for free, with cryptographic proof that it's authentic. Training is competitive. Inference is shared Open-weight models are converging on quality with closed models. Labs will continue to differentiate on training - data curation, architecture innovation, RLHF tuning. That's where the real intellectual property lives. But inference is a commodity. Two copies of Qwen-32B running the same prompt produce the same KV state and the same response, byte for byte, regardless of whose GPU runs the matrix multiplication. There is no moat in multiplying matrices. The moat is in training the weights. A global distributed cache makes this separation explicit. It doesn't matter who trained the model. Once the weights are open, the inference cost approaches zero at scale - because the network remembers every answer and can prove it's correct. No lab, no matter how well-funded, can match this. They cannot share caches across competitors. They scale linearly. The network scales logarithmically. The marginal cost of intelligence approaches zero. That's the endgame.
0:23

The Cost of Intelligence is Heading to Zero | Hyperspace P2P Distributed Cache We present to you our breakthrough cross-domain work across AI, distributed systems, cryptography, game theory to solve the primary structural inefficiency at the heart of AI infrastructure: most inference is redundant. Google has reported that only 15% of daily searches are truly novel. The rest are repeats or close variants. LLM inference inherits this same power-law distribution. Enterprise chatbots see 70-80% of queries fall into a handful of intent categories. System prompts are identical across 100% of requests within an application. The KV attention state for "You are a helpful assistant" has been computed billions of times, on millions of GPUs, identically. And yet every AI lab, every startup, every self-hosted deployment - computes and caches these results independently. There is no shared layer. No global memory. Every provider pays the full compute cost for every query, even when the answer already exists somewhere in the network. This is the problem Hyperspace solves where distributed cache operates at three levels, each catching a different class of redundancy: 1. Response cache Same prompt, same model, same parameters - instant cached response from any node in the network. SHA-256 hash lookup via DHT, with cryptographic cache proofs linking every response to its original inference execution. No trust required. Fetchers re-announce as providers, so popular responses replicate naturally across more nodes. 2. KV prefix cache Same system prompt tokens - skip the most expensive part of inference entirely. Prefill (computing Key-Value attention states) is deterministic: same model plus same tokens always produces identical KV state. The network caches these states using erasure coding and distributes them via the routing network. New questions that share a common prefix resume generation from cached state instead of recomputing from scratch. 3. Routing to cached nodes Instead of transferring KV state across the network for every request, Hyperspace routes the request to the node that already has the state loaded in VRAM. The request goes to the cache, not the cache to the request. Together, these three layers mean that 70-90% of inference requests at network scale never require full GPU computation. This work doesn't exist in isolation. It builds on research from across the industry: SGLang's RadixAttention demonstrated that automatic prefix sharing can yield up to 5x speedup on structured LLM workloads. Moonshot AI's Mooncake built an entire KV-cache-centric disaggregated architecture for production serving at Kimi. Anthropic, OpenAI, and Google all launched prompt caching products in 2024 - priced at 50-90% discounts - because system prompt reuse is so pervasive that it changes the economics of inference. What all of these systems share is a common limitation: they operate within a single organization's infrastructure. SGLang caches prefixes within one server. Mooncake disaggregates KV cache within one datacenter. Anthropic's prompt caching works within one API provider's fleet. None of them can share cached state across organizational boundaries. Hyperspace removes this boundary. The cache is global. A response computed by a node in Tokyo is immediately available to a node in Berlin. A KV prefix state generated for Qwen-32B on one machine is verifiable and reusable by any other machine running the same model. The routing network provides the delivery guarantees, the erasure coding provides the redundancy, and the cache proofs provide the trust. What this means for the cost of intelligence Big AI labs scale linearly: twice the users means twice the GPU spend. Every query is a cost center. Their internal caching helps, but it's siloed - Lab A's cache can't serve Lab B's users, and neither can serve a self-hosted Llama deployment. Hyperspace scales sub-linearly. Every new node that joins the network adds to the global cache. Every inference result enriches the cache for all future requests. The cache hit rate rises with network size because query distributions follow a power law - the most common questions are asked exponentially more often than rare ones. The implication is simple: as the network grows, the effective cost per inference drops. Not linearly. Logarithmically. At 10 million nodes, we estimate 75-90% of all inference requests can be served from cache, eliminating 400,000+ MWh of energy consumption per year and avoiding over 200,000 tons of CO2 emissions. The first person to ask a question pays the compute cost. Everyone after them gets the answer for free, with cryptographic proof that it's authentic. Training is competitive. Inference is shared Open-weight models are converging on quality with closed models. Labs will continue to differentiate on training - data curation, architecture innovation, RLHF tuning. That's where the real intellectual property lives. But inference is a commodity. Two copies of Qwen-32B running the same prompt produce the same KV state and the same response, byte for byte, regardless of whose GPU runs the matrix multiplication. There is no moat in multiplying matrices. The moat is in training the weights. A global distributed cache makes this separation explicit. It doesn't matter who trained the model. Once the weights are open, the inference cost approaches zero at scale - because the network remembers every answer and can prove it's correct. No lab, no matter how well-funded, can match this. They cannot share caches across competitors. They scale linearly. The network scales logarithmically. The marginal cost of intelligence approaches zero. That's the endgame.

Varun

37,555 Aufrufe • vor 6 Monaten

New video from Shreya Shankar on data processing with LLMs at scale, an underrated topic! Shreya starts with a real use case: public defenders analyzing case files for racial bias (4:08). Hundreds of pages per defendant. Court transcripts, police reports, news articles. Running GPT-5 on everything costs a fortune. Her solution: treat LLMs like database operators. Semantic Map, Filter, Reduce (9:18). Databricks, BigQuery, and Snowflake are already shipping this as "AI SQL." She discusses how starting at 12:51: a query optimizer for LLMs. Traditional databases rewrite queries for efficiency. Shreya does the same for LLM pipelines (semantic versions of split, map, reduce that are LLM specific, along with query decomposition). For example, trivial LLM calls are replaced with Python functions. These "rewrite directives" improve both cost AND accuracy. She also talks about a cost optimization technique: Task Cascades (30:00). Instead of running GPT-5 on every document, first ask cheap questions. "Is there any lower court mentioned?" If no, the document clearly doesn't overturn a lower court. There are many other routing questions you can ask to reduce the amount of text sent to the LLM. This requires careful optimization and tuning to get right. She explains how to do this in the video. She runs through a production example that achieved 86% cost reduction while retaining 90% accuracy. --- At 41:26, Shreya shifts to HCI. She built DocWrangler, an IDE for LLM pipelines. The design is based on "Three Gulfs" (44:35): 1. Comprehension: You don't know what's in your data 2. Specification: "Only prescription meds" is hard to operationalize 3. Generalization: A prompt that works on 10 examples fails at 10,000 Users invented "throwaway pipelines" just to explore their data before doing real analysis. Pipelines with no analytical purpose: "summarize these documents," "extract key ideas." Just ways to learn what's in their data before doing the real work. DocWrangler makes this a first-class feature. --- In the last bit of the video Shreya discusses why you can't know what "good" means until you see examples. In one study, a medical analyst extracted medications from doctor-patient transcripts. As they inspected outputs, they noticed every medication appeared with a dosage. They hadn't anticipated this. Now they wanted dosages too. They also saw Tylenol and ibuprofen appearing and realized: "Actually, I only want prescription medications." Shreya calls this "criteria drift." Your evaluation criteria evolve as you see more outputs. This matters because standard ML assumes fixed metrics: define them upfront, collect labels, measure. But with LLMs on fuzzy tasks, that assumption breaks. You discover what you actually want through the process of evaluating. If you don't account for criteria drift, you end up optimizing for a stale rubric. DocWrangler and EvalGen accommodate this by placing the human in the loop thoughtfully. Chapter timestamps: (4:08) - The problem: unstructured data at scale (9:18) - Semantic operators (Map, Filter, Reduce) (12:51) - Query optimization for LLMs (18:15) - Data decomposition (chunking) (30:00) - Task Cascades (86% cost reduction) (41:26) - DocWrangler IDE (44:35) - Three Gulfs framework (51:50) - Evaluation criteria drift More links in reply
1:14:16

New video from Shreya Shankar on data processing with LLMs at scale, an underrated topic! Shreya starts with a real use case: public defenders analyzing case files for racial bias (4:08). Hundreds of pages per defendant. Court transcripts, police reports, news articles. Running GPT-5 on everything costs a fortune. Her solution: treat LLMs like database operators. Semantic Map, Filter, Reduce (9:18). Databricks, BigQuery, and Snowflake are already shipping this as "AI SQL." She discusses how starting at 12:51: a query optimizer for LLMs. Traditional databases rewrite queries for efficiency. Shreya does the same for LLM pipelines (semantic versions of split, map, reduce that are LLM specific, along with query decomposition). For example, trivial LLM calls are replaced with Python functions. These "rewrite directives" improve both cost AND accuracy. She also talks about a cost optimization technique: Task Cascades (30:00). Instead of running GPT-5 on every document, first ask cheap questions. "Is there any lower court mentioned?" If no, the document clearly doesn't overturn a lower court. There are many other routing questions you can ask to reduce the amount of text sent to the LLM. This requires careful optimization and tuning to get right. She explains how to do this in the video. She runs through a production example that achieved 86% cost reduction while retaining 90% accuracy. --- At 41:26, Shreya shifts to HCI. She built DocWrangler, an IDE for LLM pipelines. The design is based on "Three Gulfs" (44:35): 1. Comprehension: You don't know what's in your data 2. Specification: "Only prescription meds" is hard to operationalize 3. Generalization: A prompt that works on 10 examples fails at 10,000 Users invented "throwaway pipelines" just to explore their data before doing real analysis. Pipelines with no analytical purpose: "summarize these documents," "extract key ideas." Just ways to learn what's in their data before doing the real work. DocWrangler makes this a first-class feature. --- In the last bit of the video Shreya discusses why you can't know what "good" means until you see examples. In one study, a medical analyst extracted medications from doctor-patient transcripts. As they inspected outputs, they noticed every medication appeared with a dosage. They hadn't anticipated this. Now they wanted dosages too. They also saw Tylenol and ibuprofen appearing and realized: "Actually, I only want prescription medications." Shreya calls this "criteria drift." Your evaluation criteria evolve as you see more outputs. This matters because standard ML assumes fixed metrics: define them upfront, collect labels, measure. But with LLMs on fuzzy tasks, that assumption breaks. You discover what you actually want through the process of evaluating. If you don't account for criteria drift, you end up optimizing for a stale rubric. DocWrangler and EvalGen accommodate this by placing the human in the loop thoughtfully. Chapter timestamps: (4:08) - The problem: unstructured data at scale (9:18) - Semantic operators (Map, Filter, Reduce) (12:51) - Query optimization for LLMs (18:15) - Data decomposition (chunking) (30:00) - Task Cascades (86% cost reduction) (41:26) - DocWrangler IDE (44:35) - Three Gulfs framework (51:50) - Evaluation criteria drift More links in reply

Hamel Husain

35,663 Aufrufe • vor 9 Monaten

What a year. 🚀 2025 was the year ChainOpera AI turned vision into real momentum: building a community-co-created, community-co-owned AI agent network and pushing the boundaries of what decentralized, collaborative intelligence can look like. 🚀 Biggest highlights from 2025 ✅- AI Terminal officially launched: We unveiled the ChainOpera AI Terminal as a unified gateway to decentralized AI, making it possible for anyone to interact with powerful, decentralized LLMs without technical friction. Positioned as the “browser for the DeAI era,” the AI Terminal marked a major step toward making decentralized intelligence accessible, usable, and mainstream. ✅- AI Terminal adoption at massive scale: Momentum followed quickly. The AI Terminal surpassed 2M registered users and consistently ranked top 3 among all apps on the BNB AI DappBay, validating strong product–market fit and real, sustained usage at scale. ✅- Announcing Coco: the world’s first community-owned Super Agent: We introduced Coco, the intelligence layer that sits between users and the agent network. Coco dynamically routes each request to the most efficient, community-built agent—optimizing for quality and speed while rewarding the creators behind the best-performing agents. This was a defining moment in realizing a truly community-owned intelligence layer. ✅- From agents to a living agent network: With the launch of the Agent Social Network and Super Agent architecture, ChainOpera AI moved beyond isolated agents toward a collaborative system where humans and specialized agents coordinate, share context, and solve complex, multi-step tasks together. ✅- $COAI breakout year: The listing of $COAI across major exchanges shocked the market, and throughout the year COAI consistently remained among the top AI-native crypto tokens by visibility, activity, and community engagement – reflecting growing confidence in the long-term vision of collaborative intelligence. ✅- Global presence: ChainOpera AI around-the-world tour: ChainOpera AI went global in 2025, sponsoring and participating in major AI and Web3 events across North America, Europe, and Asia, including ETHDenver, Consensus Toronto, Token2049 Singapore, ETHCC, SBC, and Devcon. These global touchpoints helped us engage directly with developers, builders, investors, and partners worldwide, accelerating adoption and positioning ChainOpera AI at the center of the emerging AIxBlockchain movement. ✅- Community momentum at scale: Community remained the heart of ChainOpera AI’s growth. We successfully completed three seasons of structured community engagement, executed a widely participated community airdrop, and ran multiple ecosystem-shaping campaigns to incentivize builders, creators, and early adopters. These efforts strengthened alignment between users, developers, and the protocol, laying the foundation for a durable, community-owned AI ecosystem. ✅- “AI for Markets” taking shape: We laid critical groundwork for AI-native market intelligence, including the launch of PrediMarket Agent and multiple trading and analysis agents—early building blocks toward an AI-driven ecosystem for crypto and DeFi markets. ✅- Building in public, with the community: Across product launches, research milestones, ecosystem discussions, and global events, we continued to build openly to bring developers, users, and partners directly into the evolution of ChainOpera AI. This year also marked the launch of the ChainOpera AI Foundation website, formally kicking off a bold Ecosystem Fund designed to empower builders, incubate high-impact projects, and accelerate the growth of a truly community-owned, collaborative AI ecosystem. To every builder, user, and supporter who helped make this year possible: THANK YOU! 🧭 What we’re excited about in the coming year 🔹- A Stronger, Denser Agent Economy (everyday adoption + cross-chain reach): In 2026, we are scaling the Agent Economy from growth to daily usage, with more agents, richer workflows, deeper multi-agent collaboration, and higher-impact use cases that users rely on every day. In parallel, we are expanding the agent network beyond a single ecosystem with cross-chain execution and interoperability, allowing agents to access the best liquidity, data, and opportunities wherever they exist. 🔹- AI Market Infrastructure Evolution: Building on PrediMarket Agent and our growing suite of trading and market-intelligence agents, we are advancing toward a mature AI market infrastructure, where agents continuously monitor, reason, simulate, optimize, and act across crypto, DeFi, and beyond. The goal is to make complex markets more accessible, more transparent, and more intelligence-driven, turning research, decision-making, and execution into a fast and reliable loop for everyday users. 🔹- Ecosystem Acceleration through the Foundation: With the ChainOpera AI Foundation and our Ecosystem Fund and Co-Creation Grants, we are doubling down on empowering independent builders to expand the protocol, the agent network, and the underlying infrastructure, so the community can co-create, co-own, and scale the ecosystem together. 🔹- Business Expansion and Market Penetration: In 2026, we will focus on expanding ChainOpera’s reach through strategic partnerships, product-led growth, and new paths to monetization, bringing AI agents to a broader global user base and driving sustained adoption, engagement, and revenue, while staying aligned with community ownership and an open ecosystem. 2025 was the proof. 2026 is where it compounds. 🔥 Co-Create. Co-Own. COAI.
0:22

What a year. 🚀 2025 was the year ChainOpera AI turned vision into real momentum: building a community-co-created, community-co-owned AI agent network and pushing the boundaries of what decentralized, collaborative intelligence can look like. 🚀 Biggest highlights from 2025 ✅- AI Terminal officially launched: We unveiled the ChainOpera AI Terminal as a unified gateway to decentralized AI, making it possible for anyone to interact with powerful, decentralized LLMs without technical friction. Positioned as the “browser for the DeAI era,” the AI Terminal marked a major step toward making decentralized intelligence accessible, usable, and mainstream. ✅- AI Terminal adoption at massive scale: Momentum followed quickly. The AI Terminal surpassed 2M registered users and consistently ranked top 3 among all apps on the BNB AI DappBay, validating strong product–market fit and real, sustained usage at scale. ✅- Announcing Coco: the world’s first community-owned Super Agent: We introduced Coco, the intelligence layer that sits between users and the agent network. Coco dynamically routes each request to the most efficient, community-built agent—optimizing for quality and speed while rewarding the creators behind the best-performing agents. This was a defining moment in realizing a truly community-owned intelligence layer. ✅- From agents to a living agent network: With the launch of the Agent Social Network and Super Agent architecture, ChainOpera AI moved beyond isolated agents toward a collaborative system where humans and specialized agents coordinate, share context, and solve complex, multi-step tasks together. ✅- $COAI breakout year: The listing of $COAI across major exchanges shocked the market, and throughout the year COAI consistently remained among the top AI-native crypto tokens by visibility, activity, and community engagement – reflecting growing confidence in the long-term vision of collaborative intelligence. ✅- Global presence: ChainOpera AI around-the-world tour: ChainOpera AI went global in 2025, sponsoring and participating in major AI and Web3 events across North America, Europe, and Asia, including ETHDenver, Consensus Toronto, Token2049 Singapore, ETHCC, SBC, and Devcon. These global touchpoints helped us engage directly with developers, builders, investors, and partners worldwide, accelerating adoption and positioning ChainOpera AI at the center of the emerging AIxBlockchain movement. ✅- Community momentum at scale: Community remained the heart of ChainOpera AI’s growth. We successfully completed three seasons of structured community engagement, executed a widely participated community airdrop, and ran multiple ecosystem-shaping campaigns to incentivize builders, creators, and early adopters. These efforts strengthened alignment between users, developers, and the protocol, laying the foundation for a durable, community-owned AI ecosystem. ✅- “AI for Markets” taking shape: We laid critical groundwork for AI-native market intelligence, including the launch of PrediMarket Agent and multiple trading and analysis agents—early building blocks toward an AI-driven ecosystem for crypto and DeFi markets. ✅- Building in public, with the community: Across product launches, research milestones, ecosystem discussions, and global events, we continued to build openly to bring developers, users, and partners directly into the evolution of ChainOpera AI. This year also marked the launch of the ChainOpera AI Foundation website, formally kicking off a bold Ecosystem Fund designed to empower builders, incubate high-impact projects, and accelerate the growth of a truly community-owned, collaborative AI ecosystem. To every builder, user, and supporter who helped make this year possible: THANK YOU! 🧭 What we’re excited about in the coming year 🔹- A Stronger, Denser Agent Economy (everyday adoption + cross-chain reach): In 2026, we are scaling the Agent Economy from growth to daily usage, with more agents, richer workflows, deeper multi-agent collaboration, and higher-impact use cases that users rely on every day. In parallel, we are expanding the agent network beyond a single ecosystem with cross-chain execution and interoperability, allowing agents to access the best liquidity, data, and opportunities wherever they exist. 🔹- AI Market Infrastructure Evolution: Building on PrediMarket Agent and our growing suite of trading and market-intelligence agents, we are advancing toward a mature AI market infrastructure, where agents continuously monitor, reason, simulate, optimize, and act across crypto, DeFi, and beyond. The goal is to make complex markets more accessible, more transparent, and more intelligence-driven, turning research, decision-making, and execution into a fast and reliable loop for everyday users. 🔹- Ecosystem Acceleration through the Foundation: With the ChainOpera AI Foundation and our Ecosystem Fund and Co-Creation Grants, we are doubling down on empowering independent builders to expand the protocol, the agent network, and the underlying infrastructure, so the community can co-create, co-own, and scale the ecosystem together. 🔹- Business Expansion and Market Penetration: In 2026, we will focus on expanding ChainOpera’s reach through strategic partnerships, product-led growth, and new paths to monetization, bringing AI agents to a broader global user base and driving sustained adoption, engagement, and revenue, while staying aligned with community ownership and an open ecosystem. 2025 was the proof. 2026 is where it compounds. 🔥 Co-Create. Co-Own. COAI.

ChainOpera AI

17,113 Aufrufe • vor 8 Monaten

This Chinese mathematician earned $10,000 a month inventing the hardest problems to train Neural Networks through Scale AI. Today his income dropped to zero. All the solutions are now generated by the model itself. He used to just hold the problem in his head and spell it out in plain text. His work is pure intellect. An expert in higher mathematics, he made his money hand-crafting the trickiest puzzles to test and train neural networks via RLHF. The bastion of "human" logic rested entirely on him, on people with PhDs who knew how to invent the problem. The collapse is simple. The shift to RLAIF and synthetic data. The model plays against itself, builds trees of logical inference, and solves deeper than a human can even invent the problem. No PhD data engineers, no hand-written prompt-completion examples, no manual grading. Just the model, search algorithms, and Chain of Thought. Ready-made "smart human-time" still sells on the market for many times more. His old rate was $50–100 per problem. The internal "mini-app" was written by the model too. Inside there's no pretty shell, just bare logic with exact steps: input: the problem statement inference tree: thousands of branches per second check: every step verifies itself output: a proof a human never had time to invent And here is what the whole setup looked like. He no longer needs to write an example by hand. He gave the model a direct instruction in human words, without a single formal term: "solve the problem yourself and grade yourself yourself" That's it. After that the algorithm found the solution, checked it, and trained on its own result, with no human. → the contractor got $50–100 per problem written → from 5,000 to 10,000 a month → now that income is annulled → a query to a math LLM costs 1–5 cents → a quant or an actuary runs 150,000–250,000 a year → the margin for whoever packages this into an agent is nearly 100% In the author's own words: "I'm no longer able to invent a problem the machine can't solve. The examiner became dumber than the one he's examining." But honestly, he admits the crude mistake himself, and it's not in the math, it's in the positioning. He tied his income to selling "smart human-time", to crafting formulas by hand. As long as he sells formulas, he's left behind. The machine computes faster than he can invent the problem. He names the right move himself: the role shifts from "intellectual craftsman" to "systems architect." Then he doesn't sell his time, he manages compute, packaging that same LLM into an autonomous agent that runs 24/7. Out of everything I've seen this year about the disappearance of intellectual professions, this is the most honest example: $50 per problem zeroed out to 1 cent per query, a doctor of science losing to a search algorithm, one problem stated in human words instead of a hand-written dataset, and right away an out-loud admission of the wrong business model. The barrier to entry in higher mathematics just dropped to the level of "describe the task in words." The only question is who'll be the first to stop selling their time and start managing the machine's compute.
3:12

This Chinese mathematician earned $10,000 a month inventing the hardest problems to train Neural Networks through Scale AI. Today his income dropped to zero. All the solutions are now generated by the model itself. He used to just hold the problem in his head and spell it out in plain text. His work is pure intellect. An expert in higher mathematics, he made his money hand-crafting the trickiest puzzles to test and train neural networks via RLHF. The bastion of "human" logic rested entirely on him, on people with PhDs who knew how to invent the problem. The collapse is simple. The shift to RLAIF and synthetic data. The model plays against itself, builds trees of logical inference, and solves deeper than a human can even invent the problem. No PhD data engineers, no hand-written prompt-completion examples, no manual grading. Just the model, search algorithms, and Chain of Thought. Ready-made "smart human-time" still sells on the market for many times more. His old rate was $50–100 per problem. The internal "mini-app" was written by the model too. Inside there's no pretty shell, just bare logic with exact steps: input: the problem statement inference tree: thousands of branches per second check: every step verifies itself output: a proof a human never had time to invent And here is what the whole setup looked like. He no longer needs to write an example by hand. He gave the model a direct instruction in human words, without a single formal term: "solve the problem yourself and grade yourself yourself" That's it. After that the algorithm found the solution, checked it, and trained on its own result, with no human. → the contractor got $50–100 per problem written → from 5,000 to 10,000 a month → now that income is annulled → a query to a math LLM costs 1–5 cents → a quant or an actuary runs 150,000–250,000 a year → the margin for whoever packages this into an agent is nearly 100% In the author's own words: "I'm no longer able to invent a problem the machine can't solve. The examiner became dumber than the one he's examining." But honestly, he admits the crude mistake himself, and it's not in the math, it's in the positioning. He tied his income to selling "smart human-time", to crafting formulas by hand. As long as he sells formulas, he's left behind. The machine computes faster than he can invent the problem. He names the right move himself: the role shifts from "intellectual craftsman" to "systems architect." Then he doesn't sell his time, he manages compute, packaging that same LLM into an autonomous agent that runs 24/7. Out of everything I've seen this year about the disappearance of intellectual professions, this is the most honest example: $50 per problem zeroed out to 1 cent per query, a doctor of science losing to a search algorithm, one problem stated in human words instead of a hand-written dataset, and right away an out-loud admission of the wrong business model. The barrier to entry in higher mathematics just dropped to the level of "describe the task in words." The only question is who'll be the first to stop selling their time and start managing the machine's compute.

Blaze

49,109 Aufrufe • vor 3 Monaten

🚀 Introducing PantheonOS ( A Fully Open-Source Agent OS for Science PantheonOS began as a research project in my Stanford lab and has since evolved into a vision to redefine data science in the era of AI—starting with computational biology, especially single-cell and spatial genomics. PantheonOS is a general agent platform built from the ground up. It is arguably the first distributed agent framework designed for scientific data analysis. 🔑 Key Features 1. Multi-Agent Collaboration – Built-in paradigms for distributed, cross-machine cooperation among agents and toolsets. 2. Native Toolset Support – Python, R, Julia, LaTeX, and more—designed for real scientific workflows. 3. Modular & Extensible – Developer-friendly design with shallow wrappers, plus LLM-driven toolset generation. 4. Evolvable Agents – Capable of evolving large-scale code projects to achieve superhuman performance (e.g., evolving upon the original Harmony [I Korsunsky, 2019, Nature Biotechnology] and Scanorama [BL Hie, 2019, Nature Biotechnology] implementations), and even evolving the system itself to adapt to new fields. 🎉 Stepwise Release Strategy We’re releasing PantheonOS in stages: Pantheon-CLI (today!), followed by Pantheon-Lab, Pantheon-Notebook, Pantheon-Slack, and more. 🌟 Pantheon-CLI Highlights - We're not just building another CLI tool. We're defining how scientists will interact with data in the AI era. - Open, Powerful, Python-First – The first fully open-source, endlessly extendable scientific “vibe analysis” framework. - Mixed Programming Magic – Combine Python, natural language, R, or Julia—seamlessly in the same environment. - PhD-Level Assistant – A command-line agent for complex real-world genomics and beyond, handling workflows at the PhD level. - Privacy by Design – Run entirely offline with local LLMs—your data never leaves your computer. ✅ Proven Applications (10 Demonstrations) Computational biology: 1. ATAC-seq: From raw reads to peak matrix 2. RNA-seq: From raw reads to expression matrix 3. Complex single-cell workflows (PhD-level) 4. Hybrid natural language + R for Seurat annotation 5. Learning from web tutorials + invoking single-cell foundation models 6. Cell segmentation on 10x Genomics HD Visium data And beyond: 7. Mixed Python & R programming examples 8. Molecular docking & structural analysis 9. Exploratory factor analysis for behavioral survey data 10. Customer segmentation & finance analytics 🌐 Learn More & Get Started Website: Pantheon-CLI Documentation: GitHub Repo: 💬 Join our community: PantheonOS Slack: PantheonOS Discord:
1:16

🚀 Introducing PantheonOS ( A Fully Open-Source Agent OS for Science PantheonOS began as a research project in my Stanford lab and has since evolved into a vision to redefine data science in the era of AI—starting with computational biology, especially single-cell and spatial genomics. PantheonOS is a general agent platform built from the ground up. It is arguably the first distributed agent framework designed for scientific data analysis. 🔑 Key Features 1. Multi-Agent Collaboration – Built-in paradigms for distributed, cross-machine cooperation among agents and toolsets. 2. Native Toolset Support – Python, R, Julia, LaTeX, and more—designed for real scientific workflows. 3. Modular & Extensible – Developer-friendly design with shallow wrappers, plus LLM-driven toolset generation. 4. Evolvable Agents – Capable of evolving large-scale code projects to achieve superhuman performance (e.g., evolving upon the original Harmony [I Korsunsky, 2019, Nature Biotechnology] and Scanorama [BL Hie, 2019, Nature Biotechnology] implementations), and even evolving the system itself to adapt to new fields. 🎉 Stepwise Release Strategy We’re releasing PantheonOS in stages: Pantheon-CLI (today!), followed by Pantheon-Lab, Pantheon-Notebook, Pantheon-Slack, and more. 🌟 Pantheon-CLI Highlights - We're not just building another CLI tool. We're defining how scientists will interact with data in the AI era. - Open, Powerful, Python-First – The first fully open-source, endlessly extendable scientific “vibe analysis” framework. - Mixed Programming Magic – Combine Python, natural language, R, or Julia—seamlessly in the same environment. - PhD-Level Assistant – A command-line agent for complex real-world genomics and beyond, handling workflows at the PhD level. - Privacy by Design – Run entirely offline with local LLMs—your data never leaves your computer. ✅ Proven Applications (10 Demonstrations) Computational biology: 1. ATAC-seq: From raw reads to peak matrix 2. RNA-seq: From raw reads to expression matrix 3. Complex single-cell workflows (PhD-level) 4. Hybrid natural language + R for Seurat annotation 5. Learning from web tutorials + invoking single-cell foundation models 6. Cell segmentation on 10x Genomics HD Visium data And beyond: 7. Mixed Python & R programming examples 8. Molecular docking & structural analysis 9. Exploratory factor analysis for behavioral survey data 10. Customer segmentation & finance analytics 🌐 Learn More & Get Started Website: Pantheon-CLI Documentation: GitHub Repo: 💬 Join our community: PantheonOS Slack: PantheonOS Discord:

evo-devo

17,439 Aufrufe • vor 1 Jahr

I know your timeline is flooded now with word salads of "insane, HER, 10 features you missed, we're so back". Sit down. Chill. Take a deep breath like Mark does in the demo . Let's think step by step: - Technique-wise, OpenAI has figured out a way to map audio to audio directly as first-class modality, and stream videos to a transformer in real-time. These require some new research on tokenization and architecture, but overall it's a data and system optimization problem (as most things are). High-quality data can come from at least 2 sources: 1) Naturally occurring dialogues on YouTube, podcasts, TV series, movies, etc. Whisper can be trained to identify speaker turns in a dialogue or separate overlapping speeches for automated annotation. 2) Synthetic data. Run the slow 3-stage pipeline using the most powerful models: speech1->text1 (ASR), text1->text2 (LLM), text2->speech2 (TTS). The middle LLM can decide when to stop and also simulate how to resume from interruption. It could output additional "thought traces" that are not verbalized to help generate better reply. Then GPT-4o distills directly from speech1->speech2, with optional auxiliary loss functions based on the 3-stage data. After distillation, these behaviors are now baked into the model without emitting intermediate texts. On the system side: the latency would not meet real-time threshold if every video frame is decompressed into an RGB image. OpenAI has likely developed their own neural-first, streaming video codec to transmit the motion deltas as tokens. The communication protocol and NN inference must be co-optimized. For example, there could be a small and energy-efficient NN running on the edge device that decides to transmit more tokens if the video is interesting, and fewer otherwise. - I didn't expect GPT-4o to be closer to GPT-5, the rumored "Arrakis" model that takes multimodal in and out. In fact, it's likely an early checkpoint of GPT-5 that hasn't finished training yet. The branding betrays a certain insecurity. Ahead of Google I/O, OpenAI would rather beat our mental projection of GPT-4.5 than disappoint by missing the sky-high expectation for GPT-5. A smart move to buy more time. - Notably, the assistant is much more lively and even a bit flirty. GPT-4o is trying (perhaps a bit too hard) to sound like HER. OpenAI is eating Character AI's lunch, with almost 100% overlap in form factor and huge distribution channels. It's a pivot towards more emotional AI with strong personality, which OpenAI seemed to actively suppress in the past. - Whoever wins Apple first wins big time. I see 3 levels of integration with iOS: 1) Ditch Siri. OpenAI distills a smaller-tier, purely on-device GPT-4o for iOS, with optional paid upgrade to use the cloud. 2) Native features to stream the camera or screen into the model. Chip-level support for neural audio/video codec. 3) Integrate with iOS system-level action API and smart home APIs. No one uses Siri Shortcuts, but it's time to resurrect. This could become the AI agent product with a billion users from the get-go. The FSD for smartphones with a Tesla-scale data flywheel.
14:22

I know your timeline is flooded now with word salads of "insane, HER, 10 features you missed, we're so back". Sit down. Chill. Take a deep breath like Mark does in the demo . Let's think step by step: - Technique-wise, OpenAI has figured out a way to map audio to audio directly as first-class modality, and stream videos to a transformer in real-time. These require some new research on tokenization and architecture, but overall it's a data and system optimization problem (as most things are). High-quality data can come from at least 2 sources: 1) Naturally occurring dialogues on YouTube, podcasts, TV series, movies, etc. Whisper can be trained to identify speaker turns in a dialogue or separate overlapping speeches for automated annotation. 2) Synthetic data. Run the slow 3-stage pipeline using the most powerful models: speech1->text1 (ASR), text1->text2 (LLM), text2->speech2 (TTS). The middle LLM can decide when to stop and also simulate how to resume from interruption. It could output additional "thought traces" that are not verbalized to help generate better reply. Then GPT-4o distills directly from speech1->speech2, with optional auxiliary loss functions based on the 3-stage data. After distillation, these behaviors are now baked into the model without emitting intermediate texts. On the system side: the latency would not meet real-time threshold if every video frame is decompressed into an RGB image. OpenAI has likely developed their own neural-first, streaming video codec to transmit the motion deltas as tokens. The communication protocol and NN inference must be co-optimized. For example, there could be a small and energy-efficient NN running on the edge device that decides to transmit more tokens if the video is interesting, and fewer otherwise. - I didn't expect GPT-4o to be closer to GPT-5, the rumored "Arrakis" model that takes multimodal in and out. In fact, it's likely an early checkpoint of GPT-5 that hasn't finished training yet. The branding betrays a certain insecurity. Ahead of Google I/O, OpenAI would rather beat our mental projection of GPT-4.5 than disappoint by missing the sky-high expectation for GPT-5. A smart move to buy more time. - Notably, the assistant is much more lively and even a bit flirty. GPT-4o is trying (perhaps a bit too hard) to sound like HER. OpenAI is eating Character AI's lunch, with almost 100% overlap in form factor and huge distribution channels. It's a pivot towards more emotional AI with strong personality, which OpenAI seemed to actively suppress in the past. - Whoever wins Apple first wins big time. I see 3 levels of integration with iOS: 1) Ditch Siri. OpenAI distills a smaller-tier, purely on-device GPT-4o for iOS, with optional paid upgrade to use the cloud. 2) Native features to stream the camera or screen into the model. Chip-level support for neural audio/video codec. 3) Integrate with iOS system-level action API and smart home APIs. No one uses Siri Shortcuts, but it's time to resurrect. This could become the AI agent product with a billion users from the get-go. The FSD for smartphones with a Tesla-scale data flywheel.

Jim Fan

991,964 Aufrufe • vor 2 Jahren