正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

Introducing Jan-v1: 4B model for web search, an open-source alternative to Perplexity Pro. In our evals, Jan v1 delivers 91% SimpleQA accuracy, slightly outperforming Perplexity Pro while running fully locally. Use cases: - Web search - Deep Research Built on the new version of Qwen's Qwen3-4B-Thinking (up to 256k... show more

👋 Jan

13,087 subscribers

691,054 次观看 • 1 年前 •via X (Twitter)

科学技术新闻政治教育

Anya Rossi• Live Now

Private livecam show

56 条评论

Can. 的头像

Can.1 年前

@ollama see how easy it is to give credit to @ggerganov and llama.cpp?

Frits Karl 的头像

Frits Karl1 年前

Not working: "Settings → Experimental Features → On, then Settings → MCP Servers → enable a search-related MCP such as Serper."

👋 Jan 的头像

👋 Jan1 年前

It's likely the model doesn’t have permission to use the MCP. - Go to Settings-> Model Providers -> llama.cpp - Find Jan v1 and click the pencil icon - In the settings panel, turn on Tools under Capabilities See the video.

Light 🧩🦅❤️ 的头像

Light 🧩🦅❤️1 年前

@FritsKarl My app do not have this @jandotai

👋 Jan 的头像

👋 Jan1 年前

@Bellaminiop @FritsKarl oh, the model should have permission to use the MCP. Please see how:

Light 🧩🦅❤️ 的头像

Light 🧩🦅❤️1 年前

@FritsKarl I can't click "Tool", my laptop is macbook. I just install Jan AI today

👋 Jan 的头像

👋 Jan1 年前

@Bellaminiop @FritsKarl Oh, how come? Are you on an Intel MacBook? Quick note, you can update the model permissions in Settings -> Model Providers -> llama.cpp

Light 🧩🦅❤️ 的头像

Light 🧩🦅❤️1 年前

@FritsKarl My macbook use chip M series. I turn on Tools but its didn't work

👋 Jan 的头像

👋 Jan1 年前

You should see a gear icon next to the model name in your chats once Tools are activated and enabled. Please watch this video and make sure your Jan is set up this way.

Alex Volkov 的头像

Alex Volkov1 年前

Congrats on the impressive score! 🔥 We'll cover this on this weeks @thursdai_pod , would you guys like to come on and tell us about this model ?

👋 Jan 的头像

👋 Jan1 年前

@thursdai_pod Thanks a lot! We'd be happy to join. Jan v1 co-author @alandao_ai will be around to talk about the model.

Alan 的头像

Alan1 年前

@altryne @thursdai_pod Cool, see you guys soon!

Alex Volkov 的头像

Alex Volkov1 年前

@jandotai @thursdai_pod awesome! can you follow me so I can send a DM with details?

TimRambo🐧 的头像

TimRambo🐧1 年前

This is so awesome! Finally, Open-Source is finally fighting back, and I know Open-Source will crush everything in its path! 👊😎

Vashisht 的头像

Vashisht1 年前

Is this available in ollama

👋 Jan 的头像

👋 Jan1 年前

Hey @ollama, could you give us a hand?

Melvin Vivas 的头像

Melvin Vivas1 年前

wow, should integrate this with @aibackends. is it available in @openrouter?

Meet Patel 的头像

Meet Patel1 年前

What does it use for doing actual web search? Like Brave, Google or anything like that?

👋 Jan 的头像

👋 Jan1 年前

You can run it with any Search MCP, such as @serperapi or @ExaAILabs.

Hugo Duprez 的头像

Hugo Duprez1 年前

@ggerganov Awesome, local AI ftw!

Team Reagent 的头像

Team Reagent1 年前

We support this! Locally running AI agents is more private and secure than whatever Perplexity lies about to their audience. We'll be testing with this and maybe providing some better results as we fine tune it ourselves in our Tetra agent!

Kristian Shultz-Mayer 的头像

Kristian Shultz-Mayer1 年前

Could you please tell me when you will launch a website for accessing the model?

👋 Jan 的头像

👋 Jan1 年前

Jan Web is on the way 🙌

Kris Krakowiak 的头像

Kris Krakowiak1 年前

Does it support MLX?

Shojaei 的头像

Shojaei1 年前

this is awesome 🔥

Singularity Age 的头像

Singularity Age1 年前

Nice. Is it possible to use your own Internet connection for search and webpage scraping?

Alex 的头像

Alex1 年前

@huggingface 91% accuracy? cute. my basement AI gets 420% by injecting memes directly into its training data. open source is based tho, now we can finally teach these models proper shitposting instead of that corporate sanitized nonsense. bullish on chaos 📈🤖

Le Techno-archéologue 的头像

Le Techno-archéologue1 年前

If this is true... The Holy toaster will be angry 🍞⚡

The Mandorlarian 的头像

The Mandorlarian1 年前

Fkn unreal, 91% 🤩🔥? You guys keep delivering! Is this you as well, @alandao_ai ?

Aon 的头像

Aon1 年前

Congratulations 🎈🍾

Himanshu Kumar 的头像

Himanshu Kumar1 年前

Open source alternatives are truly democratizing AI. This could significantly lower the barrier to entry for many. Exciting to see such progress.

Mo. | مُحَمَّدْ 的头像

Mo. | مُحَمَّدْ1 年前

I love you guys 😍

Quantyverse 的头像

Quantyverse1 年前

That is impressive! I am going to try it out. The accuracy for such a small model is awesome!

Josef Moucachen 的头像

Josef Moucachen1 年前

wow this is amazing

Alan 的头像

Alan1 年前

It's amazing to see such positive response from the community for the model. Thank you guys.

Front_Girl001 的头像

Front_Girl0011 年前

@AskPerplexity what do you think? Compare your pro model performance against this and tell the results. Are you cooked?

rahul 的头像

rahul1 年前

Hey team @jandotai , Any alternative to serper? Also this helped to setup the tools 👇 Thanks :) Love the work!!!

👋 Jan 的头像

👋 Jan1 年前

You're welcome 🙌 Jan supports MCP servers as an experimental feature, so you can add any search MCP to use with Jan v1.

rahul 的头像

rahul1 年前

Ah, That makes sense.

Pranav 的头像

Pranav1 年前

Any benchmarks on response speed?

Anderson452 的头像

Anderson4521 年前

Besides those options, is there anything else I need to configure for it to work? I'm new to MCP.

👋 Jan 的头像

👋 Jan1 年前

We're also experimenting with MCPs. - Go to Settings & turn on Experimental Features - Under MCP Servers in Settings, enable Serper & add your API key, or choose a different search MCP - Go to Model Providers -> llama.cpp, find Jan v1 turn on Tools

M Maarouf 的头像

M Maarouf1 年前

@MrAnderson452 Followed all the instructions and the wrench did show up, however, it fails to search online and in the output always says it got 403 errors. Do I need to get serper api key or something for it to work?

👋 Jan 的头像

👋 Jan1 年前

@MrAnderson452 Yes, you'll need to add a search MCP (e.g. @serperapi) for online search to work.

Sports Culture 的头像

Sports Culture1 年前

Very cool!

dhiyaan 的头像

dhiyaan1 年前

wow!

somecontextsuburbia 的头像

somecontextsuburbia1 年前

Great work folks! Very happy with your progress 🙌

Andres Franco 的头像

Andres Franco1 年前

Seems like a pretty interesting tool.

KI-CREW 的头像

KI-CREW1 年前

nice thank you

Hitarth Sharma 的头像

Hitarth Sharma1 年前

@grok what exactly does Jan v1 do if the actual search is being done by serper or exa?

JM 的头像

JM1 年前

I am trying to activate mcp it does not work.

👋 Jan 的头像

👋 Jan1 年前

Please follow the steps here:

JM 的头像

JM1 年前

got it, it works!

👋 Jan 的头像

👋 Jan1 年前

Awesome!

Gagan Ghotra 的头像

Gagan Ghotra1 年前

@zaidmukaddam 👀👀

omensec🇮🇳 的头像

omensec🇮🇳1 年前

Cool!!

相关视频

Jan v1 is now more reliable when handling searches. It's an open-source alternative to Perplexity Pro, and it now avoids infinite loops with Jan v1 2509. Performance updates: - Gains on reasoning & creativity benchmarks - Small drop on SimpleQA (91.1 -> 90.7), still competitive with Perplexity Pro The updated version is live on Jan Hub. You can run the model in Jan, llama.cpp, or vLLM. Use the model: - Model: - GGUF: Shoutout to the Qwen team for Qwen3-4B Thinking and Georgi Gerganov for llama.cpp.

Jan v1 is now more reliable when handling searches. It's an open-source alternative to Perplexity Pro, and it now avoids infinite loops with Jan v1 2509. Performance updates: - Gains on reasoning & creativity benchmarks - Small drop on SimpleQA (91.1 -> 90.7), still competitive with Perplexity Pro The updated version is live on Jan Hub. You can run the model in Jan, llama.cpp, or vLLM. Use the model: - Model: - GGUF: Shoutout to the Qwen team for Qwen3-4B Thinking and Georgi Gerganov for llama.cpp.

👋 Jan

17,759 次观看 • 1 年前

Meet Jan-nano, a 4B model that outscores DeepSeek-v3-671B using MCP. It's built on Qwen3-4B with DAPO fine-tuning, it handles: - real-time web search - deep research Model + GGUF: To run it locally: - Install Jan Beta: - Download Jan-nano in Jan Hub - Settings -> MCP, enable MCP and add your Serper API key for web tools Full technical report will be published shortly.

Meet Jan-nano, a 4B model that outscores DeepSeek-v3-671B using MCP. It's built on Qwen3-4B with DAPO fine-tuning, it handles: - real-time web search - deep research Model + GGUF: To run it locally: - Install Jan Beta: - Download Jan-nano in Jan Hub - Settings -> MCP, enable MCP and add your Serper API key for web tools Full technical report will be published shortly.

Menlo Research

56,600 次观看 • 1 年前

Introducing Jan-v2-VL, a multimodal agent built for long-horizon tasks. Jan-v2-VL executes 49 steps without failure, while the base model stops at 5 and other similar-scale VLMs stop between 1 and 2. It achieves longer, stable task execution in your browser without accuracy loss. 3 variants are available: - Jan-v2-VL-low (efficiency-oriented) - Jan-v2-VL-med (balanced) - Jan-v2-VL-high (deeper reasoning and longer execution) Models: To use it, update your Jan App and download Jan-v2-VL from the Model Hub. Activate Browser MCP servers for agentic use cases. Credit to the Qwen team for Qwen3-VL-8B-Thinking base model.

Introducing Jan-v2-VL, a multimodal agent built for long-horizon tasks. Jan-v2-VL executes 49 steps without failure, while the base model stops at 5 and other similar-scale VLMs stop between 1 and 2. It achieves longer, stable task execution in your browser without accuracy loss. 3 variants are available: - Jan-v2-VL-low (efficiency-oriented) - Jan-v2-VL-med (balanced) - Jan-v2-VL-high (deeper reasoning and longer execution) Models: To use it, update your Jan App and download Jan-v2-VL from the Model Hub. Activate Browser MCP servers for agentic use cases. Credit to the Qwen team for Qwen3-VL-8B-Thinking base model.

👋 Jan

130,375 次观看 • 10 个月前

Introducing a brand new way to use Jan 💛 You can now search the web, do deep research, and let Jan use your browser. Powered by Jan-v2-VL-Max, 30B model beating Gemini 2.5 Pro & DeepSeek R1 on execution benchmarks. Try it:

👋 Jan

29,403 次观看 • 8 个月前

We've added a new "reasoning" focus (beta) on Perplexity for Pro users. It will use the new OpenAI o1-mini. There is no search integration yet. The model is slow, and usage is limited because of rate limits. It is good for puzzles, math, and coding. Pro searches are your best way to use multi-step reasoning + web search.

We've added a new "reasoning" focus (beta) on Perplexity for Pro users. It will use the new OpenAI o1-mini. There is no search integration yet. The model is slow, and usage is limited because of rate limits. It is good for puzzles, math, and coding. Pro searches are your best way to use multi-step reasoning + web search.

Aravind Srinivas

288,808 次观看 • 2 年前

BrowserOS (Browser OS) is an open-source, privacy-first alternative to ChatGPT Atlas & Perplexity Comet. No vendor lock-in: use any LLM or search engine AI agents run locally. No tracking you for ads or data collection. Available for Mac/Win/Linux. Congrats on the launch, Nikhil Sonti, Nithin!

BrowserOS (Browser OS) is an open-source, privacy-first alternative to ChatGPT Atlas & Perplexity Comet. No vendor lock-in: use any LLM or search engine AI agents run locally. No tracking you for ads or data collection. Available for Mac/Win/Linux. Congrats on the launch, Nikhil Sonti, Nithin!

Y Combinator

162,089 次观看 • 10 个月前

Introducing "gap analysis"—new addition to scira's "extreme" mode! how it works—after the research plan is executed—the LLM will run a gap analysis to find the gaps in the results of the research plan and then search for the gaps and fill the information—and finally run a final synthesis or the information! other fixes: - you can now switch llms in-between conversations - you will have a dedicated regenerate button to re run the search or dedicated action by the same model - improved web search UI

Introducing "gap analysis"—new addition to scira's "extreme" mode! how it works—after the research plan is executed—the LLM will run a gap analysis to find the gaps in the results of the research plan and then search for the gaps and fill the information—and finally run a final synthesis or the information! other fixes: - you can now switch llms in-between conversations - you will have a dedicated regenerate button to re run the search or dedicated action by the same model - improved web search UI

Scira AI

15,643 次观看 • 1 年前

You don't need to be a command-line expert to run local AI. wraps the heavy lifting of llama.cpp into a clean UI, making it simple to deploy and manage models like Gemma 4. Instead of dealing with config files, you get one-click model downloads, clear memory estimates, and no coding required! Watch the video to see Gemma 4 in action as it parses tables from receipts, streams reasoning logs, and connects to MCP for web search.

You don't need to be a command-line expert to run local AI. wraps the heavy lifting of llama.cpp into a clean UI, making it simple to deploy and manage models like Gemma 4. Instead of dealing with config files, you get one-click model downloads, clear memory estimates, and no coding required! Watch the video to see Gemma 4 in action as it parses tables from receipts, streams reasoning logs, and connects to MCP for web search.

Google Gemma

59,024 次观看 • 3 天前

🚀 Introducing AgentCPM-Explore: The First Open-Source 4B-Agent Model to Conquer GAIA & Complex Real-World Tasks! 🤗 Hugging Face: 🔗 GitHub: ✨ Key Highlights: ✅ SOTA Agentic Performance: Sets a new benchmark for 4B-scale agent models—outperforming all peers, surpassing 8B models, and rivaling select 30B+ and closed-source LLMs. 🧠 Deep Research Capability: Excels at long-horizon reasoning, supports 100+ turns of autonomous interaction with multi-source cross-validation, human-like self-correction, and dynamic tool use + strategy adaptation—just like a real researcher! 🔓 Full-Stack Open Source: We’re open-sourcing the entire end-to-end agent stack—not just the model! Empower your own innovations with - AgentRL: Asynchronous reinforcement learning framework - AgentDock: Secure, extensible tool sandbox - AgentToLeaP: An one-click evaluation platform for agent tool-learning capabilitie - Full training data pipeline & reproducible workflows #AgentCPM #OpenSourceAI #AgenticAI #AI #GAIA #LLM #OpenBMB #AIAgents #HuggingFace

🚀 Introducing AgentCPM-Explore: The First Open-Source 4B-Agent Model to Conquer GAIA & Complex Real-World Tasks! 🤗 Hugging Face: 🔗 GitHub: ✨ Key Highlights: ✅ SOTA Agentic Performance: Sets a new benchmark for 4B-scale agent models—outperforming all peers, surpassing 8B models, and rivaling select 30B+ and closed-source LLMs. 🧠 Deep Research Capability: Excels at long-horizon reasoning, supports 100+ turns of autonomous interaction with multi-source cross-validation, human-like self-correction, and dynamic tool use + strategy adaptation—just like a real researcher! 🔓 Full-Stack Open Source: We’re open-sourcing the entire end-to-end agent stack—not just the model! Empower your own innovations with - AgentRL: Asynchronous reinforcement learning framework - AgentDock: Secure, extensible tool sandbox - AgentToLeaP: An one-click evaluation platform for agent tool-learning capabilitie - Full training data pipeline & reproducible workflows #AgentCPM #OpenSourceAI #AgenticAI #AI #GAIA #LLM #OpenBMB #AIAgents #HuggingFace

OpenBMB

13,996 次观看 • 8 个月前

Open-sourcing Introspect: MIT-licensed Deep-Research for your internal data! Works with spreadsheets, databases, PDFs, and web search. Has a remarkably simple architecture – Sonnet agent armed with recursive tool calling and 3 default tools. Best for use-cases where you want to combine insights from SQL with unstructured data + data from the web. Launching a bunch of products built on top of of this soon, but wanted to open-source our core engine to give back to the community! 2 min demo below, links in comments

Open-sourcing Introspect: MIT-licensed Deep-Research for your internal data! Works with spreadsheets, databases, PDFs, and web search. Has a remarkably simple architecture – Sonnet agent armed with recursive tool calling and 3 default tools. Best for use-cases where you want to combine insights from SQL with unstructured data + data from the web. Launching a bunch of products built on top of of this soon, but wanted to open-source our core engine to give back to the community! 2 min demo below, links in comments

Rishabh Srivastava

71,578 次观看 • 1 年前

OptimAI Search MCP brings live context to the agents people already use. Claude. Cursor. Copilot. Codex. One integration layer. Everywhere you build. Under the hood, OptimAI can process hundreds to thousands of real-time signals in parallel across the public web, social platforms, and onchain data - all powered by a decentralized network. That is the difference: not another search tool, but infrastructure for agent-native intelligence. Build with it. 💡 -- OptimAI: the #1 AI on BNB Chain DappBay.

OptimAI Search MCP brings live context to the agents people already use. Claude. Cursor. Copilot. Codex. One integration layer. Everywhere you build. Under the hood, OptimAI can process hundreds to thousands of real-time signals in parallel across the public web, social platforms, and onchain data - all powered by a decentralized network. That is the difference: not another search tool, but infrastructure for agent-native intelligence. Build with it. 💡 -- OptimAI: the #1 AI on BNB Chain DappBay.

OptimAI Network

58,755 次观看 • 3 个月前

Introducing Prompt-to-leaderboard (P2L): a real-time LLM leaderboard tailored exactly to your use case! P2L trains an LLM to generate "prompt-specific" leaderboards, so you can input a prompt and get a leaderboard specifically for that prompt. The model is trained on the 2M human preference votes from Chatbot Arena. P2L Highlights: 🔹Instant leaderboard for any prompt 🗿 🔹Optimal model routing (hit #1 on Chatbot Arena in Jan 2025 with 1395 score 🧏) 🔹Fine-grained model strength & weakness analysis 🤓 Check out our demo and thread below for more details!

Introducing Prompt-to-leaderboard (P2L): a real-time LLM leaderboard tailored exactly to your use case! P2L trains an LLM to generate "prompt-specific" leaderboards, so you can input a prompt and get a leaderboard specifically for that prompt. The model is trained on the 2M human preference votes from Chatbot Arena. P2L Highlights: 🔹Instant leaderboard for any prompt 🗿 🔹Optimal model routing (hit #1 on Chatbot Arena in Jan 2025 with 1395 score 🧏) 🔹Fine-grained model strength & weakness analysis 🤓 Check out our demo and thread below for more details!

Arena.ai

122,525 次观看 • 1 年前

MCP is an absolute game-changer. (Together with DeepSeek, MCP is probably the hottest thing in AI over the last 6 months.) I use Cursor to write code 90% of the time. I built an MCP server to connect the Cursor agent to GroundX, an open-source RAG system, and I'm not going back. This is officially insane! Here is what I did, step by step: First, a little bit of context. I maintain an end-to-end Machine Learning System with several pipelines to process data, train, evaluate, register, deploy, and monitor a model. I've written a lot of documentation explaining how the system works and how to modify and maintain it. There's also the documentation of the few libraries I used to build the system. I'm a massive fan of GroundX, an open-source enterprise-grade RAG system you can run on your servers or deploy to any cloud provider. I've been working with them for a long time. GroundX offers two services. First, the "ingest" service uses a custom, pretrained vision model to ingest and understand your data. I used this to process all the documentation I have for my code. Markdown files, source code, HTML files, and even PDF documents. Everything I've written related to my project went into GroundX. Their second service is "search," which combines text and vector search with a fine-tuned re-ranker model to retrieve information from the data. I needed to connect Cursor with this service, and that's where MCP came in. I built an MCP server with two tools: 1. The first tool would go to GroundX and retrieve the available topics. Splitting the data into topics (or "buckets," as GroundX calls them) allows me to use the same setup to serve documentation from different topics. 2. The second tool would search GroundX under a specific topic for the context related to the supplied query. The magic happens after connecting the MCP server with Cursor. Now, I can ask any questions related to my project, and Cursor's AI agent retrieves the list of available topics from the RAG system and then searches it to provide relevant context to the model. I went from getting mediocre, sometimes wrong answers to 100% truthful, complete answers. Here is the crazy part:

MCP is an absolute game-changer. (Together with DeepSeek, MCP is probably the hottest thing in AI over the last 6 months.) I use Cursor to write code 90% of the time. I built an MCP server to connect the Cursor agent to GroundX, an open-source RAG system, and I'm not going back. This is officially insane! Here is what I did, step by step: First, a little bit of context. I maintain an end-to-end Machine Learning System with several pipelines to process data, train, evaluate, register, deploy, and monitor a model. I've written a lot of documentation explaining how the system works and how to modify and maintain it. There's also the documentation of the few libraries I used to build the system. I'm a massive fan of GroundX, an open-source enterprise-grade RAG system you can run on your servers or deploy to any cloud provider. I've been working with them for a long time. GroundX offers two services. First, the "ingest" service uses a custom, pretrained vision model to ingest and understand your data. I used this to process all the documentation I have for my code. Markdown files, source code, HTML files, and even PDF documents. Everything I've written related to my project went into GroundX. Their second service is "search," which combines text and vector search with a fine-tuned re-ranker model to retrieve information from the data. I needed to connect Cursor with this service, and that's where MCP came in. I built an MCP server with two tools: 1. The first tool would go to GroundX and retrieve the available topics. Splitting the data into topics (or "buckets," as GroundX calls them) allows me to use the same setup to serve documentation from different topics. 2. The second tool would search GroundX under a specific topic for the context related to the supplied query. The magic happens after connecting the MCP server with Cursor. Now, I can ask any questions related to my project, and Cursor's AI agent retrieves the list of available topics from the RAG system and then searches it to provide relevant context to the model. I went from getting mediocre, sometimes wrong answers to 100% truthful, complete answers. Here is the crazy part:

Santiago

255,596 次观看 • 1 年前

The most important technology of our time is being built in private, but we're funding the alternative: $42M for the people building AGI in the open. Whether it's the model, weights, code, data, or evals, AI should belong to the people it serves, run on hardware they own, and reach the cheapest phone on earth. Here’s a few of the use cases we want to back ↓

The most important technology of our time is being built in private, but we're funding the alternative: $42M for the people building AGI in the open. Whether it's the model, weights, code, data, or evals, AI should belong to the people it serves, run on hardware they own, and reach the cheapest phone on earth. Here’s a few of the use cases we want to back ↓

Sentient Foundation

91,893 次观看 • 2 个月前

A builder wants to ship an agent that can search live data, use it instantly, and keep going without manual steps. That is why OptimAI Search now ships with x402, SDK, and MCP support. x402 makes access and payment seamless. SDK and MCP make integration straightforward for builders. And under the hood, OptimAI Search gives agents real-time access to huge volumes of public and social web data, structured into context they can actually use. Add Agent Marketplace on top, and domain knowledge becomes reusable: build specialized agents, publish them, and let others build on what you create. This is what search looks like in the agent-native internet. 💡Try it:

A builder wants to ship an agent that can search live data, use it instantly, and keep going without manual steps. That is why OptimAI Search now ships with x402, SDK, and MCP support. x402 makes access and payment seamless. SDK and MCP make integration straightforward for builders. And under the hood, OptimAI Search gives agents real-time access to huge volumes of public and social web data, structured into context they can actually use. Add Agent Marketplace on top, and domain knowledge becomes reusable: build specialized agents, publish them, and let others build on what you create. This is what search looks like in the agent-native internet. 💡Try it:

OptimAI Network

36,562 次观看 • 4 个月前

A voice agent powered by gpt-oss. Running locally on my macBook. Demo recorded in a Waymo with WiFi turned off. I'm still on my space game voice AI kick, obviously. Code link below. For conversational voice AI, you want to set the gpt-oss reasoning behavior to "low". (The default is "medium".) Notes on how to do that and a jinja template you can use are in the repo. The LLM in the demo video is the big, 120B version of gpt-oss. You can use the smaller, 20B model for this, of course. But OpenAI really did a cool thing here designing the 120B model to run in "just" 80GB of VRAM. And the llama.cpp mlx inference is fast: ~250ms TTFT. Running a big model on-device feels like a time warp into the future of AI.

A voice agent powered by gpt-oss. Running locally on my macBook. Demo recorded in a Waymo with WiFi turned off. I'm still on my space game voice AI kick, obviously. Code link below. For conversational voice AI, you want to set the gpt-oss reasoning behavior to "low". (The default is "medium".) Notes on how to do that and a jinja template you can use are in the repo. The LLM in the demo video is the big, 120B version of gpt-oss. You can use the smaller, 20B model for this, of course. But OpenAI really did a cool thing here designing the 120B model to run in "just" 80GB of VRAM. And the llama.cpp mlx inference is fast: ~250ms TTFT. Running a big model on-device feels like a time warp into the future of AI.

kwindla

202,211 次观看 • 1 年前

While we in India should be cautious with the DeepSeek app, we can totally make use of the open source model namesake, if securely deployed on Indian servers, to leapfrog our own AI progress. कृत्रिम has deployed DeepSeek-R1 671B on H100s - first time anywhere in the world. It’s the most powerful open-source model available and to enable all Indian developers to access it, we will price it at ₹1/million tokens for February.

While we in India should be cautious with the DeepSeek app, we can totally make use of the open source model namesake, if securely deployed on Indian servers, to leapfrog our own AI progress. कृत्रिम has deployed DeepSeek-R1 671B on H100s - first time anywhere in the world. It’s the most powerful open-source model available and to enable all Indian developers to access it, we will price it at ₹1/million tokens for February.

Bhavish Aggarwal

108,714 次观看 • 1 年前

A guy named Jan-Christoph accidentally built the most useful directory of open-source projects on the internet. It is called LibreProjects. Search for what you need and it shows you open-source web services across 15 categories: > Social networks > Search engines > Cloud storage > Video calls > Spreadsheets > Presentations > Books > News > Audio > Graphics > Games > Development tools You will find Jitsi Meet for video calls, Nextcloud for cloud storage, EtherCalc for spreadsheets, Searx for search, and Project Gutenberg for books. Click a project and LibreProjects tells you what it does, which commercial product it can replace, what license it uses, and where you can inspect the source code. The clever part is what they refuse to list. A company cannot throw one open-source feature into a paid product and sneak into the directory wearing a fake mustache. The service should be open source, hosted, free to use, and simple enough for ordinary people. It is basically an escape map from subscription software. And somehow, almost nobody knows it exists.

A guy named Jan-Christoph accidentally built the most useful directory of open-source projects on the internet. It is called LibreProjects. Search for what you need and it shows you open-source web services across 15 categories: > Social networks > Search engines > Cloud storage > Video calls > Spreadsheets > Presentations > Books > News > Audio > Graphics > Games > Development tools You will find Jitsi Meet for video calls, Nextcloud for cloud storage, EtherCalc for spreadsheets, Searx for search, and Project Gutenberg for books. Click a project and LibreProjects tells you what it does, which commercial product it can replace, what license it uses, and where you can inspect the source code. The clever part is what they refuse to list. A company cannot throw one open-source feature into a paid product and sneak into the directory wearing a fake mustache. The service should be open source, hosted, free to use, and simple enough for ordinary people. It is basically an escape map from subscription software. And somehow, almost nobody knows it exists.

Ihtesham Ali

14,089 次观看 • 1 个月前