正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

vLLM + open-webui running gpt-oss-120b on a tinybox green v2 Just `vllm serve openai/gpt-oss-120b --tensor-parallel-size 4 --async-scheduling` and you have a local OpenAI API you can trust.

the tiny corp

78,335 subscribers

36,677 次观看 • 1 年前 •via X (Twitter)

科学技术游戏财经

Anya Rossi• Live Now

Private livecam show

0 条评论

暂无评论

原始帖子的评论将显示在这里

相关视频

We’re thrilled to launch GPT-OSS—the 120B-parameter open-source LLM—on Phala Cloud! 🚀 🔒 Run the fully OSS model inside a secure TEE for end-to-end confidential AI. Explore the build 👉

We’re thrilled to launch GPT-OSS—the 120B-parameter open-source LLM—on Phala Cloud! 🚀 🔒 Run the fully OSS model inside a secure TEE for end-to-end confidential AI. Explore the build 👉

Phala

12,176 次观看 • 1 年前

🚨 YOUR GPU IS PROBABLY WASTING MORE THAN YOU THINK. vLLM is built to squeeze far more useful work out of your GPU when serving LLMs. Running an LLM at scale isn’t just about having a powerful GPU. The real problem is how efficiently you use its memory and compute. That’s where vLLM comes in. → High-throughput LLM inference and serving → PagedAttention for smarter KV-cache memory management → Continuous batching to keep GPUs busy → Prefix caching + chunked prefill → OpenAI-compatible API out of the box → Supports a huge range of modern LLM architectures → Quantization support for running models more efficiently And the crazy part? You can start an OpenAI-compatible inference server with: `vllm serve ` So your application can talk to your own model almost like it’s talking to OpenAI. The bigger idea: Don’t just buy more GPUs. Make the GPUs you already have work harder. That’s why vLLM has become such a major project in LLM inference. 🔥 #vLLM #AI #LLM #Inference #GPU #MachineLearning #AIInfrastructure #OpenSource #AIAgents #Developers

🚨 YOUR GPU IS PROBABLY WASTING MORE THAN YOU THINK. vLLM is built to squeeze far more useful work out of your GPU when serving LLMs. Running an LLM at scale isn’t just about having a powerful GPU. The real problem is how efficiently you use its memory and compute. That’s where vLLM comes in. → High-throughput LLM inference and serving → PagedAttention for smarter KV-cache memory management → Continuous batching to keep GPUs busy → Prefix caching + chunked prefill → OpenAI-compatible API out of the box → Supports a huge range of modern LLM architectures → Quantization support for running models more efficiently And the crazy part? You can start an OpenAI-compatible inference server with: `vllm serve ` So your application can talk to your own model almost like it’s talking to OpenAI. The bigger idea: Don’t just buy more GPUs. Make the GPUs you already have work harder. That’s why vLLM has become such a major project in LLM inference. 🔥 #vLLM #AI #LLM #Inference #GPU #MachineLearning #AIInfrastructure #OpenSource #AIAgents #Developers

Vikas gupta

14,138 次观看 • 10 天前

BOOOM! Both VLMs and LLMs now have a baked-in http server w/ OpenAI spec compatible API in transformers Launch it with `transformers serve` and connect your favorite apps. Here I'm running Open WebUI with local transformers. LLM, VLM, tool calling is in, STT & TTS coming soon!

BOOOM! Both VLMs and LLMs now have a baked-in http server w/ OpenAI spec compatible API in transformers Launch it with `transformers serve` and connect your favorite apps. Here I'm running Open WebUI with local transformers. LLM, VLM, tool calling is in, STT & TTS coming soon!

Lysandre

16,290 次观看 • 1 年前

We’ve seen great results using GPT-4 for content policy development and content moderation, enabling more consistent labeling, a faster feedback loop for policy refinement, and less involvement from human moderators. Built on top of the GPT-4 API:

We’ve seen great results using GPT-4 for content policy development and content moderation, enabling more consistent labeling, a faster feedback loop for policy refinement, and less involvement from human moderators. Built on top of the GPT-4 API:

OpenAI

902,486 次观看 • 3 年前

You can now do RL weight sync over HF Buckets. > Async GRPO + LoRA in TRL > train on one Job, serve vLLM on others > sync just the small delta adapter through a Bucket. Trainer, inference and storage, all on Hugging Face infra. Beautifully optimised loop by @DirhoussssiAmine 👇 3h27 → 53 min for the same reward.

You can now do RL weight sync over HF Buckets. > Async GRPO + LoRA in TRL > train on one Job, serve vLLM on others > sync just the small delta adapter through a Bucket. Trainer, inference and storage, all on Hugging Face infra. Beautifully optimised loop by @DirhoussssiAmine 👇 3h27 → 53 min for the same reward.

Adithya S K

17,961 次观看 • 3 天前

Wild that you can run a GPT-4 tier AI on a laptop and serve it to your whole network. Llama 3.3 70B is a beast.

Wild that you can run a GPT-4 tier AI on a laptop and serve it to your whole network. Llama 3.3 70B is a beast.

Aaron Ng

35,688 次观看 • 1 年前

Build with GPT-6 Astra. Launch on Product Hunt on September 18. We've teamed up with OpenAI Developers for the GPT-6 Astra Challenge. The top five launches each win $10K in OpenAI API credits and a year of ChatGPT Pro for up to two team members. If you’re already building with Astra, this is a chance to share what you’ve made and put it in front of a wider audience. You can schedule your launch for Friday, September 18.

Build with GPT-6 Astra. Launch on Product Hunt on September 18. We've teamed up with OpenAI Developers for the GPT-6 Astra Challenge. The top five launches each win $10K in OpenAI API credits and a year of ChatGPT Pro for up to two team members. If you’re already building with Astra, this is a chance to share what you’ve made and put it in front of a wider audience. You can schedule your launch for Friday, September 18.

Product Hunt 😸

442,381 次观看 • 10 天前

Introducing DeepConf: Deep Think with Confidence 🚀 First method to achieve 99.9% on AIME 2025 with open-source models! Using GPT-OSS-120B even without tools, we reached this almost-perfect accuracy while saving up to 85% generated tokens. It also delivers many strong advantages for parallel thinking: 🔥 Performance boost: ~10% accuracy across models & datasets ⚡ Ultra-efficient: Up to 85% fewer tokens generated 🔧 Plug & play: Works with ANY existing model - zero training needed (no hyperparameter tuning as well!) ⭐ Easy to deploy: Just ~50 lines of code in vLLM (see PR below) 📚 Paper: 🌐 Project: joint work with: Yichao Fu , xuewei_wang, Yuandong Tian (see details in the comments below)

Introducing DeepConf: Deep Think with Confidence 🚀 First method to achieve 99.9% on AIME 2025 with open-source models! Using GPT-OSS-120B even without tools, we reached this almost-perfect accuracy while saving up to 85% generated tokens. It also delivers many strong advantages for parallel thinking: 🔥 Performance boost: ~10% accuracy across models & datasets ⚡ Ultra-efficient: Up to 85% fewer tokens generated 🔧 Plug & play: Works with ANY existing model - zero training needed (no hyperparameter tuning as well!) ⭐ Easy to deploy: Just ~50 lines of code in vLLM (see PR below) 📚 Paper: 🌐 Project: joint work with: Yichao Fu , xuewei_wang, Yuandong Tian (see details in the comments below)

Jiawei Zhao

464,995 次观看 • 1 年前

🎉 Congrats to MiniMax (official) on releasing the open weights for MiniMax H3! Day-0 support in vLLM-Omni! One model reads text, images, video, and audio as a single context and returns video with native stereo audio. Text-to-video, first/last-frame, and multi-reference generation, 4 to 15 seconds at up to 2K and 24 FPS. The MP4 comes back with H.264 video and a synchronized stereo track already muxed in. It serves over the OpenAI-compatible /v1/videos endpoint, synchronous or async with job polling. 🔊 The video ⬇️ was made with H3, served on vLLM-Omni.

🎉 Congrats to MiniMax (official) on releasing the open weights for MiniMax H3! Day-0 support in vLLM-Omni! One model reads text, images, video, and audio as a single context and returns video with native stereo audio. Text-to-video, first/last-frame, and multi-reference generation, 4 to 15 seconds at up to 2K and 24 FPS. The MP4 comes back with H.264 video and a synchronized stereo track already muxed in. It serves over the OpenAI-compatible /v1/videos endpoint, synchronous or async with job polling. 🔊 The video ⬇️ was made with H3, served on vLLM-Omni.

vLLM

130,614 次观看 • 1 个月前

Thank you Fable 5.1 and GPT Astra OpenAI Anthropic I have finally achieved 30FPS holographic glasses free gesture based colorized segmented medical imaging from a basic black and white DICOM

Thank you Fable 5.1 and GPT Astra OpenAI Anthropic I have finally achieved 30FPS holographic glasses free gesture based colorized segmented medical imaging from a basic black and white DICOM

zero

253,368 次观看 • 12 天前

Qwen has just released a model on par with GPT-4o... And you can run it locally easily 🤯 Yep. GPT-4o level AI running offline on a laptop. - Fully open source - Only 3B active parameters - 262k context length natively Quick steps to run it on your machine and details below

Qwen has just released a model on par with GPT-4o... And you can run it locally easily 🤯 Yep. GPT-4o level AI running offline on a laptop. - Fully open source - Only 3B active parameters - 262k context length natively Quick steps to run it on your machine and details below

Paul Couvert

504,333 次观看 • 1 年前

GPT-4 is the new data scientist 🤯 With the new OpenAI code interpreter model, you just have to upload the data and provide instructions in simple english. The model does everything from cleaning data to generating insightful visualizations on autopilot!!

GPT-4 is the new data scientist 🤯 With the new OpenAI code interpreter model, you just have to upload the data and provide instructions in simple english. The model does everything from cleaning data to generating insightful visualizations on autopilot!!

Shubham Saboo

1,693,484 次观看 • 3 年前

👀 Asking all the new GPT-4.1 models from OpenAI to draw… a pelican. Then comparing with Grok 3, LLaMA 4, and Gemini 2.5 Pro. Which one do you prefer? Try it yourself on 80+ models — 👇

👀 Asking all the new GPT-4.1 models from OpenAI to draw… a pelican. Then comparing with Grok 3, LLaMA 4, and Gemini 2.5 Pro. Which one do you prefer? Try it yourself on 80+ models — 👇

Pierre Valade

123,811 次观看 • 1 年前

Guys I think OpenAI just solved sketch -> render. You can now open up a sketchpad in ChatGPT, draw whatever you want, and get a beautiful image (or design). I have zero artistic ability and am making cool stuff 👇

Guys I think OpenAI just solved sketch -> render. You can now open up a sketchpad in ChatGPT, draw whatever you want, and get a beautiful image (or design). I have zero artistic ability and am making cool stuff 👇

Justine Moore

81,751 次观看 • 13 天前

Ain't no fuckin way... you can now use Antigravity inside VS Code. 🤯 Google has launched its official Antigravity extension for VS Code. So instead of switching to another editor, you can now bring Antigravity's AI coding experience into VS Code. And the crazy part? The free plan gives you access to models like Gemini, Claude Sonnet 4.6, Claude Opus 4.6, and GPT-OSS-120B, with usage limits. Kinda wild how a few months ago people were saying VS Code was getting replaced by Antigravity... and now they're basically sitting together. 😂 Honestly, I like both :) Follow me for more amazing AI, Coding & Web Dev insights 💎

Ain't no fuckin way... you can now use Antigravity inside VS Code. 🤯 Google has launched its official Antigravity extension for VS Code. So instead of switching to another editor, you can now bring Antigravity's AI coding experience into VS Code. And the crazy part? The free plan gives you access to models like Gemini, Claude Sonnet 4.6, Claude Opus 4.6, and GPT-OSS-120B, with usage limits. Kinda wild how a few months ago people were saying VS Code was getting replaced by Antigravity... and now they're basically sitting together. 😂 Honestly, I like both :) Follow me for more amazing AI, Coding & Web Dev insights 💎

Ram Maheshwari

37,521 次观看 • 1 个月前

CRITICAL: if you are running Mosaic 2.4 on a VAX/VMS system, please be aware of this RCE that GPT-5.4 just found and exploited!

CRITICAL: if you are running Mosaic 2.4 on a VAX/VMS system, please be aware of this RCE that GPT-5.4 just found and exploited!

Brendan Dolan-Gavitt

140,217 次观看 • 5 个月前

OpenAI’s GPT-5.6 Sol model can now run inside Claude Code. There are two ways to do it, and both take minutes to set up, here’s how: Option 1, the official plugin: /plugin marketplace add openai/codex-plugin-cc /plugin install codex@openai-codex /reload-plugins, then /codex:setup That unlocks /codex:review, /codex:adversarial-review, and /codex:rescue. Claude writes, GPT critiques, you ship. Option 2, the proxy. One alias makes Sol your main model: alias claudex=‘CLAUDE_CODE_SUBAGENT_MODEL=gpt-5.6-sol CLAUDE_CODE_ALWAYS_ENABLE_EFFORT=1 claude –model gpt-5.6-sol’ Bonus: Claude Code lets you set subagent model and effort yourself. Sol Ultra runs can burn several times the tokens of a base run, so right-sizing delegated work saves real money.

OpenAI’s GPT-5.6 Sol model can now run inside Claude Code. There are two ways to do it, and both take minutes to set up, here’s how: Option 1, the official plugin: /plugin marketplace add openai/codex-plugin-cc /plugin install codex@openai-codex /reload-plugins, then /codex:setup That unlocks /codex:review, /codex:adversarial-review, and /codex:rescue. Claude writes, GPT critiques, you ship. Option 2, the proxy. One alias makes Sol your main model: alias claudex=‘CLAUDE_CODE_SUBAGENT_MODEL=gpt-5.6-sol CLAUDE_CODE_ALWAYS_ENABLE_EFFORT=1 claude –model gpt-5.6-sol’ Bonus: Claude Code lets you set subagent model and effort yourself. Sol Ultra runs can burn several times the tokens of a base run, so right-sizing delegated work saves real money.

Alvaro Cintas

90,783 次观看 • 2 个月前

This peanut-sized chinese model just dethroned Gemini at reading documents. It’s called glm-ocr. it’s a tiny 0.9b parameter vision-language model that is about to replace every expensive ocr api you use. → Handles text, tables, formulas, handwriting → Scored 94.62 on OmniDocBench V1.5 → 8 languages → vLLM, SGLang, Ollama And it’s 100% open-source.

This peanut-sized chinese model just dethroned Gemini at reading documents. It’s called glm-ocr. it’s a tiny 0.9b parameter vision-language model that is about to replace every expensive ocr api you use. → Handles text, tables, formulas, handwriting → Scored 94.62 on OmniDocBench V1.5 → 8 languages → vLLM, SGLang, Ollama And it’s 100% open-source.

Alvaro Cintas

37,577 次观看 • 5 个月前

Liquid's LFM2.5-8B-A1B smashed OpenAI's gpt-oss-20b on tool calling We ran both locally on a MacBook Pro M5 Max, 64GB, and gave each the same trip-planning request that only completes if the model fires all 7 tool calls - weather for 3 cities, two currency conversions, an email and a reminder Outputs: LFM2.5-8B-A1B: 4.8 GB RAM usage, 7/7 tool-calls, 266 tok/s, 6.9s OpenAI gpt-oss-20b: 11 GB RAM usage, 3/7 tool-calls, 146 tok/s, 15.0s The 8B used less than half the RAM and still fired all 7 calls, while the 20B silently dropped more than half of its own. It also ran ~2x faster, wrapping the full agentic request in 6.9s against 15s. That's what 38T training tokens buy: a 1B-active MoE that nails the agentic tool calls a model 2.5x its active size keeps dropping

Liquid's LFM2.5-8B-A1B smashed OpenAI's gpt-oss-20b on tool calling We ran both locally on a MacBook Pro M5 Max, 64GB, and gave each the same trip-planning request that only completes if the model fires all 7 tool calls - weather for 3 cities, two currency conversions, an email and a reminder Outputs: LFM2.5-8B-A1B: 4.8 GB RAM usage, 7/7 tool-calls, 266 tok/s, 6.9s OpenAI gpt-oss-20b: 11 GB RAM usage, 3/7 tool-calls, 146 tok/s, 15.0s The 8B used less than half the RAM and still fired all 7 calls, while the 20B silently dropped more than half of its own. It also ran ~2x faster, wrapping the full agentic request in 6.9s against 15s. That's what 38T training tokens buy: a 1B-active MoE that nails the agentic tool calls a model 2.5x its active size keeps dropping

atomic.chat

92,187 次观看 • 3 个月前