Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

vLLM + open-webui running gpt-oss-120b on a tinybox green v2 Just `vllm serve openai/gpt-oss-120b --tensor-parallel-size 4 --async-scheduling` and you have a local OpenAI API you can trust.

__tinygrad__'s profile picture

the tiny corp

78,335 subscribers

36,677 Aufrufe • vor 1 Jahr •via X (Twitter)

Wissenschaft & TechnologieGamingFinanzen
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

We’re thrilled to launch GPT-OSS—the 120B-parameter open-source LLM—on Phala Cloud! 🚀 🔒 Run the fully OSS model inside a secure TEE for end-to-end confidential AI. Explore the build 👉
0:13

We’re thrilled to launch GPT-OSS—the 120B-parameter open-source LLM—on Phala Cloud! 🚀 🔒 Run the fully OSS model inside a secure TEE for end-to-end confidential AI. Explore the build 👉

Phala

12,176 Aufrufe • vor 1 Jahr

🚨 YOUR GPU IS PROBABLY WASTING MORE THAN YOU THINK. vLLM is built to squeeze far more useful work out of your GPU when serving LLMs. Running an LLM at scale isn’t just about having a powerful GPU. The real problem is how efficiently you use its memory and compute. That’s where vLLM comes in. → High-throughput LLM inference and serving → PagedAttention for smarter KV-cache memory management → Continuous batching to keep GPUs busy → Prefix caching + chunked prefill → OpenAI-compatible API out of the box → Supports a huge range of modern LLM architectures → Quantization support for running models more efficiently And the crazy part? You can start an OpenAI-compatible inference server with: `vllm serve ` So your application can talk to your own model almost like it’s talking to OpenAI. The bigger idea: Don’t just buy more GPUs. Make the GPUs you already have work harder. That’s why vLLM has become such a major project in LLM inference. 🔥 #vLLM #AI #LLM #Inference #GPU #MachineLearning #AIInfrastructure #OpenSource #AIAgents #Developers
0:20

🚨 YOUR GPU IS PROBABLY WASTING MORE THAN YOU THINK. vLLM is built to squeeze far more useful work out of your GPU when serving LLMs. Running an LLM at scale isn’t just about having a powerful GPU. The real problem is how efficiently you use its memory and compute. That’s where vLLM comes in. → High-throughput LLM inference and serving → PagedAttention for smarter KV-cache memory management → Continuous batching to keep GPUs busy → Prefix caching + chunked prefill → OpenAI-compatible API out of the box → Supports a huge range of modern LLM architectures → Quantization support for running models more efficiently And the crazy part? You can start an OpenAI-compatible inference server with: `vllm serve ` So your application can talk to your own model almost like it’s talking to OpenAI. The bigger idea: Don’t just buy more GPUs. Make the GPUs you already have work harder. That’s why vLLM has become such a major project in LLM inference. 🔥 #vLLM #AI #LLM #Inference #GPU #MachineLearning #AIInfrastructure #OpenSource #AIAgents #Developers

Vikas gupta

14,138 Aufrufe • vor 12 Tagen

BOOOM! Both VLMs and LLMs now have a baked-in http server w/ OpenAI spec compatible API in transformers Launch it with `transformers serve` and connect your favorite apps. Here I'm running Open WebUI with local transformers. LLM, VLM, tool calling is in, STT & TTS coming soon!
0:12

BOOOM! Both VLMs and LLMs now have a baked-in http server w/ OpenAI spec compatible API in transformers Launch it with `transformers serve` and connect your favorite apps. Here I'm running Open WebUI with local transformers. LLM, VLM, tool calling is in, STT & TTS coming soon!

Lysandre

16,290 Aufrufe • vor 1 Jahr

We’ve seen great results using GPT-4 for content policy development and content moderation, enabling more consistent labeling, a faster feedback loop for policy refinement, and less involvement from human moderators. Built on top of the GPT-4 API:
0:16

We’ve seen great results using GPT-4 for content policy development and content moderation, enabling more consistent labeling, a faster feedback loop for policy refinement, and less involvement from human moderators. Built on top of the GPT-4 API:

OpenAI

902,486 Aufrufe • vor 3 Jahren

You can now do RL weight sync over HF Buckets. > Async GRPO + LoRA in TRL > train on one Job, serve vLLM on others > sync just the small delta adapter through a Bucket. Trainer, inference and storage, all on Hugging Face infra. Beautifully optimised loop by @DirhoussssiAmine 👇 3h27 → 53 min for the same reward.
0:15

You can now do RL weight sync over HF Buckets. > Async GRPO + LoRA in TRL > train on one Job, serve vLLM on others > sync just the small delta adapter through a Bucket. Trainer, inference and storage, all on Hugging Face infra. Beautifully optimised loop by @DirhoussssiAmine 👇 3h27 → 53 min for the same reward.

Adithya S K

18,231 Aufrufe • vor 5 Tagen

Wild that you can run a GPT-4 tier AI on a laptop and serve it to your whole network. Llama 3.3 70B is a beast.
0:14

Wild that you can run a GPT-4 tier AI on a laptop and serve it to your whole network. Llama 3.3 70B is a beast.

Aaron Ng

35,688 Aufrufe • vor 1 Jahr

Build with GPT-6 Astra. Launch on Product Hunt on September 18. We've teamed up with OpenAI Developers for the GPT-6 Astra Challenge. The top five launches each win $10K in OpenAI API credits and a year of ChatGPT Pro for up to two team members. If you’re already building with Astra, this is a chance to share what you’ve made and put it in front of a wider audience. You can schedule your launch for Friday, September 18.
0:13

Build with GPT-6 Astra. Launch on Product Hunt on September 18. We've teamed up with OpenAI Developers for the GPT-6 Astra Challenge. The top five launches each win $10K in OpenAI API credits and a year of ChatGPT Pro for up to two team members. If you’re already building with Astra, this is a chance to share what you’ve made and put it in front of a wider audience. You can schedule your launch for Friday, September 18.

Product Hunt 😸

442,915 Aufrufe • vor 12 Tagen

Introducing DeepConf: Deep Think with Confidence 🚀 First method to achieve 99.9% on AIME 2025 with open-source models! Using GPT-OSS-120B even without tools, we reached this almost-perfect accuracy while saving up to 85% generated tokens. It also delivers many strong advantages for parallel thinking: 🔥 Performance boost: ~10% accuracy across models & datasets ⚡ Ultra-efficient: Up to 85% fewer tokens generated 🔧 Plug & play: Works with ANY existing model - zero training needed (no hyperparameter tuning as well!) ⭐ Easy to deploy: Just ~50 lines of code in vLLM (see PR below) 📚 Paper: 🌐 Project: joint work with: Yichao Fu , xuewei_wang, Yuandong Tian (see details in the comments below)
0:16

Introducing DeepConf: Deep Think with Confidence 🚀 First method to achieve 99.9% on AIME 2025 with open-source models! Using GPT-OSS-120B even without tools, we reached this almost-perfect accuracy while saving up to 85% generated tokens. It also delivers many strong advantages for parallel thinking: 🔥 Performance boost: ~10% accuracy across models & datasets ⚡ Ultra-efficient: Up to 85% fewer tokens generated 🔧 Plug & play: Works with ANY existing model - zero training needed (no hyperparameter tuning as well!) ⭐ Easy to deploy: Just ~50 lines of code in vLLM (see PR below) 📚 Paper: 🌐 Project: joint work with: Yichao Fu , xuewei_wang, Yuandong Tian (see details in the comments below)

Jiawei Zhao

464,995 Aufrufe • vor 1 Jahr

🎉 Congrats to MiniMax (official) on releasing the open weights for MiniMax H3! Day-0 support in vLLM-Omni! One model reads text, images, video, and audio as a single context and returns video with native stereo audio. Text-to-video, first/last-frame, and multi-reference generation, 4 to 15 seconds at up to 2K and 24 FPS. The MP4 comes back with H.264 video and a synchronized stereo track already muxed in. It serves over the OpenAI-compatible /v1/videos endpoint, synchronous or async with job polling. 🔊 The video ⬇️ was made with H3, served on vLLM-Omni.
0:15

🎉 Congrats to MiniMax (official) on releasing the open weights for MiniMax H3! Day-0 support in vLLM-Omni! One model reads text, images, video, and audio as a single context and returns video with native stereo audio. Text-to-video, first/last-frame, and multi-reference generation, 4 to 15 seconds at up to 2K and 24 FPS. The MP4 comes back with H.264 video and a synchronized stereo track already muxed in. It serves over the OpenAI-compatible /v1/videos endpoint, synchronous or async with job polling. 🔊 The video ⬇️ was made with H3, served on vLLM-Omni.

vLLM

130,725 Aufrufe • vor 1 Monat

Thank you Fable 5.1 and GPT Astra OpenAI Anthropic I have finally achieved 30FPS holographic glasses free gesture based colorized segmented medical imaging from a basic black and white DICOM
0:19

Thank you Fable 5.1 and GPT Astra OpenAI Anthropic I have finally achieved 30FPS holographic glasses free gesture based colorized segmented medical imaging from a basic black and white DICOM

zero

253,368 Aufrufe • vor 14 Tagen

Qwen has just released a model on par with GPT-4o... And you can run it locally easily 🤯 Yep. GPT-4o level AI running offline on a laptop. - Fully open source - Only 3B active parameters - 262k context length natively Quick steps to run it on your machine and details below
0:15

Qwen has just released a model on par with GPT-4o... And you can run it locally easily 🤯 Yep. GPT-4o level AI running offline on a laptop. - Fully open source - Only 3B active parameters - 262k context length natively Quick steps to run it on your machine and details below

Paul Couvert

504,333 Aufrufe • vor 1 Jahr

GPT-4 is the new data scientist 🤯 With the new OpenAI code interpreter model, you just have to upload the data and provide instructions in simple english. The model does everything from cleaning data to generating insightful visualizations on autopilot!!
0:14

GPT-4 is the new data scientist 🤯 With the new OpenAI code interpreter model, you just have to upload the data and provide instructions in simple english. The model does everything from cleaning data to generating insightful visualizations on autopilot!!

Shubham Saboo

1,693,484 Aufrufe • vor 3 Jahren

👀 Asking all the new GPT-4.1 models from OpenAI to draw… a pelican. Then comparing with Grok 3, LLaMA 4, and Gemini 2.5 Pro. Which one do you prefer? Try it yourself on 80+ models — 👇
0:11

👀 Asking all the new GPT-4.1 models from OpenAI to draw… a pelican. Then comparing with Grok 3, LLaMA 4, and Gemini 2.5 Pro. Which one do you prefer? Try it yourself on 80+ models — 👇

Pierre Valade

123,811 Aufrufe • vor 1 Jahr

Guys I think OpenAI just solved sketch -> render. You can now open up a sketchpad in ChatGPT, draw whatever you want, and get a beautiful image (or design). I have zero artistic ability and am making cool stuff 👇
0:19

Guys I think OpenAI just solved sketch -> render. You can now open up a sketchpad in ChatGPT, draw whatever you want, and get a beautiful image (or design). I have zero artistic ability and am making cool stuff 👇

Justine Moore

81,751 Aufrufe • vor 14 Tagen

Ain't no fuckin way... you can now use Antigravity inside VS Code. 🤯 Google has launched its official Antigravity extension for VS Code. So instead of switching to another editor, you can now bring Antigravity's AI coding experience into VS Code. And the crazy part? The free plan gives you access to models like Gemini, Claude Sonnet 4.6, Claude Opus 4.6, and GPT-OSS-120B, with usage limits. Kinda wild how a few months ago people were saying VS Code was getting replaced by Antigravity... and now they're basically sitting together. 😂 Honestly, I like both :) Follow me for more amazing AI, Coding & Web Dev insights 💎
0:19

Ain't no fuckin way... you can now use Antigravity inside VS Code. 🤯 Google has launched its official Antigravity extension for VS Code. So instead of switching to another editor, you can now bring Antigravity's AI coding experience into VS Code. And the crazy part? The free plan gives you access to models like Gemini, Claude Sonnet 4.6, Claude Opus 4.6, and GPT-OSS-120B, with usage limits. Kinda wild how a few months ago people were saying VS Code was getting replaced by Antigravity... and now they're basically sitting together. 😂 Honestly, I like both :) Follow me for more amazing AI, Coding & Web Dev insights 💎

Ram Maheshwari

37,521 Aufrufe • vor 1 Monat

CRITICAL: if you are running Mosaic 2.4 on a VAX/VMS system, please be aware of this RCE that GPT-5.4 just found and exploited!
0:16

CRITICAL: if you are running Mosaic 2.4 on a VAX/VMS system, please be aware of this RCE that GPT-5.4 just found and exploited!

Brendan Dolan-Gavitt

140,217 Aufrufe • vor 5 Monaten

OpenAI’s GPT-5.6 Sol model can now run inside Claude Code. There are two ways to do it, and both take minutes to set up, here’s how: Option 1, the official plugin: /plugin marketplace add openai/codex-plugin-cc /plugin install codex@openai-codex /reload-plugins, then /codex:setup That unlocks /codex:review, /codex:adversarial-review, and /codex:rescue. Claude writes, GPT critiques, you ship. Option 2, the proxy. One alias makes Sol your main model: alias claudex=‘CLAUDE_CODE_SUBAGENT_MODEL=gpt-5.6-sol CLAUDE_CODE_ALWAYS_ENABLE_EFFORT=1 claude –model gpt-5.6-sol’ Bonus: Claude Code lets you set subagent model and effort yourself. Sol Ultra runs can burn several times the tokens of a base run, so right-sizing delegated work saves real money.
0:15

OpenAI’s GPT-5.6 Sol model can now run inside Claude Code. There are two ways to do it, and both take minutes to set up, here’s how: Option 1, the official plugin: /plugin marketplace add openai/codex-plugin-cc /plugin install codex@openai-codex /reload-plugins, then /codex:setup That unlocks /codex:review, /codex:adversarial-review, and /codex:rescue. Claude writes, GPT critiques, you ship. Option 2, the proxy. One alias makes Sol your main model: alias claudex=‘CLAUDE_CODE_SUBAGENT_MODEL=gpt-5.6-sol CLAUDE_CODE_ALWAYS_ENABLE_EFFORT=1 claude –model gpt-5.6-sol’ Bonus: Claude Code lets you set subagent model and effort yourself. Sol Ultra runs can burn several times the tokens of a base run, so right-sizing delegated work saves real money.

Alvaro Cintas

90,783 Aufrufe • vor 2 Monaten

This peanut-sized chinese model just dethroned Gemini at reading documents. It’s called glm-ocr. it’s a tiny 0.9b parameter vision-language model that is about to replace every expensive ocr api you use. → Handles text, tables, formulas, handwriting → Scored 94.62 on OmniDocBench V1.5 → 8 languages → vLLM, SGLang, Ollama And it’s 100% open-source.
0:15

This peanut-sized chinese model just dethroned Gemini at reading documents. It’s called glm-ocr. it’s a tiny 0.9b parameter vision-language model that is about to replace every expensive ocr api you use. → Handles text, tables, formulas, handwriting → Scored 94.62 on OmniDocBench V1.5 → 8 languages → vLLM, SGLang, Ollama And it’s 100% open-source.

Alvaro Cintas

37,577 Aufrufe • vor 5 Monaten

Liquid's LFM2.5-8B-A1B smashed OpenAI's gpt-oss-20b on tool calling We ran both locally on a MacBook Pro M5 Max, 64GB, and gave each the same trip-planning request that only completes if the model fires all 7 tool calls - weather for 3 cities, two currency conversions, an email and a reminder Outputs: LFM2.5-8B-A1B: 4.8 GB RAM usage, 7/7 tool-calls, 266 tok/s, 6.9s OpenAI gpt-oss-20b: 11 GB RAM usage, 3/7 tool-calls, 146 tok/s, 15.0s The 8B used less than half the RAM and still fired all 7 calls, while the 20B silently dropped more than half of its own. It also ran ~2x faster, wrapping the full agentic request in 6.9s against 15s. That's what 38T training tokens buy: a 1B-active MoE that nails the agentic tool calls a model 2.5x its active size keeps dropping
0:15

Liquid's LFM2.5-8B-A1B smashed OpenAI's gpt-oss-20b on tool calling We ran both locally on a MacBook Pro M5 Max, 64GB, and gave each the same trip-planning request that only completes if the model fires all 7 tool calls - weather for 3 cities, two currency conversions, an email and a reminder Outputs: LFM2.5-8B-A1B: 4.8 GB RAM usage, 7/7 tool-calls, 266 tok/s, 6.9s OpenAI gpt-oss-20b: 11 GB RAM usage, 3/7 tool-calls, 146 tok/s, 15.0s The 8B used less than half the RAM and still fired all 7 calls, while the 20B silently dropped more than half of its own. It also ran ~2x faster, wrapping the full agentic request in 6.9s against 15s. That's what 38T training tokens buy: a 1B-active MoE that nails the agentic tool calls a model 2.5x its active size keeps dropping

atomic.chat

92,187 Aufrufe • vor 3 Monaten