parakeet.cpp: native C++/ggml (ggml) inference for NVIDIA AI Developer's... Parakeet, one of the best speech-to-text models out there, from the LocalAI team. Every Parakeet model (TDT/CTC/RNNT/hybrid + cache-aware streaming), byte-for-byte identical output to NeMo, now running anywhere with no Python and even a bit faster, on CPU and GPU. Quantized GGUF on Hugging Face 🤗 Huge thanks to Georgi Gerganov for ggml and to NVIDIA AI Developer for releasing Parakeet! 🧵show more

Depth Anything 3 now runs as pure C++/ggml (ggml)... . No Python, no PyTorch, no CUDA toolkit at inference, just one self-contained GGUF. It's faster than PyTorch on CPU! and ties speed on GPU. The CPU win came from the last place..I'd have looked. Quantized GGUF on Hugging Face🤗 Shout out to Georgi Gerganov for ggml (we are building a ggml-world!❤️) and to ByteDance Open Source and Depth Anything 3 authors Bingyi Kang Jun Hao Liew Donny Y. Chen !show more

I switched from a quantized Whisper Large v3 Turbo... to Parakeet V3 in VoiceInk and the latency is much better.show more

We are open-sourcing TTSKit! Run state-of-the-art text-to-speech models on... your Mac and iPhone. The launch version supports Qwen Qwen3-TTS and generates audio faster than real-time playback with sub-200 ms time-to-first-byte. Voice cloning and advanced speed optimizations will be in the next version. Link to the GitHub repo and models on Hugging Face in comments.show more

I finally got NVIDIA AI Developer's 3D Gaussian Unscented... Transforms up and running natively on Windows via gsplat. Just in time for me to make a tutorial so people can enter NVIDIA's 3DGUT sweepstakes!!! Huge thanks to Ruilong Li for responding to my Github issues and solving my roadblocks this evening. #3D #Computervisionshow more

AI Tip: Turn on the NVIDIA Broadcast app when... live streaming for AI greenscreen + to filter out noise 📺 Tell us your fave tips & tricks for taking advantage of AI apps on your GPU below + comment #AIonRTX to win a AORUS GIGABYTE GeForce RTX 4080 16GB GAMING OC GPU!show more

As a supporter of the open weights ecosystem, we're... proud to be a post-training partner for NVIDIA Nemotron. We post-train Nemotron models for customer use cases, de-risk mainline RL runs on our AC2 platform and training stack, and contribute aggregate workload statistics for inference benchmarking. This is how open models get better, and we're excited to keep working closely with NVIDIA AI.show more

We’re releasing model weights for our 8B- parameter Dynamic... Byte Latent Transformer, an alternative to traditional tokenization methods with the potential to redefine the standards for language model efficiency and reliability. Learn more about how Dynamic Byte Latent Transformer is paving the way for groundbreaking developments in the field of language modeling. Read the research paper, and download the model and code.show more

We are continuing to advance open-source AI. Just announced... -- new tools for speech, safety, and autonomous driving. Learn about the open models, datasets, and research we’re releasing at #NeurIPS2025:show more

How to run AI models locally? -> Go to... 🤗 Hugging Face -> Grab the GGUF model link -> Drop it into 👋 Jan Hub That's all there is to it.show more

90% of "AI developers" just download pre packaged GGUF... files from Hugging Face, hit run, and call it a day. The top 10% know how to pull the raw safetensors, run the math, and quantize massive models into Q4_K_M themselves. If you think llama.cpp can only execute models, you’re missing the best part of the open source ecosystem. It’s a high performance optimization suite. Manually stripping 69% of the VRAM footprint off a brand new model architecture is where real infrastructure value is made. If you want to actually master local inference and deploy models like Google’s massive Gemma 4 12B it on consumer NVIDIA hardware using llama.cpp, you need to learn this pipeline. Let's build it. I just took the raw 22.7 GB Gemma 4 baseline and manually compressed it down to a 7.02 GB Q4_K_M GGUF artifact using llama.cpp. That is a 69% reduction in footprint. No quality loss. No VRAM bottlenecks. Just native, hardware accelerated C++ inference running a full 2,50,000 token context window on a dual NVIDIA Tesla T4 setup. Stop melting your VRAM on unoptimized weights and stop relying on other people's pipelines. Own your stack. I mapped this entire architecture from dynamic binary fetching to raw quantization and real time GPU streaming into a single, bulletproof notebook. Notebook link is in the comments below. Bookmark this blueprint for your next deployment and tell me which quantization works best for your workflow and model.show more

Qwen3.5-35B-A3B is now in Jan 🔥 It surpasses previous... Qwen3 models more than 6× its size. Get the latest Jan at Thanks to Qwen for the base model and Georgi Gerganov for llama.cpp 💛show more

Announcing our latest open medical AI models for developers:... MedGemma 1.5, which is small enough to run offline & improves performance on 3D imaging (CT & MRI), & MedASR, a speech-to-text model for medical dictation. Both available on Hugging Face + Vertex AI. #MedGemma #HealthAI #GenerativeAIshow more

Frontier world models now run on edge GPUs. ⚡... Now openly available, NVIDIA Cosmos 3 Edge brings advanced world-model capabilities to local devices. The 4-billion-parameter omnimodel can understand and generate text, image, video, ambient sound and action for physical AI across robotics, autonomous vehicles and smart infrastructure. 🔗 #SIGGRAPH2026show more

A Sun Parakeet shakes its head to the beat

Hey @NVIDIA, how do you make the perfect byte?... From working with us to tech CEO, do breakfast like Jensen Huang with new NVIDIA Breakfast Bytes—only available for order online at now through May 13 NVIDIA GTC #GTC2025show more

It’s live. AxonDAO’s GPU fleet is installed and will... soon be rentable on the open market via vast.ai. Premium GPU capacity - 8x NVIDIA B200 and 8x RTX Pro 6000 - built for large-scale AI training, inference, rendering, and research workloads. 🧵show more

WE DELIVER 24/7, DAY BY DAY! 03.06.2025! On Tuesday,... ByteWallet will be launched and available to all ByteBlockchain miners. That is the day every $BYTE miner has been waiting for. Are you ready to transfer your $BYTE to ByteWallet? How much $BYTE are you holding now? Start mining $BYTE now:show more

Want to bring open-source vision language models to the... edge? 💻 Check out our Hugging Face article on deploying NVIDIA Cosmos Reasoning 2B across the NVIDIA Jetson family with vLLM and a Live VLM WebUI. 📖show more

We’re proud to share that $vMINT is now a... member of the NVIDIA Developer Program 🌟 This will provide us with the following things (among others): Access to over 150 SDKs for advanced software development Comprehensive training resources through the Deep Learning Institute and NVIDIA On-Demand for skill enhancement Early access programs Stay tuned for more exciting updates 👀 #Volumint $vMINT #nvidia $nvda #nvda #AI #Marketmakingshow more