Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Created a GPT-Live-1 voice app template with Expo - Full-duplex audio + background calls on iOS - Choose from all supported voices - 6 Rive animations from Vercel AI Elements - Expo SDK 57 + native Expo UI controls Source below ↓

95,449 Aufrufe • vor 5 Tagen •via X (Twitter)

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

llama.cpp isn't just for text LLMs anymore. Pure C++ zero shot voice cloning just officially landed in mainline. Text generation was only step one. If you’re building autonomous local AI agents, real time voice assistants, or edge workflows, instant low latency audio is the missing piece. Thanks to PR #26254, Alibaba’s state of the art Qwen3 TTS model family is now natively supported directly inside the llama.cpp repository under the multimodal (mtmd) framework. No Python bloat. No massive PyTorch CUDA overhead. Just raw, hyper optimized C++ running GGUF voice weights. Here is why this native update is a massive deal for the open source local AI stack: # Multimodal Architecture (.gguf + mmproj) Qwen3-TTS splits the workload between the base language model backbone and a multimodal projection adapter. llama.cpp handles this using the llama-tts binary, mapping the text model alongside its --mmproj projector to process audio tokens seamlessly. # Zero Shot Voice Cloning in Seconds You don't need fine tuning or massive dataset training. Feed the C++ engine a single 5 to 10 second .wav audio sample using the --tts-speaker-file flag, and it accurately clones the exact timbre, tone, and accent on the fly. # Real World T4 GPU Benchmark & Resource FootprintRunning the 1.7B Base model in 8-bit quantization (Q8_0): - VRAM Footprint: ~7 GB peak VRAM during active zero-shot cloning. - Audio Quality: Studio grade, natural-sounding voice output in seconds. • - Execution: Direct execution via native compiled binaries or sub process calls. # Coming Next to llama-server (PR #26603) Beyond CLI execution, a native POST /tts HTTP endpoint is currently being added to llama-server, which will soon allow you to trigger voice generation directly via standard REST API requests! # quick note on Colab compilation: Because this code was merged into mainline very recently, pre-built third-party binaries haven't fully caught up yet. Compiling llama-tts directly from source on Google Colab's free CPU instance can take about 1 hour (or ~1-2 minutes if targeting single GPU arch like -DCMAKE_CUDA_ARCHITECTURES=75). Be patient during the build step, or compile it locally on your own rig for instant execution! To test this out yourself, I built a zero config Google Colab notebook that compiles llama.cpp, downloads the Q8_0 GGUF files from HuggingFace, and spins up an interactive Gradio Studio UI so you can record/upload 3 second clips and clone voices in real time. Stop sleeping on native C++ audio. The era of bulky Python audio pipelines is officially over. Links to the free Google Colab notebook and the official ggml org GGUF HuggingFace model repository are in the replies below! available in q4 and q8 both variants, 1 GB and 1.85 GBs respectively (requires additional ~500MB mmproj gguf) Are you building local voice agents yet? What does your current audio stack look like? Drop your setups below!

Alok

47,881 Aufrufe • vor 1 Monat

You don't need a GPU for fast studio grade voice cloning anymore. Qwen3 TTS (1.7B Q4_K_M) + mainline llama.cpp is officially the fastest way to generate zero shot voice clones using 100% pure CPU execution. Following up on my last post where we ran the Q8 model on a GPU, we just took local C++ voice synthesis a massive step further. The open source community quantized Alibaba's SOTA Qwen3 TTS model down to Q4_K_M GGUF, completely freeing local audio pipelines from dedicated graphics hardware. Here is the real world benchmark and hardware breakdown of running SOTA voice cloning on CPU: # Architecture & Model Setup Using Qwen3-TTS-12Hz-1.7B-Base-Q4_K_M.gguf paired with the 8 bit multimodal projector (mmproj-Q8_0.gguf), llama.cpp executes the entire pipeline in pure C++. No PyTorch, no CUDA dependencies, and no VRAM bottlenecks. # Real-World Memory Footprint - Baseline RAM: 1.6 GB system idle. - Peak Generation RAM: 8 GB RAM during active voice synthesis. - Requirement: Any basic machine with at least 8 GB of system RAM can run this easily. # Real World CPU Benchmarks - Google Colab Free Tier (Throttled 2 Core CPU): Synthesizes a 5 sec studio quality audio clip (~8 words) in 45 seconds. - Modern Consumer CPU (Intel i5/i7 13th/14th Gen or AMD Ryzen 7000/9000): generation should drop to 5 to 20 seconds (nearly 1:1 real-time generation speed!). # Zero Shot Voice Cloning Quality Pass any 5 to 20 second .wav audio sample to the C++ engine using the --tts-speaker-file flag. It yields clean, natural sounding cloned speech with virtually zero quality loss compared to unquantized FP16 weights. To make testing seamless, I built an updated zero config Google Colab notebook. It pulls the official pre built llama.cpp CPU binaries (zero compilation time!) launches a live Gradio web app right in your browser. Record a 5 second clip from your mic (or drop a .mp3, .wav file), type text, and generate cloned audio on CPU. Native C++ audio models are making edge based, offline AI voice agents a reality. Links to the free Q4 CPU Colab notebook and the Q4_K_M GGUF HuggingFace repository are in the replies below! Which models have you been running on your CPUs? What CPU hardware are you using for local inference?

Alok

103,956 Aufrufe • vor 1 Monat

If you’ve ever managed custom icons in React Native, you know the frustration of choosing between performance and developer experience. Libraries like 𝗿𝗲𝗮𝗰𝘁-𝗻𝗮𝘁𝗶𝘃𝗲-𝘀𝘃𝗴 are flexible but heavy. Every icon creates its own React subtree. The alternative? Manual icon fonts that force you into a constant back-and-forth of using web tools like IcoMoon and syncing font assets every time a design changes. 𝗦𝗼𝗳𝘁𝘄𝗮𝗿𝗲 𝗠𝗮𝗻𝘀𝗶𝗼𝗻 𝗟𝗮𝗯𝘀 just released 𝗿𝗲𝗮𝗰𝘁-𝗻𝗮𝘁𝗶𝘃𝗲-𝗻𝗮𝗻𝗼-𝗶𝗰𝗼𝗻𝘀 to solve exactly this. It’s a build-time icon font generator that gives you the performance of native fonts with the flexibility of simple SVG files. 𝗪𝗵𝗮𝘁’𝘀 𝗵𝗮𝗽𝗽𝗲𝗻𝗶𝗻𝗴? Instead of rendering complex vector paths at runtime, this library automatically converts your folder of SVGs into an optimized icon font during the build process. It essentially teaches your app to treat icons like standard text characters, which allows it to bypass React’s component tree and layout engine entirely. ➡️ 𝗕𝘂𝗶𝗹𝗱-𝘁𝗶𝗺𝗲 𝗔𝘂𝘁𝗼𝗺𝗮𝘁𝗶𝗼𝗻: It handles the entire pipeline—watching your icon folder, converting SVGs to .ttf files, and linking them to your native project automatically. ➡️ 𝗡𝗮𝘁𝗶𝘃𝗲 𝗣𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲: Because icons render as native text glyphs, it is significantly faster than traditional SVG rendering, making it the ideal choice for long, scrollable lists or icon-heavy dashboards. ➡️ 𝗘𝘅𝗽𝗼 𝗖𝗼𝗻𝗳𝗶𝗴 𝗣𝗹𝘂𝗴𝗶𝗻: It features first-class support for Expo, automating the native plumbing like Info.plist updates and asset linking during the prebuild phase. ➡️ 𝗔𝘂𝘁𝗼𝗺𝗮𝘁𝗶𝗰 𝗧𝘆𝗽𝗲 𝗦𝗮𝗳𝗲𝘁𝘆: The library generates TypeScript definitions for your icon set, providing full IDE autocomplete and ensuring you never break the UI with a misspelled icon name. 𝗪𝗵𝘆 𝗶𝘁 𝗺𝗮𝘁𝘁𝗲𝗿𝘀? In high-performance applications, small overheads add up. By shifting the heavy lifting from the mobile device to your build machine, 𝗿𝗲𝗮𝗰𝘁-𝗻𝗮𝘁𝗶𝘃𝗲-𝗻𝗮𝗻𝗼-𝗶𝗰𝗼𝗻𝘀 ensures your UI stays buttery smooth while keeping your developer workflow modern. It’s another great example of how 𝗦𝗼𝗳𝘁𝘄𝗮𝗿𝗲 𝗠𝗮𝗻𝘀𝗶𝗼𝗻 continues to solve the "last mile" of performance friction in the ecosystem. Before you migrate your entire library, keep in mind: this is designed for the 𝗡𝗲𝘄 𝗔𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲 (Fabric) and requires React Native 0.74 or higher. #ReactNative #Expo #SoftwareMansion #Icons #MobileDev #Performance #DeveloperExperience #OpenSource #JavaScript #TypeScript #SVG #Fabric

The React Native Rewind

24,992 Aufrufe • vor 4 Monaten

xAI isn't playing around. They just released the Grok Imagine API, a unified video + image generation toolkit, and it's already sitting at #1 on the Artificial Analysis Video Arena for both Text-to-Video AND Image-to-Video. It's beating: ● Google's Veo 3.1 & Veo 3 ● OpenAI's Sora 2 ● Runway Gen-4.5 ● Kling 2.5 Turbo The Numbers Don't Lie: ● 64.1% win rate against Runway Aleph in blind human evaluations ● 57% win rate against Kling o1 ● Best-in-class latency. Sub-20 second generation for 720p, 8-second videos. (up to 15-second video) ● Native audio generation baked right into video output (dialogue, music, sound effects, all synced) What Makes It Different It's built for real creative workflows: ✅ Text-to-video AND image-to-video in one API ✅ Video editing with prompt-based controls (add/remove objects, restyle scenes) ✅ Camera controls: zoom, pan, timelapse, pull-back ✅ Style transfers: cyberpunk, watercolor, anime, you name it ✅ Performance animation: map your movements onto characters ✅ Native audio-video sync (no post-production needed) Why the focus on speed and cost? The partner feedback that shaped this: "Quality alone isn't enough if latency and cost make iteration painful." So xAI optimized for all three. Speed. Cost. Quality. Already Integrated With: ● fal. ai ● ComfyUI ● InVideo ● Flora ● HeyGen xAI went from underdog to chart-topper. The Grok Imagine API is fast, affordable, and genuinely production-ready. If you're building anything with AI video, this just became the one to beat.

tetsuo

18,325 Aufrufe • vor 7 Monaten

We are in an insane run of open-weight drops. Every modality, open source is winning. This is what an open source AI summer ☀️ looks like: 🧠 LLMs & Reasoning → DeepSeek-V4-Flash-0731 (my king 👑): 304B MoE refresh, Terminal-Bench 2.1 jumps 61.8→82.7 over the preview, DeepSWE 7.3→54.4. Closes in on Opus-4.8 on Agents' Last Exam (25.2 vs 25.7). MIT. → Muse-Glimmer-30B, from Meta (they are back!!): their first open agentic model. ~29.6B dense + perception encoder, 131k+ context, built to run fully local, no cloud. Apache 2.0. → Liquid AI LFM2.5-2.6B: 2.69B params, 131k context, 220 tok/s on an M5 Max in under 2.5GB RAM. Competitive with models 4x larger on agentic tasks. → inclusionAI Ling-3.0-flash: 124B total, only 5.1B active, ~12% the size of their old 1T flagship Ring-2.6, matches it on key benchmarks. MIT. → inclusionAI Ling-3.0-tiny: 7.9B total, 1.3B active, 86-90 tok/s on an M4 Pro MacBook at ~8GB peak memory. MIT. → NVIDIA Nemotron-3.5-Lightning-30B-A3B: hybrid Mamba-2+MoE+Attention, up to 1M context, runs on a single H100 or DGX Spark, SWE-bench Verified 52.8. → deepgrove maple-preview: 20B-A1B ternary-weight reasoner, 218 tok/s on a Mac mini M4, 5.3GB checkpoint. MIT. → BigBang-v1 (endless-frontier): fine-tuned from Qwen3.6-35B-A3B via a self-evolving generator/critic synthetic-data loop. Lands aggregate performance between DeepSeek V4 Flash (284B) and V4 Pro (1.6T), at 35B. Apache 2.0. 🎬 Video → MiniMax-H3: 33B dense omni model, native stereo audio, up to 2K/15s. 3.6k+ likes already. → Minimax-H3-Turbo (lightx2v): Apache-2.0 turbo distillation of H3 for fast inference. → Lightricks LTX-2.5: image-to-video update, custom Gemma-4-12B text encoder, a markedly stronger distilled model. 🔊 Voice → NVIDIA NemotronLabs VoiceChat-11B: full-duplex speech-to-speech, ~450ms turn-taking, #2 on open VoiceBench, and the first open full-duplex model with live tool-calling mid-conversation. 🛡️ Safety → Mistral Shieldstral-1.0-3B: 3B multimodal guardrail that takes your safety policy as plain text instead of fixed categories. Beats LlamaGuard-4-12B and ShieldGemma-9B on HarmBench (99.4) and ToxicChat (84.1) at a fraction of the size. Apache 2.0.

Victor M

55,281 Aufrufe • vor 1 Monat

Goldman pays $27,000 per seat for a Bloomberg Terminal. I found 10 open source tools on GitHub that replicate almost all of it for free. Retail investors have never had this much firepower. Bookmark & Repost this one: 1. OpenBB Stocks, options, crypto, forex, and macro data in one research platform. Build your own dashboards, reports, and AI analysts on top of it. The OG of open source finance. 50K+ stars. 2. FinceptTerminal A full financial terminal: global market data, advanced charts, economic indicators, portfolio analysis, and AI research tools. Windows, Mac, and Linux. 3. Neuberg 516 drag-and-drop panels covering equities, bonds, commodities, currencies, credit, and macro. Even connects to Alpaca, Hyperliquid, and Polymarket so you can trade from the terminal itself. 4. Qlib (by Microsoft) An open source AI platform for quant investing. Train ML models, discover signals, backtest strategies, and build portfolios with the same workflow a quant desk uses. 5. FinRobot An AI equity research team on your laptop. Its agents read financial statements, build DCF valuations, debate bull vs bear cases, and generate full investment reports. 6. EdgarTools Turns the SEC database into something humans can actually use. Pull 10-Ks, 10-Qs, insider trades, executive pay, and hedge fund holdings going back to 1994. 7. LEAN (by QuantConnect) An institutional-grade engine for trading algorithms. Write strategies in Python or C#, backtest on decades of data, then connect to real brokers and go live. 8. FinanceToolkit 200+ financial ratios, valuation models, risk metrics, and economic indicators. Works on stocks, ETFs, options, currencies, commodities, and crypto from Python. 9. Ghostfolio A private wealth dashboard for stocks, ETFs, and crypto across all your accounts. Performance, allocation, diversification. Your data never leaves your machine. 10. OpenTerminalUI A self-hosted trading terminal: pro charts, screeners, options chains with live Greeks, portfolio optimization, backtesting, and an AI research agent. Runs entirely on your own hardware. Bloomberg spent 40 years building a $27,000/year moat. Open source is draining it one repo at a time. The software is free. Some live data feeds need your own API keys, but the barrier is now effort, not money. If you want the exact workflows we use to stack these tools with AI, join the AIBullss Discord:

AI Bulls

20,605 Aufrufe • vor 1 Monat