Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Experimenting with the magic of open-source! Whisper for text translation, XTTS for audio, and Video-retalker for seamless mouth sync in a short video Its not perfect, but I think we're close with open source #buildinpublic #opensource

Luis Catacora

8,813 subscribers

80,941 views • 2 years ago •via X (Twitter)

Science & Technology Arts #buildinpublic #opensource

Anya Rossi• Live Now

Private livecam show

10 Comments

Luis C2 years ago

For reference these were my runs on @replicate: Curious to see what you all come up with!

Michael Aubry — BasedLabs.ai2 years ago

Do you know how d-id or heygen works? Wondering if there is a good oss model

Bilawal Sidhu2 years ago

this is awesome, but gosh i wish it worked better for beards

Luis C2 years ago

Likewise. There seems to be limits when using the video-retalking model

AP2 years ago

Really cool! @artificialguybr

Luis C2 years ago

@artificialguybr Thanks! It was cool enough that I just had to share it

⟁ndrew V2 years ago

Pretty damn good for a first pass, impressive!

Luis C2 years ago

Right? All these models have been open source for a while too, just needed to chain them together

Crypto Industry2 years ago

The magic of open source is real, and you're the tech wizard making it happen! 🧙‍♀️

Luis C2 years ago

Thanks! Im just testing out ideas tbh

Related Videos

🎧 IAMF: The Future of Audio with Open Standards! 🎧 At CES , #VLC with #FFMPEG showcases #IAMF redefining sound: 🎵 Spatial audio for next-level immersion 🌌 Cinema-quality sound 📱 Seamless playback across devices 💻 Powered by open-source tech #VLC #CES2025 #OPENSOURCE

🎧 IAMF: The Future of Audio with Open Standards! 🎧 At CES , #VLC with #FFMPEG showcases #IAMF redefining sound: 🎵 Spatial audio for next-level immersion 🌌 Cinema-quality sound 📱 Seamless playback across devices 💻 Powered by open-source tech #VLC #CES2025 #OPENSOURCE

VideoLAN

13,326 views • 1 year ago

Open-source and free apps for coders and makers - CSS Text effects with AI prompts - Open-source self-hosted dashboard for Amazon SES - Free directory of public APIs - Open-source security framework - UI library for building terminal user interfaces

Open-source and free apps for coders and makers - CSS Text effects with AI prompts - Open-source self-hosted dashboard for Amazon SES - Free directory of public APIs - Open-source security framework - UI library for building terminal user interfaces

Csaba Kissi

27,220 views • 1 month ago

The first truly open-source audio-video model. LTX-2 is a DiT-based foundation model with all core video generation capabilities in one unified model. Designed to run locally on consumer GPUs. - text-to-video - image-to-video - and video-to-video modes 100% open-source.

The first truly open-source audio-video model. LTX-2 is a DiT-based foundation model with all core video generation capabilities in one unified model. Designed to run locally on consumer GPUs. - text-to-video - image-to-video - and video-to-video modes 100% open-source.

Akshay 🚀

66,183 views • 7 months ago

Aligned with Matan Grinberg's intuition that in the next 12 months, 90% of tokens could be going to open models. We're seeing more and more companies using frontier APIs for experimenting and open-source or private models for production and larger workloads!

Aligned with Matan Grinberg's intuition that in the next 12 months, 90% of tokens could be going to open models. We're seeing more and more companies using frontier APIs for experimenting and open-source or private models for production and larger workloads!

clem 🤗

24,138 views • 1 month ago

New open-source LoRA built in-house: LTX2.3 Audio Reactive LoRA. Built for music-driven video generation: stronger beat sync, graphic texture, color separation, light pulses, particles, and shapes that move with the track. Model:

New open-source LoRA built in-house: LTX2.3 Audio Reactive LoRA. Built for music-driven video generation: stronger beat sync, graphic texture, color separation, light pulses, particles, and shapes that move with the track. Model:

fal

16,472 views • 2 months ago

The future of AI is open-source. And ollama is the easiest way to build AI applications with open-source LLMs. Here's how to build a free, private RAG app using open-source tools. We'll use: - Ollama for LLMs and embedding models - PostgreSQL for data storage and retrieval - pgai Vectorizer for embedding creation and sync (I use Nomic for embeddings and tinnyllama as my LLM but you can substitute them for any models on Ollama)

The future of AI is open-source. And ollama is the easiest way to build AI applications with open-source LLMs. Here's how to build a free, private RAG app using open-source tools. We'll use: - Ollama for LLMs and embedding models - PostgreSQL for data storage and retrieval - pgai Vectorizer for embedding creation and sync (I use Nomic for embeddings and tinnyllama as my LLM but you can substitute them for any models on Ollama)

Avthar

34,261 views • 1 year ago

We are thrilled to announce the open-source release of Hallo2: a new avatar video generative model capable of generating stunning 4K resolution videos for up to 1 hour! 🎥✨ Explore the project for the open source code: Let's push the boundaries of avatar video generative model! #OpenSource #AI #FaceAnimation #GenerativeArt

Siyu ZHU

58,323 views • 1 year ago

Announcing SubStudio! Generate subtitles for any video in seconds with AI. 100% free & open source! Powered by Whisper on Together AI and FFmpeg via fluent-ffmpeg.

Announcing SubStudio! Generate subtitles for any video in seconds with AI. 100% free & open source! Powered by Whisper on Together AI and FFmpeg via fluent-ffmpeg.

Hassan

134,517 views • 4 months ago

Elon Musk played a key role in the creation of OpenAI, even coming up with the name “OpenAI” to reflect its commitment to being open source. But Sam Altman transformed this open-source, non-profit company into a closed-source, for-profit company.

Elon Musk played a key role in the creation of OpenAI, even coming up with the name “OpenAI” to reflect its commitment to being open source. But Sam Altman transformed this open-source, non-profit company into a closed-source, for-profit company.

DogeDesigner

2,360,489 views • 1 year ago

Elon Musk played a key role in the creation of OpenAI, even coming up with the name “OpenAI” to reflect its commitment to being open source. But Sam Altman transformed this open-source, non-profit company into a closed-source, for-profit company.

Elon Musk played a key role in the creation of OpenAI, even coming up with the name “OpenAI” to reflect its commitment to being open source. But Sam Altman transformed this open-source, non-profit company into a closed-source, for-profit company.

DogeDesigner

57,624 views • 1 year ago

Build an open-source AI video studio with this Next.js template using Veo 3 and Imagen 4 in the Gemini API. Create text-to-video, image-to-video, and edit videos in the browser for a specific time range.

Build an open-source AI video studio with this Next.js template using Veo 3 and Imagen 4 in the Gemini API. Create text-to-video, image-to-video, and edit videos in the browser for a specific time range.

Google AI Developers

41,423 views • 1 year ago

🚀 Introducing HunyuanCustom: An open-source, multimodal-driven architecture for customized video generation, powered by HunyuanVideo-13B. Outperforming existing open-source models, it rivals top closed-source solutions! 🎥 Highlights: ✅Subject Consistency: Maintains identity across single & multi-subject video generation. ✅Multimodal Inputs: Supports text, images, audio, and video for highly controlled outputs. ✅High-Quality Output: Industry-leading detail, motion smoothness, and lighting realism. Key Features: 1️⃣Single-Subject Video: Upload an image + text (e.g., “He’s walking a dog”) to create coherent videos with new actions, outfits, and scenes. 2️⃣Multi-Subject Video: Generate videos with multiple subjects (e.g., a man drinking coffee in a cozy room) from separate image inputs. 3️⃣Audio-Driven Video: Sync audio with visuals for talking or singing in any scene—perfect for digital avatars, virtual customer service, and more. 4️⃣Video-Driven Video: Seamlessly insert or replace subjects into any video for creative enhancements. Dive Deeper: 🌐Project: 📝Technical Details: 💻Code: The single-subject video capability is open-sourced and live on the Hunyuan website, with more features to be released soon! 👉 Try it now: #HunyuanCustom #VideoGeneration #AI #TencentHunyuan

🚀 Introducing HunyuanCustom: An open-source, multimodal-driven architecture for customized video generation, powered by HunyuanVideo-13B. Outperforming existing open-source models, it rivals top closed-source solutions! 🎥 Highlights: ✅Subject Consistency: Maintains identity across single & multi-subject video generation. ✅Multimodal Inputs: Supports text, images, audio, and video for highly controlled outputs. ✅High-Quality Output: Industry-leading detail, motion smoothness, and lighting realism. Key Features: 1️⃣Single-Subject Video: Upload an image + text (e.g., “He’s walking a dog”) to create coherent videos with new actions, outfits, and scenes. 2️⃣Multi-Subject Video: Generate videos with multiple subjects (e.g., a man drinking coffee in a cozy room) from separate image inputs. 3️⃣Audio-Driven Video: Sync audio with visuals for talking or singing in any scene—perfect for digital avatars, virtual customer service, and more. 4️⃣Video-Driven Video: Seamlessly insert or replace subjects into any video for creative enhancements. Dive Deeper: 🌐Project: 📝Technical Details: 💻Code: The single-subject video capability is open-sourced and live on the Hunyuan website, with more features to be released soon! 👉 Try it now: #HunyuanCustom #VideoGeneration #AI #TencentHunyuan

Tencent HY

191,908 views • 1 year ago

a short video about "gorge music", a japanese open-source genre for rock climbers and mountaineers. exploring its mythology, IDM influence and how it solves artist ego

a short video about "gorge music", a japanese open-source genre for rock climbers and mountaineers. exploring its mythology, IDM influence and how it solves artist ego

RamonPang

17,317 views • 4 months ago

Say It — private, local text-to-speech for macOS Open models 100% on-device, speak text from any app with a hotkey, and even clone your own voice. Infinite voices. Free & open-source. Listen to the video 🔊

Say It — private, local text-to-speech for macOS Open models 100% on-device, speak text from any app with a hotkey, and even clone your own voice. Infinite voices. Free & open-source. Listen to the video 🔊

calle 🟥

33,788 views • 8 days ago

Japan reports strong growth in open source value and new expectations for support, security, and governance. Watch the highlights and explore the full findings from the 2025 State of Open Source Japan report. Read the full report: #OpenSource #LFResearch #WorldOfOpenSource

Japan reports strong growth in open source value and new expectations for support, security, and governance. Watch the highlights and explore the full findings from the 2025 State of Open Source Japan report. Read the full report: #OpenSource #LFResearch #WorldOfOpenSource

The Linux Foundation

28,543 views • 7 months ago

Holy smokes! AI video just had its DeepSeek moment. LTX Video just got a massive update in both quality & speed. 13B parameters! It is a game-changing moment for AI video and... ...it’s open-source! See for yourself: 1. Fading Glory, a short I made in @LTXStudio

Holy smokes! AI video just had its DeepSeek moment. LTX Video just got a massive update in both quality & speed. 13B parameters! It is a game-changing moment for AI video and... ...it’s open-source! See for yourself: 1. Fading Glory, a short I made in @LTXStudio

Alex Patrascu

90,223 views • 1 year ago

Siri not using Whisper is one of the strangest things in tech. Whisper: open-source, years old, near-perfect speech-to-text. Siri: still mishears half of what you say. Hard to believe the engineers actually dogfood the product.

Siri not using Whisper is one of the strangest things in tech. Whisper: open-source, years old, near-perfect speech-to-text. Siri: still mishears half of what you say. Hard to believe the engineers actually dogfood the product.

AI Breakfast

42,020 views • 5 months ago

LTX-2 is natively supported in ComfyUI on Day 0 🎬🔊 The next chapter in controllability for open-source video generation. - Open-source audio-video foundation model - Generates motion, dialogue, SFX, and music together - Canny, Depth & Pose video-to-video control - Keyframe-driven generation - Native upscaling and prompt enhancement Everything in a single pass.

LTX-2 is natively supported in ComfyUI on Day 0 🎬🔊 The next chapter in controllability for open-source video generation. - Open-source audio-video foundation model - Generates motion, dialogue, SFX, and music together - Canny, Depth & Pose video-to-video control - Keyframe-driven generation - Native upscaling and prompt enhancement Everything in a single pass.

ComfyUI

108,333 views • 7 months ago

Run state-of-the-art RAG applications locally on your computer with ollama and use all the fantastic open-source models like llama3, msk's awesome models, or Command R from Cohere With Verba 1.0, we put it all in your hands 🙌 Get on board for a wild open-source ride, we're bridging any moat as open-source is here to win

Run state-of-the-art RAG applications locally on your computer with ollama and use all the fantastic open-source models like llama3, msk's awesome models, or Command R from Cohere With Verba 1.0, we put it all in your hands 🙌 Get on board for a wild open-source ride, we're bridging any moat as open-source is here to win

Philip Vollet

41,450 views • 2 years ago