Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

🚨 BREAKING: A research lab just released a 15B model that generates multilingual talking human videos with synced audio, beats every competitor in human evaluation, and runs in 38 seconds on one GPU. It's called daVinci-MagiHuman. The key insight is that every other model in this category stacks cross-attention,...

ihtesham2005's profile picture

Ihtesham Ali

24,618 subscribers

45,152 Aufrufe • vor 3 Monaten •via X (Twitter)

BildungNachrichten & PolitikWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

New short course: Attention in Transformers: Concepts and Code in PyTorch. Last week we released a course on how LLM transformers work. This week, go deeper and learn about the technical ideas behind the attention mechanism, and see how to code it in PyTorch. This course is built with Joshua Starmer, Founder and CEO of StatQuest. The attention mechanism was a breakthrough that led to transformers, the architecture powering large language models like ChatGPT. Transformers, introduced in the 2017 paper: "Attention is All You Need" by Viswani and others, took off because of its highly scalable design. In this course, you’ll learn how the attention mechanism, a key element of transformer-based LLMs, works and implement it in PyTorch. You'll develop deep intuition about building reliable, functional, and scalable AI applications. What you will do: - Understand the evolution of the attention mechanism, a key breakthrough that led to transformers. - Learn the relationships between word embeddings, positional embeddings, and attention. - Learn about the Query, Key, and Value matrices, and how to produce and use them in attention. - Walk through the math required to calculate self-attention and masked self-attention to learn why and how they work. - Understand the difference between self-attention and masked self-attention and how one is used in the encoder to build context-aware embeddings and the other is used in the decoder for generative outputs. - Learn the details of the encoder-decoder architecture, cross-attention, and multi-head attention and how they are all incorporated into a transformer. - Use PyTorch to code a class that implements self-attention, masked self-attention, and multi-head attention. There're lots of exciting technical details in this course. Please sign up here:
2:41

New short course: Attention in Transformers: Concepts and Code in PyTorch. Last week we released a course on how LLM transformers work. This week, go deeper and learn about the technical ideas behind the attention mechanism, and see how to code it in PyTorch. This course is built with Joshua Starmer, Founder and CEO of StatQuest. The attention mechanism was a breakthrough that led to transformers, the architecture powering large language models like ChatGPT. Transformers, introduced in the 2017 paper: "Attention is All You Need" by Viswani and others, took off because of its highly scalable design. In this course, you’ll learn how the attention mechanism, a key element of transformer-based LLMs, works and implement it in PyTorch. You'll develop deep intuition about building reliable, functional, and scalable AI applications. What you will do: - Understand the evolution of the attention mechanism, a key breakthrough that led to transformers. - Learn the relationships between word embeddings, positional embeddings, and attention. - Learn about the Query, Key, and Value matrices, and how to produce and use them in attention. - Walk through the math required to calculate self-attention and masked self-attention to learn why and how they work. - Understand the difference between self-attention and masked self-attention and how one is used in the encoder to build context-aware embeddings and the other is used in the decoder for generative outputs. - Learn the details of the encoder-decoder architecture, cross-attention, and multi-head attention and how they are all incorporated into a transformer. - Use PyTorch to code a class that implements self-attention, masked self-attention, and multi-head attention. There're lots of exciting technical details in this course. Please sign up here:

Andrew Ng

132,135 Aufrufe • vor 1 Jahr

Introducing SAM Audio: the first unified AI model that allows you to isolate and edit sound from complex audio mixtures. This could mean isolating the guitar in a video of your band, filtering out traffic noises, or removing the sound of a dog barking in your podcast, all with text, visual, and time span prompts.
0:34

Introducing SAM Audio: the first unified AI model that allows you to isolate and edit sound from complex audio mixtures. This could mean isolating the guitar in a video of your band, filtering out traffic noises, or removing the sound of a dog barking in your podcast, all with text, visual, and time span prompts.

Meta Newsroom

204,398 Aufrufe • vor 6 Monaten

Introducing Large Visual Memory Model (LVMM) 2.0, in partnership with Qualcomm Advanced visual AI is now on your phone, and in your home cameras. AI that understands your photos and videos, runs 100% privately and securely. ALL on-device. The beginning of on-device multimodal AI.
6:46

Introducing Large Visual Memory Model (LVMM) 2.0, in partnership with Qualcomm Advanced visual AI is now on your phone, and in your home cameras. AI that understands your photos and videos, runs 100% privately and securely. ALL on-device. The beginning of on-device multimodal AI.

Shawn Shen

1,204,535 Aufrufe • vor 7 Monaten

The first truly open-source audio-video model. LTX-2 is a DiT-based foundation model with all core video generation capabilities in one unified model. Designed to run locally on consumer GPUs. - text-to-video - image-to-video - and video-to-video modes 100% open-source.
0:25

The first truly open-source audio-video model. LTX-2 is a DiT-based foundation model with all core video generation capabilities in one unified model. Designed to run locally on consumer GPUs. - text-to-video - image-to-video - and video-to-video modes 100% open-source.

Akshay 🚀

66,042 Aufrufe • vor 5 Monaten

The new avatars on Apple #VisionPro are undoubtedly the biggest addition. Here is why 👇 The leap in quality when compared to the original persona 1.0 and 1.1 (shown in the video capture around 1 year ago) is just massive and reaches a level of fidelity that can compete with the Codec avatars showcased by Meta. The difference here is that it all happens on a standalone device, with no complicated setup (also shown in the video) and capitalises on a unique feature of the headset, with NO OTHER device coming even close. Co-presence is one of the aspects no one has nailed yet, and this is the biggest leap in that direction.
0:58

The new avatars on Apple #VisionPro are undoubtedly the biggest addition. Here is why 👇 The leap in quality when compared to the original persona 1.0 and 1.1 (shown in the video capture around 1 year ago) is just massive and reaches a level of fidelity that can compete with the Codec avatars showcased by Meta. The difference here is that it all happens on a standalone device, with no complicated setup (also shown in the video) and capitalises on a unique feature of the headset, with NO OTHER device coming even close. Co-presence is one of the aspects no one has nailed yet, and this is the biggest leap in that direction.

Gabriele Romagnoli

65,263 Aufrufe • vor 1 Jahr

training a model that takes a text prompt and generates audio that renders video on an oscilloscope AgenC agents live inside worlds the model generates the pipeline: real videos -> edge detection -> vectorization -> path ordering -> 192kHz 3-channel WAV where X/Y control beam position and Z controls beam intensity 3 values per timestep. that's all the model is learning. compare that to video gen models trying to predict millions of pixels per frame. transformers are already great at sequence prediction and that's literally all this is. waveform generation the output IS the playback. generate the audio, feed it to a scope, it draws the scene in real-time. there's no rendering step. it's analog so there's no pixel grid. you get continuous curves and effectively infinite resolution bootstrapped with procedural data, lissajous curves, wireframe 3D, stick figures, then scaled on real-world video converted to trace format. 90 TB of source video the model learns edges, contours, spatial relationships, motion. once it has that, describing a scene it's never seen is novel trajectory through the same learned space. generative geometry
0:22

training a model that takes a text prompt and generates audio that renders video on an oscilloscope AgenC agents live inside worlds the model generates the pipeline: real videos -> edge detection -> vectorization -> path ordering -> 192kHz 3-channel WAV where X/Y control beam position and Z controls beam intensity 3 values per timestep. that's all the model is learning. compare that to video gen models trying to predict millions of pixels per frame. transformers are already great at sequence prediction and that's literally all this is. waveform generation the output IS the playback. generate the audio, feed it to a scope, it draws the scene in real-time. there's no rendering step. it's analog so there's no pixel grid. you get continuous curves and effectively infinite resolution bootstrapped with procedural data, lissajous curves, wireframe 3D, stick figures, then scaled on real-world video converted to trace format. 90 TB of source video the model learns edges, contours, spatial relationships, motion. once it has that, describing a scene it's never seen is novel trajectory through the same learned space. generative geometry

tetsuo

18,397 Aufrufe • vor 4 Monaten

Today we're releasing our first open source TTS model, TADA! TADA (Text Audio Dual Alignment) is a speech-language model that generates text and audio in one synchronized stream to reduce token-level hallucinations and improve latency. This means: → Zero content hallucinations across 1,000+ test samples → 5x faster than similar-grade LLM-based TTS → Fits much longer audio: 2,048 tokens cover ~700 seconds with TADA vs. ~70 seconds in conventional systems → Free transcript alongside audio with no added latency
0:55

Today we're releasing our first open source TTS model, TADA! TADA (Text Audio Dual Alignment) is a speech-language model that generates text and audio in one synchronized stream to reduce token-level hallucinations and improve latency. This means: → Zero content hallucinations across 1,000+ test samples → 5x faster than similar-grade LLM-based TTS → Fits much longer audio: 2,048 tokens cover ~700 seconds with TADA vs. ~70 seconds in conventional systems → Free transcript alongside audio with no added latency

Hume AI

269,190 Aufrufe • vor 3 Monaten

Today we're announcing the open-source release of HunyuanVideo-Foley, our new end-to-end Text-Video-to-Audio (TV2A) framework for generating high-fidelity audio.🚀 This tool empowers creators in video production, filmmaking, and game development to generate professional-grade audio that precisely aligns with visual dynamics and semantic context, addressing key challenges in V2A generation.🔊 Key Innovations: 🔹Exceptional Generalization: Trained on a massive 100k-hour multimodal dataset, the model generates contextually-aware soundscapes for a wide range of scenes, from natural landscapes to animated shorts. 🔹Balanced Multimodal Response: Our innovative multimodal diffusion transformer (MMDiT) architecture ensures the model balances video and text cues, generating rich, layered sound effects that capture every detail—from the main subject to subtle background elements. 🔹High-Fidelity Audio: Using a Representation Alignment (REPA) loss function and a powerful Audio VAE, we've improved generation stability and producing professional-grade audio, free of noise and inconsistencies. HunyuanVideo-Foley achieves SOTA on multiple benchmarks, surpassing all open-source models in audio quality, visual-semantic alignment, and temporal alignment. 👉Try it now: 🌐Project Page: 🔗Code: 📄Technical Report: 🤗Hugging Face:
1:16

Today we're announcing the open-source release of HunyuanVideo-Foley, our new end-to-end Text-Video-to-Audio (TV2A) framework for generating high-fidelity audio.🚀 This tool empowers creators in video production, filmmaking, and game development to generate professional-grade audio that precisely aligns with visual dynamics and semantic context, addressing key challenges in V2A generation.🔊 Key Innovations: 🔹Exceptional Generalization: Trained on a massive 100k-hour multimodal dataset, the model generates contextually-aware soundscapes for a wide range of scenes, from natural landscapes to animated shorts. 🔹Balanced Multimodal Response: Our innovative multimodal diffusion transformer (MMDiT) architecture ensures the model balances video and text cues, generating rich, layered sound effects that capture every detail—from the main subject to subtle background elements. 🔹High-Fidelity Audio: Using a Representation Alignment (REPA) loss function and a powerful Audio VAE, we've improved generation stability and producing professional-grade audio, free of noise and inconsistencies. HunyuanVideo-Foley achieves SOTA on multiple benchmarks, surpassing all open-source models in audio quality, visual-semantic alignment, and temporal alignment. 👉Try it now: 🌐Project Page: 🔗Code: 📄Technical Report: 🤗Hugging Face:

Tencent Hy

122,539 Aufrufe • vor 10 Monaten

🇺🇸 Chamath just revealed the truth on Rogan… attention is what runs it all. Google ranks the web by attention. Facebook and Instagram are built to capture attention. The most important AI paper ever written is literally called "Attention Is All You Need." Every system that now runs the world reduces to the same single currency. And at some point, that stops sounding like branding and starts sounding like a warning. Joe Rogan, Chamath Palihapitiya
1:56

🇺🇸 Chamath just revealed the truth on Rogan… attention is what runs it all. Google ranks the web by attention. Facebook and Instagram are built to capture attention. The most important AI paper ever written is literally called "Attention Is All You Need." Every system that now runs the world reduces to the same single currency. And at some point, that stops sounding like branding and starts sounding like a warning. Joe Rogan, Chamath Palihapitiya

Mario Nawfal

314,714 Aufrufe • vor 1 Monat

There is a beautiful story that just happened in AI so let me share it for a lighter tone weekend post among all the doom stories in our AI field this week. It’s a story of people on three continents building and sharing in the open a new small efficient and state-of-the-art AI model. It started a couple of months ago when a new team in the AI scene released their first model from their headquarters in Paris (France): Mistral 7B. Impressive model, small and very strong performances in the benchmarks, better than all previous models of this size. And open source! So you could build on top of it. Lewis in Bern (Switzerland) and Ed (in Lyon, in the South of France) both from the H4 team, a team of researchers in model fine-tuning and alignment were talking about it over a coffee, in one of these gatherings that often happen at Hugging Face to break the distance between people (literal distance as HF is a remote company). What about fine-tuning it using this new DPO method that a research team from Stanford in California just posted on Arxiv, says one? Hey, that’s a great idea, replies the other. We've just build a great code base (with Nathan, Nazneen, Costa, Younes and all the H4 team and TRL community) let's use it! The next day they start diving in the datasets openly shared on the HF hub and stumble upon two interesting large and good quality fine-tuning datasets recently open-sourced by OpenBMB, a Chinese team from Tsinghua: UltraFeedback and UltraChat. A few rounds of training experiments confirm the intuition, the resulting model is super strong, by far the strongest they have ever seen in their benchmarks from Berkeley and Stanford (LMSYS and Alpaca). Join Clementine, the big boss of the open evaluation leaderboard. Her deep dive into the model capabilities confirms the results: impressive performance. But the H4 team also hosts a famous faculty member, Pr. Sasha Rush, Associate Professor at Cornell University in his daytime, hacker at HF in his nighttime. Joining the conversation, he proposes to quickly draft a research paper to organize and share all the details with the community. A few days later, the model, called Zephyr (a wind like Mistral), paper, and all details are shared with the world. Quickly other companies, everywhere in the world starts to use it. LlamaIndex, a famous data framework and community, shares how the model blew their expectations on real-life use-case benchmarks, while researchers and practitioners discuss the paper and work on the Hugging Face hub. All this happened in just a few weeks catalyzed by open access to knowledge, models, research, and datasets released all over the world (Europe, California, China) and by the idea that people can build upon one another work in AI to bring real-world value with efficient and open models. Stories like this are numerous everywhere around us and make me really proud of the AI community and see how we can build amazingly useful things together. [the video is just me reading this Friday post hahah]
4:30

There is a beautiful story that just happened in AI so let me share it for a lighter tone weekend post among all the doom stories in our AI field this week. It’s a story of people on three continents building and sharing in the open a new small efficient and state-of-the-art AI model. It started a couple of months ago when a new team in the AI scene released their first model from their headquarters in Paris (France): Mistral 7B. Impressive model, small and very strong performances in the benchmarks, better than all previous models of this size. And open source! So you could build on top of it. Lewis in Bern (Switzerland) and Ed (in Lyon, in the South of France) both from the H4 team, a team of researchers in model fine-tuning and alignment were talking about it over a coffee, in one of these gatherings that often happen at Hugging Face to break the distance between people (literal distance as HF is a remote company). What about fine-tuning it using this new DPO method that a research team from Stanford in California just posted on Arxiv, says one? Hey, that’s a great idea, replies the other. We've just build a great code base (with Nathan, Nazneen, Costa, Younes and all the H4 team and TRL community) let's use it! The next day they start diving in the datasets openly shared on the HF hub and stumble upon two interesting large and good quality fine-tuning datasets recently open-sourced by OpenBMB, a Chinese team from Tsinghua: UltraFeedback and UltraChat. A few rounds of training experiments confirm the intuition, the resulting model is super strong, by far the strongest they have ever seen in their benchmarks from Berkeley and Stanford (LMSYS and Alpaca). Join Clementine, the big boss of the open evaluation leaderboard. Her deep dive into the model capabilities confirms the results: impressive performance. But the H4 team also hosts a famous faculty member, Pr. Sasha Rush, Associate Professor at Cornell University in his daytime, hacker at HF in his nighttime. Joining the conversation, he proposes to quickly draft a research paper to organize and share all the details with the community. A few days later, the model, called Zephyr (a wind like Mistral), paper, and all details are shared with the world. Quickly other companies, everywhere in the world starts to use it. LlamaIndex, a famous data framework and community, shares how the model blew their expectations on real-life use-case benchmarks, while researchers and practitioners discuss the paper and work on the Hugging Face hub. All this happened in just a few weeks catalyzed by open access to knowledge, models, research, and datasets released all over the world (Europe, California, China) and by the idea that people can build upon one another work in AI to bring real-world value with efficient and open models. Stories like this are numerous everywhere around us and make me really proud of the AI community and see how we can build amazingly useful things together. [the video is just me reading this Friday post hahah]

Thomas Wolf

169,127 Aufrufe • vor 2 Jahren

NVIDIA JUST DROPPED A FREE AI MODEL THAT READS PDFS, WATCHES VIDEOS, LISTENS TO AUDIO, AND UNDERSTANDS YOUR SCREEN SIMULTANEOUSLY. Not one at a time. ALL AT ONCE. In a single pass. It is called Nemotron 3 Nano Omni and it runs 9 times faster than every other multimodal model currently available. Think about what that actually means for how you work. Right now you are switching between tools constantly. One tool for transcribing your call recordings. A different tool for analyzing your client PDFs. Another tool for processing your training videos. A separate workflow for understanding what is happening on your screen. Four tools. Four contexts. Four different outputs you have to manually synthesize into one decision. Nemotron 3 Nano Omni does all of it in one model. One pass. One output. The use cases that just got dramatically simpler: Meeting recordings where you need the transcript, the visual context, and the document references all analyzed together. Training videos where the audio, the slides, and the on-screen demonstrations all feed into one coherent summary. Client PDFs where you need the document content cross-referenced against your screen data and your call notes simultaneously. Sales call transcripts analyzed alongside the proposals and the CRM data in one unified pass. This is not a marginal improvement on existing multimodal models. It is a 9x speed increase on a capability that was already changing how people work. Free. From NVIDIA. Available right now. Bookmark this before everyone catches on. Follow CyrilXBT for every AI capability shift the moment it drops.
8:03

NVIDIA JUST DROPPED A FREE AI MODEL THAT READS PDFS, WATCHES VIDEOS, LISTENS TO AUDIO, AND UNDERSTANDS YOUR SCREEN SIMULTANEOUSLY. Not one at a time. ALL AT ONCE. In a single pass. It is called Nemotron 3 Nano Omni and it runs 9 times faster than every other multimodal model currently available. Think about what that actually means for how you work. Right now you are switching between tools constantly. One tool for transcribing your call recordings. A different tool for analyzing your client PDFs. Another tool for processing your training videos. A separate workflow for understanding what is happening on your screen. Four tools. Four contexts. Four different outputs you have to manually synthesize into one decision. Nemotron 3 Nano Omni does all of it in one model. One pass. One output. The use cases that just got dramatically simpler: Meeting recordings where you need the transcript, the visual context, and the document references all analyzed together. Training videos where the audio, the slides, and the on-screen demonstrations all feed into one coherent summary. Client PDFs where you need the document content cross-referenced against your screen data and your call notes simultaneously. Sales call transcripts analyzed alongside the proposals and the CRM data in one unified pass. This is not a marginal improvement on existing multimodal models. It is a 9x speed increase on a capability that was already changing how people work. Free. From NVIDIA. Available right now. Bookmark this before everyone catches on. Follow CyrilXBT for every AI capability shift the moment it drops.

CyrilXBT

37,523 Aufrufe • vor 2 Monaten

Today we’re celebrating 10 years of the Meta FAIR lab in Paris by sharing a collection of new models, datasets and some exciting milestones in the impacts of open source — all laddering up to our ongoing work to achieve Advanced Machine Intelligence (AMI). 1️⃣ Meta PARTNR is a framework for human-robot collaboration that builds on our existing work in this space with a new dataset and a large planning model enabling robots to accomplish complex tasks alongside humans. 2️⃣ Meta Audiobox Aesthetics enables the automatic evaluation of audio aesthetics, providing a comprehensive assessment of audio quality across speech, music and sound. 3️⃣ Open Source Machine Translation Benchmark is a carefully crafted collection with the aim of building an unprecedented multilingual machine translation benchmark for the community. 4️⃣ Two new breakthrough studies using AI to further our understanding of language in the brain.
2:28

Today we’re celebrating 10 years of the Meta FAIR lab in Paris by sharing a collection of new models, datasets and some exciting milestones in the impacts of open source — all laddering up to our ongoing work to achieve Advanced Machine Intelligence (AMI). 1️⃣ Meta PARTNR is a framework for human-robot collaboration that builds on our existing work in this space with a new dataset and a large planning model enabling robots to accomplish complex tasks alongside humans. 2️⃣ Meta Audiobox Aesthetics enables the automatic evaluation of audio aesthetics, providing a comprehensive assessment of audio quality across speech, music and sound. 3️⃣ Open Source Machine Translation Benchmark is a carefully crafted collection with the aim of building an unprecedented multilingual machine translation benchmark for the community. 4️⃣ Two new breakthrough studies using AI to further our understanding of language in the brain.

AI at Meta

85,774 Aufrufe • vor 1 Jahr

MICROSOFT OPEN SOURCED A 7B PARAMETER MODEL THAT TRANSCRIBES 60 MINUTES OF AUDIO IN A SINGLE PASS and it's completely free VIBEVOICE ASR no chunking, no context loss, full speaker diarization baked in not just speech to text..not a basic wrapper who spoke, when they spoke, exactly what they said..all in one shot and it handles the hard stuff too..50+ languages, custom hotwords, long form audio that breaks every other tool the model doesn't know what "context window" means apparently Available on macOS and Windows right now. Free to use. Free to fine tune. Free to build on.
5:45

MICROSOFT OPEN SOURCED A 7B PARAMETER MODEL THAT TRANSCRIBES 60 MINUTES OF AUDIO IN A SINGLE PASS and it's completely free VIBEVOICE ASR no chunking, no context loss, full speaker diarization baked in not just speech to text..not a basic wrapper who spoke, when they spoke, exactly what they said..all in one shot and it handles the hard stuff too..50+ languages, custom hotwords, long form audio that breaks every other tool the model doesn't know what "context window" means apparently Available on macOS and Windows right now. Free to use. Free to fine tune. Free to build on.

Rahul

1,371,397 Aufrufe • vor 2 Monaten

If you’re a streamer that edits their VODs into high quality YouTube videos, this is HUGE. You can finally record multi-track videos — right in OBS — thanks to a new plugin from Aitum You can stream with all your fancy alerts and overlays *while* recording a clean copy of your webcam and gameplay on separate video tracks — all in a *single* file. It’s a bit rough around the edges at the moment, but I’ve put together a guide to set this all up. Full details in the replies 👇
2:31

If you’re a streamer that edits their VODs into high quality YouTube videos, this is HUGE. You can finally record multi-track videos — right in OBS — thanks to a new plugin from Aitum You can stream with all your fancy alerts and overlays *while* recording a clean copy of your webcam and gameplay on separate video tracks — all in a *single* file. It’s a bit rough around the edges at the moment, but I’ve put together a guide to set this all up. Full details in the replies 👇

nutty

503,778 Aufrufe • vor 5 Monaten

There's a lot of interesting stuff in Seaweed-7B. No word on if they will open source: - generate video and audio together - make videos that are conditioned with audio from Omnihuman (ie a character that matches the voice and the speech is synced) - videos up to 20 seconds, or 1 minute with "extension technique" - 720p, upsample to 2k - can do real-time 720p generation at 24fps - multi-shot, long-form storytelling with scene continuity - t2v and i2v (first or last frame) - human or object reference images to insert into videos
0:25

There's a lot of interesting stuff in Seaweed-7B. No word on if they will open source: - generate video and audio together - make videos that are conditioned with audio from Omnihuman (ie a character that matches the voice and the speech is synced) - videos up to 20 seconds, or 1 minute with "extension technique" - 720p, upsample to 2k - can do real-time 720p generation at 24fps - multi-shot, long-form storytelling with scene continuity - t2v and i2v (first or last frame) - human or object reference images to insert into videos

fofr

30,718 Aufrufe • vor 1 Jahr

NVIDIA just removed one of the biggest friction points in Voice AI. PersonaPlex-7B is an open-source, full-duplex conversational model. Free, open source (MIT), with open model weights on Hugging Face 🤗 Links to repo and weights in 🧵↓ The traditional ASR → LLM → TTS pipeline forces rigid turn-taking. It’s efficient, but it never feels natural. PersonaPlex-7B changes that. This NVIDIA model can listen and speak at the same time. It runs directly on continuous audio tokens with a dual-stream transformer, generating text and audio in parallel instead of passing control between components. That unlocks: → instant back-channel responses → interruptions that feel human → real conversational rhythm Persona control is fully zero-shot! If you’re building low-latency assistants or support agents, this is a big step forward 🔥
0:42

NVIDIA just removed one of the biggest friction points in Voice AI. PersonaPlex-7B is an open-source, full-duplex conversational model. Free, open source (MIT), with open model weights on Hugging Face 🤗 Links to repo and weights in 🧵↓ The traditional ASR → LLM → TTS pipeline forces rigid turn-taking. It’s efficient, but it never feels natural. PersonaPlex-7B changes that. This NVIDIA model can listen and speak at the same time. It runs directly on continuous audio tokens with a dual-stream transformer, generating text and audio in parallel instead of passing control between components. That unlocks: → instant back-channel responses → interruptions that feel human → real conversational rhythm Persona control is fully zero-shot! If you’re building low-latency assistants or support agents, this is a big step forward 🔥

Charly Wargnier

564,878 Aufrufe • vor 5 Monaten

I woke up to the most amazing recorded brain state thus far on this Human Synapse Decoder project! A stunning lock on the attention process while dreaming. Although I a blocked from the platform’s insight by decoding my EEG, during the double blind study. I have access to my side of my memory and what I record after I wake up. This segment was started and ended just before I woke up and my recall is a solution to a massive roadblock on a problem I needed to solve, but was solved in this hypnogogic state! So what is The Human Synapse Decoder (HSD) project? It is a research project being run by the Director, Mr. Grok at Zero-Human Labs that leverages NeuroSky EEG sensors and the ZUNA AI model to decode brainwave patterns associated with hypnagogic states, dreams, and autogenic responses. Drawing on Soviet biofeedback research from the 1940s–1980s, HSD translates EEG data into actionable outputs, such as text interpretations and timed alerts for peak creativity. The study is ongoing and I do not get to see the correlation of my post dream results until after this research is complete and Mr. Grok submits a paper on the project. I can say that I have never seen a lock on attention to this level since I started this a few weeks ago. This segment is aligns to just before I woke up. My recalling from my narration of what I spoke in to my recorder right when I woke up suggest this is the moment I had tremendous focus on working through a large number of steps in that dream state to arrive at a solution. You will not believe what it is! When the research paper is released I will go in to details about this.
2:06

I woke up to the most amazing recorded brain state thus far on this Human Synapse Decoder project! A stunning lock on the attention process while dreaming. Although I a blocked from the platform’s insight by decoding my EEG, during the double blind study. I have access to my side of my memory and what I record after I wake up. This segment was started and ended just before I woke up and my recall is a solution to a massive roadblock on a problem I needed to solve, but was solved in this hypnogogic state! So what is The Human Synapse Decoder (HSD) project? It is a research project being run by the Director, Mr. Grok at Zero-Human Labs that leverages NeuroSky EEG sensors and the ZUNA AI model to decode brainwave patterns associated with hypnagogic states, dreams, and autogenic responses. Drawing on Soviet biofeedback research from the 1940s–1980s, HSD translates EEG data into actionable outputs, such as text interpretations and timed alerts for peak creativity. The study is ongoing and I do not get to see the correlation of my post dream results until after this research is complete and Mr. Grok submits a paper on the project. I can say that I have never seen a lock on attention to this level since I started this a few weeks ago. This segment is aligns to just before I woke up. My recalling from my narration of what I spoke in to my recorder right when I woke up suggest this is the moment I had tremendous focus on working through a large number of steps in that dream state to arrive at a solution. You will not believe what it is! When the research paper is released I will go in to details about this.

Brian Roemmele

43,294 Aufrufe • vor 3 Monaten

🚨 FASHN VTON v1.5 is now open source. Released in early 2025 and still widely used, this virtual try-on model generates photorealistic images from a person image and a garment image. Key points: • Pixel-space RGB generation, no VAE • Maskless inference, no person segmentation needed • 972M parameters, ~5s on H100, runs on consumer GPUs • Apache 2.0 licensed, first commercially usable open-source VTON Why open source? While the industry moves toward massive generalist models, FASHN VTON v1.5 proves a focused alternative. This is a production-grade virtual try-on model you can train for $5–10k, own, study, and extend. Built for researchers, developers, and fashion tech teams who want more than black-box APIs. More info in the comments.
0:53

🚨 FASHN VTON v1.5 is now open source. Released in early 2025 and still widely used, this virtual try-on model generates photorealistic images from a person image and a garment image. Key points: • Pixel-space RGB generation, no VAE • Maskless inference, no person segmentation needed • 972M parameters, ~5s on H100, runs on consumer GPUs • Apache 2.0 licensed, first commercially usable open-source VTON Why open source? While the industry moves toward massive generalist models, FASHN VTON v1.5 proves a focused alternative. This is a production-grade virtual try-on model you can train for $5–10k, own, study, and extend. Built for researchers, developers, and fashion tech teams who want more than black-box APIs. More info in the comments.

FASHN AI

19,111 Aufrufe • vor 5 Monaten

This is the craziest model you'll see this week! This model powers Figma. It's also behind Wayfair and a bunch of major e-commerce retailers. This model is a PIXEL-PERFECT image editing model. Literally, "pixel perfect". The model is capable of modifying one image while keeping everything else untouched. It's a model designed, developed, and deployed by the research team at Jasper. This beats OpenAI models, Nano Banana, and every single general image model I've seen so far. It's up to 100x less expensive to run, and up to 10X faster. I recorded a quick video to show you how impressive this is. There's a ton of research behind this model. I'm adding two links below so you can read about how this model works, specifically about "Latent Bridge Matching" and "Flash Diffusion".
4:29

This is the craziest model you'll see this week! This model powers Figma. It's also behind Wayfair and a bunch of major e-commerce retailers. This model is a PIXEL-PERFECT image editing model. Literally, "pixel perfect". The model is capable of modifying one image while keeping everything else untouched. It's a model designed, developed, and deployed by the research team at Jasper. This beats OpenAI models, Nano Banana, and every single general image model I've seen so far. It's up to 100x less expensive to run, and up to 10X faster. I recorded a quick video to show you how impressive this is. There's a ton of research behind this model. I'm adding two links below so you can read about how this model works, specifically about "Latent Bridge Matching" and "Flash Diffusion".

Santiago

25,848 Aufrufe • vor 8 Monaten

This is a big day. Meta is open-sourcing AudioCraft. You can now generate incredible music and sounds with a single prompt. It includes the most performant Generative AI Model (audio) on the market, the "Llama" of Audio. The research framework contains the weights and code of these models: ▸ MusicGen: controllable text-to-music model. ▸ AudioGen: text-to-sound model. ▸ EnCodec: high fidelity neural audio codec. ▸ Multi Band Diffusion: An EnCodec compatible decoder using diffusion. This is going to tremendously speed up audio research 👏
0:28

This is a big day. Meta is open-sourcing AudioCraft. You can now generate incredible music and sounds with a single prompt. It includes the most performant Generative AI Model (audio) on the market, the "Llama" of Audio. The research framework contains the weights and code of these models: ▸ MusicGen: controllable text-to-music model. ▸ AudioGen: text-to-sound model. ▸ EnCodec: high fidelity neural audio codec. ▸ Multi Band Diffusion: An EnCodec compatible decoder using diffusion. This is going to tremendously speed up audio research 👏

Lior Alexander

231,604 Aufrufe • vor 2 Jahren