Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

ReVISE: Self-Supervised Speech Resynthesis with Visual Input for Universal and Generalized Speech Enhancement abs: project page:

_akhaliq's profile picture

AK

400,551 subscribers

34,949 Aufrufe • vor 3 Jahren •via X (Twitter)

BildungWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

6 Kommentare

Profilbild von Alex Volkov (Thursd/AI)
Alex Volkov (Thursd/AI)vor 3 Jahren

This is so weird haah

Profilbild von RY⚡
RY⚡vor 3 Jahren

Wait.. is it..forming the speech just from the visual input? No, right?

Profilbild von GanLike
GanLikevor 3 Jahren

😁🙀

Profilbild von Clément
Clémentvor 3 Jahren

AHAHAHAHHAHAHHAHAHHA

Profilbild von Reyon Gifto Immanuel
Reyon Gifto Immanuelvor 3 Jahren

@scispace_ Summarize this

Profilbild von SciSpace
SciSpacevor 3 Jahren

@_akhaliq @ReyonGifto Here you go 👇 It proposes a unified model, ReVISE, for Generalized Speech Enhancement, with the goal of improving intelligibility, quality, and video synchronization. Results show that ReVISE greatly suppresses noise and improves quality, showing its applicability

Ähnliche Videos

📢New paper We are announcing ReVISE, the first universal audio-visual speech enhancement model powered by SSL. paper: demo: w/ Yossi Adi @TalRemez BowenShi Jacob Donley
1:08

📢New paper We are announcing ReVISE, the first universal audio-visual speech enhancement model powered by SSL. paper: demo: w/ Yossi Adi @TalRemez BowenShi Jacob Donley

Wei-Ning Hsu

46,379 Aufrufe • vor 3 Jahren

ReBotNet: Fast Real-time Video Enhancement abs: project page:
0:25

ReBotNet: Fast Real-time Video Enhancement abs: project page:

AK

111,249 Aufrufe • vor 3 Jahren

Tesla Full Self-Driving (Supervised) 13.2 exits Universal Studios
1:05

Tesla Full Self-Driving (Supervised) 13.2 exits Universal Studios

Whole Mars Catalog

117,562 Aufrufe • vor 1 Jahr

Word-As-Image for Semantic Typography abs: project page:
1:46

Word-As-Image for Semantic Typography abs: project page:

AK

385,996 Aufrufe • vor 3 Jahren

Mini-Omni 2 understands image, audio and text inputs all via end-to-end voice conversations with users 🔥 > Understands and processes images, speech, and text > Generates real-time speech responses > Supports interruptions during speech Technical Overview: > Concatenates image, audio, and text features for input. > Uses text-guided delayed parallel output for real-time speech > Involves encoder adaptation, modal alignment, and multimodal fine-tuning Best part: MIT licensed ⚡
0:54

Mini-Omni 2 understands image, audio and text inputs all via end-to-end voice conversations with users 🔥 > Understands and processes images, speech, and text > Generates real-time speech responses > Supports interruptions during speech Technical Overview: > Concatenates image, audio, and text features for input. > Uses text-guided delayed parallel output for real-time speech > Involves encoder adaptation, modal alignment, and multimodal fine-tuning Best part: MIT licensed ⚡

Vaibhav (VB) Srivastav

45,409 Aufrufe • vor 1 Jahr

SceneScape: Text-Driven Consistent Scene Generation abs: project page: text-driven perpetual view generation -- synthesizing long videos of arbitrary scenes solely from an input text describing the scene and camera poses
0:48

SceneScape: Text-Driven Consistent Scene Generation abs: project page: text-driven perpetual view generation -- synthesizing long videos of arbitrary scenes solely from an input text describing the scene and camera poses

AK

73,258 Aufrufe • vor 3 Jahren

Generative Novel View Synthesis with 3D-Aware Diffusion Models abs: project page:
0:24

Generative Novel View Synthesis with 3D-Aware Diffusion Models abs: project page:

AK

304,708 Aufrufe • vor 3 Jahren

My speech today on the Universal Credit and Personal Independence Payment Bill
2:15

My speech today on the Universal Credit and Personal Independence Payment Bill

Dr Marie Tidball MP

65,509 Aufrufe • vor 11 Monaten

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.
4:10

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

AK

290,517 Aufrufe • vor 2 Jahren

GeoCode: Interpretable Shape Programs abs: project page:
1:08

GeoCode: Interpretable Shape Programs abs: project page:

AK

1,428,749 Aufrufe • vor 3 Jahren

Introducing... Amica! 👩‍🦰🤖 Amica is an open source interface for interactive communication with 3D characters with voice synthesis, speech recognition, visual understanding, and an emotion system. 🔗
0:38

Introducing... Amica! 👩‍🦰🤖 Amica is an open source interface for interactive communication with 3D characters with voice synthesis, speech recognition, visual understanding, and an emotion system. 🔗

Arbius

41,769 Aufrufe • vor 2 Jahren

Open-source project Soundstorm (AI generated speech from Google Research) is going to give Elevenlabs a run for it's money: The text-to-speech project specializes in dialogue between multiple parties, and is available on Github:
0:20

Open-source project Soundstorm (AI generated speech from Google Research) is going to give Elevenlabs a run for it's money: The text-to-speech project specializes in dialogue between multiple parties, and is available on Github:

AI Breakfast

326,474 Aufrufe • vor 3 Jahren

Single Motion Diffusion abs: project page:
2:15

Single Motion Diffusion abs: project page:

AK

72,767 Aufrufe • vor 3 Jahren

🔥 Ming-UniAudio: The 「Nano Banana」moment for speech is here! A single model for universal understanding, generation & free-form editing. First Unified Continuous Tokenizer 「MingTok-Audio」and Unified Und & Gen Speech LLM built on it. First Universal Free-form Speech Editing Model Without Timestamp Condition. First FreeForm Audio Editing Benchmark. 💻 GitHub: 🤗 Tokenizer: 🤗 Model: base: edit: 🤗 Benchmark: 🌍 blog: #AI #Speech #SpeechLLM #LLM #GenerativeAI #Audio #ASR #TTS #SpeechEditing
2:52

🔥 Ming-UniAudio: The 「Nano Banana」moment for speech is here! A single model for universal understanding, generation & free-form editing. First Unified Continuous Tokenizer 「MingTok-Audio」and Unified Und & Gen Speech LLM built on it. First Universal Free-form Speech Editing Model Without Timestamp Condition. First FreeForm Audio Editing Benchmark. 💻 GitHub: 🤗 Tokenizer: 🤗 Model: base: edit: 🤗 Benchmark: 🌍 blog: #AI #Speech #SpeechLLM #LLM #GenerativeAI #Audio #ASR #TTS #SpeechEditing

Ant Ling

21,365 Aufrufe • vor 8 Monaten

Supervised learning has held 3D Vision back for too long. Meet RayZer — a self-supervised 3D model trained with zero 3D labels: ❌ No supervision of camera & geometry ✅ Just RGB images And the wild part? RayZer outperforms supervised methods (as 3D labels from COLMAP is noisy) 🌐 Project: (1/4)
0:27

Supervised learning has held 3D Vision back for too long. Meet RayZer — a self-supervised 3D model trained with zero 3D labels: ❌ No supervision of camera & geometry ✅ Just RGB images And the wild part? RayZer outperforms supervised methods (as 3D labels from COLMAP is noisy) 🌐 Project: (1/4)

Hanwen Jiang

69,460 Aufrufe • vor 1 Jahr

MusicLM: Generating Music From Text abs: project page:
2:16

MusicLM: Generating Music From Text abs: project page:

AK

434,142 Aufrufe • vor 3 Jahren

Announcing Universal-2, our most advanced speech-to-text model to date. Building on Universal-1's industry-leading performance, we've pushed Speech AI even further. Here's what's new:
1:01

Announcing Universal-2, our most advanced speech-to-text model to date. Building on Universal-1's industry-leading performance, we've pushed Speech AI even further. Here's what's new:

AssemblyAI

42,966 Aufrufe • vor 1 Jahr

Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion abs: project page:
1:22

Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion abs: project page:

AK

262,780 Aufrufe • vor 3 Jahren

Meet Hibiki, our simultaneous speech-to-speech translation model, currently supporting 🇫🇷➡️🇬🇧. Hibiki produces spoken and text translations of the input speech in real-time, while preserving the speaker’s voice and optimally adapting its pace based on the semantic content of the source speech. Based on objective and human evaluations, Hibiki outperforms previous systems for quality, naturalness and speaker similarity and approaches human interpreters. 🧵
1:29

Meet Hibiki, our simultaneous speech-to-speech translation model, currently supporting 🇫🇷➡️🇬🇧. Hibiki produces spoken and text translations of the input speech in real-time, while preserving the speaker’s voice and optimally adapting its pace based on the semantic content of the source speech. Based on objective and human evaluations, Hibiki outperforms previous systems for quality, naturalness and speaker similarity and approaches human interpreters. 🧵

kyutai

167,341 Aufrufe • vor 1 Jahr