Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

ReVISE: Self-Supervised Speech Resynthesis with Visual Input for Universal and Generalized Speech Enhancement abs: project page:

_akhaliq's profile picture

AK

400,551 subscribers

34,949 görüntüleme • 3 yıl önce •via X (Twitter)

EğitimBilim & Teknoloji
LIVE

Anya Rossi• Live Now

Private livecam show

6 Yorum

Alex Volkov (Thursd/AI) profil fotoğrafı
Alex Volkov (Thursd/AI)3 yıl önce

This is so weird haah

RY⚡ profil fotoğrafı
RY⚡3 yıl önce

Wait.. is it..forming the speech just from the visual input? No, right?

GanLike profil fotoğrafı
GanLike3 yıl önce

😁🙀

Clément profil fotoğrafı
Clément3 yıl önce

AHAHAHAHHAHAHHAHAHHA

Reyon Gifto Immanuel profil fotoğrafı
Reyon Gifto Immanuel3 yıl önce

@scispace_ Summarize this

SciSpace profil fotoğrafı
SciSpace3 yıl önce

@_akhaliq @ReyonGifto Here you go 👇 It proposes a unified model, ReVISE, for Generalized Speech Enhancement, with the goal of improving intelligibility, quality, and video synchronization. Results show that ReVISE greatly suppresses noise and improves quality, showing its applicability

Benzer Videolar

📢New paper We are announcing ReVISE, the first universal audio-visual speech enhancement model powered by SSL. paper: demo: w/ Yossi Adi @TalRemez BowenShi Jacob Donley
1:08

📢New paper We are announcing ReVISE, the first universal audio-visual speech enhancement model powered by SSL. paper: demo: w/ Yossi Adi @TalRemez BowenShi Jacob Donley

Wei-Ning Hsu

46,379 görüntüleme • 3 yıl önce

ReBotNet: Fast Real-time Video Enhancement abs: project page:
0:25

ReBotNet: Fast Real-time Video Enhancement abs: project page:

AK

111,249 görüntüleme • 3 yıl önce

Tesla Full Self-Driving (Supervised) 13.2 exits Universal Studios
1:05

Tesla Full Self-Driving (Supervised) 13.2 exits Universal Studios

Whole Mars Catalog

117,562 görüntüleme • 1 yıl önce

Word-As-Image for Semantic Typography abs: project page:
1:46

Word-As-Image for Semantic Typography abs: project page:

AK

385,996 görüntüleme • 3 yıl önce

Mini-Omni 2 understands image, audio and text inputs all via end-to-end voice conversations with users 🔥 > Understands and processes images, speech, and text > Generates real-time speech responses > Supports interruptions during speech Technical Overview: > Concatenates image, audio, and text features for input. > Uses text-guided delayed parallel output for real-time speech > Involves encoder adaptation, modal alignment, and multimodal fine-tuning Best part: MIT licensed ⚡
0:54

Mini-Omni 2 understands image, audio and text inputs all via end-to-end voice conversations with users 🔥 > Understands and processes images, speech, and text > Generates real-time speech responses > Supports interruptions during speech Technical Overview: > Concatenates image, audio, and text features for input. > Uses text-guided delayed parallel output for real-time speech > Involves encoder adaptation, modal alignment, and multimodal fine-tuning Best part: MIT licensed ⚡

Vaibhav (VB) Srivastav

45,409 görüntüleme • 1 yıl önce

vMAP: Vectorised Object Mapping for Neural Field SLAM abs: project page:
2:41

vMAP: Vectorised Object Mapping for Neural Field SLAM abs: project page:

AK

36,137 görüntüleme • 3 yıl önce

SceneScape: Text-Driven Consistent Scene Generation abs: project page: text-driven perpetual view generation -- synthesizing long videos of arbitrary scenes solely from an input text describing the scene and camera poses
0:48

SceneScape: Text-Driven Consistent Scene Generation abs: project page: text-driven perpetual view generation -- synthesizing long videos of arbitrary scenes solely from an input text describing the scene and camera poses

AK

73,258 görüntüleme • 3 yıl önce

Generative Novel View Synthesis with 3D-Aware Diffusion Models abs: project page:
0:24

Generative Novel View Synthesis with 3D-Aware Diffusion Models abs: project page:

AK

304,708 görüntüleme • 3 yıl önce

My speech today on the Universal Credit and Personal Independence Payment Bill
2:15

My speech today on the Universal Credit and Personal Independence Payment Bill

Dr Marie Tidball MP

65,509 görüntüleme • 11 ay önce

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.
4:10

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

AK

290,517 görüntüleme • 3 yıl önce

GeoCode: Interpretable Shape Programs abs: project page:
1:08

GeoCode: Interpretable Shape Programs abs: project page:

AK

1,428,749 görüntüleme • 3 yıl önce

Introducing... Amica! 👩‍🦰🤖 Amica is an open source interface for interactive communication with 3D characters with voice synthesis, speech recognition, visual understanding, and an emotion system. 🔗
0:38

Introducing... Amica! 👩‍🦰🤖 Amica is an open source interface for interactive communication with 3D characters with voice synthesis, speech recognition, visual understanding, and an emotion system. 🔗

Arbius

41,769 görüntüleme • 2 yıl önce

Open-source project Soundstorm (AI generated speech from Google Research) is going to give Elevenlabs a run for it's money: The text-to-speech project specializes in dialogue between multiple parties, and is available on Github:
0:20

Open-source project Soundstorm (AI generated speech from Google Research) is going to give Elevenlabs a run for it's money: The text-to-speech project specializes in dialogue between multiple parties, and is available on Github:

AI Breakfast

326,474 görüntüleme • 3 yıl önce

Single Motion Diffusion abs: project page:
2:15

Single Motion Diffusion abs: project page:

AK

72,767 görüntüleme • 3 yıl önce

🔥 Ming-UniAudio: The 「Nano Banana」moment for speech is here! A single model for universal understanding, generation & free-form editing. First Unified Continuous Tokenizer 「MingTok-Audio」and Unified Und & Gen Speech LLM built on it. First Universal Free-form Speech Editing Model Without Timestamp Condition. First FreeForm Audio Editing Benchmark. 💻 GitHub: 🤗 Tokenizer: 🤗 Model: base: edit: 🤗 Benchmark: 🌍 blog: #AI #Speech #SpeechLLM #LLM #GenerativeAI #Audio #ASR #TTS #SpeechEditing
2:52

🔥 Ming-UniAudio: The 「Nano Banana」moment for speech is here! A single model for universal understanding, generation & free-form editing. First Unified Continuous Tokenizer 「MingTok-Audio」and Unified Und & Gen Speech LLM built on it. First Universal Free-form Speech Editing Model Without Timestamp Condition. First FreeForm Audio Editing Benchmark. 💻 GitHub: 🤗 Tokenizer: 🤗 Model: base: edit: 🤗 Benchmark: 🌍 blog: #AI #Speech #SpeechLLM #LLM #GenerativeAI #Audio #ASR #TTS #SpeechEditing

Ant Ling

21,365 görüntüleme • 8 ay önce

Supervised learning has held 3D Vision back for too long. Meet RayZer — a self-supervised 3D model trained with zero 3D labels: ❌ No supervision of camera & geometry ✅ Just RGB images And the wild part? RayZer outperforms supervised methods (as 3D labels from COLMAP is noisy) 🌐 Project: (1/4)
0:27

Supervised learning has held 3D Vision back for too long. Meet RayZer — a self-supervised 3D model trained with zero 3D labels: ❌ No supervision of camera & geometry ✅ Just RGB images And the wild part? RayZer outperforms supervised methods (as 3D labels from COLMAP is noisy) 🌐 Project: (1/4)

Hanwen Jiang

69,460 görüntüleme • 1 yıl önce

Announcing Universal-2, our most advanced speech-to-text model to date. Building on Universal-1's industry-leading performance, we've pushed Speech AI even further. Here's what's new:
1:01

Announcing Universal-2, our most advanced speech-to-text model to date. Building on Universal-1's industry-leading performance, we've pushed Speech AI even further. Here's what's new:

AssemblyAI

42,966 görüntüleme • 1 yıl önce

MusicLM: Generating Music From Text abs: project page:
2:16

MusicLM: Generating Music From Text abs: project page:

AK

434,142 görüntüleme • 3 yıl önce

Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion abs: project page:
1:22

Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion abs: project page:

AK

262,780 görüntüleme • 3 yıl önce