Загрузка видео...

Не удалось загрузить видео

На главную

ReVISE: Self-Supervised Speech Resynthesis with Visual Input for Universal and Generalized Speech Enhancement abs: project page:

_akhaliq's profile picture

AK

400,551 subscribers

34,949 просмотров • 3 лет назад •via X (Twitter)

ОбразованиеНаука и технологии
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 6

Фото профиля Alex Volkov (Thursd/AI)
Alex Volkov (Thursd/AI)3 лет назад

This is so weird haah

Фото профиля RY⚡
RY⚡3 лет назад

Wait.. is it..forming the speech just from the visual input? No, right?

Фото профиля GanLike
GanLike3 лет назад

😁🙀

Фото профиля Clément
Clément3 лет назад

AHAHAHAHHAHAHHAHAHHA

Фото профиля Reyon Gifto Immanuel
Reyon Gifto Immanuel3 лет назад

@scispace_ Summarize this

Фото профиля SciSpace
SciSpace3 лет назад

@_akhaliq @ReyonGifto Here you go 👇 It proposes a unified model, ReVISE, for Generalized Speech Enhancement, with the goal of improving intelligibility, quality, and video synchronization. Results show that ReVISE greatly suppresses noise and improves quality, showing its applicability

Похожие видео

📢New paper We are announcing ReVISE, the first universal audio-visual speech enhancement model powered by SSL. paper: demo: w/ Yossi Adi @TalRemez BowenShi Jacob Donley
1:08

📢New paper We are announcing ReVISE, the first universal audio-visual speech enhancement model powered by SSL. paper: demo: w/ Yossi Adi @TalRemez BowenShi Jacob Donley

Wei-Ning Hsu

46,382 просмотров • 3 лет назад

ReBotNet: Fast Real-time Video Enhancement abs: project page:
0:25

ReBotNet: Fast Real-time Video Enhancement abs: project page:

AK

111,249 просмотров • 3 лет назад

Tesla Full Self-Driving (Supervised) 13.2 exits Universal Studios
1:05

Tesla Full Self-Driving (Supervised) 13.2 exits Universal Studios

Whole Mars Catalog

117,562 просмотров • 1 год назад

Word-As-Image for Semantic Typography abs: project page:
1:46

Word-As-Image for Semantic Typography abs: project page:

AK

385,996 просмотров • 3 лет назад

Mini-Omni 2 understands image, audio and text inputs all via end-to-end voice conversations with users 🔥 > Understands and processes images, speech, and text > Generates real-time speech responses > Supports interruptions during speech Technical Overview: > Concatenates image, audio, and text features for input. > Uses text-guided delayed parallel output for real-time speech > Involves encoder adaptation, modal alignment, and multimodal fine-tuning Best part: MIT licensed ⚡
0:54

Mini-Omni 2 understands image, audio and text inputs all via end-to-end voice conversations with users 🔥 > Understands and processes images, speech, and text > Generates real-time speech responses > Supports interruptions during speech Technical Overview: > Concatenates image, audio, and text features for input. > Uses text-guided delayed parallel output for real-time speech > Involves encoder adaptation, modal alignment, and multimodal fine-tuning Best part: MIT licensed ⚡

Vaibhav (VB) Srivastav

45,409 просмотров • 1 год назад

vMAP: Vectorised Object Mapping for Neural Field SLAM abs: project page:
2:41

vMAP: Vectorised Object Mapping for Neural Field SLAM abs: project page:

AK

36,137 просмотров • 3 лет назад

SceneScape: Text-Driven Consistent Scene Generation abs: project page: text-driven perpetual view generation -- synthesizing long videos of arbitrary scenes solely from an input text describing the scene and camera poses
0:48

SceneScape: Text-Driven Consistent Scene Generation abs: project page: text-driven perpetual view generation -- synthesizing long videos of arbitrary scenes solely from an input text describing the scene and camera poses

AK

73,258 просмотров • 3 лет назад

Generative Novel View Synthesis with 3D-Aware Diffusion Models abs: project page:
0:24

Generative Novel View Synthesis with 3D-Aware Diffusion Models abs: project page:

AK

304,708 просмотров • 3 лет назад

My speech today on the Universal Credit and Personal Independence Payment Bill
2:15

My speech today on the Universal Credit and Personal Independence Payment Bill

Dr Marie Tidball MP

65,509 просмотров • 11 месяцев назад

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.
4:10

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

AK

290,517 просмотров • 3 лет назад

GeoCode: Interpretable Shape Programs abs: project page:
1:08

GeoCode: Interpretable Shape Programs abs: project page:

AK

1,428,749 просмотров • 3 лет назад

Introducing... Amica! 👩‍🦰🤖 Amica is an open source interface for interactive communication with 3D characters with voice synthesis, speech recognition, visual understanding, and an emotion system. 🔗
0:38

Introducing... Amica! 👩‍🦰🤖 Amica is an open source interface for interactive communication with 3D characters with voice synthesis, speech recognition, visual understanding, and an emotion system. 🔗

Arbius

41,769 просмотров • 2 лет назад

Open-source project Soundstorm (AI generated speech from Google Research) is going to give Elevenlabs a run for it's money: The text-to-speech project specializes in dialogue between multiple parties, and is available on Github:
0:20

Open-source project Soundstorm (AI generated speech from Google Research) is going to give Elevenlabs a run for it's money: The text-to-speech project specializes in dialogue between multiple parties, and is available on Github:

AI Breakfast

326,474 просмотров • 3 лет назад

Single Motion Diffusion abs: project page:
2:15

Single Motion Diffusion abs: project page:

AK

72,767 просмотров • 3 лет назад

Supervised learning has held 3D Vision back for too long. Meet RayZer — a self-supervised 3D model trained with zero 3D labels: ❌ No supervision of camera & geometry ✅ Just RGB images And the wild part? RayZer outperforms supervised methods (as 3D labels from COLMAP is noisy) 🌐 Project: (1/4)
0:27

Supervised learning has held 3D Vision back for too long. Meet RayZer — a self-supervised 3D model trained with zero 3D labels: ❌ No supervision of camera & geometry ✅ Just RGB images And the wild part? RayZer outperforms supervised methods (as 3D labels from COLMAP is noisy) 🌐 Project: (1/4)

Hanwen Jiang

69,526 просмотров • 1 год назад

🔥 Ming-UniAudio: The 「Nano Banana」moment for speech is here! A single model for universal understanding, generation & free-form editing. First Unified Continuous Tokenizer 「MingTok-Audio」and Unified Und & Gen Speech LLM built on it. First Universal Free-form Speech Editing Model Without Timestamp Condition. First FreeForm Audio Editing Benchmark. 💻 GitHub: 🤗 Tokenizer: 🤗 Model: base: edit: 🤗 Benchmark: 🌍 blog: #AI #Speech #SpeechLLM #LLM #GenerativeAI #Audio #ASR #TTS #SpeechEditing
2:52

🔥 Ming-UniAudio: The 「Nano Banana」moment for speech is here! A single model for universal understanding, generation & free-form editing. First Unified Continuous Tokenizer 「MingTok-Audio」and Unified Und & Gen Speech LLM built on it. First Universal Free-form Speech Editing Model Without Timestamp Condition. First FreeForm Audio Editing Benchmark. 💻 GitHub: 🤗 Tokenizer: 🤗 Model: base: edit: 🤗 Benchmark: 🌍 blog: #AI #Speech #SpeechLLM #LLM #GenerativeAI #Audio #ASR #TTS #SpeechEditing

Ant Ling

21,365 просмотров • 8 месяцев назад

Announcing Universal-2, our most advanced speech-to-text model to date. Building on Universal-1's industry-leading performance, we've pushed Speech AI even further. Here's what's new:
1:01

Announcing Universal-2, our most advanced speech-to-text model to date. Building on Universal-1's industry-leading performance, we've pushed Speech AI even further. Here's what's new:

AssemblyAI

42,966 просмотров • 1 год назад

MusicLM: Generating Music From Text abs: project page:
2:16

MusicLM: Generating Music From Text abs: project page:

AK

434,142 просмотров • 3 лет назад

Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion abs: project page:
1:22

Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion abs: project page:

AK

262,780 просмотров • 3 лет назад