Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

We found a hack to make Whisper transcribe into any language (not just English)! ⚡️ This is an EXPERIMENTAL one-step feature: enter your audio as input, and get a transcription in another language. 🔥 Check it out here:

Vaibhav (VB) Srivastav

51,936 subscribers

74,525 Aufrufe • vor 3 Jahren •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

10 Kommentare

Profilbild von Vaibhav (VB) Srivastav

Vaibhav (VB) Srivastavvor 3 Jahren

Quick caveat: It's a task that the model isn't explicitly trained on, so test it thoroughly before us 🤗 We'll run more benchmarks this week and share our findings soon! Help us test it better? :)

Profilbild von Vaibhav (VB) Srivastav

Vaibhav (VB) Srivastavvor 3 Jahren

The audio snippet was taken from "Stupid People" comedy routine by Bill Engvall The video was generated via @mhollemans's brilliant Word-level timestamps space:

Profilbild von Eklavya

Eklavyavor 3 Jahren

It could already transcribe into 96 languages. You meant translate.

Profilbild von Vaibhav (VB) Srivastav

Vaibhav (VB) Srivastavvor 3 Jahren

Ha! Semantics - in this case we literally force it to transcribe into a target language. So, technically you can say it is translating! Whisper traditionally only translates from language “X” into English.

Profilbild von Ondřej Plátek

Ondřej Plátekvor 3 Jahren

Cool work! You might also like the work of my colleagues which work on streaming ASR/translation

Profilbild von Kolin Koehl

Kolin Koehlvor 3 Jahren

@altryne have you tried this?

Profilbild von VoiceSparks

VoiceSparksvor 3 Jahren

Thats the way Whisper (not the translation api) works. If you specify a language and speak in another it translates to the specified language. Yes, it's undocumented, but what "hack" do you mean?

Profilbild von Vaibhav (VB) Srivastav

Vaibhav (VB) Srivastavvor 3 Jahren

In my experience those results were quite suboptimal and didn’t quite result in usable transcriptions. With better decoding strategy those issues can be alleviated a bit. So the hack is more so of using contrastive search with Whisper to enable such use cases. Will run more tests soon!

Profilbild von Rober Dam

Rober Damvor 3 Jahren

When you selected portuguese and speak spanish the 2022 original versión do a very good translation, undocumented feature?

Profilbild von Vaibhav (VB) Srivastav

Vaibhav (VB) Srivastavvor 3 Jahren

Yess! Quite possible - although the accuracy of the transcriptions still needs to be checked. I’ve found it to loose context if not used with the right generation strategy! 🤗

Ähnliche Videos

Get licensed and trained for a CDL in any language so you can enter an industry where you're required to speak and understand English. What sense does that make?

Get licensed and trained for a CDL in any language so you can enter an industry where you're required to speak and understand English. What sense does that make?

Rob Carpenter

11,519 Aufrufe • vor 6 Monaten

Honorable please check out our work. We are ready to put this in parliament, we just need honest partners. Attached is an example of converting Twi audio into English - so you can speak your language and everyone else still can follow More examples on my profile

Paul Azunre

98,857 Aufrufe • vor 1 Jahr

#WATCH | While interacting with students in Rae Bareli today, Congress MP Rahul Gandhi said, "People from BJP-RSS say that one should not learn the English language. Mohan Bhagwat says we should not speak in English. But the English language is a weapon, if you learn this language, you can go anywhere, be it Tamil Nadu, Japan or Mumbai or work in any company...They want you not to learn English because where this language is used, they don't want Dalits, Adivasis and the poor not to come there. But English is your biggest weapon...Hindi is also important, it is not right to cut your roots...But English is also very important."

#WATCH | While interacting with students in Rae Bareli today, Congress MP Rahul Gandhi said, "People from BJP-RSS say that one should not learn the English language. Mohan Bhagwat says we should not speak in English. But the English language is a weapon, if you learn this language, you can go anywhere, be it Tamil Nadu, Japan or Mumbai or work in any company...They want you not to learn English because where this language is used, they don't want Dalits, Adivasis and the poor not to come there. But English is your biggest weapon...Hindi is also important, it is not right to cut your roots...But English is also very important."

ANI

291,805 Aufrufe • vor 1 Jahr

A person recording this is speaking in a different language Any Danish or Dutch out there that can verify this is not German but another language thank you. Candace Owens

A person recording this is speaking in a different language Any Danish or Dutch out there that can verify this is not German but another language thank you. Candace Owens

Alley Files

34,086 Aufrufe • vor 26 Tagen

⚡️ introducing our new synchronizer api it lets you lip-sync a video to an audio in any language why is this useful? check out the gladiator in seamless hindi, chinese, spanish, and japanese. curious? we’ll let you beta test at cost ($1/min generated). link below 👇

⚡️ introducing our new synchronizer api it lets you lip-sync a video to an audio in any language why is this useful? check out the gladiator in seamless hindi, chinese, spanish, and japanese. curious? we’ll let you beta test at cost ($1/min generated). link below 👇

Prady

39,910 Aufrufe • vor 3 Jahren

Hindi is not India’s national language. When you come to Bengaluru, you find signboards and route maps in English so that you can navigate and communicate easily. When you visit another state, is it really reasonable to expect people to conduct their affairs in your language? Respect the local language or at least learn English, which serves as a common link language. When Kannadigas visit your states, do we demand that everyone speak Kannada? Respect works both ways. Learn, adapt, and respect the local language. #NammaBengaluru #PeacefulBengaluru

Hindi is not India’s national language. When you come to Bengaluru, you find signboards and route maps in English so that you can navigate and communicate easily. When you visit another state, is it really reasonable to expect people to conduct their affairs in your language? Respect the local language or at least learn English, which serves as a common link language. When Kannadigas visit your states, do we demand that everyone speak Kannada? Respect works both ways. Learn, adapt, and respect the local language. #NammaBengaluru #PeacefulBengaluru

ಸನಾತನ (सनातन)

69,455 Aufrufe • vor 11 Tagen

Lady in pink who speaks English as her third language Says the statement from Reform UK's Nigel Farage criticising English as a second language for school children is "bang out of order" "Because you don't need English as your first language to communicate" #BBCQT

Lady in pink who speaks English as her third language Says the statement from Reform UK's Nigel Farage criticising English as a second language for school children is "bang out of order" "Because you don't need English as your first language to communicate" #BBCQT

Farrukh

332,445 Aufrufe • vor 8 Monaten

New in Runway, you can now localize ads. One image in, any language out. Input a single ad and get a version for every market. All with a single click.

New in Runway, you can now localize ads. One image in, any language out. Input a single ad and get a version for every market. All with a single click.

Runway

64,165 Aufrufe • vor 2 Monaten

AI transcribing 1 hour of audio in less than 30 seconds! Think about that! That's lightning-fast, especially when it can do that with very few mistakes. The team Deepgram released their new Nova 2 model. It's another leap forward: • 36% more accurate than OpenAI Whisper • 5-40x faster than every other alternative • 30% fewer errors than the competitors in real-time transcriptions I like to travel and can't wait for headphones with integrated live transcription and translation. We are getting close! By the way, this is the cheapest transcription service out in the market now (if you discount Whisper, which is open-source.) It costs $0.0043 to transcribe a minute of audio. When you sign up to access their API, you get $200 of credit for free, which will let you transcribe around 45,000 minutes of audio! Here is the link: I wanted to record a quick video showing their live transcription in Spanish and English. You can see it attached here. This post was sponsored by Deepgram.

AI transcribing 1 hour of audio in less than 30 seconds! Think about that! That's lightning-fast, especially when it can do that with very few mistakes. The team Deepgram released their new Nova 2 model. It's another leap forward: • 36% more accurate than OpenAI Whisper • 5-40x faster than every other alternative • 30% fewer errors than the competitors in real-time transcriptions I like to travel and can't wait for headphones with integrated live transcription and translation. We are getting close! By the way, this is the cheapest transcription service out in the market now (if you discount Whisper, which is open-source.) It costs $0.0043 to transcribe a minute of audio. When you sign up to access their API, you get $200 of credit for free, which will let you transcribe around 45,000 minutes of audio! Here is the link: I wanted to record a quick video showing their live transcription in Spanish and English. You can see it attached here. This post was sponsored by Deepgram.

Santiago

354,297 Aufrufe • vor 2 Jahren

#WATCH | Delhi: Union Home Minister Amit Shah says, "There is no opposition to any language; there should not be opposition to any foreign language, but the insistence should be to glorify our language, there should be an insistence to speak our language, and there should be an insistence to think in our language..." "We should get rid of the mentality of slavery. And until a person takes pride in his language, does not express himself in his language, we cannot be free from the mentality of slavery..."

#WATCH | Delhi: Union Home Minister Amit Shah says, "There is no opposition to any language; there should not be opposition to any foreign language, but the insistence should be to glorify our language, there should be an insistence to speak our language, and there should be an insistence to think in our language..." "We should get rid of the mentality of slavery. And until a person takes pride in his language, does not express himself in his language, we cannot be free from the mentality of slavery..."

ANI

33,193 Aufrufe • vor 1 Jahr

English Language actually Damages your Brain to a Great Extent As All the Knowledge of Your Thousands of lineages of Ancestors stored in the DNA is Sanskrit & local language made from it. Not the Broken language that was meant to make you dumb & Establish control Dr. Shah

English Language actually Damages your Brain to a Great Extent As All the Knowledge of Your Thousands of lineages of Ancestors stored in the DNA is Sanskrit & local language made from it. Not the Broken language that was meant to make you dumb & Establish control Dr. Shah

Susmit🔥AS Quotes🌌🌏🕉️

15,031 Aufrufe • vor 6 Monaten

We just brought flow maps to language modeling for one-step sequence generation 💥 Discrete diffusion is not necessary -- continuous flows over one-hot encodings achieve SoTA performance and ≥8.3× faster generation 🔥 We believe this is a major step forward for discrete generative modeling and language modeling alike. 🚀 Full thread from first author Chanhyuk Lee:

We just brought flow maps to language modeling for one-step sequence generation 💥 Discrete diffusion is not necessary -- continuous flows over one-hot encodings achieve SoTA performance and ≥8.3× faster generation 🔥 We believe this is a major step forward for discrete generative modeling and language modeling alike. 🚀 Full thread from first author Chanhyuk Lee:

Nicholas Boffi

44,416 Aufrufe • vor 6 Monaten

Landman meets this head on in Real life Situations and dismantles the Pronoun Debate using a simple English language reality check: Ainsley: “Using a plural pronoun for one person is just kind of incorrect.” TCU Counselor: “Says who” Ainsley: “Well, the English language.”

Landman meets this head on in Real life Situations and dismantles the Pronoun Debate using a simple English language reality check: Ainsley: “Using a plural pronoun for one person is just kind of incorrect.” TCU Counselor: “Says who” Ainsley: “Well, the English language.”

Johnny Midnight ⚡️

96,293 Aufrufe • vor 7 Monaten

"We ought to be able to press 1 for English and get somebody on the other end that English is their native tongue. That they understand and speak the language of THIS LAND." President Trump made English the official language of this great country, it is time for everyone to get on board or GET OUT!

"We ought to be able to press 1 for English and get somebody on the other end that English is their native tongue. That they understand and speak the language of THIS LAND." President Trump made English the official language of this great country, it is time for everyone to get on board or GET OUT!

AmericanPapaBear™

11,255 Aufrufe • vor 9 Monaten

#LanguageWar | We need a two language policy here Kannada and English and not a three language one according to #NEP: ಸಜಿತ್, Activist Prashanth G S and Bhavya Narasimhamurthy face off Harish Upadhya | #SimplySouth #Education #Kannada #TamilNadu

#LanguageWar | We need a two language policy here Kannada and English and not a three language one according to #NEP: ಸಜಿತ್, Activist Prashanth G S and Bhavya Narasimhamurthy face off Harish Upadhya | #SimplySouth #Education #Kannada #TamilNadu

News18

84,561 Aufrufe • vor 1 Jahr

We just got backed by a16z speedrun. We believe the next interface is natural language. We call it the Language User Interface (LUI) An interface that talks to your APIs, understands your product, and lets users get work done with natural language (thread below)

We just got backed by a16z speedrun. We believe the next interface is natural language. We call it the Language User Interface (LUI) An interface that talks to your APIs, understands your product, and lets users get work done with natural language (thread below)

jai

135,935 Aufrufe • vor 1 Jahr

'This is a symptom of the fact that we have more people in our schools whose English is not their first language.' Fred Kelly discusses new figures revealing that the cost of teaching English as a second language has hit £500m.

'This is a symptom of the fact that we have more people in our schools whose English is not their first language.' Fred Kelly discusses new figures revealing that the cost of teaching English as a second language has hit £500m.

GB News

10,988 Aufrufe • vor 7 Monaten

We don’t talk about this enough. Dear kids from village schools who had to learn English through reading books as their English teacher taught English in vernacular. It is normal to lack confidence or be uncomfortable with having to speak the language …get yourself a friend who is willing to help you speak the language daily.

We don’t talk about this enough. Dear kids from village schools who had to learn English through reading books as their English teacher taught English in vernacular. It is normal to lack confidence or be uncomfortable with having to speak the language …get yourself a friend who is willing to help you speak the language daily.

Village Lawyer ⚖️🇿🇦👆

48,626 Aufrufe • vor 5 Monaten