Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

We are getting closer to “Her” where conversation is the new interface. Siri couldn’t do it, so I built an e-mail summarizing feature using #GPT3 and life-like #AI generated voice on iOS. (🔈Audio on to be 🤯with voice realism!) How did I do this? 👇

Justin Alvey

17,660 subscribers

204,236 views • 3 years ago •via X (Twitter)

Science & Technology #GPT3 #AI

Anya Rossi• Live Now

Private livecam show

10 Comments

Justin Alvey3 years ago

I used the Gmail API to feed in recent unread e-mails into a prompt and send to the @OpenAI #GPT3 Completion API. Calling out details such as not “just reading them out” and other prompt tweaks gave good results

Justin Alvey3 years ago

Here are the settings I used, you can see how #GPT3 does a great job of conversationally summarizing. (For the sake of privacy I made up the e-mails shown in the demo)

Justin Alvey3 years ago

The audio model was fine-tuned on speech from the movie Her. I got good results with TorToiSe, but have also experimented with ViTS & YourTTS from @coqui_ai and more recently @ElevenLabs. None are fast enough for a snappy response together with da-vinci-003 completions, so...

Johnny Utah3 years ago

It’s crazy to think Amazon and apple are two of the largest tech companies in the world yet interacting with Alexa or Siri is 99% trash

Philipp Tsipman3 years ago

1) this is 🔥🔥 and 2) your emails are a lot more fun than my emails! 😅😅

Michael Pichardo3 years ago

Is there a simpler tutorial than just his tweet storm on how to build this?

Rational Pear3 years ago

Can you make one to help me choose a Netflix title to watch, cuz that shit gets tedious after 20-30mina of browsing?

Nathan Odle3 years ago

Good idea to pre generate audio. What sort of latency are you getting from your GPT-3 call? You might be able to afford real time speech generation if you move to local FLAN-T5-XXL which fits in 8-bit on a 24GB card. I’m moving to this strategy for my own assistant stuff.

Alexander Kyte3 years ago

We created the torment nexus from the classic tale “don’t create the torment nexus”

Writer nr. 6123 years ago

Amazing! The prompt engineering you did to get this output is just great; something to learn from!

Related Videos

We are getting closer to “Her” (part 2!) Conversationally do anything with emails, using LLM chaining & few-shot prompting for tool use ( inspired) This is now realtime (ish), thanks to #OpenAI gpt-3.5-turbo 🔈 on for voice realism! 🧵

We are getting closer to “Her” (part 2!) Conversationally do anything with emails, using LLM chaining & few-shot prompting for tool use ( inspired) This is now realtime (ish), thanks to #OpenAI gpt-3.5-turbo 🔈 on for voice realism! 🧵

Justin Alvey

481,156 views • 3 years ago

I built a service desk agent using ElevenLabs’ new Conversational AI Agents feature. Watch the video to see how responsive it is! Previously, I used ElevenLabs for cloning my voice and used my generated voice for narrations in some of my youtube videos. This feature takes elevenlabs' voice AI to a whole new level! It simplifies systems that used to require separate TTS (text-to-speech) and STT (speech-to-text) processes for both sides of the conversation. Now, it’s much simpler! Create your own agent here What will you create with this?

I built a service desk agent using ElevenLabs’ new Conversational AI Agents feature. Watch the video to see how responsive it is! Previously, I used ElevenLabs for cloning my voice and used my generated voice for narrations in some of my youtube videos. This feature takes elevenlabs' voice AI to a whole new level! It simplifies systems that used to require separate TTS (text-to-speech) and STT (speech-to-text) processes for both sides of the conversation. Now, it’s much simpler! Create your own agent here What will you create with this?

Melvin Vivas

27,722 views • 1 year ago

LLaMA voice chat + Siri TTS This example is now truly 100% offline since we are now using the built-in Siri text-to-speech available on MacOS through the "say" command

LLaMA voice chat + Siri TTS This example is now truly 100% offline since we are now using the built-in Siri text-to-speech available on MacOS through the "say" command

Georgi Gerganov

412,007 views • 3 years ago

I built an AI voice agent that called the DMV to schedule an appointment for me! ....and you can too. It took <30 minutes and two tools. (And, it was incredibly satisfying to hear my AI deal with the DMV's AI instead of sitting on the phone myself😂) How I did it 👇

I built an AI voice agent that called the DMV to schedule an appointment for me! ....and you can too. It took <30 minutes and two tools. (And, it was incredibly satisfying to hear my AI deal with the DMV's AI instead of sitting on the phone myself😂) How I did it 👇

Olivia Moore

62,403 views • 2 years ago

Was very anxious to post this, but considering both how much I loved this scene and my interest in voice acting, I wanted to do my own take on it! Obviously, this is not an impression, just my own personal take with my own aggravated voice I did for it!

Was very anxious to post this, but considering both how much I loved this scene and my interest in voice acting, I wanted to do my own take on it! Obviously, this is not an impression, just my own personal take with my own aggravated voice I did for it!

BrushyWizard

241,911 views • 5 months ago

people always ask me how they can create long, consistent AI videos with audio. well creatify aurora solves a big part of that and there are some new models coming soon too for more action + audio. I created his voice with dia tts which i think great at capturing emotional speeches. then I cloned his voice using minimax voice clone and use it on minimax tts 2.6. here is the my full conversation with claude for images and text to speech inputs:

people always ask me how they can create long, consistent AI videos with audio. well creatify aurora solves a big part of that and there are some new models coming soon too for more action + audio. I created his voice with dia tts which i think great at capturing emotional speeches. then I cloned his voice using minimax voice clone and use it on minimax tts 2.6. here is the my full conversation with claude for images and text to speech inputs:

gokaygokay

38,341 views • 8 months ago

“oh i forgot to do this” the mannerism and her voice what how did she mimick emma so accurately 😭😭

“oh i forgot to do this” the mannerism and her voice what how did she mimick emma so accurately 😭😭

fran 🎧

57,562 views • 9 months ago

Despite not being Natalia Dyer, Nancy's voice is SOOO GOOD OMG. I am so happy right now. And before anyone comments... her new face is confirmed to be getting an update to be closer to Natalia, so shush!

Despite not being Natalia Dyer, Nancy's voice is SOOO GOOD OMG. I am so happy right now. And before anyone comments... her new face is confirmed to be getting an update to be closer to Natalia, so shush!

۟

194,437 views • 7 months ago

My favorite new complaint on my videos is people thinking I'm using an AI Generated voice. I think some people are just used to YouTubers being like "heyyyyyy whattsupppitsyaboy ummm, Colt ummmm with uhhh..." Even though I'm on camera in my content! LOL

My favorite new complaint on my videos is people thinking I'm using an AI Generated voice. I think some people are just used to YouTubers being like "heyyyyyy whattsupppitsyaboy ummm, Colt ummmm with uhhh..." Even though I'm on camera in my content! LOL

colteastwood

33,921 views • 1 year ago

Sophie Nélisse on how welcoming and at ease she felt working alongside Hudson Williams on ‘Heated Rivalry’ (via matinastevis) “I owe it also a lot to Hudson. He has this kind of voice that I always said that I kinda want him to be on my Headspace app and be the meditation audio guide that I listen to when going to sleep. And that voice is the same way how he is with people; he is so generous, he is so calming, he is so present.”

Sophie Nélisse on how welcoming and at ease she felt working alongside Hudson Williams on ‘Heated Rivalry’ (via matinastevis) “I owe it also a lot to Hudson. He has this kind of voice that I always said that I kinda want him to be on my Headspace app and be the meditation audio guide that I listen to when going to sleep. And that voice is the same way how he is with people; he is so generous, he is so calming, he is so present.”

Hudson Williams Updates

163,075 views • 7 months ago

One of these voice clones *actually* sounds like the original. That’s because Mirage audio generates more expressive output than other models on the market. Our research team built Mirage audio to create more realistic voices. Launching today, this new model solves 3 common challenges in AI audio: 1. It preserves the original speaker’s accent 2. The performance feels dynamic, not flat 3. Delivery matches how people really speak Thanks to our frontier AI research, our voice clone feature preserves the little details that make your voice unique. So your voice actually sounds like you.

One of these voice clones actually sounds like the original. That’s because Mirage audio generates more expressive output than other models on the market. Our research team built Mirage audio to create more realistic voices. Launching today, this new model solves 3 common challenges in AI audio: 1. It preserves the original speaker’s accent 2. The performance feels dynamic, not flat 3. Delivery matches how people really speak Thanks to our frontier AI research, our voice clone feature preserves the little details that make your voice unique. So your voice actually sounds like you.

Mirage

14,914 views • 8 months ago

I just deployed a token on Abstract using only my voice, using AI agents. I built an open-source Discord bot that controls a wallet, can perform on-chain actions, and replies to users in the voice chat. Here's a demo:

I just deployed a token on Abstract using only my voice, using AI agents. I built an open-source Discord bot that controls a wallet, can perform on-chain actions, and replies to users in the voice chat. Here's a demo:

Jarrod Watts

131,616 views • 1 year ago

I just built an AI ghostwriter that is designed to mimic your unique voice (and no it won’t sound like an AI wrote it) here’s how it works: - you give it a tweet request - using NLP we finds similar tweets that you have previously posted about the same topic - the AI learns the topic, your content structure, your vocabulary, everything - we returns a tweet in the same fashion as if you wrote it yourself I recorded a walkthrough on how to build this yourself (including the code). reply to this tweet w/ the word “content” & I’ll send it to you (must be following so I can DM)

I just built an AI ghostwriter that is designed to mimic your unique voice (and no it won’t sound like an AI wrote it) here’s how it works: - you give it a tweet request - using NLP we finds similar tweets that you have previously posted about the same topic - the AI learns the topic, your content structure, your vocabulary, everything - we returns a tweet in the same fashion as if you wrote it yourself I recorded a walkthrough on how to build this yourself (including the code). reply to this tweet w/ the word “content” & I’ll send it to you (must be following so I can DM)

Tyler

24,007 views • 1 year ago

Colleges are using AI name readers to announce students' names during graduation ceremonies. Students scan their phones like they're in a checkout line before an AI voice reads their names. PACE University was seen using the technology during its commencement ceremony. "Our names are getting read by AI at graduation ... getting the ick at my own graduation," said one student. "It is AI. We were directly told so! We gave a website spelling and it instantly gave us an audio," said the student. "PACE told us it was AI ... We had to type out phonetic spelling, then it instantly generated audio for us to preview."

Colleges are using AI name readers to announce students' names during graduation ceremonies. Students scan their phones like they're in a checkout line before an AI voice reads their names. PACE University was seen using the technology during its commencement ceremony. "Our names are getting read by AI at graduation ... getting the ick at my own graduation," said one student. "It is AI. We were directly told so! We gave a website spelling and it instantly gave us an audio," said the student. "PACE told us it was AI ... We had to type out phonetic spelling, then it instantly generated audio for us to preview."

Collin Rugg

3,820,121 views • 1 year ago

This went viral, so I built an enhanced version: A video recorder where the AI talks back at you to give you live inspiration, via either voice or text on screen. And it can magically remove the silent parts of the video with 1 click, so you can download a clean video. I was SHOCKED when this video editing feature just worked in Google AI Studio.

This went viral, so I built an enhanced version: A video recorder where the AI talks back at you to give you live inspiration, via either voice or text on screen. And it can magically remove the silent parts of the video with 1 click, so you can download a clean video. I was SHOCKED when this video editing feature just worked in Google AI Studio.

Zara Zhang

86,365 views • 8 months ago

Dominion AI (Dominion AI) is a new, Christian-built alternative to mainstream AI models like ChatGPT and Claude. I had a conversation with the founders to learn about why they created it, what it can do, and who it's for. The full interview is now on X. Try Dominion here:

Dominion AI (Dominion AI) is a new, Christian-built alternative to mainstream AI models like ChatGPT and Claude. I had a conversation with the founders to learn about why they created it, what it can do, and who it's for. The full interview is now on X. Try Dominion here:

ZUBY:

18,223 views • 3 months ago

Music producer Matthew Tryba, who has worked with major artists like Taylor Swift, OneRepublic, Ariana Grande, Maroon 5, and more, praised Jeongwoo’s vocals, saying, “Ooh, did you hear how, when he did ‘that’s where we are,’ listen to his voice, he’s in his chest voice and on ‘are,’ he goes up to the head voice and that’s a hard transition to do.” 🥹 #박정우 🖇️

Music producer Matthew Tryba, who has worked with major artists like Taylor Swift, OneRepublic, Ariana Grande, Maroon 5, and more, praised Jeongwoo’s vocals, saying, “Ooh, did you hear how, when he did ‘that’s where we are,’ listen to his voice, he’s in his chest voice and on ‘are,’ he goes up to the head voice and that’s a hard transition to do.” 🥹 #박정우 🖇️

마기♡ (slow)

59,893 views • 11 months ago

The best way to take advantage of ChatGPT Voice. I am completely and utterly voice-pilled. Voice interaction with AI is going to change the way humans interact with computers. Here's what you can do: 1. Create a remote connection to an always-on desktop PC through the ChatGPT Desktop app (I have a Linux box in my basement) 2. Customize the ChatGPT iOS widget and add a remote voice connection 3. Tap the widget. It will open a voice chat in ChatGPT Work/Codex, on your desktop, with access to your files and all your apps. It's like an always on Jarvis that has its own computer. Amazing stuff. Try it out!

The best way to take advantage of ChatGPT Voice. I am completely and utterly voice-pilled. Voice interaction with AI is going to change the way humans interact with computers. Here's what you can do: 1. Create a remote connection to an always-on desktop PC through the ChatGPT Desktop app (I have a Linux box in my basement) 2. Customize the ChatGPT iOS widget and add a remote voice connection 3. Tap the widget. It will open a voice chat in ChatGPT Work/Codex, on your desktop, with access to your files and all your apps. It's like an always on Jarvis that has its own computer. Amazing stuff. Try it out!

Dan McAteer

42,024 views • 1 day ago