Skip navigation
This video shows a conversation between an AI-powered NPC character and an alien avatar user (me) in…
0:00 / 1:34

This video shows a conversation between an AI-powered NPC character and an alien avatar user (me) in…

Ali

@lirezaBahremand1.1K followers

Visit on X
112 likesDownloadOriginal post

14,034 views #vr #ar #GPT3

This video shows a conversation between an AI-powered NPC character and an alien avatar user (me) in VR. The AI can answer follow-up questions based on a specified personality profile. Built entirely with Unity #vr #ar #GPT3 #unity3d

17 Comments

Comments from the original post on X (Twitter)

  • Ali's profile picture

    @lirezaBahremand

    The pipeline uses #GoogleCloud's Speech-to-Text, generates a response with #GPT3's API, converts the response to audio with #Azure Text to Speech, and animates mouth movement with #Oculus/#Meta Lip Sync. The process takes ~2 seconds per prompt (sped up in the video).

    3 likes

  • Ali's profile picture

    @lirezaBahremand

    If you wanna build something similar using Unity, I highly recommend getting started with this C# wrapper for the OpenAI API:

    4 likes

  • nicrolix's profile picture

    @nicrolix

    @unity Imagine connected to VRchat people can interact with them in ur map

    2 likes

  • Ali's profile picture

    @lirezaBahremand

    @unity An AI NPC integration with VR chat would be dope, dude. However, this tech still has a ways to go. It’s easy to bug it out and break it. But the potential is what I find promising.

    1 like

  • Rodrigo's profile picture

    @rodrigon

    @unity I am working on a project similar to this one! Is there a chance for an asset based on this project? It would be a huge help!

    1 like

  • Ali's profile picture

    @lirezaBahremand

    @unity Awesome, dude. I do plan to make a template out of this when I have more free time. That said, you can reference this tweet to find all the necessary SDKs to get started:

    1 like

  • Philipp's profile picture

    @jphilipp

    @unity Nice! If you need any more inspiration feel free to browse my experiments, I'm using a similar pipeline in the later videos (but Google not Azure for text-to-speech).

    1 like

  • Ali's profile picture

    @lirezaBahremand

    @unity Dope concept, dude. What made you want to start producing these short AI-driven comedy sets? 🎭

  • Unity for Digital Twins's profile picture

    @DigitalTwin

    @unity say what now

    1 like

  • Juris's profile picture

    @Juris_Savos

    @unity I made something similar in 2D with AIML before the AI craze took off; maybe it's time to integrate a GPT model :) Great work Ali!

    1 like

  • Ali's profile picture

    @lirezaBahremand

    @unity Dude, this is awesome. I like the video format/style you got going here. Looks like a Unity scene, is that correct? What got you interested in this line of work?

  • Juris's profile picture

    @Juris_Savos

    @unity Glad to hear it! It's no where as complex as your AI but it works 😄 Yes it's Unity, and I built the chat bot for a Psychology module assignment to explore what features of an interactive chatbot increase user engagement. A big part of it was reducing the uncanny valley 🙂

  • XR_XharpRazor's profile picture

    @XR_XharpRazor

    @unity P01: Many people will say that soon AI is going to take over and destroy humanity blah blah blah. The reason why they will say it this way is because they treat AI as their enemy. I have to say thou, having a new enemy is exhausting... I prefer having a new friend...

  • Kojiro Yanoᯅ/VR Educator's profile picture

    @kojirovr

    Can I ask how you set up a personality profile?

  • Ali's profile picture

    @lirezaBahremand

    Great q. It’s nothing extraordinary. With the current GPT3 API, you can preface the question to convey a personality by specifying attributes of who the AI is. So things like “you lived here, interested in X Y Z, etc”, then the responses will acknowledge these cornerstones.

    1 like

  • Kojiro Yanoᯅ/VR Educator's profile picture

    @kojirovr

    I see. That's sounds reasonable to me. Many thanks!

  • Jeff Holmes's profile picture

    @holmesjtg

    @unity Nice work! Have you experimented with They have a plugin for #Unity and you should be able to achieve lower latency and reliable responses (based on text you provide). Might be a fun comparison: