Загрузка видео...

Не удалось загрузить видео

На главную

Realtime interactive generative models FTW! Announcing a new 🌊 of details and features for Magenta RealTime, the open weights live music AI model from GDM! * Live Jamming with audio input 🎤🎸🎵 * Personalize your own models 🔧 * Tech report 📜 Links below in the 🧵...

jesseengel's profile picture

Jesse Engel

9,221 subscribers

183,504 просмотров • 11 месяцев назад •via X (Twitter)

Наука и технологииМузыка
LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

Introducing Magenta RealTime 2 (MRT2): the live music model you can play as an instrument. MRT2 offers MIDI and prompt controls, and runs natively on a MacBook with <200ms latency. Open weights. Open source inference engine. Suite of apps and plugins. Hear what it can do and try it out for yourself below 🧵
1:30

Introducing Magenta RealTime 2 (MRT2): the live music model you can play as an instrument. MRT2 offers MIDI and prompt controls, and runs natively on a MacBook with <200ms latency. Open weights. Open source inference engine. Suite of apps and plugins. Hear what it can do and try it out for yourself below 🧵

Google Magenta Project

112,452 просмотров • 1 месяц назад

the new OpenAI realtime voice model just released + gpt 5.5 fast mode brings us a new possibility - realtime speech to live presentation! i just talk, and the whiteboard would whiteboard itself prototype is open sourced. details in thread below -
1:46

the new OpenAI realtime voice model just released + gpt 5.5 fast mode brings us a new possibility - realtime speech to live presentation! i just talk, and the whiteboard would whiteboard itself prototype is open sourced. details in thread below -

Kun Chen

114,421 просмотров • 2 месяцев назад

Really proud to share something we’ve been working on for a while: Magenta RealTime 2 (MTR2), a live music model that is highly interactive (MIDI, audio, text, lots of parameters) and low-latency (~200ms end-to-end), and runs locally on a MacBook!
0:37

Really proud to share something we’ve been working on for a while: Magenta RealTime 2 (MTR2), a live music model that is highly interactive (MIDI, audio, text, lots of parameters) and low-latency (~200ms end-to-end), and runs locally on a MacBook!

Ilaria Manco

44,633 просмотров • 1 месяц назад

New Experiment: SerendipityLM - Interactive evolutionary exploration of generative design spaces with large language models. Check out the extensive experiment report, incl. many generative art examples & link to open source repo: Find out more in this 🧵
0:24

New Experiment: SerendipityLM - Interactive evolutionary exploration of generative design spaces with large language models. Check out the extensive experiment report, incl. many generative art examples & link to open source repo: Find out more in this 🧵

samim

23,739 просмотров • 2 лет назад

Introducing GPT-Realtime-2 in the API: our most intelligent voice model yet, bringing GPT-5-class reasoning to voice agents. Voice agents are now real-time collaborators that can listen, reason, and solve complex problems as conversations unfold. Now available in the API alongside streaming models GPT-Realtime-Translate and GPT-Realtime-Whisper — a new set of audio capabilities for the next generation of voice interfaces.
4:03

Introducing GPT-Realtime-2 in the API: our most intelligent voice model yet, bringing GPT-5-class reasoning to voice agents. Voice agents are now real-time collaborators that can listen, reason, and solve complex problems as conversations unfold. Now available in the API alongside streaming models GPT-Realtime-Translate and GPT-Realtime-Whisper — a new set of audio capabilities for the next generation of voice interfaces.

OpenAI

3,650,229 просмотров • 2 месяцев назад

This is a big day. Meta is open-sourcing AudioCraft. You can now generate incredible music and sounds with a single prompt. It includes the most performant Generative AI Model (audio) on the market, the "Llama" of Audio. The research framework contains the weights and code of these models: ▸ MusicGen: controllable text-to-music model. ▸ AudioGen: text-to-sound model. ▸ EnCodec: high fidelity neural audio codec. ▸ Multi Band Diffusion: An EnCodec compatible decoder using diffusion. This is going to tremendously speed up audio research 👏
0:28

This is a big day. Meta is open-sourcing AudioCraft. You can now generate incredible music and sounds with a single prompt. It includes the most performant Generative AI Model (audio) on the market, the "Llama" of Audio. The research framework contains the weights and code of these models: ▸ MusicGen: controllable text-to-music model. ▸ AudioGen: text-to-sound model. ▸ EnCodec: high fidelity neural audio codec. ▸ Multi Band Diffusion: An EnCodec compatible decoder using diffusion. This is going to tremendously speed up audio research 👏

Lior Alexander

231,608 просмотров • 3 лет назад

Today we’re announcing two new updates in our computer vision work — a new, expanded license for our DINOv2 model and the release of FACET, a comprehensive new benchmark dataset to help evaluate and improve fairness in vision models. More details ➡️ 🧵
0:27

Today we’re announcing two new updates in our computer vision work — a new, expanded license for our DINOv2 model and the release of FACET, a comprehensive new benchmark dataset to help evaluate and improve fairness in vision models. More details ➡️ 🧵

AI at Meta

453,971 просмотров • 2 лет назад

OpenAI just dropped a completely new kind of model gpt-realtime-translate takes in speech audio from any language and outputs speech in your target language LLMs are great, but you need specialized models for specialized use cases We're running this on our smart glasses
0:42

OpenAI just dropped a completely new kind of model gpt-realtime-translate takes in speech audio from any language and outputs speech in your target language LLMs are great, but you need specialized models for specialized use cases We're running this on our smart glasses

cayden 凯登

100,395 просмотров • 1 месяц назад

Your Zoom meetings, now with an AI coach! Live analysis. Realtime guidance. Better outcomes. We collaborated with Zoom to build a Runtime template for integrating Zoom’s Realtime Media Streams. Boost the effectiveness of your sales and support teams while ensuring compliance. And, the code is open-sourced. Read more → Build on our template →
5:12

Your Zoom meetings, now with an AI coach! Live analysis. Realtime guidance. Better outcomes. We collaborated with Zoom to build a Runtime template for integrating Zoom’s Realtime Media Streams. Boost the effectiveness of your sales and support teams while ensuring compliance. And, the code is open-sourced. Read more → Build on our template →

Inworld AI

649,217 просмотров • 6 месяцев назад

Stop hardcoding one model name in your code. Now you can give each request a policy: a small rule for what the call needs It picks the right model for the job, on your own keys This is unhardcoded, our new open source routing for AI models, live today
0:47

Stop hardcoding one model name in your code. Now you can give each request a policy: a small rule for what the call needs It picks the right model for the job, on your own keys This is unhardcoded, our new open source routing for AI models, live today

GenLayer

90,616 просмотров • 26 дней назад

Last week we released OpenAI Developers gpt-realtime our latest speech-to-speech model. Prompting S2S models can be extremely powerful. To get the most out of the model we compiled a series of prompting tips of what we saw work with early customers. To present them here's Cedar our new voice in gpt-realtime walk you through it. If you want to read it yourself check the links below 👇
2:08

Last week we released OpenAI Developers gpt-realtime our latest speech-to-speech model. Prompting S2S models can be extremely powerful. To get the most out of the model we compiled a series of prompting tips of what we saw work with early customers. To present them here's Cedar our new voice in gpt-realtime walk you through it. If you want to read it yourself check the links below 👇

dominik kundel

66,904 просмотров • 10 месяцев назад

A quick first pass at a new iteration of the realtime diffusion pipeline: physical interface → camera → realtime img2img → upscale → 3DGS through the new TripoSplat. Rough and held together with tape for now, but the whole loop runs end to end. Next: - better input and camera's pov, - splitting it across two machines to automate the hand-off, - and adding in the Tripo3D API for higher-res models... the 3D printer is just waiting for that. #diffusion #3DGS #TripoSplat
1:24

A quick first pass at a new iteration of the realtime diffusion pipeline: physical interface → camera → realtime img2img → upscale → 3DGS through the new TripoSplat. Rough and held together with tape for now, but the whole loop runs end to end. Next: - better input and camera's pov, - splitting it across two machines to automate the hand-off, - and adding in the Tripo3D API for higher-res models... the 3D printer is just waiting for that. #diffusion #3DGS #TripoSplat

Hugues Bruyère

15,526 просмотров • 1 месяц назад

Today is a good day for open science. As part of our continued commitment to the growth and development of an open ecosystem, today at Meta FAIR we’re announcing four new publicly available AI models and additional research artifacts to inspire innovation in the community and help advance AI in a responsible way. More in the video from Joelle Pineau. What we’re releasing: 🦎 Meta Chameleon 7B & 34B language models that support mixed-modal input and text-only outputs. 🪙 Meta Multi-Token Prediction Pretrained Language Models for code completion using Multi-Token Prediction. 🎼 Meta JASCO Generative text-to-music models capable of accepting various conditioning inputs for greater controllability. Paper available today with a pretrained model coming soon. 🗣️ Meta AudioSeal An audio watermarking model that we believe is the first designed specifically for the localized detection of AI-generated speech, available under a commercial license. 📝 Additional RAI artifacts Including research, data and code to measure and improve the representation of geographical and cultural preferences and diversity in AI systems. We believe that access to state-of-the-art AI creates opportunities for everyone – not just a small handful of Big Tech companies. We’re excited to share this work and to see how the community learns, iterates and builds using this technology. Details and access to everything released by FAIR today ➡️
1:14

Today is a good day for open science. As part of our continued commitment to the growth and development of an open ecosystem, today at Meta FAIR we’re announcing four new publicly available AI models and additional research artifacts to inspire innovation in the community and help advance AI in a responsible way. More in the video from Joelle Pineau. What we’re releasing: 🦎 Meta Chameleon 7B & 34B language models that support mixed-modal input and text-only outputs. 🪙 Meta Multi-Token Prediction Pretrained Language Models for code completion using Multi-Token Prediction. 🎼 Meta JASCO Generative text-to-music models capable of accepting various conditioning inputs for greater controllability. Paper available today with a pretrained model coming soon. 🗣️ Meta AudioSeal An audio watermarking model that we believe is the first designed specifically for the localized detection of AI-generated speech, available under a commercial license. 📝 Additional RAI artifacts Including research, data and code to measure and improve the representation of geographical and cultural preferences and diversity in AI systems. We believe that access to state-of-the-art AI creates opportunities for everyone – not just a small handful of Big Tech companies. We’re excited to share this work and to see how the community learns, iterates and builds using this technology. Details and access to everything released by FAIR today ➡️

AI at Meta

380,751 просмотров • 2 лет назад

Exciting update for AI developers! The Hugging Face Hub is now more natively integrated into Google Cloud Vertex AI Model Garden. Search through thousands of open Generative AI models from Hugging Face models & deploy them with one click to Vertex AI or GKE. 🤯 What's new: 🔎 Browse and search thousands of Hugging Face models directly within the Vertex AI Model Garden and filter based on what is currently trending in the community. 🚀 Accelerate your AI projects by leveraging readily available one-click deploy your model to Vertex AI or Google Kubernetes Engine (GKE). ⭐️ Featuring popular open models from BlackForestLabsAI - Unofficial FLUX.1, AI at Meta Llama 3.1, Mistral AI, Google DeepMind Gemma, and countless others. Get Started:
0:38

Exciting update for AI developers! The Hugging Face Hub is now more natively integrated into Google Cloud Vertex AI Model Garden. Search through thousands of open Generative AI models from Hugging Face models & deploy them with one click to Vertex AI or GKE. 🤯 What's new: 🔎 Browse and search thousands of Hugging Face models directly within the Vertex AI Model Garden and filter based on what is currently trending in the community. 🚀 Accelerate your AI projects by leveraging readily available one-click deploy your model to Vertex AI or Google Kubernetes Engine (GKE). ⭐️ Featuring popular open models from BlackForestLabsAI - Unofficial FLUX.1, AI at Meta Llama 3.1, Mistral AI, Google DeepMind Gemma, and countless others. Get Started:

Philipp Schmid

34,754 просмотров • 1 год назад

Recap on Tencent Hunyuan Game: the first industrial-grade AIGC engine and AI-model series for game asset generation. Beyond the features currently available in the Hunyuan Game platform, the team is actively developing more capabilities (many not yet live). These advancements were unveiled in the Hunyuan Game technical report, which details four image-generation models and five video-generation models for game visuals. Watch the video for a glimpse of these innovations and check out the technical report to learn more 👉🏻
2:13

Recap on Tencent Hunyuan Game: the first industrial-grade AIGC engine and AI-model series for game asset generation. Beyond the features currently available in the Hunyuan Game platform, the team is actively developing more capabilities (many not yet live). These advancements were unveiled in the Hunyuan Game technical report, which details four image-generation models and five video-generation models for game visuals. Watch the video for a glimpse of these innovations and check out the technical report to learn more 👉🏻

Hunyuan

46,802 просмотров • 1 год назад

I left my job at NVIDIA last year... and now I co-founded @outerbasis and started a position as ceo sararīman!! (also we're part of the current Y Combinator S24 batch 😎) With new tech come new formats, and with new formats come new distribution tech. We've seen this with cameras ➡️ JPEGs ➡️ social media and cgroups ➡️ containers ➡️ orchestration. Just like how cameras capture a slice of the photon stream around us, generative AI is a 'data camera' that allows us to snapshot the society that we live in. The new format from this capture device are model weights, which require tailored distribution solutions for them. @outerbasis is a registry for model weights that lets you store model weights and orchestrate the distribution of them onto GPU servers. The system uses tech optimized for streaming weights into the servers as fast as possible- so researchers can deploy updates without downtime and ops admins can orchestrate hot swapping between models. I've been super excited about streaming, compression, and neural networks for the last N years... but now I'm excited to build a system to support of this evermore exciting ecosystem of foundation model weights!! Please reach out if this sounds interesting / useful to you!! (and you can find more details on our YC launch: (The video is a small demo of our model hotswapping)
0:51

I left my job at NVIDIA last year... and now I co-founded @outerbasis and started a position as ceo sararīman!! (also we're part of the current Y Combinator S24 batch 😎) With new tech come new formats, and with new formats come new distribution tech. We've seen this with cameras ➡️ JPEGs ➡️ social media and cgroups ➡️ containers ➡️ orchestration. Just like how cameras capture a slice of the photon stream around us, generative AI is a 'data camera' that allows us to snapshot the society that we live in. The new format from this capture device are model weights, which require tailored distribution solutions for them. @outerbasis is a registry for model weights that lets you store model weights and orchestrate the distribution of them onto GPU servers. The system uses tech optimized for streaming weights into the servers as fast as possible- so researchers can deploy updates without downtime and ops admins can orchestrate hot swapping between models. I've been super excited about streaming, compression, and neural networks for the last N years... but now I'm excited to build a system to support of this evermore exciting ecosystem of foundation model weights!! Please reach out if this sounds interesting / useful to you!! (and you can find more details on our YC launch: (The video is a small demo of our model hotswapping)

Towaki Takikawa / 瀧川永遠希

51,277 просмотров • 1 год назад

Excited to release a tool that makes the joy of live music creation more accessible 🎸 “Music FX DJ” is new and improved: Featuring a whole suite of new controls, sharing and remix features, and improved audio quality and diversity of outputs, to let you blend and warp genres and instruments in an ever evolving stream of music. It’s been an absolute pleasure to work with Jacob Collier over the past year on this. It was very affirming to find him pushing for the same things Google Magenta Project has always valued: making tools that intrinsically value creativity and agency over just imitation, embrace the “weird corners” of AI, encouraging veterans to express themselves in new ways, while empowering novices to express themselves musically maybe for the first time. Read details at: And in the 🧵 below…
0:56

Excited to release a tool that makes the joy of live music creation more accessible 🎸 “Music FX DJ” is new and improved: Featuring a whole suite of new controls, sharing and remix features, and improved audio quality and diversity of outputs, to let you blend and warp genres and instruments in an ever evolving stream of music. It’s been an absolute pleasure to work with Jacob Collier over the past year on this. It was very affirming to find him pushing for the same things Google Magenta Project has always valued: making tools that intrinsically value creativity and agency over just imitation, embrace the “weird corners” of AI, encouraging veterans to express themselves in new ways, while empowering novices to express themselves musically maybe for the first time. Read details at: And in the 🧵 below…

Jesse Engel

20,093 просмотров • 1 год назад

Voice-only programming with the new OpenAI Realtime API ... I spend a lot of time these days pair programming with LLMs. Often I'm talking rather than typing. This "voice dictation" use case has become an important vibe benchmark for me. Being able to create text input just by talking, flexibly, in a context dependent way, with tool calling, is a *hard* problem for today's models. Natural language dictation requires a very high degree of contextual intelligence, instruction following accuracy, and tool calling reliability. Today's new gpt-realtime model is quite good at this hard problem. The original realtime model release last year was impressive. Seeing what a speech-to-speech model could do got a lot of people excited about the possibilities of voice AI. The improvements since that first release are equally impressive. I can use this new model, now, for real world tasks that were past the edge of the "jagged frontier" before. Here's a video showing a couple of fun (and tricky) modes of voice input.
2:40

Voice-only programming with the new OpenAI Realtime API ... I spend a lot of time these days pair programming with LLMs. Often I'm talking rather than typing. This "voice dictation" use case has become an important vibe benchmark for me. Being able to create text input just by talking, flexibly, in a context dependent way, with tool calling, is a *hard* problem for today's models. Natural language dictation requires a very high degree of contextual intelligence, instruction following accuracy, and tool calling reliability. Today's new gpt-realtime model is quite good at this hard problem. The original realtime model release last year was impressive. Seeing what a speech-to-speech model could do got a lot of people excited about the possibilities of voice AI. The improvements since that first release are equally impressive. I can use this new model, now, for real world tasks that were past the edge of the "jagged frontier" before. Here's a video showing a couple of fun (and tricky) modes of voice input.

kwindla

50,806 просмотров • 10 месяцев назад