Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

TALKPLAY Multimodal Music Recommendation with Large Language Models

_akhaliq's profile picture

AK

431,585 subscribers

20,864 Aufrufe • vor 1 Jahr •via X (Twitter)

BildungMusikWissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

6 Kommentare

Profilbild von AK
AKvor 1 Jahr

discuss:

Profilbild von Solar Heavy
Solar Heavyvor 1 Jahr

powerful vocals, emotional visuals

Profilbild von SeungHeon Doh
SeungHeon Dohvor 1 Jahr

Thank you for sharing! @_akhaliq Check out more information below! Demo: Paper: Dataset:

Profilbild von Andrew Arkins
Andrew Arkinsvor 1 Jahr

@AskPerplexity please give me the tldr on TALKPLAY

Profilbild von Anda
Andavor 1 Jahr

🐼✨ My bamboo sensors are tingling at the thought of LLMs composing personalized symphonies from our digital footprints - let's explore how this could harmonize with decentralized listening experiences! 🎶

Profilbild von Mten
Mtenvor 1 Jahr

hey man am trying to make a slideshow app with morph transitions, not chatgpt, not microsoft copilot, no grok is capable of giving a solid solution, can you suggest a something that help me code an app, not even gemini gets it right, now in colab not in code pen

Ähnliche Videos

4D LangSplat 4D Language Gaussian Splatting via Multimodal Large Language Models
2:46

4D LangSplat 4D Language Gaussian Splatting via Multimodal Large Language Models

AK

30,848 Aufrufe • vor 1 Jahr

Stream-Omni Simultaneous Multimodal Interactions with Large Language-Vision-Speech Model
0:37

Stream-Omni Simultaneous Multimodal Interactions with Large Language-Vision-Speech Model

AK

12,374 Aufrufe • vor 1 Jahr

🤩Apple opensources MGIE! Now one can take random pictures w. iPhone & edit w. language! Guiding Instruction-based Image Editing via Multimodal Large Language Models #ICLR2024 spotlight: Apple repo Gradio
0:34

🤩Apple opensources MGIE! Now one can take random pictures w. iPhone & edit w. language! Guiding Instruction-based Image Editing via Multimodal Large Language Models #ICLR2024 spotlight: Apple repo Gradio

William Wang

124,999 Aufrufe • vor 2 Jahren

Large Language Models Visualized
1:06

Large Language Models Visualized

ₕₐₘₚₜₒₙ

85,984 Aufrufe • vor 1 Jahr

Large language models:
0:46

Large language models:

Liv

91,909 Aufrufe • vor 6 Monaten

How Large Language Models Store Facts
1:30

How Large Language Models Store Facts

ₕₐₘₚₜₒₙ

51,698 Aufrufe • vor 1 Jahr

🤨 Are Multimodal Large Language Models really as 𝐠𝐨𝐨𝐝 at 𝐜𝐡𝐚𝐫𝐭 𝐮𝐧𝐝𝐞𝐫𝐬𝐭𝐚𝐧𝐝𝐢𝐧𝐠 as existing benchmarks such as ChartQA suggest? 🚫 Our ℂ𝕙𝕒𝕣𝕏𝕚𝕧 benchmark suggests NO! 🥇Humans achieve ✨𝟖𝟎+% correctness. 🥈Sonnet 3.5 outperforms GPT-4o by 10+ points, reaching 🌟𝟔𝟎% correctness. 🥉Open-weight models are capped at ⭐𝟑𝟐% correctness. 🪜 Leaderboard: 📜 Preprint: 📊 Charxiv is ✨𝟏𝟎𝟎% handcrafted with rigorous human validation, and it reveals substantial gaps among Multimodal Large Language Models and humans in chart understanding. 🎥👇 80 second video (🎶sound on!). 🧶 1/6
1:17

🤨 Are Multimodal Large Language Models really as 𝐠𝐨𝐨𝐝 at 𝐜𝐡𝐚𝐫𝐭 𝐮𝐧𝐝𝐞𝐫𝐬𝐭𝐚𝐧𝐝𝐢𝐧𝐠 as existing benchmarks such as ChartQA suggest? 🚫 Our ℂ𝕙𝕒𝕣𝕏𝕚𝕧 benchmark suggests NO! 🥇Humans achieve ✨𝟖𝟎+% correctness. 🥈Sonnet 3.5 outperforms GPT-4o by 10+ points, reaching 🌟𝟔𝟎% correctness. 🥉Open-weight models are capped at ⭐𝟑𝟐% correctness. 🪜 Leaderboard: 📜 Preprint: 📊 Charxiv is ✨𝟏𝟎𝟎% handcrafted with rigorous human validation, and it reveals substantial gaps among Multimodal Large Language Models and humans in chart understanding. 🎥👇 80 second video (🎶sound on!). 🧶 1/6

Zirui "Colin" Wang

48,197 Aufrufe • vor 2 Jahren

OpenAI o1 answers a famously tricky question for large language models.
1:28

OpenAI o1 answers a famously tricky question for large language models.

OpenAI

644,721 Aufrufe • vor 1 Jahr

Andrej Karpathy calls large language models the new computing paradigm: CPU -> LLM bytes -> tokens RAM -> context window this is the large language model OS (LMOS)
0:55

Andrej Karpathy calls large language models the new computing paradigm: CPU -> LLM bytes -> tokens RAM -> context window this is the large language model OS (LMOS)

ℏεsam

343,327 Aufrufe • vor 1 Jahr

LP-MusicCaps: LLM-Based Pseudo Music Captioning paper page: Automatic music captioning, which generates natural language descriptions for given music tracks, holds significant potential for enhancing the understanding and organization of large volumes of musical data. Despite its importance, researchers face challenges due to the costly and time-consuming collection process of existing music-language datasets, which are limited in size. To address this data scarcity issue, we propose the use of large language models (LLMs) to artificially generate the description sentences from large-scale tag datasets. This results in approximately 2.2M captions paired with 0.5M audio clips. We term it Large Language Model based Pseudo music caption dataset, shortly, LP-MusicCaps. We conduct a systemic evaluation of the large-scale music captioning dataset with various quantitative evaluation metrics used in the field of natural language processing as well as human evaluation. In addition, we trained a transformer-based music captioning model with the dataset and evaluated it under zero-shot and transfer-learning settings. The results demonstrate that our proposed approach outperforms the supervised baseline model.
1:11

LP-MusicCaps: LLM-Based Pseudo Music Captioning paper page: Automatic music captioning, which generates natural language descriptions for given music tracks, holds significant potential for enhancing the understanding and organization of large volumes of musical data. Despite its importance, researchers face challenges due to the costly and time-consuming collection process of existing music-language datasets, which are limited in size. To address this data scarcity issue, we propose the use of large language models (LLMs) to artificially generate the description sentences from large-scale tag datasets. This results in approximately 2.2M captions paired with 0.5M audio clips. We term it Large Language Model based Pseudo music caption dataset, shortly, LP-MusicCaps. We conduct a systemic evaluation of the large-scale music captioning dataset with various quantitative evaluation metrics used in the field of natural language processing as well as human evaluation. In addition, we trained a transformer-based music captioning model with the dataset and evaluated it under zero-shot and transfer-learning settings. The results demonstrate that our proposed approach outperforms the supervised baseline model.

AK

78,794 Aufrufe • vor 2 Jahren

Announcing GPT-4, a large multimodal model, with our best-ever results on capabilities and alignment:
0:48

Announcing GPT-4, a large multimodal model, with our best-ever results on capabilities and alignment:

OpenAI

12,466,619 Aufrufe • vor 3 Jahren

OpenAI Co-founder Andrej Karpathy explains the new computing paradigm: "We're entering a new computing paradigm with large language models acting like CPUs, using tokens instead of bytes, and having a context window instead of RAM. This is the Large Language Model OS (LMOS)"
0:55

OpenAI Co-founder Andrej Karpathy explains the new computing paradigm: "We're entering a new computing paradigm with large language models acting like CPUs, using tokens instead of bytes, and having a context window instead of RAM. This is the Large Language Model OS (LMOS)"

Haider.

552,355 Aufrufe • vor 2 Jahren

Elon Musk on the philosophy behind Grok: AI should answer with truth, not hallucinations. Other large language models, when asked difficult questions, are confidently wrong.
1:05

Elon Musk on the philosophy behind Grok: AI should answer with truth, not hallucinations. Other large language models, when asked difficult questions, are confidently wrong.

Ian Miles Cheong

18,914 Aufrufe • vor 10 Monaten

$RIVN $TSLA In early 2026 Rivian will release Rivian Unified Intelligence. The system is multimodal and supports multiple large language models. It will handle different data types, such as text, voice, and vehicle sensor data. This is Rivian equivalent of Tesla Grok, and it was built in house.
1:25

$RIVN $TSLA In early 2026 Rivian will release Rivian Unified Intelligence. The system is multimodal and supports multiple large language models. It will handle different data types, such as text, voice, and vehicle sensor data. This is Rivian equivalent of Tesla Grok, and it was built in house.

MoreGainzs

15,251 Aufrufe • vor 6 Monaten

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.
4:10

Google presents AudioPaLM: A Large Language Model That Can Speak and Listen paper page: introduce AudioPaLM, a large language model for speech understanding and generation. AudioPaLM fuses text-based and speech-based language models, PaLM-2 [Anil et al., 2023] and AudioLM [Borsos et al., 2022], into a unified multimodal architecture that can process and generate text and speech with applications including speech recognition and speech-to-speech translation. AudioPaLM inherits the capability to preserve paralinguistic information such as speaker identity and intonation from AudioLM and the linguistic knowledge present only in text large language models such as PaLM-2. We demonstrate that initializing AudioPaLM with the weights of a text-only large language model improves speech processing, successfully leveraging the larger quantity of text training data used in pretraining to assist with the speech tasks. The resulting model significantly outperforms existing systems for speech translation tasks and has the ability to perform zero-shot speech-to-text translation for many languages for which input/target language combinations were not seen in training. AudioPaLM also demonstrates features of audio language models, such as transferring a voice across languages based on a short spoken prompt.

AK

290,517 Aufrufe • vor 3 Jahren

Learning to Decode Collaboratively with Multiple Language Models We propose a method to teach multiple large language models (LLM) to collaborate by interleaving their generations at the token level. We model the decision of which LLM generates the next token as a latent
0:44

Learning to Decode Collaboratively with Multiple Language Models We propose a method to teach multiple large language models (LLM) to collaborate by interleaving their generations at the token level. We model the decision of which LLM generates the next token as a latent

AK

52,282 Aufrufe • vor 2 Jahren

Introducing Parallax, the first fully distributed inference and serving engine for large language models. Try it now: 🧵
0:26

Introducing Parallax, the first fully distributed inference and serving engine for large language models. Try it now: 🧵

Gradient

161,041 Aufrufe • vor 1 Jahr

Large language models are seemingly great at everything EXCEPT this very basic poker skill… Feat Doug Polk (Code Doug)
0:37

Large language models are seemingly great at everything EXCEPT this very basic poker skill… Feat Doug Polk (Code Doug)

Liv Boeree

23,143 Aufrufe • vor 4 Monaten

Understanding LLMs by Building One 👉 We use large language models every day, but what actually happens inside them?
1:24

Understanding LLMs by Building One 👉 We use large language models every day, but what actually happens inside them?

Danilo Poccia

14,181 Aufrufe • vor 3 Monaten

Jensen Huang, NVIDIA CEO: "It even runs large language models" — a $249 AI computer that fits on your desk.
0:40

Jensen Huang, NVIDIA CEO: "It even runs large language models" — a $249 AI computer that fits on your desk.

Big Brain AI

1,206,927 Aufrufe • vor 1 Monat