Загрузка видео...

Не удалось загрузить видео

Возникла проблема при загрузке этого видео. Это может быть связано с временными проблемами сети или видео может быть недоступно.

На главную

🧑‍🍳 Experiment with a multimodal LLM in WebAR. The AI identifies ingredients, sets up bounding boxes, and prepares them for AR. I'm also using open-source models to generate recipes and corresponding images. #threejs #8thwall #generativeai

Stijn Spanhove

2,964 subscribers

61,019 просмотров • 2 лет назад •via X (Twitter)

#threejs #8thwall #generativeai

Anya Rossi• Live Now

Private livecam show

Комментарии: 10

Фото профиля Stijn Spanhove

Stijn Spanhove2 лет назад

Result from Llava 1.6 34b on @replicate

Фото профиля Stijn Spanhove

Stijn Spanhove2 лет назад

Bounding boxes

Фото профиля Michael + puppetto.com

Michael + puppetto.com2 лет назад

@MagusWazir Dude what is this future magic

Фото профиля Daniel Bigham

Daniel Bigham2 лет назад

Nice!

Фото профиля mongrov

mongrov2 лет назад

"Exciting! Integrating open-source models for recipes is a brilliant idea. I'd love to see how the multimodal LLM enhances the AR experience. Have you considered incorporating IoT to enhance user interaction or provide real-time feedback? #WebAR #AI #IoT

Фото профиля Stijn Spanhove

Stijn Spanhove2 лет назад

Did some experiments before with AR and visualising IoT data. Interesting indeed!

Фото профиля Peter VRMRCK

Peter VRMRCK2 лет назад

So good! Love it!

Фото профиля Stijn Spanhove

Stijn Spanhove2 лет назад

Thanks 🙌

Фото профиля jake nachlas

jake nachlas2 лет назад

This is really awesome!! It could also be great to see the recipes on a standard webpage so user points and scans like you have and then can lower their phone to look at the options. What do you think?

Фото профиля Stijn Spanhove

Stijn Spanhove2 лет назад

Thanks! Yes I agree, would be more user friendly. This is now running on a phone. Think could also be nice with a headset.

Похожие видео

An experiment with 𝘃𝗶𝗿𝘁𝘂𝗮𝗹 𝘁𝗿𝘆-𝗼𝗻 in WebAR 👇 👕 The GenAI-model seamlessly maps pictures onto my 3D-body in AR, all running smoothly on the web. Cool to see the results 👀 #generativeai #threejs #8thwall

An experiment with 𝘃𝗶𝗿𝘁𝘂𝗮𝗹 𝘁𝗿𝘆-𝗼𝗻 in WebAR 👇 👕 The GenAI-model seamlessly maps pictures onto my 3D-body in AR, all running smoothly on the web. Cool to see the results 👀 #generativeai #threejs #8thwall

Stijn Spanhove

42,097 просмотров • 2 лет назад

Introducing the Describe Anything Model (DAM), a powerful Multimodal LLM that generates detailed descriptions for user-specified regions in images or videos using points, boxes, scribbles, or masks. Open-source code, models, demo, data, and benchmark at:

Introducing the Describe Anything Model (DAM), a powerful Multimodal LLM that generates detailed descriptions for user-specified regions in images or videos using points, boxes, scribbles, or masks. Open-source code, models, demo, data, and benchmark at:

Yin Cui

34,990 просмотров • 1 год назад

The future of AI is open-source. And ollama is the easiest way to build AI applications with open-source LLMs. Here's how to build a free, private RAG app using open-source tools. We'll use: - Ollama for LLMs and embedding models - PostgreSQL for data storage and retrieval - pgai Vectorizer for embedding creation and sync (I use Nomic for embeddings and tinnyllama as my LLM but you can substitute them for any models on Ollama)

The future of AI is open-source. And ollama is the easiest way to build AI applications with open-source LLMs. Here's how to build a free, private RAG app using open-source tools. We'll use: - Ollama for LLMs and embedding models - PostgreSQL for data storage and retrieval - pgai Vectorizer for embedding creation and sync (I use Nomic for embeddings and tinnyllama as my LLM but you can substitute them for any models on Ollama)

Avthar

34,261 просмотров • 1 год назад

Ready to play? AI Playground curates 60+ top open-source AI models for you to explore, experiment, generate images and video, edit, chat, find answers, and more. Plus, it’s easy to use and optimized to run locally on your Intel-powered AI PC. Try it out:

Ready to play? AI Playground curates 60+ top open-source AI models for you to explore, experiment, generate images and video, edit, chat, find answers, and more. Plus, it’s easy to use and optimized to run locally on your Intel-powered AI PC. Try it out:

Intel

11,304 просмотров • 4 месяцев назад

I'm cooking up a cool tool to enhance photos with threejs and AI :) More videos in the thread! #threejs #AI

I'm cooking up a cool tool to enhance photos with threejs and AI :) More videos in the thread! #threejs #AI

Sébastien Lempens - PIXILA

88,261 просмотров • 2 лет назад

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models paper page: github: Recent advancements in text-to-image generation with diffusion models have yielded remarkable results synthesizing highly realistic and diverse images. However, these models still encounter difficulties when generating images from prompts that demand spatial or common sense reasoning. We propose to equip diffusion models with enhanced reasoning capabilities by using off-the-shelf pretrained large language models (LLMs) in a novel two-stage generation process. First, we adapt an LLM to be a text-guided layout generator through in-context learning. When provided with an image prompt, an LLM outputs a scene layout in the form of bounding boxes along with corresponding individual descriptions. Second, we steer a diffusion model with a novel controller to generate images conditioned on the layout. Both stages utilize frozen pretrained models without any LLM or diffusion model parameter optimization. We validate the superiority of our design by demonstrating its ability to outperform the base diffusion model in accurately generating images according to prompts that necessitate both language and spatial reasoning. Additionally, our method naturally allows dialog-based scene specification and is able to handle prompts in a language that is not well-supported by the underlying diffusion model.

AK

83,657 просмотров • 3 лет назад

Just built Openjourney 🚀 a vibe coded UI inspired by MidJourney built with Cursor and Replit for you to plug in your favorite models. Generate images with Imagen 4, animate them with Veo 3 or generate videos with sound. Fully open source for you to customize. Links below 👇

Just built Openjourney 🚀 a vibe coded UI inspired by MidJourney built with Cursor and Replit for you to plug in your favorite models. Generate images with Imagen 4, animate them with Veo 3 or generate videos with sound. Fully open source for you to customize. Links below 👇

Ammaar Reshi

52,820 просмотров • 1 год назад

Introducing an open-source template for broadcasting realtime transcripts. • Generate live transcripts with Scribe • Broadcast to many using Supabase • Translate with Chrome’s built-in AI Demo and open source template below.

Introducing an open-source template for broadcasting realtime transcripts. • Generate live transcripts with Scribe • Broadcast to many using Supabase • Translate with Chrome’s built-in AI Demo and open source template below.

ElevenLabs Developers

28,097 просмотров • 9 месяцев назад

Introducing private AI image generation on your Mac. Generate images with leading and open weight models, from a private workspace. Zero data retention by default. No one looking at your prompts and images. Yours only. Open Source + MIT.

Introducing private AI image generation on your Mac. Generate images with leading and open weight models, from a private workspace. Zero data retention by default. No one looking at your prompts and images. Yours only. Open Source + MIT.

OpenSoftware

137,221 просмотров • 1 месяц назад

🆕 How to run (and finetune) open source AI models with a simple API! In 5 mins, I go over how to: ◆ Generate text with DeepSeek R1 & Llama 3 ◆ Generate code with Qwen on LlamaCoder ◆ Generate images with Flux on BlinkShot ◆ Finetune a model on your own data & run it

🆕 How to run (and finetune) open source AI models with a simple API! In 5 mins, I go over how to: ◆ Generate text with DeepSeek R1 & Llama 3 ◆ Generate code with Qwen on LlamaCoder ◆ Generate images with Flux on BlinkShot ◆ Finetune a model on your own data & run it

Hassan

30,236 просмотров • 1 год назад

I am excited to introduce OpenGeoAgent, a powerful open-source multimodal AI agent for automated geospatial analysis and visualization! It supports QGIS, Jupyter notebook, and Python scripting. In this tutorial, you’ll learn how to automate GIS workflows using natural language, generate maps, analyze satellite data, and even run complex hydrological models. You can even interact with the agent using voice commands (no typing needed). Video: GitHub: QGIS Plugin: #geospatial #GeoAgent #OpenSource #AI

I am excited to introduce OpenGeoAgent, a powerful open-source multimodal AI agent for automated geospatial analysis and visualization! It supports QGIS, Jupyter notebook, and Python scripting. In this tutorial, you’ll learn how to automate GIS workflows using natural language, generate maps, analyze satellite data, and even run complex hydrological models. You can even interact with the agent using voice commands (no typing needed). Video: GitHub: QGIS Plugin: #geospatial #GeoAgent #OpenSource #AI

Qiusheng Wu

62,626 просмотров • 3 месяцев назад

Introducing `gpt-llm-trainer` ✍️ The world's simplest way to train a task-specific LLM. **Just write a sentence describing the model you want.** A chain of AI systems will generate a dataset and train a model for you. And it's open-source:

Introducing `gpt-llm-trainer` ✍️ The world's simplest way to train a task-specific LLM. Just write a sentence describing the model you want. A chain of AI systems will generate a dataset and train a model for you. And it's open-source:

Matt Shumer

337,464 просмотров • 3 лет назад

Introducing `claude-llm-trainer` ✍️ The world's simplest way to train a task-specific LLM. Just write a sentence describing the model you want. A chain of AI systems will generate a dataset and train a model for you. And it's open-source.

Introducing `claude-llm-trainer` ✍️ The world's simplest way to train a task-specific LLM. Just write a sentence describing the model you want. A chain of AI systems will generate a dataset and train a model for you. And it's open-source.

Matt Shumer

383,208 просмотров • 2 лет назад

Want to quickly craft and iterate on prompts? Check out Prompt builder in the AI Toolkit for VS Code ( Experiment with local or remote models, generate prompts using AI, and test multi-turn conversations. Plus, view the code to start building right away! 💡

Want to quickly craft and iterate on prompts? Check out Prompt builder in the AI Toolkit for VS Code ( Experiment with local or remote models, generate prompts using AI, and test multi-turn conversations. Plus, view the code to start building right away! 💡

Visual Studio Code

52,751 просмотров • 1 год назад

Today we released Meta Spirit LM — our first open source multimodal language model that freely mixes text and speech. Many existing AI voice experiences today use ASR to techniques to process speech before synthesizing with an LLM to generate text — but these approaches compromise the expressive aspects of speech. Using phonetic, pitch and tone tokens, Spirit LM models can overcome these limitations for both inputs and outputs to generate more natural sounding speech while also learning new tasks across ASR, TTS and speech classification. We hope that sharing this work will enable the research community to further new approaches for text and speech integration.

Today we released Meta Spirit LM — our first open source multimodal language model that freely mixes text and speech. Many existing AI voice experiences today use ASR to techniques to process speech before synthesizing with an LLM to generate text — but these approaches compromise the expressive aspects of speech. Using phonetic, pitch and tone tokens, Spirit LM models can overcome these limitations for both inputs and outputs to generate more natural sounding speech while also learning new tasks across ASR, TTS and speech classification. We hope that sharing this work will enable the research community to further new approaches for text and speech integration.

AI at Meta

351,764 просмотров • 1 год назад

New: AI Image Generation in Spline Hana 🧙‍♂️ - Generate images with various models - Combine or modify images and frames - Make Variations, remove BG, and Upscale Available now to try for free!

New: AI Image Generation in Spline Hana 🧙‍♂️ - Generate images with various models - Combine or modify images and frames - Make Variations, remove BG, and Upscale Available now to try for free!

Spline

21,620 просмотров • 10 месяцев назад

5/ Open source isn’t enough. Application is everything. China isn’t waiting for open-source models to catch up. They're customizing them and making them useful now in real-world settings. From factory lines to traffic systems, AI is already embedded.

5/ Open source isn’t enough. Application is everything. China isn’t waiting for open-source models to catch up. They're customizing them and making them useful now in real-world settings. From factory lines to traffic systems, AI is already embedded.

Carlo Edoardo Ferraris

75,649 просмотров • 1 год назад

Meet #DBRX: a general-purpose LLM that sets a new standard for efficient open source models. Use the DBRX model in your RAG apps or use the DBRX design to build your own custom LLMs and improve the quality of your GenAI applications.

Meet #DBRX: a general-purpose LLM that sets a new standard for efficient open source models. Use the DBRX model in your RAG apps or use the DBRX design to build your own custom LLMs and improve the quality of your GenAI applications.

Databricks

327,800 просмотров • 2 лет назад

Super proud to release our most fun AI experiment to date: AISheets 🗒️ Thousands of AI models meet spreadsheets. Build, analyze, and automate your data using open-source LLMs in one slick, fast and simple app. Surprisingly powerful! 🚀 Try it:

Super proud to release our most fun AI experiment to date: AISheets 🗒️ Thousands of AI models meet spreadsheets. Build, analyze, and automate your data using open-source LLMs in one slick, fast and simple app. Surprisingly powerful! 🚀 Try it:

Thomas Wolf

51,109 просмотров • 1 год назад