Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Google DeepMind is doing some crazy work. This is so good and is going to help so many people. They just launched SL2T (Sign Language to Text), an AI model that translates sign language directly into text. Most spoken language dictation tools rely on audio to text, but sign...

26,838 görüntüleme • 4 gün önce •via X (Twitter)

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

We’re excited to introduce Text-to-LoRA: a Hypernetwork that generates task-specific LLM adapters (LoRAs) based on a text description of the task. Catch our presentation at #ICML2025! Paper: Code: Biological systems are capable of rapid adaptation, given limited sensory cues. For example, our human visual system can quickly adapt and tune its light sensitivity to our surroundings. While modern LLMs exhibit a wide variety of capabilities and knowledge, they remain rigid when adding task-specific capabilities. Traditionally, customizing these models requires gathering large datasets and performing often expensive, time-consuming fine-tuning for specific applications. To bypass these limitations, Text-to-LoRA (T2L) meta-learns a “hypernetwork” that takes in a text description of a desired task, as a prompt, and generates a task-specific LoRA that performs well on the task. In our experiments, we show that T2L can encode hundreds of existing LoRA adapters. While the compression is lossy, T2L maintains the performance of task-specifically tuned LoRA adapters. We also show that T2L can even generalize to unseen tasks given a natural language description of the tasks. Importantly, Text-to-LoRA is parameter-efficient. It generates LoRAs in a single, inexpensive step, based solely on a simple text description of the task. This approach is a step towards dramatically lowering the technical and computational barriers, allowing non-technical users to specialize foundation models using plain language, rather than needing deep technical expertise or large compute resources.

Sakana AI

403,159 görüntüleme • 1 yıl önce

STEVE-1: A Generative Model for Text-to-Behavior in Minecraft paper page: Constructing AI models that respond to text instructions is challenging, especially for sequential decision-making tasks. This work introduces an instruction-tuned Video Pretraining (VPT) model for Minecraft called STEVE-1, demonstrating that the unCLIP approach, utilized in DALL-E 2, is also effective for creating instruction-following sequential decision-making agents. STEVE-1 is trained in two steps: adapting the pretrained VPT model to follow commands in MineCLIP's latent space, then training a prior to predict latent codes from text. This allows us to finetune VPT through self-supervised behavioral cloning and hindsight relabeling, bypassing the need for costly human text annotations. By leveraging pretrained models like VPT and MineCLIP and employing best practices from text-conditioned image generation, STEVE-1 costs just $60 to train and can follow a wide range of short-horizon open-ended text and visual instructions in Minecraft. STEVE-1 sets a new bar for open-ended instruction following in Minecraft with low-level controls (mouse and keyboard) and raw pixel inputs, far outperforming previous baselines. We provide experimental evidence highlighting key factors for downstream performance, including pretraining, classifier-free guidance, and data scaling. All resources, including our model weights, training scripts, and evaluation tools are made available for further research.

AK

144,783 görüntüleme • 3 yıl önce

I’ve been using the early version of Naval's new app Airchat for a year. Last week it went viral. And with it the haters came out in full force. Here’s why I’m still bullish on Airchat and believe it will change social media forever: 💬 Talk with real people: Twitter is the closest app to Airchat right now. But it’s completely impersonal. I’ve already built stronger connections with people on Airchat than I have from years of using Twitter. 🎤 Transcription is awesome. Voice notes are long. You don’t know if they're relevant. Having a text version of the voice note to scan makes it easier to find relevant content. 🗣️ Transcription supports multiple languages. I can record in Hebrew, and you’ll read it in English. It works great! Try it! 👥 Groups: launched this week. This focuses discussions around specific topics. Can’t wait to see more of this in the weeks ahead. 🏆 Talk to people you couldn't in other settings. Part of this is because it's so early. But you can speak to people like Naval that you wouldn't have been able to speak to otherwise. And a response to the haters: “It’s just Clubhouse” It’s not. Saying it’s Clubhouse is like saying Zoom is Clubhouse. Or a Podcast is Clubhouse. Clubhouse is sync. Airchat is async. Clubhouse is 1:many. Airchat is 1:1. “It’ll die down. Every new social app dies” Maybe. Creating a new social app is hard. It might die. But what’s that matter to you. The relationships you build last whether or not the app becomes a unicorn. “Twitter will just copy it like they copied Clubhouse” They might eventually. But what’s that matter unless you’re an investor in Airchat? I predict apps will copy it. Discord channels will offer this feature, and it will be awesome. Turning text chats into real conversations. WhatsApp is long overdue transcription for voice notes. Ideas from Airchat will get added to many apps. “I can’t use Airchat in places I can’t talk” The intimacy of Airchat is a feature, not a bug. Certain settings are meant for talking, others aren’t. Airchat is about conversations with people. You can’t do Zoom meetings everywhere either. “Twitter conversations are also conversations” Yes, but they’re far less intimate. You’re talking to a wall of text, not a person. You don’t date over text, you date over voice. It’s more intimate. You’re talking to a real person. “The content is boring” It will improve over time. There will be more content. The algorithm will improve to show you the best content. Groups will allow you to find specific content you're interested in. “I prefer Twitter” Good. Use Twitter. I use it too. “The hype will die down” Probably. But what’s that matter to you? You’ll only use apps that have 100m DAU? Successful social apps have early hype that dies down, and then grow steadily after that. 🪭 As you can tell, I’m a fan 🪭 If you’re on the app feel free to say hi. My username is `eliezer`. And feel free to DM if you need an invite.

Elie Steinbock — oss/acc

62,767 görüntüleme • 2 yıl önce