Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Training Wan 2.1-1.3B to use Qwen3-VL-2B text encoder. Doing 33% text only, 33% VL only, and 33% both. Just pretraining the two text input linear layers for now. It is amazing how quickly a model can adapt to a different text encoder. This is 25,750 steps, BS of 10.

19,095 görüntüleme • 1 gün önce •via X (Twitter)

0 Yorum

Yorum bulunmuyor

Orijinal gönderinin yorumları burada görünecek

Benzer Videolar

hey babe <33 wanna see more? text me
0:19

Sensitive content

hey babe <33 wanna see more? text me

Chalina.l

70,696 görüntüleme • 9 ay önce

STEVE-1: A Generative Model for Text-to-Behavior in Minecraft paper page: Constructing AI models that respond to text instructions is challenging, especially for sequential decision-making tasks. This work introduces an instruction-tuned Video Pretraining (VPT) model for Minecraft called STEVE-1, demonstrating that the unCLIP approach, utilized in DALL-E 2, is also effective for creating instruction-following sequential decision-making agents. STEVE-1 is trained in two steps: adapting the pretrained VPT model to follow commands in MineCLIP's latent space, then training a prior to predict latent codes from text. This allows us to finetune VPT through self-supervised behavioral cloning and hindsight relabeling, bypassing the need for costly human text annotations. By leveraging pretrained models like VPT and MineCLIP and employing best practices from text-conditioned image generation, STEVE-1 costs just $60 to train and can follow a wide range of short-horizon open-ended text and visual instructions in Minecraft. STEVE-1 sets a new bar for open-ended instruction following in Minecraft with low-level controls (mouse and keyboard) and raw pixel inputs, far outperforming previous baselines. We provide experimental evidence highlighting key factors for downstream performance, including pretraining, classifier-free guidance, and data scaling. All resources, including our model weights, training scripts, and evaluation tools are made available for further research.

AK

144,704 görüntüleme • 2 yıl önce