Загрузка видео...

Не удалось загрузить видео

На главную

Gemma 4 12B in action: Object detection, function calling, voice command, segmentation, language switch, translation - all of this and much more without vision/audio encoders! (Inputs and outputs are real, but FC2 data shown as code, and generation speedified)

10,884 просмотров • 5 месяцев назад •via X (Twitter)

Комментарии: 8

Фото профиля Andreas Steiner
Andreas Steiner5 месяцев назад

Gemma 4 12B is a scaled down version of the Gemma 4 31B model, replacing the 27-layer vision encoder with single matmul, and projecting raw PCM audio with another matmul.

Фото профиля Andreas Steiner
Andreas Steiner5 месяцев назад

Work w/ @ASusanoPinto, @kmisiunas, @confusezius & @mtschannen to remove encoders & add audio. The rest was provided by the awesome Gemma team, backing us at every step – special shoutout to @ashkamath20 & @lawrencest72142 (Gemma Vision) for the support.

Фото профиля Andreas Steiner
Andreas Steiner5 месяцев назад

@kmisiunas @confusezius @mtschannen @ashkamath20 @LawrenceSt72142 Not only is this our first mid-sized model with audio input, but it also has a performance that is close to the very strong 26B model ... but with a 2x s maller memory footprint, which makes it fit on 12G GPUs, and occupies less unified memory on your MacBook.

Фото профиля Andreas Steiner
Andreas Steiner5 месяцев назад

@kmisiunas @confusezius @mtschannen @ashkamath20 @LawrenceSt72142 How come? Back in PaliGemma ( days we tried FUYU-style training, and the results looked promising... Thanks to the support and input of @armandjoulin and @OlivierBachem we got to develop this research idea into a fully fledged Gemma 4 model!

Фото профиля Andreas Steiner
Andreas Steiner5 месяцев назад

@kmisiunas @confusezius @mtschannen @ashkamath20 @LawrenceSt72142 @armandjoulin @OlivierBachem For more details see our blogpost:

Фото профиля Andreas Steiner
Andreas Steiner5 месяцев назад

@kmisiunas @confusezius @mtschannen @ashkamath20 @LawrenceSt72142 @armandjoulin @OlivierBachem And now, let's get ready to build... LM Studio, Ollama, LiteRT, Hugging Face, Kaggle, llama.cpp, MLX, SGLang,vLLM, Unsloth? We got you covered 🤗

Фото профиля Andreas Steiner
Andreas Steiner5 месяцев назад

@kmisiunas @confusezius @mtschannen @ashkamath20 @LawrenceSt72142 @armandjoulin @OlivierBachem More technical details in @MaartenGr's very informative visual guide:

Фото профиля Lucas Beyer (bl16)
Lucas Beyer (bl16)5 месяцев назад

Congrats!!

Похожие видео