João Gante's banner

João Gante

@joao_gante • 3,393 subscribers

Research Engineer @GoogleDeepMind, DiffusionGemma prev: huggingface 🤗 (transformers team), nPlan, PhD@IST 🇵🇹

Shorts

Up to 3x faster LLM generation with no extra resources/requirements - ngram speculation has landed in 🤗 transformers! 🏎️💨 All you need to do is to add `prompt_lookup_num_tokens=10` to your `generate` call and you'll get faster LLMs🔥 🧵How does it work?

Up to 3x faster LLM generation with no extra resources/requirements - ngram speculation has landed in 🤗 transformers! 🏎️💨 All you need to do is to add `prompt_lookup_num_tokens=10` to your `generate` call and you'll get faster LLMs🔥 🧵How does it work?

107,778 views

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

LET'S GO! Cursor using local 🤗 transformers models! You can now test ANY transformers-compatible LLM against your codebase. From hacking to production, it takes only a few minutes: anything `transformers` does, you can serve into your app 🔥 Here's a demo with Qwen3 4B:

LET'S GO! Cursor using local 🤗 transformers models! You can now test ANY transformers-compatible LLM against your codebase. From hacking to production, it takes only a few minutes: anything `transformers` does, you can serve into your app 🔥 Here's a demo with Qwen3 4B:

33,536 views • 1 year ago

No more content to load