Skip navigation
BOOOM! transformers now has a baked-in http server w/ OpenAI spec compatible API
0:00 / 0:28

BOOOM! transformers now has a baked-in http server w/ OpenAI spec compatible API

Lysandre

@LysandreJik12K followers

Visit on X
99 likesDownloadOriginal post

24,806 views

BOOOM! transformers now has a baked-in http server w/ OpenAI spec compatible API Launch it with `transformers serve` and connect your favorite apps. Here I'm running 👋 Jan with local transformers and hot-swappable models. There is preliminary tool call support as well!

9 Comments

Comments from the original post on X (Twitter)

  • Lysandre's profile picture

    @LysandreJik

    Support just landed on `main` here: It ought to be rough around the edges still, so please play with it and let us know what could be improved!

    3 likes

  • Cameron R. Wolfe, Ph.D.'s profile picture

    @cwolferesearch

    @jandotai this is great for prototyping. really loving the recent product direction at HF - regaining a lot of excitement about using your tools again!

    2 likes

  • Quentin Lhoest 🤗's profile picture

    @lhoestq

    @jandotai Oh great ! Will play with it soon, see how it does for llm-based data processing :)

  • bdougie on the internet's profile picture

    @bdougieYO

    @huggingface @jandotai My side project last weekend is already out of date

  • Linus | web3 mobility network nRide's profile picture

    @dieterthemieter

    @huggingface @jandotai This is a game-changer for decentralized AI applications. Now we can focus on building protocols that aren't tied to specific vendors... the interoperability possibilities are endless! 👀

  • Luis's profile picture

    @lusxvr

    Today, we are open-sourcing our pipeline to deduplicate large-scale image datasets. On one GPU, we can deduplicate 10k images against 1M indexed test images in ~60 seconds. But how?

    491 likes

  • Omar Sanseviero's profile picture

    @osanseviero

    Want to learn about the research behind Gemma 3n? Altup - LAuReL - MatFormer - Activation sparsity - Universal Speech Model - Blog -

    546 likes

  • Niels Rogge's profile picture

    @NielsRogge

    New model alert in Transformers: EoMT! EoMT greatly simplifies the design of ViTs for image segmentation 🙌 Unlike Mask2Former and OneFormer which add complex modules like an adapter, pixel decoder and Transformer decoder on top, EoMT is just a ViT with a set of query tokens ✅

    141 likes

  • Unsloth AI's profile picture

    @UnslothAI

    We’ve teamed up with @GoogleDeepMind for a challenge with a $10,000 Unsloth prize! 🦥 Show off your best fine-tuned Gemma 3n model using Unsloth, optimized for an impactful task. The entire hackathon has $150,000 prizes to be won! Kaggle notebook:

    380 likes