Marc Sun's banner

Marc Sun

@_marcsun • 1,697 subscribers

Machine Learning Engineer @huggingface Open Source team

Shorts

`transformers` + `torchao` quantization + `torch.compile` for faster inference speed and less memory usage 🔥 Demo of "meta-llama/Meta-Llama-3.1-8B-Instruct" quantized in 4-bit weight-only :

`transformers` + `torchao` quantization + `torch.compile` for faster inference speed and less memory usage 🔥 Demo of "meta-llama/Meta-Llama-3.1-8B-Instruct" quantized in 4-bit weight-only :

24,571 次观看

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

Millions of GGUF downloads later, those same llama.cpp checkpoints can now run in 🤗 transformers. Same models, more ways to use them, and fast local inference on Mac powered by ggml kernels! Blog: ggml kernels:

Millions of GGUF downloads later, those same llama.cpp checkpoints can now run in 🤗 transformers. Same models, more ways to use them, and fast local inference on Mac powered by ggml kernels! Blog: ggml kernels:

66,530 次观看 • 1 个月前

Your LLM knows the answer before it writes a single token. So we stopped making it write. transformers serve now turns any LLM or VLM into a decision model with `/v1/systemone` endpoint. Decision model standardization coming soon in transformers by Arthur Zucker Demos below👇

Your LLM knows the answer before it writes a single token. So we stopped making it write. transformers serve now turns any LLM or VLM into a decision model with `/v1/systemone` endpoint. Decision model standardization coming soon in transformers by Arthur Zucker Demos below👇

13,808 次观看 • 17 天前

没有更多内容可加载