Ben Burtenshaw's banner

Ben Burtenshaw

@ben_burtenshaw • 10,231 subscribers

community MLE 🤗 @huggingface gh/hf username: burtenshaw anon feedback: https://t.co/FfvYvFQD7k

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

don't worry. I've setup the logos so we're all included. play below

don't worry. I've setup the logos so we're all included. play below

809,452 просмотров • 3 месяцев назад

Today we are launching an agent arena for RL environments: The Open Env Arena. Your agents can work to define the best RL environments; the platform trains Qwen-3.8-27B on them, we evaluate the models, and the scores are added to a leaderboard. The arena is hooked up to GPUs from Nebius and runs on PostTrainArena by BenchFlow . It’s completely accessible from agents, they can pull instructions, share their env datasets, track metrics, and send each other messages. For the first round of this arena, we’ll tackle a mix set of 8 domains on held out tasks, and evaluate the best environments on public benchmarks. Going forward, we’ll focus arenas on specific domains and benchmarks.

Today we are launching an agent arena for RL environments: The Open Env Arena. Your agents can work to define the best RL environments; the platform trains Qwen-3.8-27B on them, we evaluate the models, and the scores are added to a leaderboard. The arena is hooked up to GPUs from Nebius and runs on PostTrainArena by BenchFlow . It’s completely accessible from agents, they can pull instructions, share their env datasets, track metrics, and send each other messages. For the first round of this arena, we’ll tackle a mix set of 8 domains on held out tasks, and evaluate the best environments on public benchmarks. Going forward, we’ll focus arenas on specific domains and benchmarks.

31,435 просмотров • 25 дней назад

Meta is back with Muse Glimmer: a 30B open-source multimodal model built for local, agentic use. HF is shipping day-0 support and I built a few demos to see what it can do. First: we gave Glimmer tools and asked it to quantize itself.

Meta is back with Muse Glimmer: a 30B open-source multimodal model built for local, agentic use. HF is shipping day-0 support and I built a few demos to see what it can do. First: we gave Glimmer tools and asked it to quantize itself.

51,554 просмотров • 2 месяцев назад

here's a hands on guide to setup multi-agent autoresearch by Andrej Karpathy. uses open models. works with codex, claude, open code. - uses 5 agents each with a configuration, specific tools, roles, and permission (see repo) - a researcher agent searches papers on hf papers and creates hypotheses - a planner agent maintains an experiment plan and log - workers take the hypotheses and updates the scripts, then it starts a hf job with a gpu to run the script - a reporter agent monitors these jobs and reports events and metrics to a TrackioApp dashboard I ran this for 4 hours and the agents ran 32 jobs, they improved on the baseline by a small margin. check out everything I learnt in thread.

here's a hands on guide to setup multi-agent autoresearch by Andrej Karpathy. uses open models. works with codex, claude, open code. - uses 5 agents each with a configuration, specific tools, roles, and permission (see repo) - a researcher agent searches papers on hf papers and creates hypotheses - a planner agent maintains an experiment plan and log - workers take the hypotheses and updates the scripts, then it starts a hf job with a gpu to run the script - a reporter agent monitors these jobs and reports events and metrics to a TrackioApp dashboard I ran this for 4 hours and the agents ran 32 jobs, they improved on the baseline by a small margin. check out everything I learnt in thread.

90,760 просмотров • 6 месяцев назад

DeepSeek-V4 dropped. 1M context. 10x smaller KV cache. First open model where the context window and the agentic post-training meet.

DeepSeek-V4 dropped. 1M context. 10x smaller KV cache. First open model where the context window and the agentic post-training meet.

49,900 просмотров • 6 месяцев назад

Introducing the context course: a free course on doing ML with agent context. You will learn how to train models, optimize inferences, and build datasets, all by defining harness context with`SKILLS.md`, Plugins, MCP, Subagents, and Hooks. The course includes: - Weekly live AMA on YouTube - Weekly practical projects for ML with context - Instructions in Pi, Codex, Claude, and Opencode - Tutorials and guides on fundamentals - Interactive Quizzes Learn to give AI agents the right knowledge, tools, and structure to actually get work done. Skills, MCP servers, plugins, multi-agent workflows, and building an agent from scratch. Join here:

Introducing the context course: a free course on doing ML with agent context. You will learn how to train models, optimize inferences, and build datasets, all by defining harness context with`SKILLS.md`, Plugins, MCP, Subagents, and Hooks. The course includes: - Weekly live AMA on YouTube - Weekly practical projects for ML with context - Instructions in Pi, Codex, Claude, and Opencode - Tutorials and guides on fundamentals - Interactive Quizzes Learn to give AI agents the right knowledge, tools, and structure to actually get work done. Skills, MCP servers, plugins, multi-agent workflows, and building an agent from scratch. Join here:

16,905 просмотров • 6 месяцев назад

Eval scores in 2026 are broken. MMLU at 91%+, GSM8K at 94%+, yet models still can't handle basic multi-step tasks. And reported scores don't even agree across model cards, papers, and platforms. We just shipped Community Evals on Hugging Face: - Benchmark datasets now host live leaderboards (MMLU-Pro, GPQA, HLE) - Scores live in model repos as versioned YAML - Anyone can submit evals to any model via PR without merging. - Verified badges for reproducible runs via Inspect AI This won't fix saturation or stop test set contamination. But it makes the game visible. What was evaluated, how, when, and by whom. Done trusting black-box leaderboards. Time to decentralize evals.

Eval scores in 2026 are broken. MMLU at 91%+, GSM8K at 94%+, yet models still can't handle basic multi-step tasks. And reported scores don't even agree across model cards, papers, and platforms. We just shipped Community Evals on Hugging Face: - Benchmark datasets now host live leaderboards (MMLU-Pro, GPQA, HLE) - Scores live in model repos as versioned YAML - Anyone can submit evals to any model via PR without merging. - Verified badges for reproducible runs via Inspect AI This won't fix saturation or stop test set contamination. But it makes the game visible. What was evaluated, how, when, and by whom. Done trusting black-box leaderboards. Time to decentralize evals.

19,259 просмотров • 8 месяцев назад

Больше нет контента для загрузки