Ivan Fioravanti
@ivanfioravanti • 43,760 subscribers
GenAI/LLM addicted, Apple MLX, Cloud computing, Kubernetes, Technology Advisor, Investor and Co-Founder & Board Member of CoreView. Local AI Ambassador.
Shorts
Videos
GLM-4.7-8bit (350GB) running at 19 toks/s on two M3 Ultra 512GB using Tensor Parallelism with EXO - MLX, versus 14 toks/s with single node. 🚀 Now context benchmarking & then OpenCode tests 🔥 Note: this is from sources, I had to change things to run it.
Ivan Fioravanti ᯅ327,687 Aufrufe • vor 9 Monaten
A video showing impact of max frequency change on my 2 x DGX Spark cluster with DeepSeek V4 Flash 0731 nvfp4 running like crazy at c4 using Mia recipe! Power consumption at wall is surely higher than the one reported by nvtop, but it gives you an idea: - 2455 MHz ~ 47W ~72º C - 2300 MHz ~34W ~67º C - 2200 MHz ~32W ~66º C - 2000 MHz ~27W ~63º C - 1800 MHz ~23W ~61º C This has minimal impact on decode speed that is memory bound, but there is impact on prefill and other compute bound activities. set it with: sudo nvidia-smi -lgc 0,2200 reset it with: sudo nvidia-smi -rgc I need to thank Paul to let me discover this magical trick!
Ivan Fioravanti ᯅ76,968 Aufrufe • vor 2 Monaten
MLX GLM 5.2 Distributed on two M3 Ultra 512GB 🔥 One M3 Ultra: 18.8 tokens/sec Two M3 Ultra: 23.4 tokens/sec Context: - PR by Pedro Cuenca is still open and probably there is room for improvement: - basic generation test to measure decoding performance here, I will do a full context benchmarking once PR is more mature - nvfp4 quantization used - Video alternates standard speed and x20, with one Mac first and distributed later. Enjoy! 🙌🏻
Ivan Fioravanti ᯅ88,821 Aufrufe • vor 3 Monaten
Qwen3.8 Flash Next q4 on DwarfStar one shot creation of "Super Human" a Super Mario NES-style platformer, but themed as Humanity's last stand vs. AI. It took 2 hours and 10M tokens using omp on M3 Ultra. Now I'm trying to improve sprites, let's see how it goes! Things like these were impossible few months ago using Local AI! Prompt: Build a single self-contained HTML file (inline CSS + JS, no external assets, no build step) called "Super Human" a Super Mario NES-style platformer, but themed as Humanity's last stand vs. AI. The player is "Super Human" (a tiny pixel worker in a red shirt + denim overalls); the enemies are mascots of famous AI labs. GOAL OF THE GAME: Run right across one long level, stomp/shoot AI mascots, collect data- coins, smash blocks, grab powerups, and reach the flag. Title screen → play → game over / win screens. Full HUD (score, coins, world, time, lives).
Ivan Fioravanti13,701 Aufrufe • vor 14 Tagen
deepseek-v4.1-flash I'm really curious to understand the architecture behind the scenes. 🤔 Model is really fast as you can see in the video! I've got an average decoding speed of around 350 t/s. BTW it is clearly experimental and: - currently, the model is under heavy load and failing quite often. - it thinks like there is not tomorrow - it keeps failing editing file - after 4M tokens my frogger game is still in progress - it can't see images out of the box on DeepSeek Harness, you have to add these config lines - id: deepseek-v4.1-flash-expires-on-0910 name: deepseek-v4.1-flash-expires-on-0910 contextWindow: 1000000 inputModalities: - text - image imagePixelBudget: 640000 imageMaxBytes: 1048576 So play with it, but be aware of the very early stage.
Ivan Fioravanti27,967 Aufrufe • vor 1 Monat
So what's your feeling on DeepSeek V4 Flash 0731? Surely great model for Local AI and coding but clearly not at GLM 5.2 or Kimi K3 level. Obviously, because it's a different category. For Frogger I tried 3 times using omp, but I always get a game too complex to play or with small issues here and there. I've used the mxfp4 version. But it worked great with web search, html and CSS issues with custom plugins on TMRNL X with Hermes Agent.
Ivan Fioravanti46,976 Aufrufe • vor 2 Monaten
Repo Prompt + o3 = Mind Blowing results! Plain video, no edit!
Ivan Fioravanti ᯅ180,129 Aufrufe • vor 1 Jahr
I respectfully disagree for several reasons. Calling a customer, whether free or paying, an idiot is simply wrong. OpenCode, like any other coding agent, clearly tries to preserve the prompt cache as much as possible. Otherwise, it would be painfully slow. The “Stop Using OpenCode” article, which I believe Dax is referring to, focused on specific cases that can invalidate the cached prefix. These issues become much more visible and painful when using local models. Modifying AGENTS.md is one example, as shown in this video. During this coding session, cache efficiency is extremely high. But the moment I modify AGENTS.md, boom: 62K tokens need to be processed again as new input. The date change is another valid point raised in the article. I understand that both issues may sound trivial, and personally I've never been impacted by them, but their impact on the OpenCode end-user experience can be significant when using local models. I agree that both the feedback and the article were too direct and were probably written by the author out of frustration. However, they also contained constructive points that could help improve the product. I use OpenCode alongside several other tools, and I like it overall. But I really dislike this kind of reaction to user feedback, regardless of how that feedback is expressed. I would have preferred a response focused more on listening, understanding the problem, and improving the product.
Ivan Fioravanti ᯅ37,594 Aufrufe • vor 2 Monaten
I'm sure many already knows: you can use DeepSeek V3 in Cursor right now: - add as OpenAI Base URL - use deepseek/deepseek-chat as model - use it in chat (not in composer) Here the combo created a snake game eating AI companies. 3 turns needed to get it working 1/2 🧵 Sonnet below
Ivan Fioravanti ᯅ153,359 Aufrufe • vor 1 Jahr
