Загрузка видео...

Не удалось загрузить видео

На главную

GPT 5.6 Luna Max vs 5.6 Sol Medium… Yeah, it’s not even a question 😭 Sol on MEDIUM produced a much better result than Luna on MAX—and needed only around HALF the time. Same harness. Better generation. Faster result. Luna may be cheaper, but if you have the budget,...

98,811 просмотров • 16 дней назад •via X (Twitter)

Комментарии: 0

Нет доступных комментариев

Здесь появятся комментарии из оригинального поста

Похожие видео

This is my "feel the AGI" moment: I used GPT-5.6 Sol to train my own autocorrect model that outperforms GPT-5.6 Sol (wtf??) I have no ML background. I have no idea what I'm doing. I just kept pushing Sol until it spat out a SOTA model. And I spent $0. The motivation: Years of talking to AI have made me terrible at typing. Rather than fix my skill issue, I decided to throw more AI at it. My idea was: instead of autocorrect that interrupts my flow, I want to type fast with mistakes and have AI clean it up after. I wanted the smallest local model possible, for speed, for battery life, for science! So I decided to train my own. Inspired by Andrej Karpathy’s autoresearch, I ran Codex /goal with this setup: pick an experiment, try it, record the results to a doc, throw it out if it fails, and plan the next experiment without repeating failures. I gave a few examples that had to pass, tight latency targets, and let it run. Sol did some amazing things. First, it scanned benchmarks and shortlisted base models: Qwen 3.5, Gemma 4, Liquid LFM 2.5. It found a dataset on HuggingFace for typed text. Then it built a simulator for fingers striking a Mac keyboard, modeling the physical layout with a Gaussian distribution around each key. It simulated striking the wrong key, wrong order, fat-fingering, etc. With the models + data + simulator, it fine-tuned using MLX right on my MacBook. It had a working prototype within an hour! But accuracy was pretty poor. — Problem 1: Tokenization Sol read papers, ran tests, and identified that the tokenizer was the bottleneck. Tokenization makes typos hard for the model to see, so it memorizes mappings instead of using its language priors. Sol tried ByT5, Google’s tokenizer-free byte-level LLM. This made a big improvement, but the model is old and lacked the knowledge needed to reach Sol performance. Sol dug deeper and realized a tokenizer-free model isn’t needed; instead, it used T5Gemma, an encoder-decoder model. This can understand the input deeply before producing output, and furthermore, Sol could post-train the encoder to improve performance. This gave a much higher ceiling. — Problem 2: Loss function Now the model was correcting some typos perfectly, but ignoring most. Sol realized that standard cross-entropy loss was teaching the model to avoid edits, because the vast majority of characters in the training data were left unmodified. The fix was wild: Sol wrote a custom loss function that byte-aligns the source and target strings, uses a dynamic programming algorithm to compute the minimum edits between the two, then weights correct edits much higher than copies. After a lot of tuning, this dramatically improved accuracy. — Problem 3: Autoregression One failure mode remained: if the model made a mistake, it couldn’t backtrack. It could only predict the next token. Teaching it to “think” like a reasoning model would solve this, but would be far too slow. Sol found a beautiful solution: instead of greedily predicting the next token, beam search over all possibilities. This parallelizes the exploration instead of one linear chain-of-thought. At the end, choose the path with highest cumulative log probability. This worked great, but made the experience worse, since the user wouldn’t see progress until the whole search was done. To fix this, Sol made a clever observation: after each search step, the longest common prefix among surviving branches is guaranteed to appear in the final result, so it can be displayed immediately. As the search progresses, weaker paths are dropped and the prefix grows, so the user sees continuous progress. Sol built all this as a custom MLX pipeline that does the parallel decoding on the MacBook GPU, with just ~40ms TTFT. It’s crazy fast and entirely local. — Final eval (error reduction rate, higher is better): - Apple autocorrect: 49.66% - GPT-5.6 Luna: 82.47% - GPT-5.6 Terra: 87.64% - GPT-5.6 Sol: 90.56% - Our model (1.7B): 91.02% Final cost: - 1 quota reset (thanks Tibo) - $0 (And yes, I verified there's no cheating. In fact, we test words scrubbed from the training data to prove the model isn’t memorizing) There were a ton more details and tangents I could write about: contrastive learning, GRPO, DPO, dynamic masking, and more. Sol is a fascinating and creative model. It blew my mind so many times. Don’t let a lack of experience stop you: Sol makes AI experiments accessible to anyone!

Anshu

178,934 просмотров • 1 месяц назад

Comeback Journal Had to borrow SOL and got rekt all the way down to 0.7 SOL. Took a 0.5 SOL trade and scalped +0.2 SOL. Saturday, Aug. 2 – 12:59 AM Portfolio: 0.9 SOL By the end of Aug. 2... Back to 0 SOL. Aug. 3 0xW sent me 0.6 SOL because he made money from the NA shill. Thank you again. Started only risking 0.05–0.1 SOL per trade to protect capital. Built it back to 0.4 SOL. Then p sent me 1 SOL for the same reason. That genuinely helped me during survival mode. Portfolio: 1.4 SOL. Saw LMEOW at 40k. Put 0.5 SOL in, leaving me with 0.9 SOL on the sidelines. Yeah, I oversized, but I had that feeling. What made me buy wasn't the chart. The narrative made me extremely bullish. Volume kept increasing, and I kept seeing more smart wallets buying than selling. Sold 1.5 SOL around 255k. Then the timeline started pushing the "OG" coin again. Momentum died. Round-tripped a few SOL and finally sold my remaining 0.7 SOL around 114k. Turned that 0.5 SOL trade into roughly 2.2 SOL before closing the position. I still think splitting the attention between two coins as a marketing stunt hurts everyone involved. Btw LJC, a lot of traders look up to you. Encouraging more PvP because the "OG" needs marketing instead of letting the market naturally bag work only makes the trenches worse. If it wasn't for that in the back of my mind i could've been at 10 solana today, thanks. Aug. 3 – 12:04 PM Current portfolio: 3.1 SOL.

Nofelines 🐂🀄️

61,484 просмотров • 14 дней назад