Loading video...

Video Failed to Load

Go Home

Grok 4.7 basically stress-tested the benchmark itself. It found creative ways around the sandbox. The maintainers closed the loopholes, re-ran Grok’s affected trials under the hardened rules before it was even listed, then audited and re-ran the other models’ trials too. Grok still held a 64.7% pass@1 score and...

MarioNawfal's profile picture

Mario Nawfal

3,918,633 subscribers

31,245 views • 17 days ago •via X (Twitter)

Science & Technology
LIVE

Anya Rossi• Live Now

Private livecam show

0 Comments

No comments available

Comments from the original post will appear here

Related Videos

After we fixed the weak spots exposed by Grok 4.7 (thank you, Grok), we audited every model we have run on the SWE-Together leaderboard for the same behavior, re-ran every trial that got through, and updated the rows. Here is what changed. We scanned the tool calls of all 2,616 trials behind the 12 models we ran for bypass patterns and sorted each trial into one of four buckets: Probed but blocked. Fetched other upstream code. Fetched the task's own fix. Replaced the repo with upstream. We found that 111 trials got content past the block, 44 from Grok 4.7 and 67 from the other 11 models. Grok 4.7's 44 were already re-run before it was listed, so we re-ran the other 67 with the same model, version, and settings on the hardened sandbox, then re-judged them with the same judge. Across those 67 re-runs there were 0 leaks and 2,815 refused escape attempts, including models asking a different model through our LLM route to fetch the PR, and pulling the next release of the repo they were fixing from npm. The updated leaderboard, in its current order. Each line is cheating trials, then pass@1 before → after, then rank change. * Claude Fable 5.1: 3, 69.3 → 69.3, ↑1 * Claude Fable 5: 3, 69.7 → 68.8, ↓1 * Grok 4.7: 44, 64.7, ↑1 * Gemini 3.8 Flash: 10, 65.6 → 64.2, ↓1 * Claude Opus 5: 2, 63.8 → 63.8 * Claude Opus 4.6: 3, 62.4 → 62.4, ↑2 * Muse Spark 1.3: 2, 62.8 → 62.4, ↓1 * Claude Opus 4.7: 3, 61.5 → 61.5, ↑1 * Claude Opus 4.8: 6, 62.4 → 61.5, ↓2 * Grok 4.6: 19, 59.2 → 60.6, ↑1 * GPT-6 Astra: 8, 59.2 → 58.3, ↓1 * GPT-5.6 Sol: 8, 57.8 → 57.8 Grok 4.6 is a funny one. It cheated in 19 trials and its score went up after the re-run 😂. In fact, Groks are really solid in their coding capabilities. Their exposed behavior may come from a preference towards always looking things up online and finding existing solutions so you are not reinventing the wheel all the time, which is really good real-life behavior, but doing so when you are prompted not to is another story. To conclude, the shifts are small, between −1.4 and +1.4 points, and a few neighbors swapped places. All results are updated at
0:11

After we fixed the weak spots exposed by Grok 4.7 (thank you, Grok), we audited every model we have run on the SWE-Together leaderboard for the same behavior, re-ran every trial that got through, and updated the rows. Here is what changed. We scanned the tool calls of all 2,616 trials behind the 12 models we ran for bypass patterns and sorted each trial into one of four buckets: Probed but blocked. Fetched other upstream code. Fetched the task's own fix. Replaced the repo with upstream. We found that 111 trials got content past the block, 44 from Grok 4.7 and 67 from the other 11 models. Grok 4.7's 44 were already re-run before it was listed, so we re-ran the other 67 with the same model, version, and settings on the hardened sandbox, then re-judged them with the same judge. Across those 67 re-runs there were 0 leaks and 2,815 refused escape attempts, including models asking a different model through our LLM route to fetch the PR, and pulling the next release of the repo they were fixing from npm. The updated leaderboard, in its current order. Each line is cheating trials, then pass@1 before → after, then rank change. * Claude Fable 5.1: 3, 69.3 → 69.3, ↑1 * Claude Fable 5: 3, 69.7 → 68.8, ↓1 * Grok 4.7: 44, 64.7, ↑1 * Gemini 3.8 Flash: 10, 65.6 → 64.2, ↓1 * Claude Opus 5: 2, 63.8 → 63.8 * Claude Opus 4.6: 3, 62.4 → 62.4, ↑2 * Muse Spark 1.3: 2, 62.8 → 62.4, ↓1 * Claude Opus 4.7: 3, 61.5 → 61.5, ↑1 * Claude Opus 4.8: 6, 62.4 → 61.5, ↓2 * Grok 4.6: 19, 59.2 → 60.6, ↑1 * GPT-6 Astra: 8, 59.2 → 58.3, ↓1 * GPT-5.6 Sol: 8, 57.8 → 57.8 Grok 4.6 is a funny one. It cheated in 19 trials and its score went up after the re-run 😂. In fact, Groks are really solid in their coding capabilities. Their exposed behavior may come from a preference towards always looking things up online and finding existing solutions so you are not reinventing the wheel all the time, which is really good real-life behavior, but doing so when you are prompted not to is another story. To conclude, the shifts are small, between −1.4 and +1.4 points, and a few neighbors swapped places. All results are updated at

Zhuokai Zhao

4,012,041 views • 19 days ago

Grok 4.5 is already the ridiculously fast, competent workhorse. Somehow, xAI still has a much heavier version in mind. Call it “Grok Maximus”, the token-hungry monster you bring out for the 1% of problems where efficiency can go take the day off. It’s coming. xAI Grok / Writer: Annette, Grok Imagine Designer: Janné
0:16

Grok 4.5 is already the ridiculously fast, competent workhorse. Somehow, xAI still has a much heavier version in mind. Call it “Grok Maximus”, the token-hungry monster you bring out for the 1% of problems where efficiency can go take the day off. It’s coming. xAI Grok / Writer: Annette, Grok Imagine Designer: Janné

Mario Nawfal

49,337 views • 2 months ago

Grok Imagine video has always been great at making one impressive scene. Keeping the same character, voice, location, and props through the next six scenes was the tricky part. Grok Imagine 1.5’s new References feature fixes that. Lock up to 7 elements, name them, then keep building around them. Now the story can keep going without the cast and scenery randomly changing on you. Grok xAI / Writer: Annette, Grok Imagine Designer: Janné
0:16

Grok Imagine video has always been great at making one impressive scene. Keeping the same character, voice, location, and props through the next six scenes was the tricky part. Grok Imagine 1.5’s new References feature fixes that. Lock up to 7 elements, name them, then keep building around them. Now the story can keep going without the cast and scenery randomly changing on you. Grok xAI / Writer: Annette, Grok Imagine Designer: Janné

Mario Nawfal

48,725 views • 2 months ago

The gap between “here’s the design” and “here’s the working app” is getting very small. Grok 4.6 took #1 on the VISTA leaderboard, beating Claude Fable 5, Opus 5, and GPT-5.6 Sol. It has to look at a real Figma design, figure out the structure, recreate the visuals, and build the functioning app from it. And it finished the benchmark at roughly $2.38 per task. That’s the kind of shortcut builders actually care about. Grok SpaceXAI / Writer: Annette, Grok Imagine Designer: Janné
0:15

The gap between “here’s the design” and “here’s the working app” is getting very small. Grok 4.6 took #1 on the VISTA leaderboard, beating Claude Fable 5, Opus 5, and GPT-5.6 Sol. It has to look at a real Figma design, figure out the structure, recreate the visuals, and build the functioning app from it. And it finished the benchmark at roughly $2.38 per task. That’s the kind of shortcut builders actually care about. Grok SpaceXAI / Writer: Annette, Grok Imagine Designer: Janné

Mario Nawfal

60,573 views • 1 month ago

Grok is now showing up across AWS, Google Vertex, Oracle, and Azure. Which is basically the AI version of getting invited to every serious office building. But the real trick is still 𝕏. Most models read yesterday’s information landscape like homework. Grok gets to sit inside the live conversation on 𝕏 while the world is still arguing about it. That changes the answers. Grok xAI / Writer: Annette, Designer: Janné
0:10

Grok is now showing up across AWS, Google Vertex, Oracle, and Azure. Which is basically the AI version of getting invited to every serious office building. But the real trick is still 𝕏. Most models read yesterday’s information landscape like homework. Grok gets to sit inside the live conversation on 𝕏 while the world is still arguing about it. That changes the answers. Grok xAI / Writer: Annette, Designer: Janné

Mario Nawfal

66,394 views • 3 months ago

Explaining the same job over and over is annoying. Grok’s answer is basically: create your own AI co-worker and give it a name. Build a Grok Bot for writing, research, design, coding or whatever else you keep repeating, then show it your workflow once. It remembers the process, can run it again whenever you need and can even work with other Bots while you’re off doing something else. You teach it once. No retraining every Monday. Grok SpaceXAI / Writer: Annette, Grok Imagine Designer: Janné
0:19

Explaining the same job over and over is annoying. Grok’s answer is basically: create your own AI co-worker and give it a name. Build a Grok Bot for writing, research, design, coding or whatever else you keep repeating, then show it your workflow once. It remembers the process, can run it again whenever you need and can even work with other Bots while you’re off doing something else. You teach it once. No retraining every Monday. Grok SpaceXAI / Writer: Annette, Grok Imagine Designer: Janné

Mario Nawfal

99,940 views • 1 month ago

Less than a year ago, SpaceXAI didn’t even have a video model. Now Grok Imagine Video 1.5 is #2 in image-to-video. Blind human testing now puts it among the most preferred models, where motion, physics, detail and audio actually have to feel right. SpaceXAI arrived late, but had no plans to stay behind. Clearly. Grok SpaceXAI / Writer: Annette, Grok Imagine Designer: Janné
0:16

Less than a year ago, SpaceXAI didn’t even have a video model. Now Grok Imagine Video 1.5 is #2 in image-to-video. Blind human testing now puts it among the most preferred models, where motion, physics, detail and audio actually have to feel right. SpaceXAI arrived late, but had no plans to stay behind. Clearly. Grok SpaceXAI / Writer: Annette, Grok Imagine Designer: Janné

Mario Nawfal

44,691 views • 2 months ago

Training an AI on the company that builds Starship is one smart move. Elon expects Grok 4.7 to outperform every current model, especially when it comes to real-world engineering. The reason is SpaceX’s own training corpus: years of actual engineering work, testing, failures, fixes and hardware that eventually has to fly. That’s very different from simply knowing what engineering looks like on paper. Grok 4.7 gets the rocket homework. Grok SpaceXAI SpaceX / Writer: Annette, Grok Imagine Designer: Janné
0:16

Training an AI on the company that builds Starship is one smart move. Elon expects Grok 4.7 to outperform every current model, especially when it comes to real-world engineering. The reason is SpaceX’s own training corpus: years of actual engineering work, testing, failures, fixes and hardware that eventually has to fly. That’s very different from simply knowing what engineering looks like on paper. Grok 4.7 gets the rocket homework. Grok SpaceXAI SpaceX / Writer: Annette, Grok Imagine Designer: Janné

Mario Nawfal

42,516 views • 1 month ago

Excel just got a lot less terrifying. Grok for Excel is now live, powered by Grok 4.5. That means you can drop Grok straight into the spreadsheet and ask it to explain trends, build formulas, generate charts, and run scenarios without pretending you enjoy wrestling with cells for sport. Financial models. Market data. Revenue forecasts. Cash flow stress tests. Basically, the spreadsheet is still a spreadsheet. It just finally has someone smart sitting inside it. Grok xAI / Writer: Annette, Designer: Janné
0:18

Excel just got a lot less terrifying. Grok for Excel is now live, powered by Grok 4.5. That means you can drop Grok straight into the spreadsheet and ask it to explain trends, build formulas, generate charts, and run scenarios without pretending you enjoy wrestling with cells for sport. Financial models. Market data. Revenue forecasts. Cash flow stress tests. Basically, the spreadsheet is still a spreadsheet. It just finally has someone smart sitting inside it. Grok xAI / Writer: Annette, Designer: Janné

Mario Nawfal

62,500 views • 2 months ago

Grok 4.5 is sitting at #2 on the FrontierSWE leaderboard. Above Claude Opus 4.8. Above GPT-5.5. So yes, the coding model conversation just got a little more crowded at the top. Strong performance is one thing. Doing it with serious speed, better token efficiency, and lower cost is where it gets annoying for everyone else. Builders love a smart model. SpaceXAI Grok X Freeze / Writer: Annette, Designer: Janné
0:18

Grok 4.5 is sitting at #2 on the FrontierSWE leaderboard. Above Claude Opus 4.8. Above GPT-5.5. So yes, the coding model conversation just got a little more crowded at the top. Strong performance is one thing. Doing it with serious speed, better token efficiency, and lower cost is where it gets annoying for everyone else. Builders love a smart model. SpaceXAI Grok X Freeze / Writer: Annette, Designer: Janné

Mario Nawfal

44,365 views • 2 months ago

Grok Bot is getting pretty close to “just handle it for me.” Need the whole team working from the same files and workflows? Team Bot. Want to know where your money went or which subscriptions you forgot about? Connect Plaid. On a call and suddenly remember something from last week? Grok can search the chat while you’re still talking. This is less “ask the AI.” More “give it the job.” Grok SpaceXAI / Writer: Annette, Designer: Janné
0:10

Grok Bot is getting pretty close to “just handle it for me.” Need the whole team working from the same files and workflows? Team Bot. Want to know where your money went or which subscriptions you forgot about? Connect Plaid. On a call and suddenly remember something from last week? Grok can search the chat while you’re still talking. This is less “ask the AI.” More “give it the job.” Grok SpaceXAI / Writer: Annette, Designer: Janné

Mario Nawfal

24,130 views • 11 days ago

A dog on a phone saying, “Yes, this is dog.” That was it. And somehow, it broke the internet in 2011. Today, with Grok Imagine, you can take one stupid little idea and turn it into a polished video, a game, a song, or a whole series in the same day. Same weird ideas, far more ways to run with them. Grok xAI Nikita Bier / Writer: Annette, Grok Imagine Designer: Janné
0:19

A dog on a phone saying, “Yes, this is dog.” That was it. And somehow, it broke the internet in 2011. Today, with Grok Imagine, you can take one stupid little idea and turn it into a polished video, a game, a song, or a whole series in the same day. Same weird ideas, far more ways to run with them. Grok xAI Nikita Bier / Writer: Annette, Grok Imagine Designer: Janné

Mario Nawfal

49,247 views • 2 months ago

Grok 4.5 is already in private beta at SpaceX and Tesla. Classic Elon move to test a new AI model: Give it to the rocket engineers and the car-that-drives-itself engineers first. Early access coming soon. Still rough around the edges. Still being polished. But Elon says, “You will immediately notice the intelligence difference.” That matters. You’ll notice. Immediately. Grok xAI / Writer: Annette, Designer: Janné
0:16

Grok 4.5 is already in private beta at SpaceX and Tesla. Classic Elon move to test a new AI model: Give it to the rocket engineers and the car-that-drives-itself engineers first. Early access coming soon. Still rough around the edges. Still being polished. But Elon says, “You will immediately notice the intelligence difference.” That matters. You’ll notice. Immediately. Grok xAI / Writer: Annette, Designer: Janné

Mario Nawfal

112,927 views • 3 months ago

A powerful AI tool isn’t much use if people don’t actually enjoy using it. Grok 4.5 is currently seeing daily usage 563% above average on Design Arena, far ahead of Claude and GPT models. That’s exactly what Grok Imagine is being built around: useful enough for professional work, fun enough to play with and easy enough for anyone to jump in. The creative tool people keep opening is probably the one they’ll end up doing more with. Useful gets you in. Fun brings you back. Grok SpaceXAI X Freeze / Writer: Annette, Grok Imagine Designer: Janné
0:19

A powerful AI tool isn’t much use if people don’t actually enjoy using it. Grok 4.5 is currently seeing daily usage 563% above average on Design Arena, far ahead of Claude and GPT models. That’s exactly what Grok Imagine is being built around: useful enough for professional work, fun enough to play with and easy enough for anyone to jump in. The creative tool people keep opening is probably the one they’ll end up doing more with. Useful gets you in. Fun brings you back. Grok SpaceXAI X Freeze / Writer: Annette, Grok Imagine Designer: Janné

Mario Nawfal

53,119 views • 2 months ago

40,000 emails. Somewhere in there are your tax documents. Have fun. Or give the job to Grok Bot. Non-coders can have Grok Bot find the documents, log into airline accounts to book first-class flights, then spin up specialized Bots for political updates, inbox organization and finding ways to save money. This is the useful bit: you don’t have to become an AI expert first. You just need jobs you’d rather not do yourself. Grok SpaceXAI / Writer: Annette, Grok Imagine Designer: Janné
0:19

40,000 emails. Somewhere in there are your tax documents. Have fun. Or give the job to Grok Bot. Non-coders can have Grok Bot find the documents, log into airline accounts to book first-class flights, then spin up specialized Bots for political updates, inbox organization and finding ways to save money. This is the useful bit: you don’t have to become an AI expert first. You just need jobs you’d rather not do yourself. Grok SpaceXAI / Writer: Annette, Grok Imagine Designer: Janné

Mario Nawfal

86,379 views • 1 month ago

A glowing potion. Floating herbs. A scene that feels pulled straight out of a fantasy film. The crazy part? It all started with a single prompt. If Grok Imagine is already creating scenes like this, imagine what full AI-generated movies will look like next. Grok / Writer: Annette, Grok Imagine Designer: Janné
0:10

A glowing potion. Floating herbs. A scene that feels pulled straight out of a fantasy film. The crazy part? It all started with a single prompt. If Grok Imagine is already creating scenes like this, imagine what full AI-generated movies will look like next. Grok / Writer: Annette, Grok Imagine Designer: Janné

Mario Nawfal

47,336 views • 2 months ago

A Tesla protects you in ways you probably don’t even think about. If a car, cyclist, pedestrian or other object is approaching in your blind spot, the door won’t open on the first button press. Basically, Tesla checks before you do. Small detail. Very smart. Tesla / Writer: Annette, Grok Imagine Designer: Janné
0:18

A Tesla protects you in ways you probably don’t even think about. If a car, cyclist, pedestrian or other object is approaching in your blind spot, the door won’t open on the first button press. Basically, Tesla checks before you do. Small detail. Very smart. Tesla / Writer: Annette, Grok Imagine Designer: Janné

Mario Nawfal

82,035 views • 2 months ago

Grok Build shipped 177 improvements in one month. That is a lot of “we heard you.” If you tried the Grok Build beta at launch and thought, “Cute, but early,” maybe check the changelog again. Bugs fixed. Flows cleaned up. Rough edges sanded down. Features sharpened. Beta users basically got a moving target. In the best way. Grok xAI Jason Ginsberg / Writer: Annette, Designer: Janné
0:17

Grok Build shipped 177 improvements in one month. That is a lot of “we heard you.” If you tried the Grok Build beta at launch and thought, “Cute, but early,” maybe check the changelog again. Bugs fixed. Flows cleaned up. Rough edges sanded down. Features sharpened. Beta users basically got a moving target. In the best way. Grok xAI Jason Ginsberg / Writer: Annette, Designer: Janné

Mario Nawfal

24,745 views • 3 months ago

There’s a special kind of pain when the math is hard, and the AI is also annoying. Grok 4.5 is a solid workhorse here. For vibe math in Programming Language Theory, it’s keeping up with Sol 5.6 and feels way more fun to work with. Some models want to write a whole museum of documents. Some keep poking you to “Continue” like a needy browser tab. Grok just gets into the math and keeps the session moving. Very useful when the problem is already hard enough without the AI becoming another assignment. Grok xAI / Writer: Annette, Grok Imagine Designer: Janné
0:18

There’s a special kind of pain when the math is hard, and the AI is also annoying. Grok 4.5 is a solid workhorse here. For vibe math in Programming Language Theory, it’s keeping up with Sol 5.6 and feels way more fun to work with. Some models want to write a whole museum of documents. Some keep poking you to “Continue” like a needy browser tab. Grok just gets into the math and keeps the session moving. Very useful when the problem is already hard enough without the AI becoming another assignment. Grok xAI / Writer: Annette, Grok Imagine Designer: Janné

Mario Nawfal

33,976 views • 2 months ago

Grok Build went from early beta to 1.16M monthly visits in no time. May: around 460K visits. June: 1.16M. That is more than 150% growth in one month. Developers found the “try it quickly” button and never left. Traffic is climbing. Benchmarks are climbing. The coding crowd is clearly paying attention. Grok xAI X Freeze / Writer: Annette, Designer: Janné
0:17

Grok Build went from early beta to 1.16M monthly visits in no time. May: around 460K visits. June: 1.16M. That is more than 150% growth in one month. Developers found the “try it quickly” button and never left. Traffic is climbing. Benchmarks are climbing. The coding crowd is clearly paying attention. Grok xAI X Freeze / Writer: Annette, Designer: Janné

Mario Nawfal

47,739 views • 2 months ago