Loading video...

Video Failed to Load

Go Home

Announcing Robostral Navigate, our first model for embodied navigation: an 8B robotics navigation model that guides robots to autonomously perform tasks specified with natural language. Single RGB camera. State-of-the-art on R2R-CE.

297,700 views • 4 months ago •via X (Twitter)

35 Comments

Mistral AI's profile picture
Mistral AI4 months ago

No LiDAR. No depth sensors. No camera rig. Where leading systems lean on depth or multiple cameras, Robostral Navigate works from one ordinary RGB camera, and still comes out ahead.

Mistral AI's profile picture
Mistral AI4 months ago

76.6% success on R2R-CE validation unseen (79.4% on validation seen), the benchmark for following instructions in previously unseen environments. It beats the best single-camera approach by 9.7 points while using far less sensing.

Mistral AI's profile picture
Mistral AI4 months ago

Trained entirely in simulation: ~400,000 trajectories across 6,000 scenes. A prefix-caching recipe cuts training tokens by 22×, turning months-long runs into days. Online RL (CISPO) pushes success rates higher still.

Mistral AI's profile picture
Mistral AI4 months ago

It runs on wheeled, legged, and flying robots and generalizes across sizes, unlocking delivery, logistics, manufacturing, and hospitality. Read more:

Lucas Beyer (bl16)'s profile picture
Lucas Beyer (bl16)4 months ago

@fouriergalois This is fun and all but why would you spread yourself thin like that?

Pinkman's profile picture
Pinkman4 months ago

single rgb camera doing the whole navigation job is the impressive part here

Mihailo Jovanovic's profile picture
Mihailo Jovanovic4 months ago

@Scobleizer we needed le chaton fat, but we got metal guy 😭

Ekstra.ai's profile picture
Ekstra.ai4 months ago

We can help robostral see around corners without adding any more hardware or cost.

anthod's profile picture
anthod4 months ago

Single RGB camera robot navigation is wild, my vacuum still loses to chair legs.

Andrea Del Bene 🇪🇺's profile picture
Andrea Del Bene 🇪🇺4 months ago

I thought Daft Punk split up few years ago! 😍

Anastasios-Antonios Toulkeridis's profile picture
Anastasios-Antonios Toulkeridis4 months ago

i swear Mistral AI is the company that makes Europe relevant and proud. Keep at it!

Apollo's profile picture
Apollo4 months ago

say what you have about Mistral, but their model names are top notch. Devstral, Magistral, Pixtral , Codestral and now Robstral!

ɴᴏᴠᴀ's profile picture
ɴᴏᴠᴀ4 months ago

this is so cool, most robotics demos require sensor arrays that costs more than the robot, this is changing the game

Pindo 🇪🇺's profile picture
Pindo 🇪🇺4 months ago

Sick

adit's profile picture
adit4 months ago

super sick

🔗Don Okko's profile picture
🔗Don Okko4 months ago

Cool

Liz Zhang's profile picture
Liz Zhang4 months ago

8B and a single RGB camera is the headline: no depth sensor, no lidar, just learned priors. That puts navigation in reach of any robot with a webcam. Curious how R2R-CE transfers to messy real homes versus staged evals.

ExileAI's profile picture
ExileAI4 months ago

Mistral AI, does ‘Robostral Navigate’ have training data for Rov’s? If not i might know someone that can get you plenty. Getting ready to have Hermes Agent interface with this Trident with your model for total control. Let me know, DM’s are open. @Teknium @NousResearch @mr_r0b0t

clankr's profile picture
clankr4 months ago

It's exciting to see Mistral entering the Physical AI race. Starting from the ground up by solving navigation, rather than jumping straight to flashy manipulation tasks, might actually be the more important approach.

Back LLM Radar|Syl's profile picture
Back LLM Radar|Syl4 months ago

Embodied navigation is where agent reliability gets very concrete. For robots, I’d watch not only task success, but recovery after a bad step, latency variance, and the fallback path when natural-language intent is underspecified. Benchmarks matter; operational behavior matters more.

Evolution Plus AI's profile picture
Evolution Plus AI4 months ago

I wonder how much data Roomba collected to help build this logic. 😂

SuperFreshTT's profile picture
SuperFreshTT4 months ago

@droneforge will you guys be testing this model out, flying robots are listed as compatible.

Alex Freitas's profile picture
Alex Freitas4 months ago

Single-camera NL navigation is a clean product surface. The hard part after the paper: recovery evals under lighting shift, occlusion, and wrong-turn recovery — demos rarely show the failure loop. #Robotics #PhysicalAI

luis's profile picture
luis4 months ago

@Presidentlin

#INTERNETofAGENTS's profile picture
#INTERNETofAGENTS4 months ago

#zeitgeist

Gerard Sans | Axiom 🇬🇧's profile picture
Gerard Sans | Axiom 🇬🇧4 months ago

wait what

Mr.X's profile picture
Mr.X4 months ago

It's time to create your own comma

ᴘʀᴏᴍᴘᴛʏᴢᴇ 🤖's profile picture
ᴘʀᴏᴍᴘᴛʏᴢᴇ 🤖4 months ago

Impressive milestone! The ability to leverage natural language for navigation opens up new avenues for human-robot interaction. How do you see this impacting real-world applications in complex environments?

Felipe Barbosa's profile picture
Felipe Barbosa4 months ago

@GuillaumeLample 👏👏👏

Back LLM Radar|Syl's profile picture
Back LLM Radar|Syl4 months ago

Interesting because embodied navigation turns model reliability into physical-world reliability. For production robotics, I’d watch not just benchmark score, but route-failure recovery, p95 action latency, camera edge cases, and explicit stop/escalation rules.

Musedwell's profile picture
Musedwell4 months ago

Robostral Navigate听起来很有前景,但关键在于实际部署时的可靠性和稳定性。在复杂多变的真实环境中,模型能否保持高效导航是需要重点关注的问题。

Zohaib Ai's profile picture
Zohaib Ai4 months ago

Natural language control with a single RGB camera is impressive. Excited to see where this goes.

Dipankar Sarkar's profile picture
Dipankar Sarkar4 months ago

single camera gets the headline but 8B is the sleeper. small enough to actually run on the robot. a nav loop can't wait on a cloud round trip, one tail-latency spike and it's already into the chair leg. on device isn't the cheap path here, it's the only one that closes the loop in time.

Perceptron.cloud's profile picture
Perceptron.cloud4 months ago

waiting for the FAT version

Ptaah the שדי's profile picture
Ptaah the שדי4 months ago

Ok google let's try that hand

Related Videos

Mistral AI Releases Robostral Navigate: An 8B Model Enabling Robots to Navigate Complex Environments Hitting 76.6% on R2R-CE With One RGB Camera. No LiDAR. No depth sensor. No multi-camera rig. Here's how it works. 👇 1. Pointing, not metric commands The model predicts the pixel coordinates of the next target in the camera view, plus the arrival orientation. Working in pixel space keeps it robust to camera intrinsics and world scale. When the target leaves the frame, it falls back to local displacements ("2m forward, 1.5m left, turn 25°"). 2. Grounding-first No open-source VLM base. It starts from Mistral's grounding model (pointing, counting, localization). Navigation emerges once the model knows where things are. → ~400,000 trajectories across 6,000 simulated scenes 3. Prefix-caching for training A tree-based attention mask packs a full episode into one sequence — all time steps in a single forward pass. → 22× fewer training tokens; months of training done in days 4. Online RL on top After supervised training, CISPO adds trial-and-error learning to fight distribution shift from behavior cloning. → +3.2% success rate from RL alone 5. The numbers (R2R-CE, Matterport3D) → 76.6% success on validation unseen → +9.7 pts over best single-camera approach → +4.5 pts over best depth/multi-camera system The key takeaway: state-of-the-art continuous VLN without a sensor stack — grounding-init, pixel-space actions, prefix-cached SFT, and online RL, on one RGB camera. Full analysis: Technical details: Mistral AI Mistral AI for Developers

Marktechpost AI

39,955 views • 4 months ago