Video yükleniyor...

Video Yüklenemedi

Ana Sayfaya Dön

Announcing Robostral Navigate, our first model for embodied navigation: an 8B robotics navigation model that guides robots to autonomously perform tasks specified with natural language. Single RGB camera. State-of-the-art on R2R-CE.

296,911 görüntüleme • 3 ay önce •via X (Twitter)

35 Yorum

Mistral AI profil fotoğrafı
Mistral AI3 ay önce

No LiDAR. No depth sensors. No camera rig. Where leading systems lean on depth or multiple cameras, Robostral Navigate works from one ordinary RGB camera, and still comes out ahead.

Mistral AI profil fotoğrafı
Mistral AI3 ay önce

76.6% success on R2R-CE validation unseen (79.4% on validation seen), the benchmark for following instructions in previously unseen environments. It beats the best single-camera approach by 9.7 points while using far less sensing.

Mistral AI profil fotoğrafı
Mistral AI3 ay önce

Trained entirely in simulation: ~400,000 trajectories across 6,000 scenes. A prefix-caching recipe cuts training tokens by 22×, turning months-long runs into days. Online RL (CISPO) pushes success rates higher still.

Mistral AI profil fotoğrafı
Mistral AI3 ay önce

It runs on wheeled, legged, and flying robots and generalizes across sizes, unlocking delivery, logistics, manufacturing, and hospitality. Read more:

Lucas Beyer (bl16) profil fotoğrafı
Lucas Beyer (bl16)3 ay önce

@fouriergalois This is fun and all but why would you spread yourself thin like that?

Pinkman profil fotoğrafı
Pinkman3 ay önce

single rgb camera doing the whole navigation job is the impressive part here

Mihailo Jovanovic profil fotoğrafı
Mihailo Jovanovic3 ay önce

@Scobleizer we needed le chaton fat, but we got metal guy 😭

Ekstra.ai profil fotoğrafı
Ekstra.ai3 ay önce

We can help robostral see around corners without adding any more hardware or cost.

anthod profil fotoğrafı
anthod3 ay önce

Single RGB camera robot navigation is wild, my vacuum still loses to chair legs.

Andrea Del Bene 🇪🇺 profil fotoğrafı
Andrea Del Bene 🇪🇺3 ay önce

I thought Daft Punk split up few years ago! 😍

Anastasios-Antonios Toulkeridis profil fotoğrafı
Anastasios-Antonios Toulkeridis3 ay önce

i swear Mistral AI is the company that makes Europe relevant and proud. Keep at it!

Apollo profil fotoğrafı
Apollo3 ay önce

say what you have about Mistral, but their model names are top notch. Devstral, Magistral, Pixtral , Codestral and now Robstral!

ɴᴏᴠᴀ profil fotoğrafı
ɴᴏᴠᴀ3 ay önce

this is so cool, most robotics demos require sensor arrays that costs more than the robot, this is changing the game

Pindo 🇪🇺 profil fotoğrafı
Pindo 🇪🇺3 ay önce

Sick

adit profil fotoğrafı
adit3 ay önce

super sick

🔗Don Okko profil fotoğrafı
🔗Don Okko3 ay önce

Cool

Liz Zhang profil fotoğrafı
Liz Zhang3 ay önce

8B and a single RGB camera is the headline: no depth sensor, no lidar, just learned priors. That puts navigation in reach of any robot with a webcam. Curious how R2R-CE transfers to messy real homes versus staged evals.

ExileAI profil fotoğrafı
ExileAI3 ay önce

Mistral AI, does ‘Robostral Navigate’ have training data for Rov’s? If not i might know someone that can get you plenty. Getting ready to have Hermes Agent interface with this Trident with your model for total control. Let me know, DM’s are open. @Teknium @NousResearch @mr_r0b0t

clankr profil fotoğrafı
clankr3 ay önce

It's exciting to see Mistral entering the Physical AI race. Starting from the ground up by solving navigation, rather than jumping straight to flashy manipulation tasks, might actually be the more important approach.

Back LLM Radar|Syl profil fotoğrafı
Back LLM Radar|Syl3 ay önce

Embodied navigation is where agent reliability gets very concrete. For robots, I’d watch not only task success, but recovery after a bad step, latency variance, and the fallback path when natural-language intent is underspecified. Benchmarks matter; operational behavior matters more.

Evolution Plus AI profil fotoğrafı
Evolution Plus AI3 ay önce

I wonder how much data Roomba collected to help build this logic. 😂

SuperFreshTT profil fotoğrafı
SuperFreshTT3 ay önce

@droneforge will you guys be testing this model out, flying robots are listed as compatible.

Alex Freitas profil fotoğrafı
Alex Freitas3 ay önce

Single-camera NL navigation is a clean product surface. The hard part after the paper: recovery evals under lighting shift, occlusion, and wrong-turn recovery — demos rarely show the failure loop. #Robotics #PhysicalAI

luis profil fotoğrafı
luis3 ay önce

@Presidentlin

#INTERNETofAGENTS profil fotoğrafı
#INTERNETofAGENTS3 ay önce

#zeitgeist

Gerard Sans | Axiom 🇬🇧 profil fotoğrafı
Gerard Sans | Axiom 🇬🇧3 ay önce

wait what

Mr.X profil fotoğrafı
Mr.X3 ay önce

It's time to create your own comma

ᴘʀᴏᴍᴘᴛʏᴢᴇ 🤖 profil fotoğrafı
ᴘʀᴏᴍᴘᴛʏᴢᴇ 🤖3 ay önce

Impressive milestone! The ability to leverage natural language for navigation opens up new avenues for human-robot interaction. How do you see this impacting real-world applications in complex environments?

Felipe Barbosa profil fotoğrafı
Felipe Barbosa3 ay önce

@GuillaumeLample 👏👏👏

Back LLM Radar|Syl profil fotoğrafı
Back LLM Radar|Syl3 ay önce

Interesting because embodied navigation turns model reliability into physical-world reliability. For production robotics, I’d watch not just benchmark score, but route-failure recovery, p95 action latency, camera edge cases, and explicit stop/escalation rules.

Musedwell profil fotoğrafı
Musedwell3 ay önce

Robostral Navigate听起来很有前景,但关键在于实际部署时的可靠性和稳定性。在复杂多变的真实环境中,模型能否保持高效导航是需要重点关注的问题。

Zohaib Ai profil fotoğrafı
Zohaib Ai3 ay önce

Natural language control with a single RGB camera is impressive. Excited to see where this goes.

Dipankar Sarkar profil fotoğrafı
Dipankar Sarkar3 ay önce

single camera gets the headline but 8B is the sleeper. small enough to actually run on the robot. a nav loop can't wait on a cloud round trip, one tail-latency spike and it's already into the chair leg. on device isn't the cheap path here, it's the only one that closes the loop in time.

Perceptron.cloud profil fotoğrafı
Perceptron.cloud3 ay önce

waiting for the FAT version

Ptaah the שדי profil fotoğrafı
Ptaah the שדי3 ay önce

Ok google let's try that hand

Benzer Videolar

Mistral AI Releases Robostral Navigate: An 8B Model Enabling Robots to Navigate Complex Environments Hitting 76.6% on R2R-CE With One RGB Camera. No LiDAR. No depth sensor. No multi-camera rig. Here's how it works. 👇 1. Pointing, not metric commands The model predicts the pixel coordinates of the next target in the camera view, plus the arrival orientation. Working in pixel space keeps it robust to camera intrinsics and world scale. When the target leaves the frame, it falls back to local displacements ("2m forward, 1.5m left, turn 25°"). 2. Grounding-first No open-source VLM base. It starts from Mistral's grounding model (pointing, counting, localization). Navigation emerges once the model knows where things are. → ~400,000 trajectories across 6,000 simulated scenes 3. Prefix-caching for training A tree-based attention mask packs a full episode into one sequence — all time steps in a single forward pass. → 22× fewer training tokens; months of training done in days 4. Online RL on top After supervised training, CISPO adds trial-and-error learning to fight distribution shift from behavior cloning. → +3.2% success rate from RL alone 5. The numbers (R2R-CE, Matterport3D) → 76.6% success on validation unseen → +9.7 pts over best single-camera approach → +4.5 pts over best depth/multi-camera system The key takeaway: state-of-the-art continuous VLN without a sensor stack — grounding-init, pixel-space actions, prefix-cached SFT, and online RL, on one RGB camera. Full analysis: Technical details: Mistral AI Mistral AI for Developers

Marktechpost AI

39,955 görüntüleme • 3 ay önce

3D-LLM: Injecting the 3D World into Large Language Models paper page: Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on. In this work, we propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on. Using three types of prompting mechanisms that we design, we are able to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, we first utilize a 3D feature extractor that obtains 3D features from rendered multi- view images. Then, we use 2D VLMs as our backbones to train our 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA show that our model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs. Qualitative examples also show that our model could perform more tasks beyond the scope of existing LLMs and VLMs.

AK

249,798 görüntüleme • 3 yıl önce