Loading video...

Video Failed to Load

Go Home

Got nerd-sniped into robotics research 2 months ago with no prior experience Ended up reimplementing computer vision on my Roomba with SLAM and now it's mapping the lab like a mini-explorer Here are my 3 most surprising takeaways:

179,221 views • 2 years ago •via X (Twitter)

10 Comments

Pavan Jayasinha's profile picture
Pavan Jayasinha2 years ago

#1 - Many tasks in robotics are better done without ML Coming from a software/ML background, I asked myself why traditional robotics hasn't been replaced with ML. I learned that ML often comes at the cost of speed & predictability over many traditional statistical approaches.

Pavan Jayasinha's profile picture
Pavan Jayasinha2 years ago

#2 - ROS[2] makes robot SW a lot more fun To get this to work, I had to convert an existing SLAM algo developed by Kevin Hu (VIP Lab) here at UWaterloo to ROS2. It made me realize how much of the grunt work ROS2 alleviates from someone just trying to ship a robotics project.

Pavan Jayasinha's profile picture
Pavan Jayasinha2 years ago

#3 - You learn much more by debugging and trying to figure shit out than following a course The main difficulty in this project was surprisingly a network issue through docker and navigating a complex build system. None of these are taught in ROS courses. ...

Pavan Jayasinha's profile picture
Pavan Jayasinha2 years ago

I did 3 out of a 22-hour ROS course ( and just jumped into porting the SLAM codebase I was given to ROS2. It takes much less time to get started than I thought.

Pavan Jayasinha's profile picture
Pavan Jayasinha2 years ago

I'd highly recommend playing around with robotics - think of an easy project in actuation/perception/control and take the first step. I'm very bullish on humanoid robotics, and psyched to play around with more robotics stuff when I'm back in the loo post-coop 🥘

λux's profile picture
λux2 years ago

banger...that's so cool build brother 🔥

Pavan Jayasinha's profile picture
Pavan Jayasinha2 years ago

🫡

Heskel Balas 🚁's profile picture
Heskel Balas 🚁2 years ago

You should check out what @mehul is up to with @maticrobots

albet's profile picture
albet2 years ago

So let’s say I want to make a robot to pick up/shovel a furball off the floor in a simple rectangular room. How impossible is that

Pavan Jayasinha's profile picture
Pavan Jayasinha2 years ago

1. Use SLAM (what I just implemented) to map the room and locate the robot 2. Add object detection to identify the furball 3. Use path planning to navigate to the furball 4. Implement a simple grasping or shoveling mechanism (matic robots implements something like this afaik)

Related Videos

Sergey Levine (Sergey Levine) is one of the world's top robotics researchers and co-founder of Physical Intelligence. We talked about where humanoid robotics is today, thoughts on the Chinese robotics ecosystem, and his predictions for future timelines. In this episode: • Current state of robotics and surprising capabilities so far • Chinese robotics compared to US ecosystem • If OpenAI and Anthropic got into robotics • His top robotics research paper recommendation • Predictions for when humanoid robotics will land Where to watch: • YouTube - • Spotify - • Apple Podcasts - • Transcript - Thank you to the sponsor of this episode for supporting my work: • WorkOS: makes your app Enterprise Ready with easy to use APIs to add SSO, SCIM, RBAC, and more in just a few lines of code, check them out at Chapters: 00:00 Intro 00:37 Where are we today 04:20 Most surprising capabilities so far 07:03 The most inspiring real world robotics 08:36 If OpenAI or Anthropic got into robotics 10:22 Chinese robotics 13:15 Will one lab breakout from the rest 16:59 Thoughts on a concrete roadmap 21:03 Generalization and demonstrating it 26:04 Types of data and which is best for robotics 34:34 Why humanoid robotics differs from Waymo 37:10 If humanoid robotics failed here is why 39:55 Are there hot take modeling architectures in robotics 42:05 Thoughts on AI safety in robotics 46:44 Top robotics research paper recommendation 49:35 Why is Boston Dynamics less top of mind 53:47 Advice for his younger self 56:42 Outro

Ryan Peterman

69,730 views • 18 days ago