Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

My team is hiring in multimodal representation learning. We are working on tactile representations to equip robots with a physical understanding of the world. Bored of scraping disembodied internet data and looking for a new challenge? Apply here:

Mustafa Mukadam

2,056 subscribers

56,704 Aufrufe • vor 2 Jahren •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

3 Kommentare

Profilbild von Mustafa Mukadam

Mustafa Mukadamvor 2 Jahren

Some recent works from my team along these lines - Tactile to Depth: - Tactile Diffusion: - Tactile-VLM: - Tactile Seq-to-Seq SSM:

Profilbild von Alexander Soare

Alexander Soarevor 2 Jahren

I'll apply if PhD is not a hard requirement. LI for professional creds GH for whatever personal projects I can fit in

Profilbild von Ram Koppu

Ram Koppuvor 2 Jahren

@chris_j_paxton Are there any end to end open source foundation models for Robotics?

Ähnliche Videos

My team at FAIR AI at Meta is hiring postdocs. We are working in the space of dexterous manipulation and tactile sensing. If you are interested reach out / apply:

My team at FAIR AI at Meta is hiring postdocs. We are working in the space of dexterous manipulation and tactile sensing. If you are interested reach out / apply:

Mustafa Mukadam

40,587 Aufrufe • vor 2 Jahren

Does GPT understand the world? Here is what Ilya Sutskever, co-founder of OpenAI, says during a discussion with Jensen Huang, CEO of Nvidia: (1) When we train a large neural network to accurately predict the next word in lots of different texts from the internet, the AI is learning a world model. (2) On the surface, it may look like learning correlations in text, but it turns out that to 'just learn' statistical correlations in text, to compress information really well, what the neural network learns is some representation of the process that produced the text. (3) This text is a projection of the world...what the neural network is learning is aspects of the world, of people, of the human conditions, their hopes, dreams, motivations, their interactions...the situations we are in. The neural network learns a compressed, abstract, usable representation." Do you think learning representations = understanding? Are large language models simply stochastic parrots, or are they much more?

Does GPT understand the world? Here is what Ilya Sutskever, co-founder of OpenAI, says during a discussion with Jensen Huang, CEO of Nvidia: (1) When we train a large neural network to accurately predict the next word in lots of different texts from the internet, the AI is learning a world model. (2) On the surface, it may look like learning correlations in text, but it turns out that to 'just learn' statistical correlations in text, to compress information really well, what the neural network learns is some representation of the process that produced the text. (3) This text is a projection of the world...what the neural network is learning is aspects of the world, of people, of the human conditions, their hopes, dreams, motivations, their interactions...the situations we are in. The neural network learns a compressed, abstract, usable representation." Do you think learning representations = understanding? Are large language models simply stochastic parrots, or are they much more?

Alex Ker 🔭

1,367,023 Aufrufe • vor 2 Jahren

🤖ROBOTS ARE GETTING SMARTER AT TOUCHING THE REAL WORLD Researchers from UC Berkeley, NVIDIA and Stanford introduced T-Rex, a framework that combines vision, language and tactile sensing. Instead of relying on cameras alone, robots can now respond to physical contact in real time. Robots are no longer just seeing objects. They’re learning how to FEEL them.

🤖ROBOTS ARE GETTING SMARTER AT TOUCHING THE REAL WORLD Researchers from UC Berkeley, NVIDIA and Stanford introduced T-Rex, a framework that combines vision, language and tactile sensing. Instead of relying on cameras alone, robots can now respond to physical contact in real time. Robots are no longer just seeing objects. They’re learning how to FEEL them.

Coin Bureau

27,388 Aufrufe • vor 22 Tagen

Language models process data as one-dimensional sequences of tokens — ideal for text, but limited for understanding the physical world. In contrast, spatial intelligence embraces a 3D representation, unlocking a deeper understanding of space and time. More from martin_casado's interview with Fei-Fei Li and Justin Johnson:

Language models process data as one-dimensional sequences of tokens — ideal for text, but limited for understanding the physical world. In contrast, spatial intelligence embraces a 3D representation, unlocking a deeper understanding of space and time. More from martin_casado's interview with Fei-Fei Li and Justin Johnson:

a16z

64,662 Aufrufe • vor 1 Jahr

We are delighted to announce the publication of the Physical Literacy Consensus Statement for England! 📢 It's been developed to facilitate a shared understanding of physical literacy for those working in the sport, education, physical activity, health and youth sectors.

We are delighted to announce the publication of the Physical Literacy Consensus Statement for England! 📢 It's been developed to facilitate a shared understanding of physical literacy for those working in the sport, education, physical activity, health and youth sectors.

Sport England

122,900 Aufrufe • vor 2 Jahren

Bernt Børnich says teleoperation data and synthetic data are like a crutch to bootstrap your way to robots learning in the real world.

Bernt Børnich says teleoperation data and synthetic data are like a crutch to bootstrap your way to robots learning in the real world.

The Humanoid Hub

27,977 Aufrufe • vor 1 Jahr

Introducing ESM Cambrian. Unsupervised learning can invert biology at scale to reveal the hidden structure of the natural world. We’ve scaled up compute and data to train a new generation of protein language models. ESM C defines a new state of the art for protein representation learning.

Introducing ESM Cambrian. Unsupervised learning can invert biology at scale to reveal the hidden structure of the natural world. We’ve scaled up compute and data to train a new generation of protein language models. ESM C defines a new state of the art for protein representation learning.

Alex Rives

207,033 Aufrufe • vor 1 Jahr

Keith Pelley on what he is looking for with the possibility of hiring both a president and GM for the Leafs. "They have to be data centric."

Keith Pelley on what he is looking for with the possibility of hiring both a president and GM for the Leafs. "They have to be data centric."

TSN

37,532 Aufrufe • vor 3 Monaten

Watch our co-founder, , explain how the future of machine learning will lead us to a continuously updated digital representation of the world on the internet, and why this requires shifts in the way we access and allocate computational resources.

Watch our co-founder, , explain how the future of machine learning will lead us to a continuously updated digital representation of the world on the internet, and why this requires shifts in the way we access and allocate computational resources.

gensyn

29,885 Aufrufe • vor 1 Jahr

Introducing a new look, for a new world. At Verse, we’re on a mission to redefine digital environments. We aim to bridge the gap between the physical and digital worlds by providing our users with an experience that closely mimics real-life interactions. This is a world built for players, brands and communities to thrive in a new era of experiential gameplay and engagement. Welcome to the Internet of Reality. Welcome to Verse World.

Introducing a new look, for a new world. At Verse, we’re on a mission to redefine digital environments. We aim to bridge the gap between the physical and digital worlds by providing our users with an experience that closely mimics real-life interactions. This is a world built for players, brands and communities to thrive in a new era of experiential gameplay and engagement. Welcome to the Internet of Reality. Welcome to Verse World.

Verse World

64,546 Aufrufe • vor 1 Jahr

Haven't uploaded an updated Showreel in a while, so here we go! Still on the lookout for a new challenge and new team to join, so if you're looking for a gameplay animator with plenty of experience digging around in blueprints, state machines and sequencer, please reach out! 🌟

Haven't uploaded an updated Showreel in a while, so here we go! Still on the lookout for a new challenge and new team to join, so if you're looking for a gameplay animator with plenty of experience digging around in blueprints, state machines and sequencer, please reach out! 🌟

🌟 Lukas Lenniger 🌟

13,009 Aufrufe • vor 1 Jahr

Announcing a new Memory system for robots on Dimensional Robots in production generate thousands of hours of video, lidar, odometry, far too large to fit into your Agent context SpatialMemory2 builds a multimodal data store in latent space for your Agents Fully open source

Announcing a new Memory system for robots on Dimensional Robots in production generate thousands of hours of video, lidar, odometry, far too large to fit into your Agent context SpatialMemory2 builds a multimodal data store in latent space for your Agents Fully open source

stash

228,249 Aufrufe • vor 2 Monaten

Over 6,000 STEAM Labs are being established in government schools across Punjab, empowering students with hands-on learning in robotics, coding, and data analytics. Here are real moments from Govt. High School for Girls, Jallo Morh, where young minds are building robots and exploring the world of tech. Punjab is the trailblazer.

Over 6,000 STEAM Labs are being established in government schools across Punjab, empowering students with hands-on learning in robotics, coding, and data analytics. Here are real moments from Govt. High School for Girls, Jallo Morh, where young minds are building robots and exploring the world of tech. Punjab is the trailblazer.

Maryam Nawaz Sharif

25,235 Aufrufe • vor 10 Monaten

Today we are excited to open up Neuracore to the academic community! Neuracore is a new data foundation built to accelerate robot learning by removing one of the field’s biggest bottlenecks: capturing and working with high-fidelity multimodal robotics data. For the first time, researchers can store, view, and work with robotics data in a cloud-native system built specifically for large-scale learning, and we are making this core platform completely free for academia. The platform lets teams capture every sensor at its native rate, store and visualize data without loss, and then train and deploy models locally using our open-source code (Link in the comments). We are rolling out access to select academic institutions first. Anyone with an academic email can sign up, and if your institution is not part of the initial rollout, you will be able to join the waitlist directly. Beyond providing this infrastructure, we see an opportunity to build a global community where engineers and researchers can share, collaborate, and advance the frontier of robot learning together. Supported by our recent $3M pre-seed round led by Earlybird VC, we are excited to take this mission even further. Our long-term goal is for Neuracore to become the natural home for cutting-edge robot learning algorithms and real-world robotics experimentation, helping accelerate the next wave of Physical AI.

Today we are excited to open up Neuracore to the academic community! Neuracore is a new data foundation built to accelerate robot learning by removing one of the field’s biggest bottlenecks: capturing and working with high-fidelity multimodal robotics data. For the first time, researchers can store, view, and work with robotics data in a cloud-native system built specifically for large-scale learning, and we are making this core platform completely free for academia. The platform lets teams capture every sensor at its native rate, store and visualize data without loss, and then train and deploy models locally using our open-source code (Link in the comments). We are rolling out access to select academic institutions first. Anyone with an academic email can sign up, and if your institution is not part of the initial rollout, you will be able to join the waitlist directly. Beyond providing this infrastructure, we see an opportunity to build a global community where engineers and researchers can share, collaborate, and advance the frontier of robot learning together. Supported by our recent $3M pre-seed round led by Earlybird VC, we are excited to take this mission even further. Our long-term goal is for Neuracore to become the natural home for cutting-edge robot learning algorithms and real-world robotics experimentation, helping accelerate the next wave of Physical AI.

Neuracore

40,620 Aufrufe • vor 7 Monaten

Experiments in progress. The one on the right has been learning for ~3 hours, the one in the middle for ~1 hour, and the one on the left just started a few minutes ago. The initial motivation for making the physical Atari was just to commit ourselves to a subset of algorithms that can make progress in this setup. This commitment rules out algorithms that require billions of samples to learn (or worse, require multiple environments running in parallel). Atari games are simple enough that we should be able to show learning on them in a short amount of time with no prior knowledge. Since then, I've realized that this setup is also a good way to compare different paradigms in robotics in a principled way. These paradigms are sim2real, learning from tele-operated data, and learning directly on the robots. So far, I have observed that getting sim2real to work reliably is hard. It requires tweaks that don't scale. Policies that can play perfectly in simulation fall apart because of latencies and the messiness of the real world. These aspects could be modeled to improve the simulation, but not without sinking significant human engineering hours. I have higher hopes for learning from tele-operated data, but that requires a human to learn the task first. These experiments are on my to-do list. I have to learn to play some of the games well through the robot. I’m half-decent at playing Pong and Ms Pacman now. Learning directly on robots is looking like the most promising approach. This approach takes away pesky distribution shifts and makes it possible to have algorithms that continually improve with more data and time without any human intervention. It feels great to let experiments run overnight and wake up to find improved policies. With learning on robots, I should, in principle, be able to go on a long vacation and come back to find better policies for complex tasks beyond Atari games. Whether that is possible with current learning algorithms is a different question.

Experiments in progress. The one on the right has been learning for ~3 hours, the one in the middle for ~1 hour, and the one on the left just started a few minutes ago. The initial motivation for making the physical Atari was just to commit ourselves to a subset of algorithms that can make progress in this setup. This commitment rules out algorithms that require billions of samples to learn (or worse, require multiple environments running in parallel). Atari games are simple enough that we should be able to show learning on them in a short amount of time with no prior knowledge. Since then, I've realized that this setup is also a good way to compare different paradigms in robotics in a principled way. These paradigms are sim2real, learning from tele-operated data, and learning directly on the robots. So far, I have observed that getting sim2real to work reliably is hard. It requires tweaks that don't scale. Policies that can play perfectly in simulation fall apart because of latencies and the messiness of the real world. These aspects could be modeled to improve the simulation, but not without sinking significant human engineering hours. I have higher hopes for learning from tele-operated data, but that requires a human to learn the task first. These experiments are on my to-do list. I have to learn to play some of the games well through the robot. I’m half-decent at playing Pong and Ms Pacman now. Learning directly on robots is looking like the most promising approach. This approach takes away pesky distribution shifts and makes it possible to have algorithms that continually improve with more data and time without any human intervention. It feels great to let experiments run overnight and wake up to find improved policies. With learning on robots, I should, in principle, be able to go on a long vacation and come back to find better policies for complex tasks beyond Atari games. Whether that is possible with current learning algorithms is a different question.

Khurram Javed

52,110 Aufrufe • vor 7 Monaten

Delhi CM Rekha Gupta said about working with Parvesh Verma: "We are a team and we work together. There’s mutual understanding, and our focus is on delivering results. Together, we are committed to serving the people of Delhi efficiently."

Delhi CM Rekha Gupta said about working with Parvesh Verma: "We are a team and we work together. There’s mutual understanding, and our focus is on delivering results. Together, we are committed to serving the people of Delhi efficiently."

Times Algebra

18,539 Aufrufe • vor 11 Monaten

We are hiring for Harvey's Retrieval and Data team to expand Harvey to over 1,000 data sources in the 50+ countries we operate in. Our clients use Harvey to do research over legal, financial, tax, and regulatory data around the world. Thread on search + data at Harvey:

We are hiring for Harvey's Retrieval and Data team to expand Harvey to over 1,000 data sources in the 50+ countries we operate in. Our clients use Harvey to do research over legal, financial, tax, and regulatory data around the world. Thread on search + data at Harvey:

Gabe Pereyra

77,941 Aufrufe • vor 11 Monaten

Φ-SO : Physical Symbolic Optimization - Learning Physics from Data 🧠 The Physical Symbolic Optimization package uses deep reinforcement learning to discover physical laws from data. Here is Φ-SO discovering the analytical expression of a damped harmonic oscillator.

Φ-SO : Physical Symbolic Optimization - Learning Physics from Data 🧠 The Physical Symbolic Optimization package uses deep reinforcement learning to discover physical laws from data. Here is Φ-SO discovering the analytical expression of a damped harmonic oscillator.

Jousef Murad

433,149 Aufrufe • vor 2 Jahren

I was really impressed by the UMI gripper (Cheng Chi et al.), but a key limitation is that **force-related data wasn’t captured**: humans feel haptic feedback through the mechanical springs, but the robot couldn’t leverage that info, limiting the data’s value for fine-grained manipulation tasks. Led by my amazing students Yolanda Zhu and Binghao Huang, we designed a **portable visuo-tactile gripper** by integrating our dense, flexible tactile arrays with the UMI gripper to enable large-scale in-the-wild data collection. 🔗 We demonstrate **cross-modal representation learning** and **downstream policy learning** on tasks requiring in-hand state estimation (e.g., test tube reorientation) and fine-grained force sensing (e.g., pipette fluid transfer). Key takeaways: - Our flexible tactile arrays store the rich haptic information humans perceive as dense tactile signals. - Portability and robustness are key for in-the-wild data collection; our portable gripper is compact, lightweight, and durable. - Touch provides precise, robust measurements of in-hand object pose, invariant to lighting and viewpoint. - Cross-modal pretraining on large-scale in-the-wild data significantly improves policy robustness and sample efficiency (as shown many times before — and verified again here!). Also check out our previous investigations of dense, flexible tactile grids for understanding human-robot-environment interactions: - Dense tactile glove (Nature ’19): - 3D-ViTac (CoRL ’24):

I was really impressed by the UMI gripper (Cheng Chi et al.), but a key limitation is that force-related data wasn’t captured: humans feel haptic feedback through the mechanical springs, but the robot couldn’t leverage that info, limiting the data’s value for fine-grained manipulation tasks. Led by my amazing students Yolanda Zhu and Binghao Huang, we designed a portable visuo-tactile gripper by integrating our dense, flexible tactile arrays with the UMI gripper to enable large-scale in-the-wild data collection. 🔗 We demonstrate cross-modal representation learning and downstream policy learning on tasks requiring in-hand state estimation (e.g., test tube reorientation) and fine-grained force sensing (e.g., pipette fluid transfer). Key takeaways: - Our flexible tactile arrays store the rich haptic information humans perceive as dense tactile signals. - Portability and robustness are key for in-the-wild data collection; our portable gripper is compact, lightweight, and durable. - Touch provides precise, robust measurements of in-hand object pose, invariant to lighting and viewpoint. - Cross-modal pretraining on large-scale in-the-wild data significantly improves policy robustness and sample efficiency (as shown many times before — and verified again here!). Also check out our previous investigations of dense, flexible tactile grids for understanding human-robot-environment interactions: - Dense tactile glove (Nature ’19): - 3D-ViTac (CoRL ’24):

Yunzhu Li

13,188 Aufrufe • vor 1 Jahr