Loading video...

Video Failed to Load

There was a problem loading this video. This could be due to a temporary network issue or the video might be unavailable.

Cool research by Reality Labs at Meta to allow touch typing on any surface using egocentric camera-based hand-tracking as input. Full paper here:

jessica

2,750 subscribers

34,313 views • 1 year ago •via X (Twitter)

Anya Rossi• Live Now

Private livecam show

8 Comments

Morten Haulik ᯅ1 year ago

@RealityLabs They just published that in October? Wild! I remember they did it with the Q2.

PDF GPT2 years ago

This is my favorite AI tool for reviewing reports. Just upload a report, ask for a summary, and get one in seconds. It's like ChatGPT, but built for documents. Try it for free.

asim ᯅ1 year ago

@RealityLabs this is so cool

xulipa1 year ago

@RealityLabs So excited about this

Lidoor Joseph1 year ago

@RealityLabs Cool

Ⓚutaka Kujua 🦈1 year ago

@RealityLabs Oh I LOVE this.

blingerang1 year ago

@RealityLabs When is this coming :/

Opic Technologies, Inc.1 year ago

@RealityLabs Cool! 😎

Related Videos

Steam Link for Meta Quest now has hand tracking. But you need to be opted into both the SteamVR Beta on your PC and the Quest App Beta It currently just passes hand tracking input to games using gesture based emulation via Skeletal Input

Steam Link for Meta Quest now has hand tracking. But you need to be opted into both the SteamVR Beta on your PC and the Quest App Beta It currently just passes hand tracking input to games using gesture based emulation via Skeletal Input

Brad Lynch

77,833 views • 2 years ago

Released by Reality Labs at Meta Research at #ECCV2024, Nymeria is a large-scale multimodal egocentric dataset for full-body motion understanding with potential applications in VR/MR headsets, smart glasses and more. More on this work + access to the dataset ➡️

Released by Reality Labs at Meta Research at #ECCV2024, Nymeria is a large-scale multimodal egocentric dataset for full-body motion understanding with potential applications in VR/MR headsets, smart glasses and more. More on this work + access to the dataset ➡️

AI at Meta

52,408 views • 1 year ago

A new paper by Disney Research introduces a 2-stage technique for motion tracking in physics-based character animation, using a pretrained latent space and RL to control diverse and unseen motions on both virtual and real-world humanoid robots. Paper:

A new paper by Disney Research introduces a 2-stage technique for motion tracking in physics-based character animation, using a pretrained latent space and RL to control diverse and unseen motions on both virtual and real-world humanoid robots. Paper:

The Humanoid Hub

20,083 views • 2 years ago

Been cooking on this for a while, but it's nearly ready. Ive built an iOS app using Snap Camera Kit to provide full body, hand, & face tracking into any app using VMC/OSC. No calibration, no weird setup. Just set the IP of your PC &use your fav VTube app (Using Warudo here).

Been cooking on this for a while, but it's nearly ready. Ive built an iOS app using Snap Camera Kit to provide full body, hand, & face tracking into any app using VMC/OSC. No calibration, no weird setup. Just set the IP of your PC &use your fav VTube app (Using Warudo here).

Brielle Garcia

213,760 views • 2 years ago

Last week we released Meta Chameleon: a new mixed-modal research model from Meta FAIR. Get the models ➡️ The 7B & 34B safety tuned models we’ve released can take any combination of text and images as input and produce text outputs using a new early fusion approach. While some LLMs have separate image and text encoders or decoders, Chameleon is one of the first publicly released approaches using a single unified architecture. We’re releasing Chameleon models under a research license to help democratize access to foundational mixed-modal models & further research on early fusion. Approach & training details in the paper ➡️

Last week we released Meta Chameleon: a new mixed-modal research model from Meta FAIR. Get the models ➡️ The 7B & 34B safety tuned models we’ve released can take any combination of text and images as input and produce text outputs using a new early fusion approach. While some LLMs have separate image and text encoders or decoders, Chameleon is one of the first publicly released approaches using a single unified architecture. We’re releasing Chameleon models under a research license to help democratize access to foundational mixed-modal models & further research on early fusion. Approach & training details in the paper ➡️

AI at Meta

54,428 views • 2 years ago

George Hotz explains his thoughts on working at OpenAI vs. META: > why would any researcher want to work at openai? > paper clip maxxing > can’t share your research > you’re on the BAD team > meanwhile who would’ve thought.. > zucc is the GOOD guy > open sourcing is the only way to fight the paper clip maximizers > you think you’ll be able to control superintelligence? you won’t. BASED AND CHECKED

George Hotz explains his thoughts on working at OpenAI vs. META: > why would any researcher want to work at openai? > paper clip maxxing > can’t share your research > you’re on the BAD team > meanwhile who would’ve thought.. > zucc is the GOOD guy > open sourcing is the only way to fight the paper clip maximizers > you think you’ll be able to control superintelligence? you won’t. BASED AND CHECKED

NIK

445,191 views • 1 year ago

Cold Plasma Fusion Reactor by ENG8 Based on Exotic Vacuum Object/plasmoid research. Produces electrical overunity of 3x+ and getting more efficient. Pretty cool to look at too.

Cold Plasma Fusion Reactor by ENG8 Based on Exotic Vacuum Object/plasmoid research. Produces electrical overunity of 3x+ and getting more efficient. Pretty cool to look at too.

Ashton Forbes

75,660 views • 1 year ago

Meta Develops Non-Invasive Neural Interface for Next-Generation Interaction Meta Reality Labs is advancing wrist-based neural interface technology designed to enable more natural and intuitive interaction with future computing platforms. The non-invasive device leverages finger-tracking to interpret user intent. Meta’s CEO has indicated that the technology could reach commercialization within the next few years. Video source: Meta #Meta #VR #AR #NeuralWristband #tech ----------------------- Wanna get your company on Wevolver too? Learn how:

Meta Develops Non-Invasive Neural Interface for Next-Generation Interaction Meta Reality Labs is advancing wrist-based neural interface technology designed to enable more natural and intuitive interaction with future computing platforms. The non-invasive device leverages finger-tracking to interpret user intent. Meta’s CEO has indicated that the technology could reach commercialization within the next few years. Video source: Meta #Meta #VR #AR #NeuralWristband #tech ----------------------- Wanna get your company on Wevolver too? Learn how:

Wevolver

332,635 views • 5 months ago

Today we’re introducing SceneScript, a novel method for reconstructing environments and representing the layout of physical spaces from Reality Labs at Meta Research. Details ➡️ SceneScript is able to directly infer a room’s geometry using end-to-end machine learning and represent it using language. Compared to previous approaches, this results in representations of physical scenes that are compact, complete, interpretable and extensible.

Today we’re introducing SceneScript, a novel method for reconstructing environments and representing the layout of physical spaces from Reality Labs at Meta Research. Details ➡️ SceneScript is able to directly infer a room’s geometry using end-to-end machine learning and represent it using language. Compared to previous approaches, this results in representations of physical scenes that are compact, complete, interpretable and extensible.

AI at Meta

334,490 views • 2 years ago

We’re taking action to put patients first. By using an activity-based funding model, healthcare facilities will be paid based on the number of operations they perform. This will help reduce waitlists and allow more people to receive their surgeries as quickly as possible.

We’re taking action to put patients first. By using an activity-based funding model, healthcare facilities will be paid based on the number of operations they perform. This will help reduce waitlists and allow more people to receive their surgeries as quickly as possible.

Danielle Smith

59,215 views • 1 year ago

Typing was NOT the most surprising part of the surface keyboard pushed in #MetaQuest latest update. Let me start by telling you how to try it yourself (available only on Quest 3) 1- On your phone, open the Meta Horizon app. 2- Go to Devices and select your headset 3- Tap Headset settings, then tap Advanced settings. 4- Tap the toggle next to Public Test Channel to join Meta Quest PTC (Public Test Channel) 5- After go in the headset settings > experimental features > surface keyboard ⌨️ Typing felt very responsive but what caught me off guard was the touchpad. 🖱️Even when looking straight at the screen the cursor moves accurately as if you are using an actual trackpad. Tapping acts as a click and it just works. ✌️ It also responds to the scrolling motion using 2 Fingers!

Typing was NOT the most surprising part of the surface keyboard pushed in #MetaQuest latest update. Let me start by telling you how to try it yourself (available only on Quest 3) 1- On your phone, open the Meta Horizon app. 2- Go to Devices and select your headset 3- Tap Headset settings, then tap Advanced settings. 4- Tap the toggle next to Public Test Channel to join Meta Quest PTC (Public Test Channel) 5- After go in the headset settings > experimental features > surface keyboard ⌨️ Typing felt very responsive but what caught me off guard was the touchpad. 🖱️Even when looking straight at the screen the cursor moves accurately as if you are using an actual trackpad. Tapping acts as a click and it just works. ✌️ It also responds to the scrolling motion using 2 Fingers!

Gabriele Romagnoli

20,670 views • 6 months ago

New AI research from Meta – CoTracker3 Simpler and Better Point Tracking by Pseudo-Labelling Real Videos. More details ➡️ Demo on Hugging Face ➡️ Building on our previous work on CoTracker, this new model demonstrates impressive tracking results where points can be tracked for a long time even when they're occluded or leave the field of view. CoTracker3 achieves state-of-the-art, outperforming all recent point tracking approaches on standard benchmarks — often by a substantial margin. We've released the research paper, code and a demo on Hugging Face — along with models available under an A-NC license to support further research in this space.

New AI research from Meta – CoTracker3 Simpler and Better Point Tracking by Pseudo-Labelling Real Videos. More details ➡️ Demo on Hugging Face ➡️ Building on our previous work on CoTracker, this new model demonstrates impressive tracking results where points can be tracked for a long time even when they're occluded or leave the field of view. CoTracker3 achieves state-of-the-art, outperforming all recent point tracking approaches on standard benchmarks — often by a substantial margin. We've released the research paper, code and a demo on Hugging Face — along with models available under an A-NC license to support further research in this space.

AI at Meta

218,996 views • 1 year ago

⛳️ Tiny Golf - A Very Cool Meta Horizon Start Developer Competition Submission! Mixed Reality games are challenging because they must adapt to real-world spaces with dynamic content, but they don’t have to be complex. Tiny Golf is a great example, using a minimalistic aesthetic and 🙌 hand-tracking pinch-and-pull gestures to launch the golf ball. I hope MR/VR games like this inspire many of you to build, there is just so much opportunity with XR today and 2026 is the year to start! 😉 📌 To get started, you can use: Meta XR All-In-One SDK, or a leaner option: Meta XR Interaction SDK, which pulls in the Meta XR Core SDK package and includes advanced hand-tracking features and passthrough.

⛳️ Tiny Golf - A Very Cool Meta Horizon Start Developer Competition Submission! Mixed Reality games are challenging because they must adapt to real-world spaces with dynamic content, but they don’t have to be complex. Tiny Golf is a great example, using a minimalistic aesthetic and 🙌 hand-tracking pinch-and-pull gestures to launch the golf ball. I hope MR/VR games like this inspire many of you to build, there is just so much opportunity with XR today and 2026 is the year to start! 😉 📌 To get started, you can use: Meta XR All-In-One SDK, or a leaner option: Meta XR Interaction SDK, which pulls in the Meta XR Core SDK package and includes advanced hand-tracking features and passthrough.

Dilmer 👓🔜 GDC

31,749 views • 7 months ago

What happens when robot world models learn from human experience at scale? 🤔 DreamDojo from NVIDIA Research is a generalist robot world model pretrained on 44K hours of egocentric human videos and then post-trained on robot data to generalize across new objects and environments. After distillation, it runs at 10 FPS for live teleoperation, policy evaluation, and model-based planning. Read the ICML paper to learn more 📄

What happens when robot world models learn from human experience at scale? 🤔 DreamDojo from NVIDIA Research is a generalist robot world model pretrained on 44K hours of egocentric human videos and then post-trained on robot data to generalize across new objects and environments. After distillation, it runs at 10 FPS for live teleoperation, policy evaluation, and model-based planning. Read the ICML paper to learn more 📄

NVIDIA Robotics

22,732 views • 1 month ago

An AI research assistant where you can ask a question about any file and automatically get the answer. #ChatGPT for your files. Here I use it on a research paper produced by my old lab at Stanford. #GPT3 #AI Jay Cummings Andrew D. Huberman, Ph.D. Lex Fridman

An AI research assistant where you can ask a question about any file and automatically get the answer. #ChatGPT for your files. Here I use it on a research paper produced by my old lab at Stanford. #GPT3 #AI Jay Cummings Andrew D. Huberman, Ph.D. Lex Fridman

Humata

35,157 views • 3 years ago

Twitch streamer RebeuDeter won a Lamborghini by outsmarting MrBeast Streamers had to keep their hand on the Lamborghini while blindfolded and take it off as close as possible to the 10 minute countdown His friend helped him win by using a vlog camera timer to tell him exactly when to take his hand off

Twitch streamer RebeuDeter won a Lamborghini by outsmarting MrBeast Streamers had to keep their hand on the Lamborghini while blindfolded and take it off as close as possible to the 10 minute countdown His friend helped him win by using a vlog camera timer to tell him exactly when to take his hand off

Jack

4,359,672 views • 4 months ago

[BLOOPER] More fun with our automated camera friends on the BBC News at One today. Presenter Ben Brown was undoubtedly glad he had his paper script to hand, as his autocue disappeared out of view.

[BLOOPER] More fun with our automated camera friends on the BBC News at One today. Presenter Ben Brown was undoubtedly glad he had his paper script to hand, as his autocue disappeared out of view.

The TV Room

14,805 views • 3 years ago

📣 This solo developer who currently works at Pixar as a Technical Director is building a VR rhythm game called Sock Puppet Superstar for Quest as an indie project, and over the last week he has been going viral on TikTok! 🤖Features/Tech: - Using Unity, OpenXR, hand tracking, and a custom voice synthesizer built in FMOD - Raise and lower your puppet to control the pitch - Works with hand tracking or controllers - Customize your sock puppet with colors, fabrics, accessories, and voices 📌 Devs can achieve this using the Meta XR Interaction SDK or Unity XR Toolkit (XR Hands) to add full hand support like what’s shown here. 🚀 Share this to give this solo indie dev more visibility! I’ll continue this bi-weekly with more indie devs who deserve the amplification!

📣 This solo developer who currently works at Pixar as a Technical Director is building a VR rhythm game called Sock Puppet Superstar for Quest as an indie project, and over the last week he has been going viral on TikTok! 🤖Features/Tech: - Using Unity, OpenXR, hand tracking, and a custom voice synthesizer built in FMOD - Raise and lower your puppet to control the pitch - Works with hand tracking or controllers - Customize your sock puppet with colors, fabrics, accessories, and voices 📌 Devs can achieve this using the Meta XR Interaction SDK or Unity XR Toolkit (XR Hands) to add full hand support like what’s shown here. 🚀 Share this to give this solo indie dev more visibility! I’ll continue this bi-weekly with more indie devs who deserve the amplification!

Dilmer

150,843 views • 3 months ago

Today at Meta FAIR we’re announcing three new cutting-edge developments in robotics and touch perception — and releasing a collection of artifacts to empower the community to build on this work. Details on all of this new work ➡️ 1️⃣ Meta Sparsh is the first general-purpose encoder for vision-based tactile sensing that works across many tactile sensors and many tasks. Trained on 460K+ tactile images using self-supervised learning. 2️⃣ Meta Digit 360 is a breakthrough artificial fingertip-based tactile sensor, equipped with 18+ sensing features to deliver detailed touch data with human-level precision and touch-sensing capabilities. 3️⃣ Meta Digit Plexus is a standardized platform for robotic sensor connections and interactions. It provides a hardware-software solution to integrate tactile sensors on a single robot hand and enables seamless data collection, control and analysis over a single cable. The potential impact of expanding capabilities and components like these for the open source community ranges from medical research to supply chain, manufacturing and much more. We’re excited to continue this work with the broader community.

Today at Meta FAIR we’re announcing three new cutting-edge developments in robotics and touch perception — and releasing a collection of artifacts to empower the community to build on this work. Details on all of this new work ➡️ 1️⃣ Meta Sparsh is the first general-purpose encoder for vision-based tactile sensing that works across many tactile sensors and many tasks. Trained on 460K+ tactile images using self-supervised learning. 2️⃣ Meta Digit 360 is a breakthrough artificial fingertip-based tactile sensor, equipped with 18+ sensing features to deliver detailed touch data with human-level precision and touch-sensing capabilities. 3️⃣ Meta Digit Plexus is a standardized platform for robotic sensor connections and interactions. It provides a hardware-software solution to integrate tactile sensors on a single robot hand and enables seamless data collection, control and analysis over a single cable. The potential impact of expanding capabilities and components like these for the open source community ranges from medical research to supply chain, manufacturing and much more. We’re excited to continue this work with the broader community.

AI at Meta

453,343 views • 1 year ago