Загрузка видео...

Не удалось загрузить видео

На главную

Cool research by Reality Labs at Meta to allow touch typing on any surface using egocentric camera-based hand-tracking as input. Full paper here:

spatiallyjess's profile picture

jessica

2,722 subscribers

34,325 просмотров • 1 год назад •via X (Twitter)

LIVE

Anya Rossi• Live Now

Private livecam show

Комментарии: 8

Фото профиля Morten Haulik ᯅ
Morten Haulik ᯅ1 год назад

@RealityLabs They just published that in October? Wild! I remember they did it with the Q2.

Фото профиля PDF GPT
PDF GPT2 лет назад

This is my favorite AI tool for reviewing reports. Just upload a report, ask for a summary, and get one in seconds. It's like ChatGPT, but built for documents. Try it for free.

Фото профиля asim ᯅ
asim ᯅ1 год назад

@RealityLabs this is so cool

Фото профиля xulipa
xulipa1 год назад

@RealityLabs So excited about this

Фото профиля Lidoor Joseph
Lidoor Joseph1 год назад

@RealityLabs Cool

Фото профиля Ⓚ​utaka Kujua 🦈
Ⓚ​utaka Kujua 🦈1 год назад

@RealityLabs Oh I LOVE this.

Фото профиля blingerang
blingerang1 год назад

@RealityLabs When is this coming :/

Фото профиля Opic Technologies, Inc.
Opic Technologies, Inc.1 год назад

@RealityLabs Cool! 😎

Похожие видео

Steam Link for Meta Quest now has hand tracking. But you need to be opted into both the SteamVR Beta on your PC and the Quest App Beta It currently just passes hand tracking input to games using gesture based emulation via Skeletal Input
0:53

Steam Link for Meta Quest now has hand tracking. But you need to be opted into both the SteamVR Beta on your PC and the Quest App Beta It currently just passes hand tracking input to games using gesture based emulation via Skeletal Input

Brad Lynch

77,833 просмотров • 2 лет назад

Released by Reality Labs at Meta Research at #ECCV2024, Nymeria is a large-scale multimodal egocentric dataset for full-body motion understanding with potential applications in VR/MR headsets, smart glasses and more. More on this work + access to the dataset ➡️
0:25

Released by Reality Labs at Meta Research at #ECCV2024, Nymeria is a large-scale multimodal egocentric dataset for full-body motion understanding with potential applications in VR/MR headsets, smart glasses and more. More on this work + access to the dataset ➡️

AI at Meta

52,408 просмотров • 1 год назад

EgoExoMoCap: Distributed Ego-Exo Human Motion Capture ECCV 2026 (Oral) Meta Reality Labs/ETH Zürich With two or more people wearing smart glasses, EgoExoMoCap combines each wearer’s egocentric (Ego) and exocentric (Exo) views to estimate full-body motion. Without bulky multi-camera setups or mocap suits, it reconstructs 3D human motion even under occlusions using head/wrist tracking and DINOv3-based visual features.
0:20

EgoExoMoCap: Distributed Ego-Exo Human Motion Capture ECCV 2026 (Oral) Meta Reality Labs/ETH Zürich With two or more people wearing smart glasses, EgoExoMoCap combines each wearer’s egocentric (Ego) and exocentric (Exo) views to estimate full-body motion. Without bulky multi-camera setups or mocap suits, it reconstructs 3D human motion even under occlusions using head/wrist tracking and DINOv3-based visual features.

Ryohei Sasaki@engineer

25,765 просмотров • 10 дней назад

A new paper by Disney Research introduces a 2-stage technique for motion tracking in physics-based character animation, using a pretrained latent space and RL to control diverse and unseen motions on both virtual and real-world humanoid robots. Paper:
0:46

A new paper by Disney Research introduces a 2-stage technique for motion tracking in physics-based character animation, using a pretrained latent space and RL to control diverse and unseen motions on both virtual and real-world humanoid robots. Paper:

The Humanoid Hub

20,088 просмотров • 2 лет назад

Been cooking on this for a while, but it's nearly ready. Ive built an iOS app using Snap Camera Kit to provide full body, hand, & face tracking into any app using VMC/OSC. No calibration, no weird setup. Just set the IP of your PC &use your fav VTube app (Using Warudo here).
0:36

Been cooking on this for a while, but it's nearly ready. Ive built an iOS app using Snap Camera Kit to provide full body, hand, & face tracking into any app using VMC/OSC. No calibration, no weird setup. Just set the IP of your PC &use your fav VTube app (Using Warudo here).

Brielle Garcia

213,760 просмотров • 2 лет назад

Last week we released Meta Chameleon: a new mixed-modal research model from Meta FAIR. Get the models ➡️ The 7B & 34B safety tuned models we’ve released can take any combination of text and images as input and produce text outputs using a new early fusion approach. While some LLMs have separate image and text encoders or decoders, Chameleon is one of the first publicly released approaches using a single unified architecture. We’re releasing Chameleon models under a research license to help democratize access to foundational mixed-modal models & further research on early fusion. Approach & training details in the paper ➡️
0:40

Last week we released Meta Chameleon: a new mixed-modal research model from Meta FAIR. Get the models ➡️ The 7B & 34B safety tuned models we’ve released can take any combination of text and images as input and produce text outputs using a new early fusion approach. While some LLMs have separate image and text encoders or decoders, Chameleon is one of the first publicly released approaches using a single unified architecture. We’re releasing Chameleon models under a research license to help democratize access to foundational mixed-modal models & further research on early fusion. Approach & training details in the paper ➡️

AI at Meta

54,428 просмотров • 2 лет назад

George Hotz explains his thoughts on working at OpenAI vs. META: > why would any researcher want to work at openai? > paper clip maxxing > can’t share your research > you’re on the BAD team > meanwhile who would’ve thought.. > zucc is the GOOD guy > open sourcing is the only way to fight the paper clip maximizers > you think you’ll be able to control superintelligence? you won’t. BASED AND CHECKED
1:20

George Hotz explains his thoughts on working at OpenAI vs. META: > why would any researcher want to work at openai? > paper clip maxxing > can’t share your research > you’re on the BAD team > meanwhile who would’ve thought.. > zucc is the GOOD guy > open sourcing is the only way to fight the paper clip maximizers > you think you’ll be able to control superintelligence? you won’t. BASED AND CHECKED

NIK

445,191 просмотров • 1 год назад

Cold Plasma Fusion Reactor by ENG8 Based on Exotic Vacuum Object/plasmoid research. Produces electrical overunity of 3x+ and getting more efficient. Pretty cool to look at too.
0:41

Cold Plasma Fusion Reactor by ENG8 Based on Exotic Vacuum Object/plasmoid research. Produces electrical overunity of 3x+ and getting more efficient. Pretty cool to look at too.

Ashton Forbes

75,660 просмотров • 1 год назад

Meta Develops Non-Invasive Neural Interface for Next-Generation Interaction Meta Reality Labs is advancing wrist-based neural interface technology designed to enable more natural and intuitive interaction with future computing platforms. The non-invasive device leverages finger-tracking to interpret user intent. Meta’s CEO has indicated that the technology could reach commercialization within the next few years. Video source: Meta #Meta #VR #AR #NeuralWristband #tech ----------------------- Wanna get your company on Wevolver too? Learn how:
1:00

Meta Develops Non-Invasive Neural Interface for Next-Generation Interaction Meta Reality Labs is advancing wrist-based neural interface technology designed to enable more natural and intuitive interaction with future computing platforms. The non-invasive device leverages finger-tracking to interpret user intent. Meta’s CEO has indicated that the technology could reach commercialization within the next few years. Video source: Meta #Meta #VR #AR #NeuralWristband #tech ----------------------- Wanna get your company on Wevolver too? Learn how:

Wevolver

332,830 просмотров • 5 месяцев назад

Today we’re introducing SceneScript, a novel method for reconstructing environments and representing the layout of physical spaces from Reality Labs at Meta Research. Details ➡️ SceneScript is able to directly infer a room’s geometry using end-to-end machine learning and represent it using language. Compared to previous approaches, this results in representations of physical scenes that are compact, complete, interpretable and extensible.
0:45

Today we’re introducing SceneScript, a novel method for reconstructing environments and representing the layout of physical spaces from Reality Labs at Meta Research. Details ➡️ SceneScript is able to directly infer a room’s geometry using end-to-end machine learning and represent it using language. Compared to previous approaches, this results in representations of physical scenes that are compact, complete, interpretable and extensible.

AI at Meta

334,490 просмотров • 2 лет назад

Typing was NOT the most surprising part of the surface keyboard pushed in #MetaQuest latest update. Let me start by telling you how to try it yourself (available only on Quest 3) 1- On your phone, open the Meta Horizon app. 2- Go to Devices and select your headset 3- Tap Headset settings, then tap Advanced settings. 4- Tap the toggle next to Public Test Channel to join Meta Quest PTC (Public Test Channel) 5- After go in the headset settings > experimental features > surface keyboard ⌨️ Typing felt very responsive but what caught me off guard was the touchpad. 🖱️Even when looking straight at the screen the cursor moves accurately as if you are using an actual trackpad. Tapping acts as a click and it just works. ✌️ It also responds to the scrolling motion using 2 Fingers!
0:45

Typing was NOT the most surprising part of the surface keyboard pushed in #MetaQuest latest update. Let me start by telling you how to try it yourself (available only on Quest 3) 1- On your phone, open the Meta Horizon app. 2- Go to Devices and select your headset 3- Tap Headset settings, then tap Advanced settings. 4- Tap the toggle next to Public Test Channel to join Meta Quest PTC (Public Test Channel) 5- After go in the headset settings > experimental features > surface keyboard ⌨️ Typing felt very responsive but what caught me off guard was the touchpad. 🖱️Even when looking straight at the screen the cursor moves accurately as if you are using an actual trackpad. Tapping acts as a click and it just works. ✌️ It also responds to the scrolling motion using 2 Fingers!

Gabriele Romagnoli

20,670 просмотров • 6 месяцев назад

New AI research from Meta – CoTracker3 Simpler and Better Point Tracking by Pseudo-Labelling Real Videos. More details ➡️ Demo on Hugging Face ➡️ Building on our previous work on CoTracker, this new model demonstrates impressive tracking results where points can be tracked for a long time even when they're occluded or leave the field of view. CoTracker3 achieves state-of-the-art, outperforming all recent point tracking approaches on standard benchmarks — often by a substantial margin. We've released the research paper, code and a demo on Hugging Face — along with models available under an A-NC license to support further research in this space.
0:27

New AI research from Meta – CoTracker3 Simpler and Better Point Tracking by Pseudo-Labelling Real Videos. More details ➡️ Demo on Hugging Face ➡️ Building on our previous work on CoTracker, this new model demonstrates impressive tracking results where points can be tracked for a long time even when they're occluded or leave the field of view. CoTracker3 achieves state-of-the-art, outperforming all recent point tracking approaches on standard benchmarks — often by a substantial margin. We've released the research paper, code and a demo on Hugging Face — along with models available under an A-NC license to support further research in this space.

AI at Meta

219,026 просмотров • 1 год назад

⛳️ Tiny Golf - A Very Cool Meta Horizon Start Developer Competition Submission! Mixed Reality games are challenging because they must adapt to real-world spaces with dynamic content, but they don’t have to be complex. Tiny Golf is a great example, using a minimalistic aesthetic and 🙌 hand-tracking pinch-and-pull gestures to launch the golf ball. I hope MR/VR games like this inspire many of you to build, there is just so much opportunity with XR today and 2026 is the year to start! 😉 📌 To get started, you can use: Meta XR All-In-One SDK, or a leaner option: Meta XR Interaction SDK, which pulls in the Meta XR Core SDK package and includes advanced hand-tracking features and passthrough.
0:43

⛳️ Tiny Golf - A Very Cool Meta Horizon Start Developer Competition Submission! Mixed Reality games are challenging because they must adapt to real-world spaces with dynamic content, but they don’t have to be complex. Tiny Golf is a great example, using a minimalistic aesthetic and 🙌 hand-tracking pinch-and-pull gestures to launch the golf ball. I hope MR/VR games like this inspire many of you to build, there is just so much opportunity with XR today and 2026 is the year to start! 😉 📌 To get started, you can use: Meta XR All-In-One SDK, or a leaner option: Meta XR Interaction SDK, which pulls in the Meta XR Core SDK package and includes advanced hand-tracking features and passthrough.

Dilmer 👓🔜 GDC

31,749 просмотров • 8 месяцев назад

What happens when robot world models learn from human experience at scale? 🤔 DreamDojo from NVIDIA Research is a generalist robot world model pretrained on 44K hours of egocentric human videos and then post-trained on robot data to generalize across new objects and environments. After distillation, it runs at 10 FPS for live teleoperation, policy evaluation, and model-based planning. Read the ICML paper to learn more 📄
0:32

What happens when robot world models learn from human experience at scale? 🤔 DreamDojo from NVIDIA Research is a generalist robot world model pretrained on 44K hours of egocentric human videos and then post-trained on robot data to generalize across new objects and environments. After distillation, it runs at 10 FPS for live teleoperation, policy evaluation, and model-based planning. Read the ICML paper to learn more 📄

NVIDIA Robotics

22,958 просмотров • 1 месяц назад

An AI research assistant where you can ask a question about any file and automatically get the answer. #ChatGPT for your files. Here I use it on a research paper produced by my old lab at Stanford. #GPT3 #AI Jay Cummings Andrew D. Huberman, Ph.D. Lex Fridman
0:35

An AI research assistant where you can ask a question about any file and automatically get the answer. #ChatGPT for your files. Here I use it on a research paper produced by my old lab at Stanford. #GPT3 #AI Jay Cummings Andrew D. Huberman, Ph.D. Lex Fridman

Humata

35,185 просмотров • 3 лет назад

Twitch streamer RebeuDeter won a Lamborghini by outsmarting MrBeast Streamers had to keep their hand on the Lamborghini while blindfolded and take it off as close as possible to the 10 minute countdown His friend helped him win by using a vlog camera timer to tell him exactly when to take his hand off
3:03

Twitch streamer RebeuDeter won a Lamborghini by outsmarting MrBeast Streamers had to keep their hand on the Lamborghini while blindfolded and take it off as close as possible to the 10 minute countdown His friend helped him win by using a vlog camera timer to tell him exactly when to take his hand off

Jack

4,359,830 просмотров • 4 месяцев назад

[BLOOPER] More fun with our automated camera friends on the BBC News at One today. Presenter Ben Brown was undoubtedly glad he had his paper script to hand, as his autocue disappeared out of view.
0:38

[BLOOPER] More fun with our automated camera friends on the BBC News at One today. Presenter Ben Brown was undoubtedly glad he had his paper script to hand, as his autocue disappeared out of view.

The TV Room

14,812 просмотров • 3 лет назад

📣 This solo developer who currently works at Pixar as a Technical Director is building a VR rhythm game called Sock Puppet Superstar for Quest as an indie project, and over the last week he has been going viral on TikTok! 🤖Features/Tech: - Using Unity, OpenXR, hand tracking, and a custom voice synthesizer built in FMOD - Raise and lower your puppet to control the pitch - Works with hand tracking or controllers - Customize your sock puppet with colors, fabrics, accessories, and voices 📌 Devs can achieve this using the Meta XR Interaction SDK or Unity XR Toolkit (XR Hands) to add full hand support like what’s shown here. 🚀 Share this to give this solo indie dev more visibility! I’ll continue this bi-weekly with more indie devs who deserve the amplification!
0:35

📣 This solo developer who currently works at Pixar as a Technical Director is building a VR rhythm game called Sock Puppet Superstar for Quest as an indie project, and over the last week he has been going viral on TikTok! 🤖Features/Tech: - Using Unity, OpenXR, hand tracking, and a custom voice synthesizer built in FMOD - Raise and lower your puppet to control the pitch - Works with hand tracking or controllers - Customize your sock puppet with colors, fabrics, accessories, and voices 📌 Devs can achieve this using the Meta XR Interaction SDK or Unity XR Toolkit (XR Hands) to add full hand support like what’s shown here. 🚀 Share this to give this solo indie dev more visibility! I’ll continue this bi-weekly with more indie devs who deserve the amplification!

Dilmer

150,843 просмотров • 4 месяцев назад

Today at Meta FAIR we’re announcing three new cutting-edge developments in robotics and touch perception — and releasing a collection of artifacts to empower the community to build on this work. Details on all of this new work ➡️ 1️⃣ Meta Sparsh is the first general-purpose encoder for vision-based tactile sensing that works across many tactile sensors and many tasks. Trained on 460K+ tactile images using self-supervised learning. 2️⃣ Meta Digit 360 is a breakthrough artificial fingertip-based tactile sensor, equipped with 18+ sensing features to deliver detailed touch data with human-level precision and touch-sensing capabilities. 3️⃣ Meta Digit Plexus is a standardized platform for robotic sensor connections and interactions. It provides a hardware-software solution to integrate tactile sensors on a single robot hand and enables seamless data collection, control and analysis over a single cable. The potential impact of expanding capabilities and components like these for the open source community ranges from medical research to supply chain, manufacturing and much more. We’re excited to continue this work with the broader community.
2:35

Today at Meta FAIR we’re announcing three new cutting-edge developments in robotics and touch perception — and releasing a collection of artifacts to empower the community to build on this work. Details on all of this new work ➡️ 1️⃣ Meta Sparsh is the first general-purpose encoder for vision-based tactile sensing that works across many tactile sensors and many tasks. Trained on 460K+ tactile images using self-supervised learning. 2️⃣ Meta Digit 360 is a breakthrough artificial fingertip-based tactile sensor, equipped with 18+ sensing features to deliver detailed touch data with human-level precision and touch-sensing capabilities. 3️⃣ Meta Digit Plexus is a standardized platform for robotic sensor connections and interactions. It provides a hardware-software solution to integrate tactile sensors on a single robot hand and enables seamless data collection, control and analysis over a single cable. The potential impact of expanding capabilities and components like these for the open source community ranges from medical research to supply chain, manufacturing and much more. We’re excited to continue this work with the broader community.

AI at Meta

453,343 просмотров • 1 год назад