Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

Open source code for Augmented-Reality Control of the Boston Dynamics Spot with MS Microsoft HoloLens, incl. gesture control, smart following & cognitive offloading. Holo-SpoK papers, videos & code at: -- superb #AR work by Rodrigo Chacon Personal Robotics @ImperialEEE

Yiannis Demiris

1,301 subscribers

13,016 Aufrufe • vor 1 Jahr •via X (Twitter)

Wissenschaft & Technologie #AR

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Open-source robot arm meets hand tracking [📍GitHub below] It is designed with an industrial mindset but built as a 3D-printed desktop system. PAROL6 paired with a LEAP Motion controller is a nice example of how accessible robot teleoperation has become. • Hand motion is streamed to the robot at 100 Hz via UDP • A pneumatic gripper is controlled by simple fist open and close gestures • The entire robot stack is open source, from mechanics to control software Combine that with low-latency hand tracking and you get a very practical platform for learning manipulation, teleoperation, and human-robot interfaces. This kind of setup is great for experimentation, teleop, data collection, and teaching robots by demonstration All without proprietary hardware or locked software. Credit to SourceRobotics 📍Code: —— Weekly robotics and AI insights. Subscribe free:

Open-source robot arm meets hand tracking [📍GitHub below] It is designed with an industrial mindset but built as a 3D-printed desktop system. PAROL6 paired with a LEAP Motion controller is a nice example of how accessible robot teleoperation has become. • Hand motion is streamed to the robot at 100 Hz via UDP • A pneumatic gripper is controlled by simple fist open and close gestures • The entire robot stack is open source, from mechanics to control software Combine that with low-latency hand tracking and you get a very practical platform for learning manipulation, teleoperation, and human-robot interfaces. This kind of setup is great for experimentation, teleop, data collection, and teaching robots by demonstration All without proprietary hardware or locked software. Credit to SourceRobotics 📍Code: —— Weekly robotics and AI insights. Subscribe free:

Ilir Aliu

32,025 Aufrufe • vor 6 Monaten

New short course: Prompt Engineering with Llama 2, built in collaboration with Meta AI at Meta, and taught by Amit Sangani! Meta's Llama 2 has been game-changing for AI. Building with open source lets you control your own data, scrutinize errors, update (or not) the models as you please, and work alongside the global community advancing open models. Llama isn't a single model, it's a collection of models. In this course, you'll: - Learn the differences between different Llama 2 flavors, and when to use each. - Prompt the Llama chat models -- you'll also see how Llama's instruction tags work -- so they can help you with day-to-day tasks, like writing or summarization. - Use advanced prompting, like few-shot prompting for classification, and chain-of-thought prompting for solving logic problems. - Use specialized models in the Llama collection for specific tasks, like Code Llama to help you write, analyze, and improve code, and Llama Guard, which checks prompts and model responses for harmful content. The course also touches on how to run Llama 2 locally on your own computer. I hope you’ll take this course and try out these powerful, open models!

New short course: Prompt Engineering with Llama 2, built in collaboration with Meta AI at Meta, and taught by Amit Sangani! Meta's Llama 2 has been game-changing for AI. Building with open source lets you control your own data, scrutinize errors, update (or not) the models as you please, and work alongside the global community advancing open models. Llama isn't a single model, it's a collection of models. In this course, you'll: - Learn the differences between different Llama 2 flavors, and when to use each. - Prompt the Llama chat models -- you'll also see how Llama's instruction tags work -- so they can help you with day-to-day tasks, like writing or summarization. - Use advanced prompting, like few-shot prompting for classification, and chain-of-thought prompting for solving logic problems. - Use specialized models in the Llama collection for specific tasks, like Code Llama to help you write, analyze, and improve code, and Llama Guard, which checks prompts and model responses for harmful content. The course also touches on how to run Llama 2 locally on your own computer. I hope you’ll take this course and try out these powerful, open models!

Andrew Ng

162,842 Aufrufe • vor 2 Jahren

Check out our latest work, "Actor-Critic Model Predictive Control: Differentiable Optimization meets Reinforcement Learning for Agile Flight," published in the IEEE Transactions on Robotics, where we reconcile #OptimalControl and #ReinforcementLearning, achieving the same super-human performance, but with superior generalizability, as our previous model-free deep RL! Code released! PDF: Code: Full Video: Model-free #ReinforcementLearning (RL) is known for its strong task performance and flexibility in optimizing general reward formulations. On the other hand, #ModelPredictiveControl (MPC) provides robustness, constraint handling, and powerful online replanning capabilities. In this work, we extend our previous AC-MPC paper (Romero, ICRA'24) by taking a deeper look at how both approaches can be unified. We introduce and extend Actor-Critic Model Predictive Control (AC-MPC), a framework that embeds a differentiable MPC inside an Actor-Critic RL architecture. This integration allows the MPC-based actor to perform short-term predictive optimization, while the critic facilitates long-horizon learning and exploration. We conduct a comprehensive study that highlights AC-MPC’s key advantages: - Better out-of-distribution generalization, both against unknown disturbances and changes in the quadrotor dynamics - Improved sample efficiency - A novel empirical analysis uncovering a relationship between the critic’s value function and the MPC cost function, providing deeper insight into their interplay. We validate our method in simulation and the real world on a quadcopter flying at superhuman speeds of up to 21 m/s, matching state-of-the-art model-free RL performance, and retaining the predictive structure of MPC for more reliable out-of-distribution behavior. Reference: Actor-Critic Model Predictive Control: Differentiable Optimization meets Reinforcement Learning for Agile Flight IEEE Transactions on Robotics (T-RO), 2025 PDF: Full Video: Code: Kudos to Ángel Romero, Elie Aljalbout, Yunlong Song! University of Zurich UZH Science UZH Space Hub AUTOASSESS European Research Council (ERC) UZHai

Check out our latest work, "Actor-Critic Model Predictive Control: Differentiable Optimization meets Reinforcement Learning for Agile Flight," published in the IEEE Transactions on Robotics, where we reconcile #OptimalControl and #ReinforcementLearning, achieving the same super-human performance, but with superior generalizability, as our previous model-free deep RL! Code released! PDF: Code: Full Video: Model-free #ReinforcementLearning (RL) is known for its strong task performance and flexibility in optimizing general reward formulations. On the other hand, #ModelPredictiveControl (MPC) provides robustness, constraint handling, and powerful online replanning capabilities. In this work, we extend our previous AC-MPC paper (Romero, ICRA'24) by taking a deeper look at how both approaches can be unified. We introduce and extend Actor-Critic Model Predictive Control (AC-MPC), a framework that embeds a differentiable MPC inside an Actor-Critic RL architecture. This integration allows the MPC-based actor to perform short-term predictive optimization, while the critic facilitates long-horizon learning and exploration. We conduct a comprehensive study that highlights AC-MPC’s key advantages: - Better out-of-distribution generalization, both against unknown disturbances and changes in the quadrotor dynamics - Improved sample efficiency - A novel empirical analysis uncovering a relationship between the critic’s value function and the MPC cost function, providing deeper insight into their interplay. We validate our method in simulation and the real world on a quadcopter flying at superhuman speeds of up to 21 m/s, matching state-of-the-art model-free RL performance, and retaining the predictive structure of MPC for more reliable out-of-distribution behavior. Reference: Actor-Critic Model Predictive Control: Differentiable Optimization meets Reinforcement Learning for Agile Flight IEEE Transactions on Robotics (T-RO), 2025 PDF: Full Video: Code: Kudos to Ángel Romero, Elie Aljalbout, Yunlong Song! University of Zurich UZH Science UZH Space Hub AUTOASSESS European Research Council (ERC) UZHai

Davide Scaramuzza

27,090 Aufrufe • vor 6 Monaten

Can AI agents adapt zero-shot, to complex multi-step language instructions in open-ended environments? We present MaestroMotif, a method for AI-assisted skill design that produces highly capable and steerable hierarchical agents. To the best of our knowledge, it is the first method that, without expert labeled datasets, solves compositional tasks requiring hundreds of steps for completion. All the modules within MaestroMotif are learned from interaction: from the highest level of planning to the lowest-level of sensorimotor control. On the open-ended domain of NetHack, it surpasses existing approaches, including those that are fine-tuned specifically for each task. At the heart of MaestroMotif is the idea that decomposing a task into subtasks significantly helps decision making. MaestroMotif leverages an agent designer's intuition about a domain to identify important skills and describe them in natural language. These short descriptions then get converted into adaptable hierarchical agents through AI feedback and in-context learning. Our paper was recently published at ICLR 2025 and we open-source the whole project including the code, prompts and pre-trained models. Paper: Code: NotebookLM Podcast: This work was done with the amazing Mikael Henaff, Roberta Raileanu, Shagun Sodhani, Pascal Vincent, Amy Zhang, Pierre-Luc Bacon, Doina Precup, with equal supervision by Marlos C. Machado and Pierluca D'Oro. Take a look at the following thread:

Can AI agents adapt zero-shot, to complex multi-step language instructions in open-ended environments? We present MaestroMotif, a method for AI-assisted skill design that produces highly capable and steerable hierarchical agents. To the best of our knowledge, it is the first method that, without expert labeled datasets, solves compositional tasks requiring hundreds of steps for completion. All the modules within MaestroMotif are learned from interaction: from the highest level of planning to the lowest-level of sensorimotor control. On the open-ended domain of NetHack, it surpasses existing approaches, including those that are fine-tuned specifically for each task. At the heart of MaestroMotif is the idea that decomposing a task into subtasks significantly helps decision making. MaestroMotif leverages an agent designer's intuition about a domain to identify important skills and describe them in natural language. These short descriptions then get converted into adaptable hierarchical agents through AI feedback and in-context learning. Our paper was recently published at ICLR 2025 and we open-source the whole project including the code, prompts and pre-trained models. Paper: Code: NotebookLM Podcast: This work was done with the amazing Mikael Henaff, Roberta Raileanu, Shagun Sodhani, Pascal Vincent, Amy Zhang, Pierre-Luc Bacon, Doina Precup, with equal supervision by Marlos C. Machado and Pierluca D'Oro. Take a look at the following thread:

Martin Klissarov

80,280 Aufrufe • vor 1 Jahr

Today we are excited to open up Neuracore to the academic community! Neuracore is a new data foundation built to accelerate robot learning by removing one of the field’s biggest bottlenecks: capturing and working with high-fidelity multimodal robotics data. For the first time, researchers can store, view, and work with robotics data in a cloud-native system built specifically for large-scale learning, and we are making this core platform completely free for academia. The platform lets teams capture every sensor at its native rate, store and visualize data without loss, and then train and deploy models locally using our open-source code (Link in the comments). We are rolling out access to select academic institutions first. Anyone with an academic email can sign up, and if your institution is not part of the initial rollout, you will be able to join the waitlist directly. Beyond providing this infrastructure, we see an opportunity to build a global community where engineers and researchers can share, collaborate, and advance the frontier of robot learning together. Supported by our recent $3M pre-seed round led by Earlybird VC, we are excited to take this mission even further. Our long-term goal is for Neuracore to become the natural home for cutting-edge robot learning algorithms and real-world robotics experimentation, helping accelerate the next wave of Physical AI.

Today we are excited to open up Neuracore to the academic community! Neuracore is a new data foundation built to accelerate robot learning by removing one of the field’s biggest bottlenecks: capturing and working with high-fidelity multimodal robotics data. For the first time, researchers can store, view, and work with robotics data in a cloud-native system built specifically for large-scale learning, and we are making this core platform completely free for academia. The platform lets teams capture every sensor at its native rate, store and visualize data without loss, and then train and deploy models locally using our open-source code (Link in the comments). We are rolling out access to select academic institutions first. Anyone with an academic email can sign up, and if your institution is not part of the initial rollout, you will be able to join the waitlist directly. Beyond providing this infrastructure, we see an opportunity to build a global community where engineers and researchers can share, collaborate, and advance the frontier of robot learning together. Supported by our recent $3M pre-seed round led by Earlybird VC, we are excited to take this mission even further. Our long-term goal is for Neuracore to become the natural home for cutting-edge robot learning algorithms and real-world robotics experimentation, helping accelerate the next wave of Physical AI.

Neuracore

40,620 Aufrufe • vor 8 Monaten

LimX Dynamics just released LimX Luna, officially described by the company as the first mass-deliverable full-size female humanoid robot. 🤖 This is not aimed at warehouses; it is built for malls, theme parks, auto shows, brand events, and other commercial spaces where attention is the product. Luna stands 160 cm tall, has 27 body DoF excluding end effectors, and combines whole-body motion control with voice, vision, facial expressions, gesture interaction, and motion imitation. The sharper part is the content stack: users can upload dance videos, build scenes with zero-code tools in LimX Studio, and run synchronized shows with up to 200 robots. That makes Luna less like a one-off demo machine and more like a programmable entertainment platform for foot traffic, performances, and social clips. For humanoids, this is a useful shift: not only “can it do labor,” but “can it create a consumer experience people actually stop to watch.”

RoboHub🤖

13,909 Aufrufe • vor 2 Monaten

Building humanoids just got a whole lot easier. 🤖🛠️ Roboparty has officially released the full-stack source code and hardware blueprints for their Roboto_Original bipedal robot. This isn't just another GitHub repo—it’s a complete roadmap designed to slash development costs by a projected 80% and turn "secret sauce" into shared infrastructure. ➤ High-Performance Specs: Standing 1.2m (3.9ft) and weighing 30kg (66lbs), this bot hits a running speed of 3m/s (6.7mph). It features an auto-grade structure and modular joints, proving open-source hardware can absolutely compete at a top-tier level. ➤ The Ultimate Dev Kit: They’ve opened up everything: full structural drawings, EBOMs, a supplier list, and even their AMP motion control algorithms. It even supports SMPL-X to leverage human motion capture data, making gait training way more efficient. ➤ No More Reinventing the Wheel: By sharing their Sim2Real gap solutions and a "Know-how" knowledge base, Roboparty is helping the community skip the trial-and-error phase. The goal is to move from "showy stunts" to actual real-world utility. ➤ Ready to fork the future? Check out the resources: GitHub: Gitee: Docs: Source: Roboparty #HumanoidRobots #OpenSource #EmbodiedAI #Roboparty #Robotics #BuildInPublic

Building humanoids just got a whole lot easier. 🤖🛠️ Roboparty has officially released the full-stack source code and hardware blueprints for their Roboto_Original bipedal robot. This isn't just another GitHub repo—it’s a complete roadmap designed to slash development costs by a projected 80% and turn "secret sauce" into shared infrastructure. ➤ High-Performance Specs: Standing 1.2m (3.9ft) and weighing 30kg (66lbs), this bot hits a running speed of 3m/s (6.7mph). It features an auto-grade structure and modular joints, proving open-source hardware can absolutely compete at a top-tier level. ➤ The Ultimate Dev Kit: They’ve opened up everything: full structural drawings, EBOMs, a supplier list, and even their AMP motion control algorithms. It even supports SMPL-X to leverage human motion capture data, making gait training way more efficient. ➤ No More Reinventing the Wheel: By sharing their Sim2Real gap solutions and a "Know-how" knowledge base, Roboparty is helping the community skip the trial-and-error phase. The goal is to move from "showy stunts" to actual real-world utility. ➤ Ready to fork the future? Check out the resources: GitHub: Gitee: Docs: Source: Roboparty #HumanoidRobots #OpenSource #EmbodiedAI #Roboparty #Robotics #BuildInPublic

RoboHub🤖

63,085 Aufrufe • vor 6 Monaten

I rebuilt my viral TikTok AI agent with Claude Code in 37 minutes 🤯 A full research-to-brief pipeline that scrapes TikTok, analyzes videos with AI, and generates creative briefs for your clients. All inside Claude Code + Replit. Perfect for creative agencies and DTC brands who want to turn competitor research into briefs without the manual grind. Look, all e-commerce brands & agencies should have at least one person on their team who is their "AI expert" & can vibe-code apps & workflows like this. Here's what the Claude Code version of my TikTok Agent does: → Search TikTok by keyword, date range, and video count → Pull engagement metrics, captions, and thumbnails → Gemini actually watches the video and analyzes the hook → AI scrapes comments for common questions and insights → Generate a full creative brief based on your template + brand bible No watching videos manually. No copying notes into docs. No rewriting briefs from scratch. What you control: - Multiple client projects with separate brand bibles - Your own creative brief template - Which videos to analyze and brief - Full customization through Replit's AI agent Research → Analysis → Brief. One workflow, running a custom, mini-SaaS inside your company I recorded a full walkthrough showing exactly how I built this from scratch. Want the full tutorial? > Like this post > Comment "CLAUDE" And I'll send it over (must be following so I can DM)

Mike Futia

74,512 Aufrufe • vor 6 Monaten

Boston Dynamics just dropped the production-ready Atlas at CES 2026, and the specs are next-level. This isn't just a prototype; it's a rugged, enterprise-grade worker built for the grind of real-world factories.🤖🦾 With 56 degrees of freedom and fully rotational joints, Atlas moves with "superhuman" flexibility. It features human-scale hands with tactile sensing in every finger and palm, allowing for high-precision manipulation. Plus, its 360-degree vision system ensures it always knows who's approaching, keeping human-robot collaboration safe. ➤ Rugged Performance: Lifts up to 110 lbs (approx. 50kg) and operates in extreme temps from -4°F to 104°F (-20°C to 40°C). It’s even water-resistant for industrial wash-downs. ➤ Smart Endurance: Runs for 4 hours on dual swappable batteries. When power gets low, it autonomously heads back to the station to swap its own batteries and get right back to work. ➤ Cloud Intelligence: New tasks can be trained in less than a day. Thanks to the Orbit platform, once one Atlas learns a skill, it’s instantly shared across the entire fleet. The scale is massive: the entire 2026 supply is already spoken for by Hyundai and a new AI partner. Hyundai is also building a specialized "Robotics Metal Plant" and a new factory capable of churning out 30,000 Atlas units annually. We’re officially watching the birth of a humanoid workforce at scale. Source: Boston Dynamics & Hyundai Group

Boston Dynamics just dropped the production-ready Atlas at CES 2026, and the specs are next-level. This isn't just a prototype; it's a rugged, enterprise-grade worker built for the grind of real-world factories.🤖🦾 With 56 degrees of freedom and fully rotational joints, Atlas moves with "superhuman" flexibility. It features human-scale hands with tactile sensing in every finger and palm, allowing for high-precision manipulation. Plus, its 360-degree vision system ensures it always knows who's approaching, keeping human-robot collaboration safe. ➤ Rugged Performance: Lifts up to 110 lbs (approx. 50kg) and operates in extreme temps from -4°F to 104°F (-20°C to 40°C). It’s even water-resistant for industrial wash-downs. ➤ Smart Endurance: Runs for 4 hours on dual swappable batteries. When power gets low, it autonomously heads back to the station to swap its own batteries and get right back to work. ➤ Cloud Intelligence: New tasks can be trained in less than a day. Thanks to the Orbit platform, once one Atlas learns a skill, it’s instantly shared across the entire fleet. The scale is massive: the entire 2026 supply is already spoken for by Hyundai and a new AI partner. Hyundai is also building a specialized "Robotics Metal Plant" and a new factory capable of churning out 30,000 Atlas units annually. We’re officially watching the birth of a humanoid workforce at scale. Source: Boston Dynamics & Hyundai Group

RoboHub🤖

28,554 Aufrufe • vor 6 Monaten

NVIDIA just unleashed SANA-WM and it’s an absolute MONSTER for the future of open source AI! A blazing-fast 2.6B-parameter open-source world model that doesn’t just generate video… it creates controllable, physics-rich, high-fidelity worlds on demand. Why this is insanely powerful: • One image + text prompt + 6-DoF camera trajectory → generates 720p videos up to 60 seconds long with buttery-smooth, precisely controlled camera movement. You’re not just watching, you’re piloting the simulation. • Runs locally on a single consumer GPU (RTX 5090 level) thanks to heavy distillation + NVFP4 quantization. Full 60-second clip denoised in ~34 seconds. No massive clusters required. • 36× higher throughput than previous open models while rivaling (or beating) closed industrial giants in visual quality and consistency. • Trained lightning-fast: ~213K public videos in just 15 days on 64 H100s. • Built with next-level tech: Hybrid Linear Attention, dual-branch camera control, two-stage pipeline, and rock-solid metric-scale pose understanding. This is a true open world model, the foundation for embodied AI, robotics, autonomous systems, and hyper-realistic simulations that can run anywhere. Project: At our Zero-Human Company, we’re already running SANA-WM live in our core pipelines. It’s supercharging autonomous agent training, generating unlimited synthetic training data, and powering full end-to-end simulation loops, zero humans in the loop. The speed and control let us test thousands of edge-case scenarios overnight, iterate at lightspeed, and push our fully autonomous operations further than ever before. This is the kind of breakthrough that turns science fiction into daily reality. World models just leveled up — hard. The age of personal, local, controllable universes is here.

NVIDIA just unleashed SANA-WM and it’s an absolute MONSTER for the future of open source AI! A blazing-fast 2.6B-parameter open-source world model that doesn’t just generate video… it creates controllable, physics-rich, high-fidelity worlds on demand. Why this is insanely powerful: • One image + text prompt + 6-DoF camera trajectory → generates 720p videos up to 60 seconds long with buttery-smooth, precisely controlled camera movement. You’re not just watching, you’re piloting the simulation. • Runs locally on a single consumer GPU (RTX 5090 level) thanks to heavy distillation + NVFP4 quantization. Full 60-second clip denoised in ~34 seconds. No massive clusters required. • 36× higher throughput than previous open models while rivaling (or beating) closed industrial giants in visual quality and consistency. • Trained lightning-fast: ~213K public videos in just 15 days on 64 H100s. • Built with next-level tech: Hybrid Linear Attention, dual-branch camera control, two-stage pipeline, and rock-solid metric-scale pose understanding. This is a true open world model, the foundation for embodied AI, robotics, autonomous systems, and hyper-realistic simulations that can run anywhere. Project: At our Zero-Human Company, we’re already running SANA-WM live in our core pipelines. It’s supercharging autonomous agent training, generating unlimited synthetic training data, and powering full end-to-end simulation loops, zero humans in the loop. The speed and control let us test thousands of edge-case scenarios overnight, iterate at lightspeed, and push our fully autonomous operations further than ever before. This is the kind of breakthrough that turns science fiction into daily reality. World models just leveled up — hard. The age of personal, local, controllable universes is here.

Brian Roemmele

618,431 Aufrufe • vor 2 Monaten

Open-source magnetic tactile sensor for $5! 🧲 Researchers introduced a magnetic tactile sensor that's low-cost, and easy to fabricate, democratizing tactile sensing for robotics. Operating in unstructured environments like homes and offices requires robots to sense forces during physical interaction. Yet the lack of a versatile, accessible tactile sensor has led to fragmented solutions and often force-unaware, sensorless approaches. Building an eFlesh sensor requires four components: a hobbyist 3D printer, off-the-shelf magnets (less than $5), a CAD model, and a magnetometer circuit board. The sensor is 3D printed with magnets embedded in the middle layer. Based on chosen mechanical properties, magnets displace in response to contact forces, measured by a magnetometer underneath. An open-source design tool converts simple OBJ/STL files into 3D-printable STLs. This enables application-specific sensors for robot hands, grippers, quadruped feet, and more. Slip detection generalizes to unseen objects with 95% accuracy. Visual-tactile control policies improve manipulation by 40% over vision-only baselines, achieving 90% success on precise tasks like plug insertion and credit card swiping. All design files, code, trained models, and conversion tools are openly available. Project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Open-source magnetic tactile sensor for $5! 🧲 Researchers introduced a magnetic tactile sensor that's low-cost, and easy to fabricate, democratizing tactile sensing for robotics. Operating in unstructured environments like homes and offices requires robots to sense forces during physical interaction. Yet the lack of a versatile, accessible tactile sensor has led to fragmented solutions and often force-unaware, sensorless approaches. Building an eFlesh sensor requires four components: a hobbyist 3D printer, off-the-shelf magnets (less than $5), a CAD model, and a magnetometer circuit board. The sensor is 3D printed with magnets embedded in the middle layer. Based on chosen mechanical properties, magnets displace in response to contact forces, measured by a magnetometer underneath. An open-source design tool converts simple OBJ/STL files into 3D-printable STLs. This enables application-specific sensors for robot hands, grippers, quadruped feet, and more. Slip detection generalizes to unseen objects with 95% accuracy. Visual-tactile control policies improve manipulation by 40% over vision-only baselines, achieving 90% success on precise tasks like plug insertion and credit card swiping. All design files, code, trained models, and conversion tools are openly available. Project page: ~~ ♻️ Join the weekly robotics newsletter, and never miss any news →

Lukas Ziegler

289,055 Aufrufe • vor 3 Monaten

The corporate system wants you trading time for a paycheck. The alternative is building automated leverage. You do not need a team of engineers. You just need the right open-source architecture. Here are 10 GitHub repos to automate your workflows, replace manual labor, and direct your own reality: 1. n8n Bypass expensive SaaS subscriptions. Build custom AI automation workflows that run on your own servers. 2. Ollama Stop sending your private data to massive API providers. Run heavy AI models locally on your own machine. Complete privacy. 3. Open Interpreter Let language models control your computer. Automate the repetitive corporate tasks they pay you to do manually. 4. Aider An AI pair programmer that lives in your terminal. Stop writing boilerplate code and focus strictly on the architecture. 5. Dify An open-source LLM app development platform. Build and deploy functional AI agents in minutes, not months. 6. Flowise A drag-and-drop UI to build customized LLM flows. You do not need to be a senior developer to build massive leverage. 7. Supabase Spin up a Postgres database, authentication, and instant APIs. Own your backend entirely. 8. Auto-GPT Give an AI an objective and let it execute. It browses the web, writes code, and chains thoughts together autonomously. 9. Outline An open-source knowledge base for your personal leverage. Stop losing your documentation in arbitrary corporate systems. 10. NocoDB Turn any database into a smart spreadsheet. Keep your data on your own infrastructure and stop paying for convenience. The secret to tech survival? Stop playing by their rules. Build your own systems and take your leverage with you.

The corporate system wants you trading time for a paycheck. The alternative is building automated leverage. You do not need a team of engineers. You just need the right open-source architecture. Here are 10 GitHub repos to automate your workflows, replace manual labor, and direct your own reality: 1. n8n Bypass expensive SaaS subscriptions. Build custom AI automation workflows that run on your own servers. 2. Ollama Stop sending your private data to massive API providers. Run heavy AI models locally on your own machine. Complete privacy. 3. Open Interpreter Let language models control your computer. Automate the repetitive corporate tasks they pay you to do manually. 4. Aider An AI pair programmer that lives in your terminal. Stop writing boilerplate code and focus strictly on the architecture. 5. Dify An open-source LLM app development platform. Build and deploy functional AI agents in minutes, not months. 6. Flowise A drag-and-drop UI to build customized LLM flows. You do not need to be a senior developer to build massive leverage. 7. Supabase Spin up a Postgres database, authentication, and instant APIs. Own your backend entirely. 8. Auto-GPT Give an AI an objective and let it execute. It browses the web, writes code, and chains thoughts together autonomously. 9. Outline An open-source knowledge base for your personal leverage. Stop losing your documentation in arbitrary corporate systems. 10. NocoDB Turn any database into a smart spreadsheet. Keep your data on your own infrastructure and stop paying for convenience. The secret to tech survival? Stop playing by their rules. Build your own systems and take your leverage with you.

Katyayani Shukla

16,964 Aufrufe • vor 3 Monaten

XBOW raised $117M to build AI hacking agents. Now someone just open-sourced it for FREE. Strix deploys autonomous AI agents that act like real hackers - they run your code dynamically, find vulnerabilities, and validate them through actual proof-of-concepts. Why it matters: The biggest problem with traditional security testing is that it doesn't keep up with development speed. Strix solves this by integrating directly into your workflow: ↳ Run it in CI/CD to catch vulnerabilities before production ↳ Get real proof-of-concepts, not false positives from static analysis ↳ Test everything: injection attacks, access control, business logic flaws The best part? You don't need to be a security expert. Strix includes a complete hacker toolkit - HTTP proxy, browser automation, and Python runtime for exploit development. It's like having a security team that works at the speed of your CI/CD pipeline. The best part is that the tool runs locally in Docker containers, so your code never leaves your environment. Getting started is simple: - pipx install strix-agent - Point it at your codebase (app, repo, or directory) Everything is 100% open-source! I've shared link to the GitHub repo in the replies!

XBOW raised $117M to build AI hacking agents. Now someone just open-sourced it for FREE. Strix deploys autonomous AI agents that act like real hackers - they run your code dynamically, find vulnerabilities, and validate them through actual proof-of-concepts. Why it matters: The biggest problem with traditional security testing is that it doesn't keep up with development speed. Strix solves this by integrating directly into your workflow: ↳ Run it in CI/CD to catch vulnerabilities before production ↳ Get real proof-of-concepts, not false positives from static analysis ↳ Test everything: injection attacks, access control, business logic flaws The best part? You don't need to be a security expert. Strix includes a complete hacker toolkit - HTTP proxy, browser automation, and Python runtime for exploit development. It's like having a security team that works at the speed of your CI/CD pipeline. The best part is that the tool runs locally in Docker containers, so your code never leaves your environment. Getting started is simple: - pipx install strix-agent - Point it at your codebase (app, repo, or directory) Everything is 100% open-source! I've shared link to the GitHub repo in the replies!

Akshay 🚀

96,762 Aufrufe • vor 8 Monaten

We trained a robot dog to balance and walk on top of a yoga ball purely in simulation, and then transfer zero-shot to the real world. No fine-tuning. Just works. I’m excited to announce DrEureka, an LLM agent that writes code to train robot skills in simulation, and writes more code to bridge the difficult simulation-reality gap. It fully automates the pipeline from new skill learning to real-world deployment. The Yoga ball task is particularly hard because it is not possible to accurately simulate the bouncy ball surface. Yet DrEureka has no trouble searching over a vast space of sim-to-real configurations, and enables the dog to steer the ball on various terrains, even walking sideways! Traditionally, the sim-to-real transfer is achieved by domain randomization, a tedious process that requires expert human roboticists to stare at every parameter and adjust by hand. Frontier LLMs like GPT-4 have tons of built-in physical intuition for friction, damping, stiffness, gravity, etc. We are (mildly) surprised to find that DrEureka can tune these parameters competently and explain its reasoning well. DrEureka builds on our prior work Eureka, the algorithm that teaches a 5-finger robot hand to do pen spinning. It takes one step further on our quest to automate the entire robot learning pipeline by an AI agent system. One model that outputs strings will supervise another model that outputs torque control. We open-source everything! Welcome you all to check out the paper, more videos, and try the codebase today: Code:

We trained a robot dog to balance and walk on top of a yoga ball purely in simulation, and then transfer zero-shot to the real world. No fine-tuning. Just works. I’m excited to announce DrEureka, an LLM agent that writes code to train robot skills in simulation, and writes more code to bridge the difficult simulation-reality gap. It fully automates the pipeline from new skill learning to real-world deployment. The Yoga ball task is particularly hard because it is not possible to accurately simulate the bouncy ball surface. Yet DrEureka has no trouble searching over a vast space of sim-to-real configurations, and enables the dog to steer the ball on various terrains, even walking sideways! Traditionally, the sim-to-real transfer is achieved by domain randomization, a tedious process that requires expert human roboticists to stare at every parameter and adjust by hand. Frontier LLMs like GPT-4 have tons of built-in physical intuition for friction, damping, stiffness, gravity, etc. We are (mildly) surprised to find that DrEureka can tune these parameters competently and explain its reasoning well. DrEureka builds on our prior work Eureka, the algorithm that teaches a 5-finger robot hand to do pen spinning. It takes one step further on our quest to automate the entire robot learning pipeline by an AI agent system. One model that outputs strings will supervise another model that outputs torque control. We open-source everything! Welcome you all to check out the paper, more videos, and try the codebase today: Code:

Jim Fan

908,872 Aufrufe • vor 2 Jahren

🚀We are thrilled to open-source Hunyuan-GameCraft, a high-dynamic interactive game video generation framework built on HunyuanVideo. It generates playable and physically realistic videos from a single scene image and user action signals, empowering creators and developers to "direct" games with first-person or third-person perspectives. Key Advantages: 🔹High Dynamics: Unifies standard keyboard inputs into a shared continuous action space, enabling high-precision control over velocity and angle. This allows for the exploration of complex trajectories, overcoming the stiff, limited motion of traditional models. It can also generate dynamic environmental content like moving clouds, rain, snow, and water flow. 🔹Long-term Consistency: Uses hybrid history condition to preserve the original scene information after significant movement. 🔹Significant Cost Reduction: No need for expensive modeling/rendering. PCM distillation compresses inference steps, boosting speed and lowering costs. This allows the quantized 13B model to run on consumer-grade GPUs like the RTX 4090. Project Page: Code: Technical Report: Hugging Face：

🚀We are thrilled to open-source Hunyuan-GameCraft, a high-dynamic interactive game video generation framework built on HunyuanVideo. It generates playable and physically realistic videos from a single scene image and user action signals, empowering creators and developers to "direct" games with first-person or third-person perspectives. Key Advantages: 🔹High Dynamics: Unifies standard keyboard inputs into a shared continuous action space, enabling high-precision control over velocity and angle. This allows for the exploration of complex trajectories, overcoming the stiff, limited motion of traditional models. It can also generate dynamic environmental content like moving clouds, rain, snow, and water flow. 🔹Long-term Consistency: Uses hybrid history condition to preserve the original scene information after significant movement. 🔹Significant Cost Reduction: No need for expensive modeling/rendering. PCM distillation compresses inference steps, boosting speed and lowering costs. This allows the quantized 13B model to run on consumer-grade GPUs like the RTX 4090. Project Page: Code: Technical Report: Hugging Face：

Tencent Hy

169,638 Aufrufe • vor 11 Monaten

Generative AI is cool and all, but procedural 3D modeling just hits different. Check out this Houdini setup by Pepe Buendia. Why this is cool: Instead of manually placing every building and car, this system generates an NYC-style city that builds itself -- automatically spawning buildings with unique variations, plus a traffic system where cars actually obey traffic lights and avoid crashing into each other (all orchestrated through houdini VEX code). Each car knows exactly which road it's on and makes real-time decisions about navigation. The whole thing runs on custom algorithms that handle everything from road directions to building placement. If you did this with generative AI, even approaches to conditioning and taming the chaos -- you'd still get a ton of variance which might be an issue for visual and spatial consistency. To me this is procedural modeling at its finest - precise control meets infinite scalability. Immensely useful for media & entertainment, but also synthetic training data for robotics and augmented reality. I of course can't wait until multimodal LLMs to have the spatial understanding to write these kind of controllable 3D worlds for us. Claude counting 2D pixels on a screen is a step in this direction. Soon they'll be able to manipulate 2D ortho viewports, and eventually spawn procedural 3D worlds.

Generative AI is cool and all, but procedural 3D modeling just hits different. Check out this Houdini setup by Pepe Buendia. Why this is cool: Instead of manually placing every building and car, this system generates an NYC-style city that builds itself -- automatically spawning buildings with unique variations, plus a traffic system where cars actually obey traffic lights and avoid crashing into each other (all orchestrated through houdini VEX code). Each car knows exactly which road it's on and makes real-time decisions about navigation. The whole thing runs on custom algorithms that handle everything from road directions to building placement. If you did this with generative AI, even approaches to conditioning and taming the chaos -- you'd still get a ton of variance which might be an issue for visual and spatial consistency. To me this is procedural modeling at its finest - precise control meets infinite scalability. Immensely useful for media & entertainment, but also synthetic training data for robotics and augmented reality. I of course can't wait until multimodal LLMs to have the spatial understanding to write these kind of controllable 3D worlds for us. Claude counting 2D pixels on a screen is a step in this direction. Soon they'll be able to manipulate 2D ortho viewports, and eventually spawn procedural 3D worlds.

Bilawal Sidhu

527,289 Aufrufe • vor 1 Jahr

Can an inexpensive, off-the-shelf IMU be the only sensor to estimate the full state (position, velocity, orientation) of a quadrotor flying through a track at high speed and even be on-pair with vision-based localization? The answer is yes, within certain limitations! In this #RAL2023 paper, we propose a learning-based odometry algorithm that couples a model-based filter driven by the inertial measurements with a learning-based module with access to the control commands. Our system outperforms by a large margin the state-of-the-art visual-inertial odometry (#VIO) algorithms and the state-of-the-art learned-inertial odometry algorithm, #TLIO, for the task of drone racing. Additionally, we show that our system is as accurate as a VIO algorithm that uses a camera to localize to a known map of the racing track. The main limitation of our approach is that it cannot generalize to trajectories that have not been seen at training time. However, in drone racing competitions, the track is known beforehand. Human pilots spend hours or even days of practice on the race track before the competition. Similarly, our system can be trained with the data collected during practice time and deployed during the competition. Future work will investigate how to generalize to trajectories not seen at training time. The code is released! Paper: Video: Code: Kudos to Giovanni Cioffi Leonard Bauersfeld Elia Kaufmann European Research Council (ERC) University of Zurich UZH Science UZH Space Hub NCCR Robotics Aerial Core #RAL2023 #IROS2023 #SLAM

Can an inexpensive, off-the-shelf IMU be the only sensor to estimate the full state (position, velocity, orientation) of a quadrotor flying through a track at high speed and even be on-pair with vision-based localization? The answer is yes, within certain limitations! In this #RAL2023 paper, we propose a learning-based odometry algorithm that couples a model-based filter driven by the inertial measurements with a learning-based module with access to the control commands. Our system outperforms by a large margin the state-of-the-art visual-inertial odometry (#VIO) algorithms and the state-of-the-art learned-inertial odometry algorithm, #TLIO, for the task of drone racing. Additionally, we show that our system is as accurate as a VIO algorithm that uses a camera to localize to a known map of the racing track. The main limitation of our approach is that it cannot generalize to trajectories that have not been seen at training time. However, in drone racing competitions, the track is known beforehand. Human pilots spend hours or even days of practice on the race track before the competition. Similarly, our system can be trained with the data collected during practice time and deployed during the competition. Future work will investigate how to generalize to trajectories not seen at training time. The code is released! Paper: Video: Code: Kudos to Giovanni Cioffi Leonard Bauersfeld Elia Kaufmann European Research Council (ERC) University of Zurich UZH Science UZH Space Hub NCCR Robotics Aerial Core #RAL2023 #IROS2023 #SLAM

Davide Scaramuzza

37,061 Aufrufe • vor 2 Jahren

Be Smart as Karpathy Andrej Karpathy with Teamily AI 🧠 Your Personal Knowledge Base: ✅ Built in One Chat. 📈 Compounded via Conversations. Karpathy’s insight is spot on ( It attracts 10 million views in a few days. The idea is simple: AI should build personal knowledge from everything you feed it, so it stops rediscovering things from scratch like a Retrieval-Augmented Generation (RAG). But here’s the reality — most people aren’t Stanford PhD-level geeks like Karpathy. For the rest of us, operating a hacky collection of scripts and tools (Obsidian Web Clipper, Marp, Dataview, etc.) as seen in Karpathy’s idea file is far too complex ( The Internet needs an intuitive product where a personal knowledge base is a persistent, compounding artifact — one that grows alongside the content you consume, the contexts you inhabit, and the questions you ask. Teamily AI ( is the answer. The conversation IS the knowledge base. It’s an AI-native messenger where AI teammates join your chats. They remember your past discussions, your preferences, and your team’s context — getting smarter the more you talk. No setup. No complicated workflows. Just text as you normally do. Whether you’re saving articles and videos, brainstorming at work, or collaborating with colleagues, your AI teammates are right there. They listen, remember, and help — not from scratch every time, but by building a personal knowledge graph of everything you’re involved in. In essence, your knowledge compounds automatically. ✨ The user experience is effortless. Whenever you need a well-organized view of your data, just ask the "Personal AI" at the top of the Teamily window: "Visualize my personal knowledge base" Want to customize the style or indexes? Just chat with it. You define how you manage your knowledge. Our co-founder Aiden has prepared a short video to show you just how easy it is. 📽️

Be Smart as Karpathy Andrej Karpathy with Teamily AI 🧠 Your Personal Knowledge Base: ✅ Built in One Chat. 📈 Compounded via Conversations. Karpathy’s insight is spot on ( It attracts 10 million views in a few days. The idea is simple: AI should build personal knowledge from everything you feed it, so it stops rediscovering things from scratch like a Retrieval-Augmented Generation (RAG). But here’s the reality — most people aren’t Stanford PhD-level geeks like Karpathy. For the rest of us, operating a hacky collection of scripts and tools (Obsidian Web Clipper, Marp, Dataview, etc.) as seen in Karpathy’s idea file is far too complex ( The Internet needs an intuitive product where a personal knowledge base is a persistent, compounding artifact — one that grows alongside the content you consume, the contexts you inhabit, and the questions you ask. Teamily AI ( is the answer. The conversation IS the knowledge base. It’s an AI-native messenger where AI teammates join your chats. They remember your past discussions, your preferences, and your team’s context — getting smarter the more you talk. No setup. No complicated workflows. Just text as you normally do. Whether you’re saving articles and videos, brainstorming at work, or collaborating with colleagues, your AI teammates are right there. They listen, remember, and help — not from scratch every time, but by building a personal knowledge graph of everything you’re involved in. In essence, your knowledge compounds automatically. ✨ The user experience is effortless. Whenever you need a well-organized view of your data, just ask the "Personal AI" at the top of the Teamily window: "Visualize my personal knowledge base" Want to customize the style or indexes? Just chat with it. You define how you manage your knowledge. Our co-founder Aiden has prepared a short video to show you just how easy it is. 📽️

Teamily AI

15,517 Aufrufe • vor 3 Monaten

We are excited to share #PDF2Audio, an open-source alternative to the #podcast feature of #NotebookLM with flexibility & tailored outputs that you can precisely control in the app: You can make a podcast, lecture, discussions, short/long form summaries & more, including the use of the amazing🍓o1 model (Sam Altman OpenAI: with stunning results!). Code & HF Space: You can find #PDF2Audio on GitHub for local use or try the Hugging Face space, all featuring Gradio. Link to the repo & HF space in the reply. Thank you @knowsuchagencyfor the great work on #promptic and #pdf2podcast, as well as LiteLLM (YC W23), & AK for helping us with the Hugging Face spaces version. We hope that this tool is useful for the community. Background: Developing audio podcasts, lectures, & summaries from complex documents & data has become an exciting trend with impacts from research to education to business. Our open-source #PDF2Audio tool that allows users to utilize various models such as #o1 or local/open-source models, to develop deep-dives into technical content. Example application - material design analysis: As an example to show what the system can do, check the video for a detailed 13-minute analysis of one of the designs created by #SciAgents merging silk & dandelion pigments, created using 🍓o1. The conversation describes the new material, an integration of silk proteins & luteolin/dandelion pigments to create a new biomaterial. Silk, a natural #nanostructured protein-based fiber known for its strength & flexibility, is combined with dandelion pigments like luteolin, which offer unique optical properties. By merging these components at the nanoscale level, the resulting material displays structural coloration—vibrant, tunable colors created by the material's structure rather than synthetic dyes, and leverages silk's hierarchical organization as a scaffold for the pigments, ensuring uniform distribution and non-covalent bonding at the molecular level. Key technical features include: ➡️Low-temperature processing to maintain the integrity of both silk and pigments while reducing energy consumption by 30%. ➡️Enhanced mechanical properties, with tensile strength up to 1.5 gigapascals. ➡️Potential self-healing capabilities and environmental responsiveness, allowing the material to repair minor damage and change color based on environmental conditions. ➡️UV protection and antimicrobial properties, which make this material ideal for smart textiles, eco-friendly coatings, and medical applications. This development opens new doors for sustainable materials, offering an eco-friendly alternative to synthetic fibers with applications in various industries, from fashion to healthcare.

We are excited to share #PDF2Audio, an open-source alternative to the #podcast feature of #NotebookLM with flexibility & tailored outputs that you can precisely control in the app: You can make a podcast, lecture, discussions, short/long form summaries & more, including the use of the amazing🍓o1 model (Sam Altman OpenAI: with stunning results!). Code & HF Space: You can find #PDF2Audio on GitHub for local use or try the Hugging Face space, all featuring Gradio. Link to the repo & HF space in the reply. Thank you @knowsuchagencyfor the great work on #promptic and #pdf2podcast, as well as LiteLLM (YC W23), & AK for helping us with the Hugging Face spaces version. We hope that this tool is useful for the community. Background: Developing audio podcasts, lectures, & summaries from complex documents & data has become an exciting trend with impacts from research to education to business. Our open-source #PDF2Audio tool that allows users to utilize various models such as #o1 or local/open-source models, to develop deep-dives into technical content. Example application - material design analysis: As an example to show what the system can do, check the video for a detailed 13-minute analysis of one of the designs created by #SciAgents merging silk & dandelion pigments, created using 🍓o1. The conversation describes the new material, an integration of silk proteins & luteolin/dandelion pigments to create a new biomaterial. Silk, a natural #nanostructured protein-based fiber known for its strength & flexibility, is combined with dandelion pigments like luteolin, which offer unique optical properties. By merging these components at the nanoscale level, the resulting material displays structural coloration—vibrant, tunable colors created by the material's structure rather than synthetic dyes, and leverages silk's hierarchical organization as a scaffold for the pigments, ensuring uniform distribution and non-covalent bonding at the molecular level. Key technical features include: ➡️Low-temperature processing to maintain the integrity of both silk and pigments while reducing energy consumption by 30%. ➡️Enhanced mechanical properties, with tensile strength up to 1.5 gigapascals. ➡️Potential self-healing capabilities and environmental responsiveness, allowing the material to repair minor damage and change color based on environmental conditions. ➡️UV protection and antimicrobial properties, which make this material ideal for smart textiles, eco-friendly coatings, and medical applications. This development opens new doors for sustainable materials, offering an eco-friendly alternative to synthetic fibers with applications in various industries, from fashion to healthcare.

Markus J. Buehler

208,353 Aufrufe • vor 1 Jahr

SHE CHARGES $7500 FOR A SITE YOU CONTROL BY WAVING AT YOUR WEBCAM no mouse, no scroll, no clicking anything, the camera tracks your hand and the whole site responds as you move it through the air how she built it: render the base clip in seedance 2 open replit and attach your design reference tell it to wire hand tracking with mediapipe map the hand position to the video in a single prompt still in the same chat, switch it to slide mode or a scroll transform whenever you want she never wrote one line of the tracking code, replit stood up the entire gesture layer from that one instruction meanwhile your local dev still thinks interactive means a button that lights up when the cursor grazes it the exact mediapipe prompt is in the guide below, save it before this becomes the standard

SHE CHARGES $7500 FOR A SITE YOU CONTROL BY WAVING AT YOUR WEBCAM no mouse, no scroll, no clicking anything, the camera tracks your hand and the whole site responds as you move it through the air how she built it: render the base clip in seedance 2 open replit and attach your design reference tell it to wire hand tracking with mediapipe map the hand position to the video in a single prompt still in the same chat, switch it to slide mode or a scroll transform whenever you want she never wrote one line of the tracking code, replit stood up the entire gesture layer from that one instruction meanwhile your local dev still thinks interactive means a button that lights up when the cursor grazes it the exact mediapipe prompt is in the guide below, save it before this becomes the standard

Bounce

58,238 Aufrufe • vor 21 Tagen