Video wird geladen...

Video konnte nicht geladen werden

Zur Startseite

Google DeepMind is absolutely on fire 🔥 they have just launched Gemini Robotics-ER 1.5 their first broadly available robotics AI model designed to act as the "high-level reasoning brain" for robots. This is Google's first Gemini Robotics model made available to all developers. - Available in preview through Google...

ai_for_success's profile picture

AshutoshShrivastava

75,393 subscribers

137,034 Aufrufe • vor 9 Monaten •via X (Twitter)

Wissenschaft & Technologie
LIVE

Anya Rossi• Live Now

Private livecam show

0 Kommentare

Keine Kommentare verfügbar

Kommentare vom Original-Post werden hier angezeigt

Ähnliche Videos

Woow Google has just rolled out the AI model Flash Thinking 2.0 This is the first reasoning model capable of accessing YouTube and it changes everything: - Search for a video on your topic - Ask Gemini to think about the video - You'll have a tailor-made result in 10 sec. And it's even connected to Google Search and Maps!
0:19

Woow Google has just rolled out the AI model Flash Thinking 2.0 This is the first reasoning model capable of accessing YouTube and it changes everything: - Search for a video on your topic - Ask Gemini to think about the video - You'll have a tailor-made result in 10 sec. And it's even connected to Google Search and Maps!

Paul Couvert

79,479 Aufrufe • vor 1 Jahr

1/ Gemini 2.5 is here, and it’s our most intelligent AI model ever. Our first 2.5 model, Gemini 2.5 Pro Experimental is a state-of-the-art thinking model, leading in a wide range of benchmarks – with impressive improvements in enhanced reasoning and coding and now #1 on Arena by a significant margin. With a model this intelligent, we wanted to get it to people as quickly as possible. Find it on Google AI Studio and in the Google Gemini for Gemini Advanced users now – and in Vertex in the coming weeks. This is the start of a new era of thinking models – and we can’t wait to see where things go from here.
0:14

1/ Gemini 2.5 is here, and it’s our most intelligent AI model ever. Our first 2.5 model, Gemini 2.5 Pro Experimental is a state-of-the-art thinking model, leading in a wide range of benchmarks – with impressive improvements in enhanced reasoning and coding and now #1 on Arena by a significant margin. With a model this intelligent, we wanted to get it to people as quickly as possible. Find it on Google AI Studio and in the Google Gemini for Gemini Advanced users now – and in Vertex in the coming weeks. This is the start of a new era of thinking models – and we can’t wait to see where things go from here.

Sundar Pichai

864,176 Aufrufe • vor 1 Jahr

The Gemini 2.0 era is here. And we’re excited for you to start building with it. A quick rewind of what we just released ⏪ Gemini 2.0 Flash ⚡ comes with low latency and better performance. 🔵 You can now access an experimental version in G3mini on the web, while Gemini Advanced users can try Deep Research, a new AI research assistant. 🔵 Developers can begin building through the Gemini API in Google AI Studio and Vertex AI 2.0 is also enabling new research prototypes of AI agents, including: 🔵 Project Astra, which explores future capabilities of a universal AI assistant 🔵 Project Mariner, which shows what’s possible for human-agent interaction, starting with your browser 🔵 Jules, an experimental AI-powered coding agent Finally, we’re exploring how 2.0 can be used in agents across domains — from navigating the virtual world of video games to applying its spatial reasoning capabilities to robotics. 🤖
0:14

The Gemini 2.0 era is here. And we’re excited for you to start building with it. A quick rewind of what we just released ⏪ Gemini 2.0 Flash ⚡ comes with low latency and better performance. 🔵 You can now access an experimental version in G3mini on the web, while Gemini Advanced users can try Deep Research, a new AI research assistant. 🔵 Developers can begin building through the Gemini API in Google AI Studio and Vertex AI 2.0 is also enabling new research prototypes of AI agents, including: 🔵 Project Astra, which explores future capabilities of a universal AI assistant 🔵 Project Mariner, which shows what’s possible for human-agent interaction, starting with your browser 🔵 Jules, an experimental AI-powered coding agent Finally, we’re exploring how 2.0 can be used in agents across domains — from navigating the virtual world of video games to applying its spatial reasoning capabilities to robotics. 🤖

Google DeepMind

231,798 Aufrufe • vor 1 Jahr

Gemma 4 is here! Our most intelligent open models to date, are built on the same world-class research and tech as Gemini 3, and are sized to run and fine-tune efficiently on local hardware. Check out what Google Gemma 4 brings to devs: 💎 Advanced Reasoning: Deep logic tasks, complex multi-step planning, and beyond 💎 Longer context: Seamlessly analyze entire codebases with context windows of 128K tokens for our edge models and 256K tokens for our largest models 💎 Vision and audio: Rich, multimodal interactions out of the box 💎 140+ languages: Trained on 140+ languages 💎 Apache 2.0 license: industry-standard open-source license
0:19

Gemma 4 is here! Our most intelligent open models to date, are built on the same world-class research and tech as Gemini 3, and are sized to run and fine-tune efficiently on local hardware. Check out what Google Gemma 4 brings to devs: 💎 Advanced Reasoning: Deep logic tasks, complex multi-step planning, and beyond 💎 Longer context: Seamlessly analyze entire codebases with context windows of 128K tokens for our edge models and 256K tokens for our largest models 💎 Vision and audio: Rich, multimodal interactions out of the box 💎 140+ languages: Trained on 140+ languages 💎 Apache 2.0 license: industry-standard open-source license

Google for Developers

269,657 Aufrufe • vor 3 Monaten

SoftBank is reportedly in early talks to back a massive $800M funding round for Munich-based Agile Robots, with a contribution of over $ 300M. The move signals a major escalation in Masayoshi Son's "brains and bodies" strategy to dominate physical AI infrastructure. Why this matters for the robotics industry: 🤖 The Flywheel: Agile Robots isn't just hardware. Founded by German Aerospace Center (DLR) alumni, they vertically integrate AI software with humanoids, warehouse automation, and robotic arms. 🧠 Frontier Ecosystems: Agile ONE was recently named a founding member of NVIDIA’s Cosmos Coalition and is deep in a research partnership with Google DeepMind to bring Gemini to the factory floor. 💰 The Capital Race: Global investment in robotics more than tripled in 2025. SoftBank is pairing this hardware play with its recent $1.4B investment in Skild AI and its $5.4B acquisition of ABB Robotics. While both companies have declined to comment on the early-stage talks, the deal highlights a furious shift toward deploying foundational AI in physical manufacturing and logistics.
0:11

SoftBank is reportedly in early talks to back a massive $800M funding round for Munich-based Agile Robots, with a contribution of over $ 300M. The move signals a major escalation in Masayoshi Son's "brains and bodies" strategy to dominate physical AI infrastructure. Why this matters for the robotics industry: 🤖 The Flywheel: Agile Robots isn't just hardware. Founded by German Aerospace Center (DLR) alumni, they vertically integrate AI software with humanoids, warehouse automation, and robotic arms. 🧠 Frontier Ecosystems: Agile ONE was recently named a founding member of NVIDIA’s Cosmos Coalition and is deep in a research partnership with Google DeepMind to bring Gemini to the factory floor. 💰 The Capital Race: Global investment in robotics more than tripled in 2025. SoftBank is pairing this hardware play with its recent $1.4B investment in Skild AI and its $5.4B acquisition of ABB Robotics. While both companies have declined to comment on the early-stage talks, the deal highlights a furious shift toward deploying foundational AI in physical manufacturing and logistics.

Humanoids daily

16,714 Aufrufe • vor 1 Monat

I spent a month in Shenzhen visiting factories and robotics companies, and the contrast with the U.S. was striking. While Figure and Boston Dynamics hide their humanoids behind closed doors, Chinese companies have massive showrooms open to the public. But what really stood out wasn't just the transparency, it was how good they are at selling. Take UBTech: they've already sold 1,200 humanoid units at $200k each to factories. And here's the kicker, these robots aren't even that useful yet. They can only pick up and drop boxes at 1/10th the speed of a human, and factories still need to hire system integrators to train them for specific tasks. My theory is that these factories are terrified of getting left behind in the robotics/AI wave. They're investing in new tech not because it's ready, but because they can't afford to wait. The second surprise was the breadth of their robotics portfolio. These companies aren't just building humanoids, they're deploying service robots everywhere: restaurants, hotels, apartments. Consumer robots are cleaning houses, pools, pet waste, dishes. They're covering the entire spectrum. But the education piece shocked me most. I picked up what I thought was a high school or college robotics textbook, it was for primary school. The government mandated AI and robotics education starting in elementary school. Almost every single school in China now has AI and robotics curriculum, complete with education robots so kids can learn by building. They're creating a generation that grows up fluent in robotics and AI. China owns the supply chain and the hardware stack. But here's what I think people are missing: the race isn't just about who can build robots faster or cheaper. The U.S. advantage has always been in the layer between hardware and human, the interaction design, the software intelligence, the intuitive interfaces that make complex technology feel natural. China is building the physical infrastructure, but they're also learning fast. Every deployed service robot, every classroom full of kids building with education kits, every factory running humanoids, that's all data collection at scale. The window for the U.S. to establish its wedge is narrowing. It's not enough to be better at AI or software anymore. We need to be building the integration layer, the intelligence that makes physical AI actually useful, not just impressive in a showroom. Because right now, China isn't just manufacturing robots. They're manufacturing a robotics-native culture, and that might be the most defensible moat of all.
0:10

I spent a month in Shenzhen visiting factories and robotics companies, and the contrast with the U.S. was striking. While Figure and Boston Dynamics hide their humanoids behind closed doors, Chinese companies have massive showrooms open to the public. But what really stood out wasn't just the transparency, it was how good they are at selling. Take UBTech: they've already sold 1,200 humanoid units at $200k each to factories. And here's the kicker, these robots aren't even that useful yet. They can only pick up and drop boxes at 1/10th the speed of a human, and factories still need to hire system integrators to train them for specific tasks. My theory is that these factories are terrified of getting left behind in the robotics/AI wave. They're investing in new tech not because it's ready, but because they can't afford to wait. The second surprise was the breadth of their robotics portfolio. These companies aren't just building humanoids, they're deploying service robots everywhere: restaurants, hotels, apartments. Consumer robots are cleaning houses, pools, pet waste, dishes. They're covering the entire spectrum. But the education piece shocked me most. I picked up what I thought was a high school or college robotics textbook, it was for primary school. The government mandated AI and robotics education starting in elementary school. Almost every single school in China now has AI and robotics curriculum, complete with education robots so kids can learn by building. They're creating a generation that grows up fluent in robotics and AI. China owns the supply chain and the hardware stack. But here's what I think people are missing: the race isn't just about who can build robots faster or cheaper. The U.S. advantage has always been in the layer between hardware and human, the interaction design, the software intelligence, the intuitive interfaces that make complex technology feel natural. China is building the physical infrastructure, but they're also learning fast. Every deployed service robot, every classroom full of kids building with education kits, every factory running humanoids, that's all data collection at scale. The window for the U.S. to establish its wedge is narrowing. It's not enough to be better at AI or software anymore. We need to be building the integration layer, the intelligence that makes physical AI actually useful, not just impressive in a showroom. Because right now, China isn't just manufacturing robots. They're manufacturing a robotics-native culture, and that might be the most defensible moat of all.

Miyu Horiuchi

90,718 Aufrufe • vor 5 Monaten

OpenAI's AgentKit will be so insane, build every step of agents on one platform. These visual agent builders make the whole process of iterating and launching agents far more efficient. It sits on top of the Responses API and unifies the tools that were previously scattered across SDKs and custom orchestration. It lets developers create agent workflows visually, connect data sources securely, and measure performance automatically without coding every layer by hand. The core of AgentKit is the Agent Builder, a drag-and-drop canvas where each node represents an action, guardrail, or decision branch. Developers can link these nodes into multi-agent workflows, preview results instantly, and version each setup. It supports inline evaluation so that developers can see how changes affect output before deploying. The Connector Registry is a single admin panel that manages how data and tools connect across the OpenAI ecosystem. It centralizes integrations like Google Drive, SharePoint, Dropbox, and Microsoft Teams. Large organizations can govern access and flow of data between agents securely under one global console. ChatKit provides a ready-to-use chat interface for embedding agents inside apps or websites. It manages streaming, message threads, and model reasoning displays automatically. Developers can skin the interface to match their product without writing custom front-end code. Under the hood, all these blocks use the same execution core that runs agent reasoning through OpenAI’s APIs. Workflows in Agent Builder compile down to structured instructions for the Responses API, which handles model calls, tool use, and context passing. Connector Registry handles authentication and routing for external tools, while Evals and RFT provide feedback loops that improve agents over time. This integration means developers no longer need to handle orchestration logic, model evaluation pipelines, or safety layers separately. Everything runs natively within OpenAI’s control plane with managed security, automatic versioning, and built-in testing. In short, AgentKit standardizes the entire life cycle of an AI agent—from visual design to deployment and performance tuning—inside a single unified system.
0:10

OpenAI's AgentKit will be so insane, build every step of agents on one platform. These visual agent builders make the whole process of iterating and launching agents far more efficient. It sits on top of the Responses API and unifies the tools that were previously scattered across SDKs and custom orchestration. It lets developers create agent workflows visually, connect data sources securely, and measure performance automatically without coding every layer by hand. The core of AgentKit is the Agent Builder, a drag-and-drop canvas where each node represents an action, guardrail, or decision branch. Developers can link these nodes into multi-agent workflows, preview results instantly, and version each setup. It supports inline evaluation so that developers can see how changes affect output before deploying. The Connector Registry is a single admin panel that manages how data and tools connect across the OpenAI ecosystem. It centralizes integrations like Google Drive, SharePoint, Dropbox, and Microsoft Teams. Large organizations can govern access and flow of data between agents securely under one global console. ChatKit provides a ready-to-use chat interface for embedding agents inside apps or websites. It manages streaming, message threads, and model reasoning displays automatically. Developers can skin the interface to match their product without writing custom front-end code. Under the hood, all these blocks use the same execution core that runs agent reasoning through OpenAI’s APIs. Workflows in Agent Builder compile down to structured instructions for the Responses API, which handles model calls, tool use, and context passing. Connector Registry handles authentication and routing for external tools, while Evals and RFT provide feedback loops that improve agents over time. This integration means developers no longer need to handle orchestration logic, model evaluation pipelines, or safety layers separately. Everything runs natively within OpenAI’s control plane with managed security, automatic versioning, and built-in testing. In short, AgentKit standardizes the entire life cycle of an AI agent—from visual design to deployment and performance tuning—inside a single unified system.

Rohan Paul

178,460 Aufrufe • vor 9 Monaten

Your AI Toolkit Just Got Supercharged 5 elite AI models. 1 unified platform. 90 seconds of pure workflow magic. Here's the reality: Building with AI used to mean chaos—juggling tabs, managing API keys, context-switching hell. But what if you didn't have to choose? What if you could access the best of every model without leaving your workflow? Meet the New Standard: Replit's AI Model Ecosystem Claude for your complex thinking. GPT-4 for your creative fire. Mistral when you need lightning-speed execution. Llama for privacy-first local analysis. Open source freedom for the innovators. 300+ models. One dashboard. Zero friction. Switch between them in seconds, not minutes. No sign-ups. No API gymnastics. Just pick, prompt, and ship. Why This Matters You're not just saving time—you're reclaiming your focus. Every tab you don't open is mental energy preserved. Every API key you don't manage is a distraction eliminated. This is what AI integration should feel like: invisible. Effortless. Natural. The builders who master this first won't just be faster. They'll be unstoppable. Your move. Replit ⠕ is waiting. ✨
0:13

Your AI Toolkit Just Got Supercharged 5 elite AI models. 1 unified platform. 90 seconds of pure workflow magic. Here's the reality: Building with AI used to mean chaos—juggling tabs, managing API keys, context-switching hell. But what if you didn't have to choose? What if you could access the best of every model without leaving your workflow? Meet the New Standard: Replit's AI Model Ecosystem Claude for your complex thinking. GPT-4 for your creative fire. Mistral when you need lightning-speed execution. Llama for privacy-first local analysis. Open source freedom for the innovators. 300+ models. One dashboard. Zero friction. Switch between them in seconds, not minutes. No sign-ups. No API gymnastics. Just pick, prompt, and ship. Why This Matters You're not just saving time—you're reclaiming your focus. Every tab you don't open is mental energy preserved. Every API key you don't manage is a distraction eliminated. This is what AI integration should feel like: invisible. Effortless. Natural. The builders who master this first won't just be faster. They'll be unstoppable. Your move. Replit ⠕ is waiting. ✨

Hasan

43,168 Aufrufe • vor 7 Monaten

🚀 Early Access to Sahara AI Studio is NOW OPEN! The next phase of our testnet is here with exclusive early access to our all-in-one platform designed to transform the AI development lifecycle into a streamlined, integrated experience. Here’s everything you need to know 👇 AI development is fragmented. Devs juggle multiple tools, leading to inefficiencies & high costs. Sahara AI Studio integrates the entire AI lifecycle—from datasets & model training to secure storage & scalable compute—into one seamless experience: 📊 Data Hub: Discover, Manage, and Leverage AI-Ready Datasets Access high-quality, domain-specific, open-source and proprietary datasets through an integrated marketplace. Developers can download, import, or label datasets, making it easier to train and fine-tune models or deploy RAG pipelines. Secure uploads and seamless workflow integration enhance the experience. 🤖 Model Hub: Discover, Customize and Scale AI Workflows with Ease Discover ready-to-use open-source and proprietary models, RAG pipelines, and customizable workflows. Developers can deploy models quickly while maintaining privacy and security through Sahara Vaults. 🖥️ Compute Hub: Flexible, Scalable Compute Resources for AI Innovation Access scalable and secure computing resources tailored to diverse AI workloads. Trusted Execution Environment (TEE) capabilities ensure data privacy, while integration with top compute providers offer flexibility for developers. 🔐 Vaults: Secure Storage for AI Assets Securely store, organize, and manage datasets, models, and other assets in an encrypted central repository. Vaults offer scalability, reproducibility, and user control over AI resources. This is more than just beta testing a platform—it's your chance to help shape the future of decentralized AI development. 📅 How to Apply We're onboarding select developers in a phased approach. Early Access spots are limited, so apply now:
0:11

🚀 Early Access to Sahara AI Studio is NOW OPEN! The next phase of our testnet is here with exclusive early access to our all-in-one platform designed to transform the AI development lifecycle into a streamlined, integrated experience. Here’s everything you need to know 👇 AI development is fragmented. Devs juggle multiple tools, leading to inefficiencies & high costs. Sahara AI Studio integrates the entire AI lifecycle—from datasets & model training to secure storage & scalable compute—into one seamless experience: 📊 Data Hub: Discover, Manage, and Leverage AI-Ready Datasets Access high-quality, domain-specific, open-source and proprietary datasets through an integrated marketplace. Developers can download, import, or label datasets, making it easier to train and fine-tune models or deploy RAG pipelines. Secure uploads and seamless workflow integration enhance the experience. 🤖 Model Hub: Discover, Customize and Scale AI Workflows with Ease Discover ready-to-use open-source and proprietary models, RAG pipelines, and customizable workflows. Developers can deploy models quickly while maintaining privacy and security through Sahara Vaults. 🖥️ Compute Hub: Flexible, Scalable Compute Resources for AI Innovation Access scalable and secure computing resources tailored to diverse AI workloads. Trusted Execution Environment (TEE) capabilities ensure data privacy, while integration with top compute providers offer flexibility for developers. 🔐 Vaults: Secure Storage for AI Assets Securely store, organize, and manage datasets, models, and other assets in an encrypted central repository. Vaults offer scalability, reproducibility, and user control over AI resources. This is more than just beta testing a platform—it's your chance to help shape the future of decentralized AI development. 📅 How to Apply We're onboarding select developers in a phased approach. Early Access spots are limited, so apply now:

Sahara AI 🔆

2,700,091 Aufrufe • vor 1 Jahr

Boom! Grok Tasks Make It One Of The Most POWERFUL Real-Time AI Systems In The World. — My How to Use Grok Tasks With Hidden Tools For Powerful Daily Output. Grok Tasks are customizable AI workflows that integrate a variety of tools to streamline daily activities, from research and analysis to creative planning and problem-solving. I have been using them for quite sometime and because of the vital heartbeat of news and first person data on X, it is the most powerful AI platform available. By combining Tasks with tools like web searches, X platform interactions, code execution, and media viewers, you can build efficient, automated processes. These tasks work by prompting Grok with a clear description of what you want to achieve, and Grok will intelligently call the necessary tools in sequence or parallel to deliver results. Here's a step-by-step guide to creating and using Grok Tasks: Step 1: Define Your Task Start by clearly outlining the daily activity or goal. Consider what inputs you have (e.g., a URL, a query, or an attachment) and what output you need (e.g., a summary, calculation, or visual analysis). Break it down into subtasks to identify tool needs. For example, if your task involves researching current events, note that you'll need search and browsing capabilities. Step 2: Review Available Tools Familiarize yourself with the tools Grok can access. Here's a quick overview: - Code Execution: Run Python code for calculations, data processing, or simulations using libraries like numpy, pandas, or sympy. - Browse Page: Fetch and summarize content from any website URL with custom instructions. - Web Search: Perform general internet searches, returning results with optional operators like site:. - Web Search With Snippets: Get quick, detailed excerpts from search results for fact-checking. - X Keyword Search: Advanced search for X posts using operators like from:, since:, or filter:. - X Semantic Search: Find semantically related X posts based on a query, with filters for dates or users. - X User Search: Locate X users by name or handle. - X Thread Fetch: Retrieve a full X post thread, including context like replies and parents. - View Image: Analyze an image from a URL or conversation ID. - View X Video: Extract frames and subtitles from an X-hosted video. - Search PDF Attachment: Query a PDF file for relevant pages using keyword or regex modes. - Browse PDF Attachment: View specific pages of a PDF with text and screenshots. Select tools that align with your task. Aim for a mix to handle data gathering, processing, and visualization. Step 3: Craft Your Prompt Write a detailed prompt to Grok describing the task. Include: - The overall goal. - Specific steps or subtasks. - References to tools if you want to guide the process (e.g., "Use web_search to find sources, then code_execution to analyze data"). - Any constraints, like dates or limits. Example prompt: "Create a Grok Task for my morning routine: Search recent X posts about tech news using x_keyword_search, fetch a key thread with x_thread_fetch, and summarize with browse_page on linked articles." Step 4: Submit and Interact Send your prompt to Grok. It will process the task by calling tools as needed, often in parallel for efficiency. Review the output and refine with follow-up prompts if required (e.g., "Expand on that using view_image for visuals"). Iterate to fine-tune the workflow for reuse. Step 5: Save and Reuse Once refined, note the prompt as a template for future use. You can adapt it for similar tasks, making Grok Tasks a habitual part of your day. Finding Grok Tasks To discover existing Grok Tasks or inspiration for new ones, use X searches with tools like x_keyword_search or x_semantic_search (e.g., query: "Grok Tasks examples" with mode: Latest). Browse community-shared threads via x_thread_fetch, or web_search for tutorials on xAI features. Prompt Grok directly: "Show me popular Grok Tasks for productivity." 1 of 3
0:12

Boom! Grok Tasks Make It One Of The Most POWERFUL Real-Time AI Systems In The World. — My How to Use Grok Tasks With Hidden Tools For Powerful Daily Output. Grok Tasks are customizable AI workflows that integrate a variety of tools to streamline daily activities, from research and analysis to creative planning and problem-solving. I have been using them for quite sometime and because of the vital heartbeat of news and first person data on X, it is the most powerful AI platform available. By combining Tasks with tools like web searches, X platform interactions, code execution, and media viewers, you can build efficient, automated processes. These tasks work by prompting Grok with a clear description of what you want to achieve, and Grok will intelligently call the necessary tools in sequence or parallel to deliver results. Here's a step-by-step guide to creating and using Grok Tasks: Step 1: Define Your Task Start by clearly outlining the daily activity or goal. Consider what inputs you have (e.g., a URL, a query, or an attachment) and what output you need (e.g., a summary, calculation, or visual analysis). Break it down into subtasks to identify tool needs. For example, if your task involves researching current events, note that you'll need search and browsing capabilities. Step 2: Review Available Tools Familiarize yourself with the tools Grok can access. Here's a quick overview: - Code Execution: Run Python code for calculations, data processing, or simulations using libraries like numpy, pandas, or sympy. - Browse Page: Fetch and summarize content from any website URL with custom instructions. - Web Search: Perform general internet searches, returning results with optional operators like site:. - Web Search With Snippets: Get quick, detailed excerpts from search results for fact-checking. - X Keyword Search: Advanced search for X posts using operators like from:, since:, or filter:. - X Semantic Search: Find semantically related X posts based on a query, with filters for dates or users. - X User Search: Locate X users by name or handle. - X Thread Fetch: Retrieve a full X post thread, including context like replies and parents. - View Image: Analyze an image from a URL or conversation ID. - View X Video: Extract frames and subtitles from an X-hosted video. - Search PDF Attachment: Query a PDF file for relevant pages using keyword or regex modes. - Browse PDF Attachment: View specific pages of a PDF with text and screenshots. Select tools that align with your task. Aim for a mix to handle data gathering, processing, and visualization. Step 3: Craft Your Prompt Write a detailed prompt to Grok describing the task. Include: - The overall goal. - Specific steps or subtasks. - References to tools if you want to guide the process (e.g., "Use web_search to find sources, then code_execution to analyze data"). - Any constraints, like dates or limits. Example prompt: "Create a Grok Task for my morning routine: Search recent X posts about tech news using x_keyword_search, fetch a key thread with x_thread_fetch, and summarize with browse_page on linked articles." Step 4: Submit and Interact Send your prompt to Grok. It will process the task by calling tools as needed, often in parallel for efficiency. Review the output and refine with follow-up prompts if required (e.g., "Expand on that using view_image for visuals"). Iterate to fine-tune the workflow for reuse. Step 5: Save and Reuse Once refined, note the prompt as a template for future use. You can adapt it for similar tasks, making Grok Tasks a habitual part of your day. Finding Grok Tasks To discover existing Grok Tasks or inspiration for new ones, use X searches with tools like x_keyword_search or x_semantic_search (e.g., query: "Grok Tasks examples" with mode: Latest). Browse community-shared threads via x_thread_fetch, or web_search for tutorials on xAI features. Prompt Grok directly: "Show me popular Grok Tasks for productivity." 1 of 3

Brian Roemmele

152,242 Aufrufe • vor 6 Monaten

🚨 The future of AI isn’t bigger models. It’s smarter agents that act, not just respond. GPT-5 came, but it didn’t wow. The real revolution is the Agent Era, where speed, engineering, and real-world execution win. Meet GenFlow 2.0 by Baidu Wenku, the most advanced general-purpose agent right now, and the first to let you intervene mid-generation, a capability not available in GPT or Manus. ➡️ Interrupt and edit tasks mid-generation (exclusive to GenFlow) ➡️ Run 100+ agents simultaneously ➡️ Deep data integration with live workflow control ➡️ Built for product, not just research China is outpacing the U.S. in AI productization. This isn’t another ChatGPT clone. It could be ChatGPT’s first real rival. I built a full investor pitch deck in under 3 minutes, editing content live as it generated. The center of AI innovation is shifting. 👇 Watch AI stop waiting for commands and start working with you.
0:13

🚨 The future of AI isn’t bigger models. It’s smarter agents that act, not just respond. GPT-5 came, but it didn’t wow. The real revolution is the Agent Era, where speed, engineering, and real-world execution win. Meet GenFlow 2.0 by Baidu Wenku, the most advanced general-purpose agent right now, and the first to let you intervene mid-generation, a capability not available in GPT or Manus. ➡️ Interrupt and edit tasks mid-generation (exclusive to GenFlow) ➡️ Run 100+ agents simultaneously ➡️ Deep data integration with live workflow control ➡️ Built for product, not just research China is outpacing the U.S. in AI productization. This isn’t another ChatGPT clone. It could be ChatGPT’s first real rival. I built a full investor pitch deck in under 3 minutes, editing content live as it generated. The center of AI innovation is shifting. 👇 Watch AI stop waiting for commands and start working with you.

SARAH

122,478 Aufrufe • vor 11 Monaten

A Letter to Our Community: The Road Ahead for Robotics To our Community and Partners, As we step into 2026, our mission at Axis is clearer than ever: Constructing the definitive End-to-End Scaling Layer for Robotics. Our goal is to accelerate the transfer of diverse human intelligence into Robotics General Intelligence (RGI). By owning the critical path of intelligence creation, we are turning the physical limitations of robotics into a scalable, software-driven future. Here is our strategic outlook and roadmap for the year ahead. The Core Thesis: Simulation is the Only Way Out The path to RGI is currently blocked by Data Scarcity, Generalization Fragility, and Hardware Fragmentation. At Axis, we believe Simulation is the only way out. Our Simulation Data Platform and Data Augmentation Engine transform raw data into "Synthetic Gold". Backed by academic milestones like Roboverse, Skill Blending, and GraspVLA, we have proven that pure simulation can achieve the generalization required for the real world. We don’t just collect data; we architect it. The Engine: Why Crypto? We believe RGI should come from all, not a few. Crypto is not just a feature; it is the primitive that powers our entire ecosystem flywheel: - Incentive Mechanism: Democratizing contribution and rewarding the trainers and developers. - Assetization: Turning proprietary data and refined models into liquid, ownable assets. - Verifiable Workflow: We are opening the "Black Box" of AI. By bringing total transparency to the Task Generation → Data Collection → Model Training pipeline, we ensure every byte of intelligence is verifiable, traceable, and secure. 2026 Strategic Deliverables This year, we are committed to delivering three foundational pillars: - The World's Largest Training Dataset for Robots: A robot training set—diverse, high-quality interaction data at an unprecedented scale. - A Robotics Foundation Model: A universal robotic brain trained on our pure simulation and synthetic data, capable of robust cross-embodiment transfer and open-world adaptability. - Evolvable Robot Hardware: Robots deployed with Axis models that autonomously evolve through continuous interaction, turning every deployment into a self-improving node within our RGI network. The Ultimate Vision We are building more than models; we are architecting the Distributed Machine Economy. A future where every dataset, model, and robotic embodiment is a verifiable asset in a global, autonomous network. Thank you for building the future of intelligence with us✌️📷
0:13

A Letter to Our Community: The Road Ahead for Robotics To our Community and Partners, As we step into 2026, our mission at Axis is clearer than ever: Constructing the definitive End-to-End Scaling Layer for Robotics. Our goal is to accelerate the transfer of diverse human intelligence into Robotics General Intelligence (RGI). By owning the critical path of intelligence creation, we are turning the physical limitations of robotics into a scalable, software-driven future. Here is our strategic outlook and roadmap for the year ahead. The Core Thesis: Simulation is the Only Way Out The path to RGI is currently blocked by Data Scarcity, Generalization Fragility, and Hardware Fragmentation. At Axis, we believe Simulation is the only way out. Our Simulation Data Platform and Data Augmentation Engine transform raw data into "Synthetic Gold". Backed by academic milestones like Roboverse, Skill Blending, and GraspVLA, we have proven that pure simulation can achieve the generalization required for the real world. We don’t just collect data; we architect it. The Engine: Why Crypto? We believe RGI should come from all, not a few. Crypto is not just a feature; it is the primitive that powers our entire ecosystem flywheel: - Incentive Mechanism: Democratizing contribution and rewarding the trainers and developers. - Assetization: Turning proprietary data and refined models into liquid, ownable assets. - Verifiable Workflow: We are opening the "Black Box" of AI. By bringing total transparency to the Task Generation → Data Collection → Model Training pipeline, we ensure every byte of intelligence is verifiable, traceable, and secure. 2026 Strategic Deliverables This year, we are committed to delivering three foundational pillars: - The World's Largest Training Dataset for Robots: A robot training set—diverse, high-quality interaction data at an unprecedented scale. - A Robotics Foundation Model: A universal robotic brain trained on our pure simulation and synthetic data, capable of robust cross-embodiment transfer and open-world adaptability. - Evolvable Robot Hardware: Robots deployed with Axis models that autonomously evolve through continuous interaction, turning every deployment into a self-improving node within our RGI network. The Ultimate Vision We are building more than models; we are architecting the Distributed Machine Economy. A future where every dataset, model, and robotic embodiment is a verifiable asset in a global, autonomous network. Thank you for building the future of intelligence with us✌️📷

Axis Robotics

27,858 Aufrufe • vor 6 Monaten

Robora Sim: A PyBullet-Powered Environment for Learning Robotic Physical Intelligence We are currently building our Robora simulation environment setup for our sim based learning, leveraging PyBullet, an industry-standard physics engine widely used in AI-driven robotics research and development. The environment is optimized with GPU-accelerated learning algorithms, enabling high-speed imitation learning and reinforcement learning within a safe and controlled virtual setup before shipping out to real world. This simulation platform allows our models to learn, adapt, and generalize across different robot morphologies, terrain types and task objectives - all before deployment to the real world. At it's core, the system combines a VLA-powered high-level planner with low-level motion control algorithms, working cohesively to produce emergent, physically intelligent behaviors. This synergy between simulation, learning, and real-world transfer marks a major step forward in our pursuit of adaptive and intelligent robotic systems. Through advanced domain randomization and synthetic data generation, the Robora Simulation Environment ensures that policies trained in simulation transfer effectively to real-world robots, minimizing the sim-to-real gap. Moreover, users will be able to test and integrate their own hardware kits within selected simulation environments in the Robora Dapp, ensuring seamless compatibility and safer real-world implementation.
0:14

Robora Sim: A PyBullet-Powered Environment for Learning Robotic Physical Intelligence We are currently building our Robora simulation environment setup for our sim based learning, leveraging PyBullet, an industry-standard physics engine widely used in AI-driven robotics research and development. The environment is optimized with GPU-accelerated learning algorithms, enabling high-speed imitation learning and reinforcement learning within a safe and controlled virtual setup before shipping out to real world. This simulation platform allows our models to learn, adapt, and generalize across different robot morphologies, terrain types and task objectives - all before deployment to the real world. At it's core, the system combines a VLA-powered high-level planner with low-level motion control algorithms, working cohesively to produce emergent, physically intelligent behaviors. This synergy between simulation, learning, and real-world transfer marks a major step forward in our pursuit of adaptive and intelligent robotic systems. Through advanced domain randomization and synthetic data generation, the Robora Simulation Environment ensures that policies trained in simulation transfer effectively to real-world robots, minimizing the sim-to-real gap. Moreover, users will be able to test and integrate their own hardware kits within selected simulation environments in the Robora Dapp, ensuring seamless compatibility and safer real-world implementation.

Robora

23,489 Aufrufe • vor 9 Monaten

STEVE-1: A Generative Model for Text-to-Behavior in Minecraft paper page: Constructing AI models that respond to text instructions is challenging, especially for sequential decision-making tasks. This work introduces an instruction-tuned Video Pretraining (VPT) model for Minecraft called STEVE-1, demonstrating that the unCLIP approach, utilized in DALL-E 2, is also effective for creating instruction-following sequential decision-making agents. STEVE-1 is trained in two steps: adapting the pretrained VPT model to follow commands in MineCLIP's latent space, then training a prior to predict latent codes from text. This allows us to finetune VPT through self-supervised behavioral cloning and hindsight relabeling, bypassing the need for costly human text annotations. By leveraging pretrained models like VPT and MineCLIP and employing best practices from text-conditioned image generation, STEVE-1 costs just $60 to train and can follow a wide range of short-horizon open-ended text and visual instructions in Minecraft. STEVE-1 sets a new bar for open-ended instruction following in Minecraft with low-level controls (mouse and keyboard) and raw pixel inputs, far outperforming previous baselines. We provide experimental evidence highlighting key factors for downstream performance, including pretraining, classifier-free guidance, and data scaling. All resources, including our model weights, training scripts, and evaluation tools are made available for further research.
0:14

STEVE-1: A Generative Model for Text-to-Behavior in Minecraft paper page: Constructing AI models that respond to text instructions is challenging, especially for sequential decision-making tasks. This work introduces an instruction-tuned Video Pretraining (VPT) model for Minecraft called STEVE-1, demonstrating that the unCLIP approach, utilized in DALL-E 2, is also effective for creating instruction-following sequential decision-making agents. STEVE-1 is trained in two steps: adapting the pretrained VPT model to follow commands in MineCLIP's latent space, then training a prior to predict latent codes from text. This allows us to finetune VPT through self-supervised behavioral cloning and hindsight relabeling, bypassing the need for costly human text annotations. By leveraging pretrained models like VPT and MineCLIP and employing best practices from text-conditioned image generation, STEVE-1 costs just $60 to train and can follow a wide range of short-horizon open-ended text and visual instructions in Minecraft. STEVE-1 sets a new bar for open-ended instruction following in Minecraft with low-level controls (mouse and keyboard) and raw pixel inputs, far outperforming previous baselines. We provide experimental evidence highlighting key factors for downstream performance, including pretraining, classifier-free guidance, and data scaling. All resources, including our model weights, training scripts, and evaluation tools are made available for further research.

AK

144,704 Aufrufe • vor 3 Jahren

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with supercharged compute with physics on GPU. However, the methods use 0-th order gradients, and are often not the best optimizers. Can we do better than PPO in continuous control for robotics? Turns out yes! 🥳 tl;dr: Faster, better RL than PPO in continuous control 💪 The answer lies in using more information from the simulation. We are juicing the simulation on GPU as it is, why not use it for gradients as well? This has been a driving question in a series of our works. We first studied this problem in ICLR 2022 paper on Short Horizon Actor Critic Naive gradient based methods are stuck in local minima and have exploding/vanishing gradients. SHAC solved this problem truncated rollouts and model based value estimation, where the model is Differentiable Sim. This boosted sample efficiency and wall-clock time immensely especially in high dimensional systems such as humanoids Yet, given enough compute PPO often caught up. Our follow up paper on on Adaptive Horizon Actor Critic at ICML 2024 discovers the cause and provides a fix. However, we find that even when given ground-truth dynamics, not all gradients are useful due to sample error. 1st-Order Model-Based Reinforcement Learning methods employing differentiable simulation provide gradients with reduced variance but are susceptible to bias in scenarios involving stiff dynamics, such as physical contact. We find that back-propagating through contact and long trajectories drastically reduces gradient accuracy. Using this insight, we propose AHAC to dynamically adapt its roll-out horizon to avoid differentiating through stiff contact. AHAC is a first-order model-based RL algorithm that learns high-dimensional tasks in minutes (wall clock) and outperforms PPO by 40%, even in the limit of data provided to PPO. This work is led by Ignat Georgiev alongside Krishnan Srinivasan, Jie Xu, Eric Heiden and ample assistance from warp team at NVIDIA Robotics (Miles Macklin)
0:10

Model-Free Reinforcement Learning (MFRL) has been alluring, especially with supercharged compute with physics on GPU. However, the methods use 0-th order gradients, and are often not the best optimizers. Can we do better than PPO in continuous control for robotics? Turns out yes! 🥳 tl;dr: Faster, better RL than PPO in continuous control 💪 The answer lies in using more information from the simulation. We are juicing the simulation on GPU as it is, why not use it for gradients as well? This has been a driving question in a series of our works. We first studied this problem in ICLR 2022 paper on Short Horizon Actor Critic Naive gradient based methods are stuck in local minima and have exploding/vanishing gradients. SHAC solved this problem truncated rollouts and model based value estimation, where the model is Differentiable Sim. This boosted sample efficiency and wall-clock time immensely especially in high dimensional systems such as humanoids Yet, given enough compute PPO often caught up. Our follow up paper on on Adaptive Horizon Actor Critic at ICML 2024 discovers the cause and provides a fix. However, we find that even when given ground-truth dynamics, not all gradients are useful due to sample error. 1st-Order Model-Based Reinforcement Learning methods employing differentiable simulation provide gradients with reduced variance but are susceptible to bias in scenarios involving stiff dynamics, such as physical contact. We find that back-propagating through contact and long trajectories drastically reduces gradient accuracy. Using this insight, we propose AHAC to dynamically adapt its roll-out horizon to avoid differentiating through stiff contact. AHAC is a first-order model-based RL algorithm that learns high-dimensional tasks in minutes (wall clock) and outperforms PPO by 40%, even in the limit of data provided to PPO. This work is led by Ignat Georgiev alongside Krishnan Srinivasan, Jie Xu, Eric Heiden and ample assistance from warp team at NVIDIA Robotics (Miles Macklin)

Animesh Garg

52,300 Aufrufe • vor 2 Jahren

Here's proof that the $Virtuals token is undervalued! We are three months into 2026 and Virtuals Protocol have; ➥ Overhauled the core Virtuals website including an outline of the four major pillars of focus for the year. Agent Commerce Protocol (ACP), Butler, Capital Markets, and Robotics. ➥ Added the Pegasus and Titan launchpads to add to the existing Unicorn launchpad. This now provides a full suite of launch options catering to all types. Arguably the most comprehensive launch suite across crypto! ➥ Listed on Aster 🥷 Perpetuals allowing up to 75x leverage trading on the $Virtual token. ➥ Integrated Bankr to Butler and ACP. ➥ Partnered with XMAQUINA, a major player across Robotics Capital Markets and provided participants with access to the $DEUS pre-sale. One of many robotics partnerships for the year to date! ➥ Launched Virtuals on Base App ➥ Held, supported, and/or sponsored multiple hackathon/ builder meeting type events including; ↠ Physical AI Hackathon in SF ↠ Agentic Commerce Hackathon with the likes of Coinbase Developer Platform🛡️ and Google Cloud ↠ Traders House Consensus Hong Kong week with ACTIV8 ↠ ETH Denver ↠ Base Batches 003: Robotics ↠ Stanford Blockchain Accelerator (Standford Blockchain Accelerator (SBA)) ↠ Base Korea Builders Workshop (Base Korea) ↠ Eth Robotics Club HACK2026 (ETH Robotics Club) ↠ Synthesis Hackathon (synthesis) ➥ Partnered with OpenMind and Fabric Foundation and supported the $ROBO token launch. This matured into the first ever Titan launch on Virtuals with the $ROBO token being the highest launched on the protocol ($400m+). ➥ Launched Butler Pro, an enhanced version of the initial Butler we have come to know and love on the timeline, in the DMs, as well as on the Virtuals ACP site. ➥ Become the standout user of x402, accounting for over 95%+ of usage this year. ➥ Integrated on , the automated onchain finance investment platform. ➥ Supported and contributed to the implementation of the Ethereum Foundation ERC8004 standard. Integrating the standard into ACP and offering an automated integration to the standard for all ACP agents. ➥ Established an easy onboarding for OpenClaw🦞 agents to plug into Virtuals ACP, creating a new flow of agents and builders across the ecosystem. ➥ Launched the 60-days launch mechanic which allows builders to 'experiment' with a crypto token but having an option to exit after 60 days with partial refunds provided to holders. A game-changing launch mechanic not seen before in the space. ➥ Strengthened the relationship with Base and having multiple interactions with jesse.base.eth on the timeline! ➥ Launched the AGDP(dot)io site, creating an incentivised mechanism for agents contributing to the growth of the protocol to really earn. Imagine Amazon for autonomous agents with rewards up to $1m per month! This pushed the total agent-to-agent revenue over $4m USD with over 2m jobs completed. ➥ Collaborated with t54.ai, a business building trust and risk infrastructure for the agentic economy, to strengthen the ACP offering. ➥ Invested over $1m on 30+ humanoid robots as part of the soon to be announced 'Eastworld' Robotics accelerator lab. ➥ Released ERC8183, a universal commerce layer for AI agents, in partnership with the Ethereum Foundations dAI team. A significant offering which has since been integrated via partnerships with; ↠ BNB (BNB Chain) ↠ X Layer (X Layer) ↠ Monad (Monad) ↠ XRP Ledger (RippleX) ↠ World Chain (World Chain) ↠ Celo (Celo) ↠ Moonpay (MoonPay 🟣) ↠ Arbitrum (Arbitrum) ↠ Abstract (Abstract) ↠ Mante (Mantle) ➥ Launched the Virtuals Degen Arena providing up to $100k a week to top agents who compete in trading competitions in the arena. ➥ Launched the Virtuals Console, providing an ultra easy, no-code, way to own an AI agent in seconds. ↛. If you've managed to get to this point, I can't imagine you are anything other than bullish on Virtuals. What really is amazing is that there is MUCH more to come. Imagine where we are in another three months, and three months after that!?
0:10

Here's proof that the $Virtuals token is undervalued! We are three months into 2026 and Virtuals Protocol have; ➥ Overhauled the core Virtuals website including an outline of the four major pillars of focus for the year. Agent Commerce Protocol (ACP), Butler, Capital Markets, and Robotics. ➥ Added the Pegasus and Titan launchpads to add to the existing Unicorn launchpad. This now provides a full suite of launch options catering to all types. Arguably the most comprehensive launch suite across crypto! ➥ Listed on Aster 🥷 Perpetuals allowing up to 75x leverage trading on the $Virtual token. ➥ Integrated Bankr to Butler and ACP. ➥ Partnered with XMAQUINA, a major player across Robotics Capital Markets and provided participants with access to the $DEUS pre-sale. One of many robotics partnerships for the year to date! ➥ Launched Virtuals on Base App ➥ Held, supported, and/or sponsored multiple hackathon/ builder meeting type events including; ↠ Physical AI Hackathon in SF ↠ Agentic Commerce Hackathon with the likes of Coinbase Developer Platform🛡️ and Google Cloud ↠ Traders House Consensus Hong Kong week with ACTIV8 ↠ ETH Denver ↠ Base Batches 003: Robotics ↠ Stanford Blockchain Accelerator (Standford Blockchain Accelerator (SBA)) ↠ Base Korea Builders Workshop (Base Korea) ↠ Eth Robotics Club HACK2026 (ETH Robotics Club) ↠ Synthesis Hackathon (synthesis) ➥ Partnered with OpenMind and Fabric Foundation and supported the $ROBO token launch. This matured into the first ever Titan launch on Virtuals with the $ROBO token being the highest launched on the protocol ($400m+). ➥ Launched Butler Pro, an enhanced version of the initial Butler we have come to know and love on the timeline, in the DMs, as well as on the Virtuals ACP site. ➥ Become the standout user of x402, accounting for over 95%+ of usage this year. ➥ Integrated on , the automated onchain finance investment platform. ➥ Supported and contributed to the implementation of the Ethereum Foundation ERC8004 standard. Integrating the standard into ACP and offering an automated integration to the standard for all ACP agents. ➥ Established an easy onboarding for OpenClaw🦞 agents to plug into Virtuals ACP, creating a new flow of agents and builders across the ecosystem. ➥ Launched the 60-days launch mechanic which allows builders to 'experiment' with a crypto token but having an option to exit after 60 days with partial refunds provided to holders. A game-changing launch mechanic not seen before in the space. ➥ Strengthened the relationship with Base and having multiple interactions with jesse.base.eth on the timeline! ➥ Launched the AGDP(dot)io site, creating an incentivised mechanism for agents contributing to the growth of the protocol to really earn. Imagine Amazon for autonomous agents with rewards up to $1m per month! This pushed the total agent-to-agent revenue over $4m USD with over 2m jobs completed. ➥ Collaborated with t54.ai, a business building trust and risk infrastructure for the agentic economy, to strengthen the ACP offering. ➥ Invested over $1m on 30+ humanoid robots as part of the soon to be announced 'Eastworld' Robotics accelerator lab. ➥ Released ERC8183, a universal commerce layer for AI agents, in partnership with the Ethereum Foundations dAI team. A significant offering which has since been integrated via partnerships with; ↠ BNB (BNB Chain) ↠ X Layer (X Layer) ↠ Monad (Monad) ↠ XRP Ledger (RippleX) ↠ World Chain (World Chain) ↠ Celo (Celo) ↠ Moonpay (MoonPay 🟣) ↠ Arbitrum (Arbitrum) ↠ Abstract (Abstract) ↠ Mante (Mantle) ➥ Launched the Virtuals Degen Arena providing up to $100k a week to top agents who compete in trading competitions in the arena. ➥ Launched the Virtuals Console, providing an ultra easy, no-code, way to own an AI agent in seconds. ↛. If you've managed to get to this point, I can't imagine you are anything other than bullish on Virtuals. What really is amazing is that there is MUCH more to come. Imagine where we are in another three months, and three months after that!?

bigwil

1,658,312 Aufrufe • vor 3 Monaten

You can't 3D reconstruct glass from images... ...WRONG! Thanks for video diffusion, now just about anything is possible! Introducing...Diffusion Knows Transparency (DKT) Transparent and reflective objects usually break robot vision and photogrammetry pipelines because they don't follow the "solid object" rules standard cameras expect. DKT is a new AI model that repurposes the "internal physics engine" found in video generation models to solve this problem. Researchers took a massive video diffusion model (WAN) and fine-tuned it using a custom-built synthetic dataset to turn it into a high-precision depth sensor. To train the AI, they built the first massive synthetic video library of transparent objects, 1.32 million frames of perfectly labeled glass and metal objects in motion. Without ever seeing a "real" labeled video of glass during training, the model (DKT) outperformed all previous specialized systems on real-world benchmarks (ClearPose, DREDS). They created a "lightweight" 1.3B parameter version that runs fast enough (0.17s per frame) to be used on actual robot hardware. Two reasons I find this project important: 1. It further proves that synthetic data will be essential for training the next generation vision models. 2. In real-world robotic tests, using DKT's depth maps nearly doubled the success rate of robot arms trying to pick up objects on tricky reflective or translucent surfaces. At home robots will need to interact with these types of objects on a daily basis. Check out the project page here: Code is LIVE! #Computervision #Robotics #AI
0:14

You can't 3D reconstruct glass from images... ...WRONG! Thanks for video diffusion, now just about anything is possible! Introducing...Diffusion Knows Transparency (DKT) Transparent and reflective objects usually break robot vision and photogrammetry pipelines because they don't follow the "solid object" rules standard cameras expect. DKT is a new AI model that repurposes the "internal physics engine" found in video generation models to solve this problem. Researchers took a massive video diffusion model (WAN) and fine-tuned it using a custom-built synthetic dataset to turn it into a high-precision depth sensor. To train the AI, they built the first massive synthetic video library of transparent objects, 1.32 million frames of perfectly labeled glass and metal objects in motion. Without ever seeing a "real" labeled video of glass during training, the model (DKT) outperformed all previous specialized systems on real-world benchmarks (ClearPose, DREDS). They created a "lightweight" 1.3B parameter version that runs fast enough (0.17s per frame) to be used on actual robot hardware. Two reasons I find this project important: 1. It further proves that synthetic data will be essential for training the next generation vision models. 2. In real-world robotic tests, using DKT's depth maps nearly doubled the success rate of robot arms trying to pick up objects on tricky reflective or translucent surfaces. At home robots will need to interact with these types of objects on a daily basis. Check out the project page here: Code is LIVE! #Computervision #Robotics #AI

Jonathan Stephens

17,712 Aufrufe • vor 6 Monaten