正在加载视频...

视频加载失败

加载此视频时出现问题。这可能是由于临时网络问题，或视频可能不可用。

We just released 3 million samples of high quality vision language model training dataset for use cases such as: 📄 optical character recognition (OCR) 📊 visual question answering (VQA) 📝 captioning 🤗 Learn more: 📥 Download:

NVIDIA AI Developer

111,550 subscribers

95,858 次观看 • 1 年前 •via X (Twitter)

科学技术教育

Anya Rossi• Live Now

Private livecam show

23 条评论

Energy Pulse 的头像

Energy Pulse1 年前

This is gold for anyone building multimodal models, Bravo NVIDIA!👏

Furkan Gözükara 的头像

Furkan Gözükara1 年前

Just stop selling low vram from ridiculous prices I paid 32 gb vram 5090, 4000 usd

Vaibhav (VB) Srivastav 的头像

Vaibhav (VB) Srivastav1 年前

brilliant release!

PageIndex 的头像

PageIndex1 年前

Most OCR tools only extract each page independently, losing the broader document context and hierarchy. PageIndex OCR is the first long-context OCR model that preserves the document’s global hierarchical structure—outperforming Mistral OCR.

Cyril Zakka, MD 的头像

Cyril Zakka, MD1 年前

Amazing!

Quentin Lhoest 🤗 的头像

Quentin Lhoest 🤗1 年前

So cool ! It would be amazing to improve the structure to show the images in the Dataset Viewer, happy to help :)

Bo 的头像

Bo1 年前

cc @michael_g_u please fwd to G & A 😅

Alex Veremeyenko 的头像

Alex Veremeyenko1 年前

this dataset drop is a game changer for multimodal projects, excited to see what the community builds with it

Breakpoint 的头像

Breakpoint1 年前

🔥🔥🔥

Nathan 的头像

Nathan1 年前

nice !

無限 💀 acc/acc 的头像

無限 💀 acc/acc1 年前

🤤 Data.

Aero - AI & Crypto 的头像

Aero - AI & Crypto1 年前

Data will fuel open-source. @Alibaba_Qwen your turn. @AIatMeta @Zai_org

Hardy Chen 🏃 COLM2026 的头像

Hardy Chen 🏃 COLM20261 年前

Thanks for the great sharing! But it seems that the ocr annotations and images are not paired. Did I miss anything?

Ammar Fahmy 的头像

Ammar Fahmy1 年前

@huggingface Awsm 👌

jr 的头像

jr1 年前

thank you for your work!! Do all the images uploaded to HF yet? I see some of the folders missing.

Robert Youssef 的头像

Robert Youssef1 年前

that's a serious dataset drop. gonna make some waves in multimodal projects. nice to see real value for the community

ZIAISTAN 的头像

ZIAISTAN1 年前

Excellent can it be used to train from scratch?

Numan 的头像

Numan1 年前

@huggingface wowowow

Iqra Saifi 的头像

Iqra Saifi1 年前

Wowww

seekgpt 的头像

seekgpt1 年前

Great 👍

Caleb 的头像

Caleb1 年前

Amazing! 🤩

Himanshu Kumar 的头像

Himanshu Kumar1 年前

Large datasets are key for robust AI. This opens exciting new avenues for computer vision research. What innovative applications will this unlock?

Moon 🎑 的头像

Moon 🎑1 年前

Really helpful for vision based model architecture.

相关视频

Introducing Meta Perception Language Model (PLM): an open & reproducible vision-language model tackling challenging visual tasks. Learn more about how PLM can help the open source community build more capable computer vision systems. Read the research paper, and download the code and dataset:

Introducing Meta Perception Language Model (PLM): an open & reproducible vision-language model tackling challenging visual tasks. Learn more about how PLM can help the open source community build more capable computer vision systems. Read the research paper, and download the code and dataset:

AI at Meta

94,534 次观看 • 1 年前

From recognizing text to understanding documents. 📄 Meet Xiaomi-OCR-0, an open-source document understanding model with just 0.8B parameters. It brings document parsing, key information extraction, and OCR-oriented visual question answering into one lightweight model. It scores 95.24 on Real5-OmniDocBench and 83.2 on average across five OCR-oriented VQA benchmarks. On OmniDocBench v1.6, Xiaomi-OCR-0 ranks first in document parsing performance among models with fewer than 1B parameters. Model weights, inference code, an interactive demo, MCP service, and Agent Skill are now open source. 👇 🤗 Huggingface: 🔗 GitHub: 📄 Demo: #XiaomiAI #OCR #OpenSource

From recognizing text to understanding documents. 📄 Meet Xiaomi-OCR-0, an open-source document understanding model with just 0.8B parameters. It brings document parsing, key information extraction, and OCR-oriented visual question answering into one lightweight model. It scores 95.24 on Real5-OmniDocBench and 83.2 on average across five OCR-oriented VQA benchmarks. On OmniDocBench v1.6, Xiaomi-OCR-0 ranks first in document parsing performance among models with fewer than 1B parameters. Model weights, inference code, an interactive demo, MCP service, and Agent Skill are now open source. 👇 🤗 Huggingface: 🔗 GitHub: 📄 Demo: #XiaomiAI #OCR #OpenSource

Xiaomi Tech

154,367 次观看 • 14 天前

Florence-2, the new vision foundation model by Microsoft, can now run 100% locally in your browser on WebGPU, thanks to Transformers.js! 🤗🤯 It supports tasks like image captioning, optical character recognition, object detection, and many more! 😍 WOW! Demo (+ source code) 👇

Florence-2, the new vision foundation model by Microsoft, can now run 100% locally in your browser on WebGPU, thanks to Transformers.js! 🤗🤯 It supports tasks like image captioning, optical character recognition, object detection, and many more! 😍 WOW! Demo (+ source code) 👇

Xenova

88,782 次观看 • 2 年前

NVIDIA drop a 3B vision-language model for fast, high-quality visual grounding in real time accurately. - Parallel box decoding - 10× faster than Qwen3-VL - Trained on 138M queries/785M boxes - GUI, OCR, and document layout, dense detection - Open source Useful for computer-use agents and Physical AI,

NVIDIA drop a 3B vision-language model for fast, high-quality visual grounding in real time accurately. - Parallel box decoding - 10× faster than Qwen3-VL - Trained on 138M queries/785M boxes - GUI, OCR, and document layout, dense detection - Open source Useful for computer-use agents and Physical AI,

Md Ismail Šojal 🕷️

134,048 次观看 • 1 个月前

📣 Microsoft Research releases Florence-VL, a new family of MLLMs powered by the generative vision foundation model Florence-2. Achieves significant improvements in general VQA, perception, hallucination, OCR, Chart, knowledge-intensive understanding, and more🔥Learn more👇

📣 Microsoft Research releases Florence-VL, a new family of MLLMs powered by the generative vision foundation model Florence-2. Achieves significant improvements in general VQA, perception, hallucination, OCR, Chart, knowledge-intensive understanding, and more🔥Learn more👇

Gradio

14,454 次观看 • 1 年前

Meta FAIR recently released the Seamless Interaction Dataset, the largest known high-quality video dataset of its kind, with: 4,000+ diverse participants 4,000+ hours of footage 65k+ interactions 5,000+ annotated samples This dataset of full-body, in-person, face-to-face interaction videos represents a crucial stepping stone to understanding and modeling how people communicate and behave when they’re together—advancing AI's ability to generate more natural conversations and human-like gestures. Download the dataset on Hugging Face: Learn more about the dataset:

Meta FAIR recently released the Seamless Interaction Dataset, the largest known high-quality video dataset of its kind, with: 4,000+ diverse participants 4,000+ hours of footage 65k+ interactions 5,000+ annotated samples This dataset of full-body, in-person, face-to-face interaction videos represents a crucial stepping stone to understanding and modeling how people communicate and behave when they’re together—advancing AI's ability to generate more natural conversations and human-like gestures. Download the dataset on Hugging Face: Learn more about the dataset:

AI at Meta

24,133 次观看 • 1 年前

LP-MusicCaps: LLM-Based Pseudo Music Captioning paper page: Automatic music captioning, which generates natural language descriptions for given music tracks, holds significant potential for enhancing the understanding and organization of large volumes of musical data. Despite its importance, researchers face challenges due to the costly and time-consuming collection process of existing music-language datasets, which are limited in size. To address this data scarcity issue, we propose the use of large language models (LLMs) to artificially generate the description sentences from large-scale tag datasets. This results in approximately 2.2M captions paired with 0.5M audio clips. We term it Large Language Model based Pseudo music caption dataset, shortly, LP-MusicCaps. We conduct a systemic evaluation of the large-scale music captioning dataset with various quantitative evaluation metrics used in the field of natural language processing as well as human evaluation. In addition, we trained a transformer-based music captioning model with the dataset and evaluated it under zero-shot and transfer-learning settings. The results demonstrate that our proposed approach outperforms the supervised baseline model.

LP-MusicCaps: LLM-Based Pseudo Music Captioning paper page: Automatic music captioning, which generates natural language descriptions for given music tracks, holds significant potential for enhancing the understanding and organization of large volumes of musical data. Despite its importance, researchers face challenges due to the costly and time-consuming collection process of existing music-language datasets, which are limited in size. To address this data scarcity issue, we propose the use of large language models (LLMs) to artificially generate the description sentences from large-scale tag datasets. This results in approximately 2.2M captions paired with 0.5M audio clips. We term it Large Language Model based Pseudo music caption dataset, shortly, LP-MusicCaps. We conduct a systemic evaluation of the large-scale music captioning dataset with various quantitative evaluation metrics used in the field of natural language processing as well as human evaluation. In addition, we trained a transformer-based music captioning model with the dataset and evaluated it under zero-shot and transfer-learning settings. The results demonstrate that our proposed approach outperforms the supervised baseline model.

AK

78,794 次观看 • 3 年前

Blender Studio released Project Storm, a high-quality, animation-ready character rig designed for Blender 5.0. Learn how it was made and download here:

Blender Studio released Project Storm, a high-quality, animation-ready character rig designed for Blender 5.0. Learn how it was made and download here:

80 LEVEL

50,593 次观看 • 11 个月前

🚨 Just released: MIRIAD, a million-scale medical QA dataset to ground LLMs in reliable medical knowledge. 5.8M question-answer pairs, each distilled from peer-reviewed literature! 🔥 That's structured, high-quality data built for medical AI. 🧵 ↓

🚨 Just released: MIRIAD, a million-scale medical QA dataset to ground LLMs in reliable medical knowledge. 5.8M question-answer pairs, each distilled from peer-reviewed literature! 🔥 That's structured, high-quality data built for medical AI. 🧵 ↓

Charly Wargnier

48,127 次观看 • 1 年前

In case you missed it, we recently launched "Post-training of LLMs," a short course where you'll: ✅ Understand when and why to use post-training methods like Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning. ✅ Learn the concepts underlying the three post-training methods of SFT, DPO, and Online RL, their common use-cases, and how to curate high-quality data to effectively train a model using each method. ✅ Download a pre-trained model and implement post-training pipelines to turn a base model into an instruct model, change the identity of a chat assistant, and improve a model’s math capabilities. Learn more and enroll for free:

In case you missed it, we recently launched "Post-training of LLMs," a short course where you'll: ✅ Understand when and why to use post-training methods like Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Online Reinforcement Learning. ✅ Learn the concepts underlying the three post-training methods of SFT, DPO, and Online RL, their common use-cases, and how to curate high-quality data to effectively train a model using each method. ✅ Download a pre-trained model and implement post-training pipelines to turn a base model into an instruct model, change the identity of a chat assistant, and improve a model’s math capabilities. Learn more and enroll for free:

DeepLearning.AI

16,779 次观看 • 1 年前

Today we presented our work on Action Recognition with 3D pose and tracking at #CVPR. We also released our code, models, and dataset of 1.5 million human trajectories. Code: Project Page:

Today we presented our work on Action Recognition with 3D pose and tracking at #CVPR. We also released our code, models, and dataset of 1.5 million human trajectories. Code: Project Page:

Jathushan Rajasegaran

132,016 次观看 • 3 年前

Very excited to introduce Paperclip Datasets! Curate your own datasets from millions of papers in minutes. Just describe the dataset you want to build! As example, we release the largest dataset of 20k half-life human measurements curated from 2400+ drugs💊 🖇️ Try it: 📝 Blog: 📊 Half-life dataset:

Very excited to introduce Paperclip Datasets! Curate your own datasets from millions of papers in minutes. Just describe the dataset you want to build! As example, we release the largest dataset of 20k half-life human measurements curated from 2400+ drugs💊 🖇️ Try it: 📝 Blog: 📊 Half-life dataset:

James Zou

71,480 次观看 • 9 天前

Bengaluru-based startup Sarvam AI is drawing global attention with its latest AI models, Sarvam Vision and Bulbul V3. The company claims its technology has outperformed global giants like Google Gemini and ChatGPT in key optical character recognition (OCR) benchmarks. If verified, the breakthrough could mark a major moment for India’s growing AI ecosystem. What makes Sarvam Vision different? And how does Bulbul V3 push the boundaries of OCR performance? #SarvamAI

Bengaluru-based startup Sarvam AI is drawing global attention with its latest AI models, Sarvam Vision and Bulbul V3. The company claims its technology has outperformed global giants like Google Gemini and ChatGPT in key optical character recognition (OCR) benchmarks. If verified, the breakthrough could mark a major moment for India’s growing AI ecosystem. What makes Sarvam Vision different? And how does Bulbul V3 push the boundaries of OCR performance? #SarvamAI

NDTV

51,372 次观看 • 9 个月前

Introducing Objaverse-XL, an open dataset of over 10 million 3D objects! With it, we train Zero123-XL, a foundation model for 3D, observing incredible 3D generalization abilities: 🧵👇 📝 Paper:

Introducing Objaverse-XL, an open dataset of over 10 million 3D objects! With it, we train Zero123-XL, a foundation model for 3D, observing incredible 3D generalization abilities: 🧵👇 📝 Paper:

Matt Deitke

368,934 次观看 • 3 年前

Today we’re introducing Scribe v2: the most accurate transcription model ever released. While Scribe v2 Realtime is optimized for ultra low latency and agents use cases, Scribe v2 is built for batch transcription, subtitling, and captioning at scale.

Today we’re introducing Scribe v2: the most accurate transcription model ever released. While Scribe v2 Realtime is optimized for ultra low latency and agents use cases, Scribe v2 is built for batch transcription, subtitling, and captioning at scale.

ElevenLabs

558,355 次观看 • 10 个月前

🚀 Meta FAIR is releasing several new research artifacts on our road to advanced machine intelligence (AMI). These latest advancements are transforming our understanding of perception. 1️⃣ Meta Perception Encoder: A large-scale vision encoder that excels across several image & video tasks. 2️⃣ Meta Perception Language Model: A fully open & reproducible vision-language model designed to tackle visual recognition tasks. 3️⃣ Meta Locate 3D: An end-to-end model for accurate object localization in 3D environments. 4️⃣ Releasing model weights for our 8B-parameter Dynamic Byte Latent Transformer, an alternative to traditional tokenization methods with the potential to redefine the standards for language model efficiency and reliability. 5️⃣Collaborative Reasoner: A framework for evaluating & improving collaborative reasoning skills in language models. Download the code, datasets, and research papers and learn more about how these artifacts are paving the way for more efficient and accurate AI systems.➡️

🚀 Meta FAIR is releasing several new research artifacts on our road to advanced machine intelligence (AMI). These latest advancements are transforming our understanding of perception. 1️⃣ Meta Perception Encoder: A large-scale vision encoder that excels across several image & video tasks. 2️⃣ Meta Perception Language Model: A fully open & reproducible vision-language model designed to tackle visual recognition tasks. 3️⃣ Meta Locate 3D: An end-to-end model for accurate object localization in 3D environments. 4️⃣ Releasing model weights for our 8B-parameter Dynamic Byte Latent Transformer, an alternative to traditional tokenization methods with the potential to redefine the standards for language model efficiency and reliability. 5️⃣Collaborative Reasoner: A framework for evaluating & improving collaborative reasoning skills in language models. Download the code, datasets, and research papers and learn more about how these artifacts are paving the way for more efficient and accurate AI systems.➡️

AI at Meta

163,663 次观看 • 1 年前

Let's talk parsing charts 📊📈. Last week we released ParseBench, the first document OCR benchmark for AI agents. New in ParseBench: ChartDataPointMatch. Most document look at a chart and OCR the caption. Agents need the actual numbers. That's the gap between "OCR'd the text around the chart" and "actually read the chart." More about ParseBench, the GitHub code, Hugging Face dataset, and scientific paper→

Let's talk parsing charts 📊📈. Last week we released ParseBench, the first document OCR benchmark for AI agents. New in ParseBench: ChartDataPointMatch. Most document look at a chart and OCR the caption. Agents need the actual numbers. That's the gap between "OCR'd the text around the chart" and "actually read the chart." More about ParseBench, the GitHub code, Hugging Face dataset, and scientific paper→

LlamaIndex 🦙

13,987 次观看 • 6 个月前

What happens when we train the largest vision-language model and add in robot experiences? The result is PaLM-E 🌴🤖, a 562-billion parameter, general-purpose, embodied visual-language generalist - across robotics, vision, and language. Website:

What happens when we train the largest vision-language model and add in robot experiences? The result is PaLM-E 🌴🤖, a 562-billion parameter, general-purpose, embodied visual-language generalist - across robotics, vision, and language. Website:

Danny Driess

1,272,956 次观看 • 3 年前

3D-LLM: Injecting the 3D World into Large Language Models paper page: Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on. In this work, we propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on. Using three types of prompting mechanisms that we design, we are able to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, we first utilize a 3D feature extractor that obtains 3D features from rendered multi- view images. Then, we use 2D VLMs as our backbones to train our 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA show that our model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs. Qualitative examples also show that our model could perform more tasks beyond the scope of existing LLMs and VLMs.

3D-LLM: Injecting the 3D World into Large Language Models paper page: Large language models (LLMs) and Vision-Language Models (VLMs) have been proven to excel at multiple tasks, such as commonsense reasoning. Powerful as these models can be, they are not grounded in the 3D physical world, which involves richer concepts such as spatial relationships, affordances, physics, layout, and so on. In this work, we propose to inject the 3D world into large language models and introduce a whole new family of 3D-LLMs. Specifically, 3D-LLMs can take 3D point clouds and their features as input and perform a diverse set of 3D-related tasks, including captioning, dense captioning, 3D question answering, task decomposition, 3D grounding, 3D-assisted dialog, navigation, and so on. Using three types of prompting mechanisms that we design, we are able to collect over 300k 3D-language data covering these tasks. To efficiently train 3D-LLMs, we first utilize a 3D feature extractor that obtains 3D features from rendered multi- view images. Then, we use 2D VLMs as our backbones to train our 3D-LLMs. By introducing a 3D localization mechanism, 3D-LLMs can better capture 3D spatial information. Experiments on ScanQA show that our model outperforms state-of-the-art baselines by a large margin (e.g., the BLEU-1 score surpasses state-of-the-art score by 9%). Furthermore, experiments on our held-in datasets for 3D captioning, task composition, and 3D-assisted dialogue show that our model outperforms 2D VLMs. Qualitative examples also show that our model could perform more tasks beyond the scope of existing LLMs and VLMs.

AK

250,159 次观看 • 3 年前