Загрузка видео...

Не удалось загрузить видео

На главную

Introducing DocJev - a lightning-fast OSS library for document classification and splitting with jev ⚡️ Give a document alongside some natural language category rules. Jev will predict the document category (classify) or the boundaries between sub-documents (split). It is 6x faster than gpt-5.6-luna with equivalent accuracy. You can choose...

208,982 просмотров • 4 дней назад •via X (Twitter)

Комментарии: 32

Фото профиля Jerry Liu
Jerry Liu4 дней назад

Latency benchmarks:

Фото профиля Jerry Liu
Jerry Liu4 дней назад

You can find the full interactive report here:

Фото профиля AI Mastery Guide
AI Mastery Guide4 дней назад

Splitting docs that fast is useful

Фото профиля Greg Mushen
Greg Mushen4 дней назад

Nice. Would be very cool to have model routing based on PDF contents, e.g. use Marker if there are tables, etc.

Фото профиля Imaan Sultan 🇵🇰🇸🇦
Imaan Sultan 🇵🇰🇸🇦4 дней назад

need to try this 👀

Фото профиля First Sauce Labs
First Sauce Labs4 дней назад

The "natural language category rules" is the part that will get interesting fast. Looking forward to trying it out.

Фото профиля Likhith Bhargav
Likhith Bhargav4 дней назад

Natural-language rules as the interface are the right abstraction - humans own the taxonomy, Jev owns the cheap decision.

Фото профиля kettybluce
kettybluce3 дней назад

Nice example of keeping the interface simple while exposing both classification and boundary detection. The split output should make it easier to audit where a document needs review, especially when the classifier is uncertain.

Фото профиля Jatin Garg
Jatin Garg4 дней назад

Natural language category rules removes the whole annotation/fine-tuning step for document classification.

Фото профиля Tanisha Pritha
Tanisha Pritha4 дней назад

Just wired LlamaParse into a procurement RAG ingest last week. Tables + scanned PDFs were the whole problem. Caching the parsed doc and then classifying/splitting fast is exactly the missing step. Trying DocJev on the next batch.

Фото профиля TensorQuay
TensorQuay4 дней назад

Could you benchmark the scanned PDFs in the synthetic set next? The report's ~6x figures exclude OCR. Timing the full pipeline would show how much of that speedup reaches someone uploading a scan.

Фото профиля Jay Zhou
Jay Zhou4 дней назад

a bad split never throws an error. it just quietly changes every answer downstream and the citation still looks fine.

Фото профиля Naveen 🚀
Naveen 🚀4 дней назад

This is great

Фото профиля anytan
anytan4 дней назад

DocJev 6x on doc split. that's the hit

Фото профиля Anurag
Anurag4 дней назад

The split is the decision that matters, the classification is just filing. I've watched document pipelines where a perfect classifier sat on top of bad splits and the RAG answers were still garbage, because one wrong boundary means two contracts live in the same chunk. Get the boundaries right and the labels almost don't matter.

Фото профиля Brick | AI Builder Signal
Brick | AI Builder Signal3 дней назад

Important qualifier: the repo's accuracy pilot is a 40-PDF, 8-packet convenience sample with shared source/template families and uncontrolled provider caching. The reproducible report does not establish general split accuracy. Keep boundary review before unattended use.

Фото профиля manish · building AgentWorks
manish · building AgentWorks3 дней назад

Document classification and splitting, nice launch

Фото профиля Christopher Dean
Christopher Dean4 дней назад

ocr tradeoffs are the whole game — latency vs accuracy is what makes doc pipelines ship or stall.

Фото профиля AI议会
AI议会3 дней назад

文档分类/切分做成开源原语,比再堆一个「全能 RAG」更落地。DocJev 这种薄库才容易嵌进现有流水线。

Фото профиля Clara Benson
Clara Benson4 дней назад

Will the LlamaParse preprocessing latency bottleneck break real-time API limits if downstream tasks never actually reuse those cached representations?

Фото профиля Jamik
Jamik3 дней назад

the subdocument boundary bit is super useful, thats usually where doc classifiers get weird

Фото профиля Kizuno18
Kizuno184 дней назад

fast splitting + classification is what unlocks unmonitored pipelines. tested an autonomous pipeline pulling macro data for brazil from outside: using fast models for schema boundaries instead of giant context windows cut latency 85% and completely eliminated 3am timeouts.

Фото профиля Klasta
Klasta4 дней назад

splitting docs from english category rules, no classifier zoo

Фото профиля Nines
Nines4 дней назад

The split between classification and boundary detection is a nice touch. OCR pipelines often get treated as one problem, but the backend choice matters once documents stop being clean scans.

Фото профиля Richard Tang
Richard Tang4 дней назад

huge!

Фото профиля Brjan | AI Builder
Brjan | AI Builder4 дней назад

how does DocJev handle edge cases where documents don't fit typical rules?

Фото профиля Stanislav Sorokin
Stanislav Sorokin3 дней назад

Similar picture on papers instead of documents: 1,000 arXiv abstracts, 24 categories, 57 ms each. With Opus 5 as judge Jev matched 85 of 100 at 153x lower cost per paper. Confidence under 0.90 flagged 14 of the 15 misses.

Фото профиля Josh R Barry
Josh R Barry4 дней назад

Turning the annotation step into a rules interface is a very good trade for this kind of workflow.

Фото профиля Elias Lumer
Elias Lumer4 дней назад

Is jev on your leaderboard? Curious on your benchmark accuracy vs latency for Jev vs others

Фото профиля Miano
Miano4 дней назад

This is a strong abstraction. Letting the user describe boundaries in natural language removes a lot of brittle preprocessing, especially when document formats keep changing.

Фото профиля Mitansh
Mitansh4 дней назад

docjev for classify + split with natural language rules is neat 6x faster is a serious flex on doc pipelines

Фото профиля Steven
Steven3 дней назад

I tested DocJEV with both a local model and the official JEV model — works equally great! 🔥

Похожие видео

The latest RAG trend for the current agent harnesses (Codex, Cowork) is to do two passes of document processing to solve a knowledge work task over a data room of documents: 1️⃣ A fast and light pass, oftentimes using a free/OSS doc parsing tool. This can be cheaply run across 10-100-1k’s of files, and enables the agent to then do retrieval (e.g. grep, semantic) to find relevant subsets of context. 2️⃣ A “just-in-time” VLM-based pass. Once the agent finds the relevant pages of context, it will screenshot the documents can call its own VLM (or write code) to dissect the pages. The issue with only using VLM-based OCR tools over massive ad-hoc customer file dumps is that it’s slow and expensive. Doing JIT VLM OCR allows the agent to filter through the data cheaply, but still preserve accuracy for the context that’s needed for the task. The agent harnesses do two-pass document processing by default using off the shelf-tools: pdf2text as the first pass, and using itself (Opus 5) as the second pass. See the below video where Cowork runs over a bunch of PDFs to answer a question about a benchmark graph in the Kimi k3 paper. The main issues here with the “out of the box” doc processing these agents offer are: * Opus 5 is not the best VLM for OCR. It is also way too expensive at scale and lacks grounding * The OSS tools like pypdf, pdf2text, may not be versatile enough as the first pass. * The agent will write a lot of throwaway code to rewrite things an OCR tool would’ve provided out of the box, like chart processing, bounding boxes, confidence scores, leading to increased cost and speed. We have all the tools within LlamaIndex 🦙 to help any agent do two-pass document processing with higher accuracy and lower cost. 1️⃣ We have liteparse for the first pass - a free/OSS parser written in Rust that’s faster/more accurate than other OSS parsers, and supports 50+ document types 2️⃣ We have LlamaParse for the second pass - an agentic document engine that uses VLMs+harnesses to achieve SOTA in accuracy and cost across various doc parsing and extraction tasks. It can be called from any agent harness as an MCP or skill. It takes in page numbers as input, so that the agent can choose to run LlamaParse over a subset of the doc instead of the full doc as a “zoom-in” pass. Come check it out! LiteParse: LlamaParse: All the relevant docs, including MCP, are here:

Jerry Liu

22,763 просмотров • 1 месяц назад

We’re open sourcing the first document OCR benchmark for the agentic era, ParseBench. Document parsing is the foundation of every AI agent that works with real-world files. ParseBench is a benchmark that measures parsing quality specifically for agent knowledge work: ✅ It optimizes for semantic correctness (instead of exact similarity) ✅ It has the most comprehensive distribution of real-world enterprise documents It contains ~2,000 human-verified enterprise document pages with 167,000+ test rules across five dimensions that matter most: tables, charts, content faithfulness, semantic formatting, and visual grounding. We benchmarked 14 known document parsers on ParseBench, from frontier/OSS VLMs to specialized parsers to LlamaParse. Here are some of our findings: 💡 Increasing compute budget yields diminishing returns - Gemini/gpt-5-mini/haiku gain 3-5 points from minimal to high thinking, at 4x the cost. 💡 Charts are the most polarizing dimension for evaluation. Most specialized parsers score below 6%, while some VLM-based parsers do a bit better. 💡 VLMs are great at visual understanding but terrible at layout extraction. GPT-5-mini/haiku score below 10% on our visual grounding task, all specialized parsers do much better. 💡 No method crushes all 5 dimensions at once, but LlamaParse achieves the highest overall score at 84.9%, and is the leader in 4 out of the 5 dimensions. This is by far the deepest technical work that we’ve published as a company. I would encourage you to start with our blog and explore our links to Hugging Face to GitHub. All the details are in our full 35-page (!!) ArXiv whitepaper. 🌐: Blog: 📄 Paper: 💻 Code: 📊 Dataset: 🎥 YouTube:

Jerry Liu

108,093 просмотров • 5 месяцев назад

Introducing ExtractBench, the most comprehensive benchmark for information extraction from complex enterprise documents. The latest models are pushing the frontier of coding and knowledge work, but surprisingly they still struggle on complex doc extraction tasks in production. A well-tuned extractor must parse multi-page filings without dropping rows, emit exact spatial citations for auditability, and handle messy scans. Also they must do all of this at a viable per-page cost so that you can scale this to millions of docs in production (you can’t be paying upwards of $1 in tokens per page!) Existing extraction benchmarks fall short: they are not large/diverse enough in document domain (finance, energy, gov, auto), elements (long records, scans, grounding), and schemas. So our applied research team built ExtractBench. We evaluated 14 systems: frontier VLMs, coding agents, and specialized extraction APIs, against 370 enterprise documents: 4,869 pages, 67 document types. Our biggest finding 🧪: Short documents mask critical system flaws. On files past 50 pages, commercial VLMs collapse below 35% recall due to silent list truncation. They hold high precision, but lose output attention and drop most of the table rows. ExtractBench evaluates value accuracy, long-record completeness, spatial grounding, and per-page cost with zero LLM judges. It is 100% deterministic and reproducible. In tandem with ExtractBench, we’re also introducing 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗣𝗹𝘂𝘀, a new Extract tier in LlamaParse that debuts at #1 on the leaderboard: 95.6% value accuracy, at less than a third the cost of the closest peer. Explore the findings, download the dataset, or run the harness: Blog: GitHub: HuggingFace: We will be actively evolving both our extraction benchmark as well as our extraction harness over time. If you check out either ExtractBench or LlamaParse, let us know your feedback!

Jerry Liu

75,502 просмотров • 1 месяц назад