PDF parsing is still painful because LLMs reorder text in complex layouts, break tables across pages, and fail on graphs or images. 💡Testing the new open-source OCRFlux model, and here the results are really good for a change. So OCRFlux is a multimodal, LLM based toolkit for converting PDFs... and images into clean, readable, plain Markdown text. Because the underlying VLM is only 3B param, it runs even on a 3090 GPU. The model is available on Hugging Face . The engine that powers the OCRFlux, teaches the model to rebuild every page and then stitch fragments across pages into one clean Markdown file. It bundles one vision language model with 3B parameters that was fine-tuned from Qwen 2.5-VL-3B-Instruct for both page parsing and cross-page merging. OCRFlux reads raw page images and, guided by task prompts, outputs Markdown for each page and merges split elements across pages. The evaluation shows Edit Distance Similarity (EDS) 0.967 and cross‑page table Tree Edit Distance 0.950, so the parser is both accurate and layout aware. How it works while parsing each page - Convert into text with a natural reading order, even in the presence of multi-column layouts, figures, and insets - Support for complicated tables and equations - Automatically removes headers and footers Cross-page table/paragraph merging - Cross-page table merging - Cross-page paragraph merging A compact vision‑language models can beat bigger models once cross‑page context is added. 🧵 1/n Read on 👇show more

🧵 2/n 📄 The problem space Most open tools lose structure on pages that mix text blocks, figures and multi‑column tables. They also ignore the fact that a PDF page boundary can cut tables or paragraphs in half, so their final Markdown keeps fragments and duplicated headers. These limits slow downstream document understanding because text has to be fixed by hand.

🧵 3/n 🛠️ Model design OCRFlux fine tunes Qwen2.5‑VL‑3B with two prompt templates, one for single page parsing and one for cross‑page merging. Only the rendered page image enters the prompt, not any external layout metadata, which keeps context length short and avoids errors from faulty OCR blocks.

🧵 4/n 🗂️ Single‑page parsing method Ground truth tables are stored as HTML so the model learns rowspan and colspan patterns. During inference the model returns Markdown where tables appear in HTML form, which preserves merged cells and column spans while staying copy‑friendly.

🧵 5/n 🔗 Cross‑page detection and merging The same model receives two consecutive page Markdown lists and predicts which elements belong together. If the elements are tables, the model rewrites a full table, deletes repeated headers and realigns cells split by the page cut, reaching 98.6% F1 on the benchmark.

🧵 6/n 📊 Benchmark results On 2 000 mixed‑language pages OCRFlux scores 0.967 Edit Distance Similarity, beating olmOCR by 0.095 and Nanonets‑OCR‑s by 0.109. On 9064 PubTabNet tables the model scores 0.861 average Tree Edit Distance, staying ahead for simple tables and remaining close on complex ones.

🧵 7/n Evaluation for cross-page paragraphs/tables merging

🧵 8/n Here’s a simple example to run the model on your machine. The helper runs OCRFlux in 2 steps. First it calls ocrflux.pipeline to parse the input PDF or image with the chosen 3B model and store JSONL outputs in a workspace directory. Next it invokes ocrflux.jsonl_to_markdown to turn those JSONL records into a clean Markdown document. It returns and prints the path of that final file, so other code can open or share it easily.

Check them out here Huggingface - Github - Official Site:

OCRFlux seems promising. PDF parsing is still far from solved. The more open-source solutions that tackle it the better.

yes