Vision-based(Colapli) RAG is becoming popular, so we built a platform to compare: - Simple OCR RAG - VisionRAG - Colpali - Hybrid Colpali 🚀 Introducing VARAG – the Vision-First RAG Engine (Vision Augmented Retrieval and Generation).

A few days ago, I wanted to figure out the best vision-based retrieval techniques for a product I was working on. Ended up building a library out of it Tech used : - colpali engine - @lancedb which was a joy to use - its easy to use - very flexible and developer friendly

Some Experiments and Observations I ingested 30 research papers, and here is how long each technique took to process them.

note that in the colpali implementation the score is being calculated across all the vectors in the database thus the time taken is high Accuracy and other RAG metrics yet to test out Retrival Speed:

colpali does everything. what did you build on top of it?

Colpali-engine mainly provides functions to generate embeddings and calculate the score. Varag primarily offers an abstraction layer on top of it and provides a uniform interface for each retrieval technique, making it easier to compare and modify when needed.

@adithya_s_k What is hybrid colpali RAG. Is it extract metadata from images and indexing ?

Its a combination of plain visionRAG and colpali plain vision RAG uses clip/jina-clip as the embedding model where we retrive the initial top n images then we use colpali to rank within those top n to get top k

Hey 👋🏻 there is a collective repo for RAG papers implementation, would you like to join hands?

This is cool