A 30M-parameter model, on CPUs, inside Postgres It matches a 0.6B dense embedding model on top-100 recall, to within 0.004, and finds more in the top 1,000 Meet Evoke. Open source today.

How: the model turns text into weighted words, including related words the text never uses Those go into the same inverted index as BM25 keywords. One lookup, one score, one ranked list. Nothing to fuse

The model runs on ordinary CPUs in shared background workers Writes commit straight away and get their meaning a moment later, so nothing waits on the model. No GPU, and no text leaves the database. Air-gapped installs work too

On BEIR15, Recall@100 goes from 0.563 with BM25 alone to 0.667 That's within 0.004 of a 0.6B dense model with ~20x the parameters, and Evoke places more relevant documents in the top 1,000 Built on IBM's Granite-Embedding-30M-Sparse. Apache-2.0

Read how it works: Code, Docker image and model:

@EMostaque Every cpu should come with a small one like this embedded in the ASIC

When the embedding model lives next to the rows, keep ANN index rebuilds off the write path. Queue refreshes, and after every model swap measure recall@k on a held-out set before you trust the new vectors in production.

Oh snap

multi-language?

This is really cool, do have some tabular data in this? No SQL and SQL. Like a fast analyzer, or is more like a re ranker? I was thinking of this like a another face of a model that complements something like tabpfn2

awesome

Keyword and meaning-based search in one index, running on CPUs. That’s an appealing combination. Secondly , the small model is interesting but having less to set up and maintain is what catches my eye.

30M parameters achieving that on CPUs is seriously impressive efficiency

That’s so cool! ☝️☝️

nice, no GPU is a big deal for small teams. how much does the index grow over plain BM25?