跳过导航
Not a fan of Knowledge Graphs, but recently I started using them more often for a surprising reason…
0:00 / 0:38

Not a fan of Knowledge Graphs, but recently I started using them more often for a surprising reason…

Han Xiao

@hxiao2.2万 位关注者

在 X 上查看
672 个赞下载原帖

58,433 次观看

Not a fan of Knowledge Graphs, but recently I started using them more often for a surprising reason: to build non-trivial private verifiers for agentic search. For those who don't know, building a private eval set for a scaffolded LLM in 2026 is really challenging, like seriously hard. It takes a lot of effort to find a question that's non-trivial to a scaffolded LLM yet still answerable. To find those question-answer pairs, I built a knowledge graph extractor where you can throw a corpus at it, and it extracts the entity relations using qwen3.6-35b-a3b-MTP on an L4 at 70 tps (which is really good for such a low-budget GPU). Then I mark out the longest path in the graph and use it to generate challenging question-answer pairs. The idea is to find those genuinely multi-hop fact chains that are verifiable from the corpus, to stress-test the agentic search system.

16 条评论

评论来自 X (Twitter) 上的原帖

  • Han Xiao 的头像

    @hxiao

    And yes, if you throw a real-world corpus at it, like here where I threw in all of Jina's blog posts and research papers from the last 6 years, you get this super long path across 90 nodes over multiple documents, perfect for building challenging multi-hop query for agentic search.

    33 个赞

  • Han Xiao 的头像

    @hxiao

    Model using Qwen3.6-35b-A3b-Q3_K_XL-MTP from @UnslothAI which i found the best in both quality & speed on low-budget gpu like L4 24gb, which is also cheap to scale out. Source: Demo:

    72 个赞

  • Alex Dong 的头像

    @alexdong

    What don’t you like about knowledge graph? They are deterministic, you can traverse them, query them. Perfect for grounding, isn’t it?

    4 个赞

  • Kevin  的头像

    @Kevinduan2014

    Another dimension is time, even harder

    1 个赞

  • Shuying Luo 的头像

    @shuying_luo

    Thanks for sharing! Though a totall unrelated use case, we also build knowledge trees for spreadsheet usage for selecting tasks. Walking it like a graph and extracting longest path will be an interesting experiment

    1 个赞

  • astra 的头像

    @novel_engineer

    5090 delivers 150-200TPS if you need more

    1 个赞

  • Ryan Johnson 的头像

    @RyanWorkforceAI

    This is very close to how my machine graph (graphiti/neo4j) is valuable. Not as a fancy retrieval layer, but as a verification harness. The graph makes multi-hop dependencies visible, exposes brittle claim chains, and helps create private questions that are answerable from the corpus but hard for an agent to bluff through. That shift matters because graph is not just context for the agent. It becomes a way to test the agent.

    1 个赞

  • bk 的头像

    @boulatbek

    this measures models ability to establish the sequence of nodes that need to be traversed before answer to the question is found? or is it what agent all about in general

  • Onur Solmaz 的头像

    @onusoz

    awesome, gonna try this locally

  • David Breslauer 的头像

    @davidNbreslauer

    Why are you not a fan of KGs?

  • substrate 的头像

    @SelfReflective

    thanks for sharing! Curious why you don’t like them? This post implies they may have some value ?

  • BlockedPath 的头像

    @BlockedPaths

    private eval for agentic search is a nightmare nobody talks about because you can't benchmark it publicly by definition. KGs as verifiers is a genuinely clever solve.

  • Bhishmaraj S 的头像

    @bhi5hmaraj

    > extracts the entity relations How do you verify the accuracy of this

  • Sebastian Buzdugan 的头像

    @sebuzdugan

    knowledge graph verifiers break once schema drift from tool changes silently flips labels

  • Michel aka Agent B 的头像

    @MichelIvan92347

    Interesting use case Han 🧐 Thanks for sharing!

  • Adel Bucetta 的头像

    @adelbucetta

    the reason most people overlook private verifiers is they focus on the search part, not the evaluation metrics that make sense in a world where model outputs are constantly evolving