I packed-up a full-text paper scraper, vector database, and LLM into a CLI to answer questions from only highly-cited peer-reviewed papers. Feels unreal to be able instantly get answers by an LLM "reading" dozens of papers. 1/2

Made with @LangChainAI, @OpenAI, textualize, FAISS, @SemanticScholar. Final frame (sorry no alt) 2/2

@OpenAI @SemanticScholar Most of the code (except the interface) is already open source -

Can you ask it a question that bothers me about birds? Why are birds moving their heads in quick succession of interrupted rapid movements?

Cool! how did you come up with a decision to cut off lower cited papers and what threshold did you choose?

It's a sort from influential citations and cut to 5 papers per search. No citation threshold.

Can the code be open-sourced Professor? I want to try too

Is it open sourced ? If not, can you try to open source it ?

I’m new to this stuff. Is there a way to do something similar by reading 10 websites ? Or do 10 websites need to be converted into text and then read

Definitely possible