We introduce representative generative benchmarking—custom eval sets built from your own data that reflect real user queries. thank you for collaborating! link to report in replies

link to technical report: Grounded in experiments with production data, our method captures performance differences that public benchmarks like MTEB miss.

API testing of interfaces is critical to determine if they meet requirements for functionality, reliability, performance, and security. Check out RTTS - the automated testing experts since 1996. #API #testautomation #integrationtest

@weights_biases this is super cool! at 🤗Huggingface we introduced a generative open source system but for full LLM evals instead! would be great to collab!

@weights_biases you guys somehow made notebooks look good, incredible

@weights_biases YES YES YES YES

@weights_biases Jealous of that @HermanMiller chair, how can I get a job at @trychroma.

@weights_biases Perhaps, you should use AI to reformat your message/paper?

@weights_biases This is what I need to do manually at the moment. very curious to see what this is capable of.

@weights_biases Kelly is legit