正在加载视频...

视频加载失败

introducing rd-signal-2: a frontier classification model that is 1600x cheaper than GPT 5.6 Sol. free to try in Raindrop, and available via a new API for training/hosting custom classifiers with Zero Data Retention.

162,078 次观看 • 1 个月前 •via X (Twitter)

57 条评论

ben hylak 的头像
ben hylak1 个月前

read more at: rd-signal-2 balances precision/recall at a price that is still 280x lower than GPT-5.6 Luna xhigh. it powers signals and real-time issue detection in raindrop, allowing our customers to detect and accurately track sparse behaviors.

ben hylak 的头像
ben hylak1 个月前

most LLM observability providers require customers to sample a very small % of traffic (and pay for the judges!) but agent failures are often rare. it's common for a behavioral anomaly to be < 1% of traffic. sampling can completely hide the problem.

ben hylak 的头像
ben hylak1 个月前

you can say "the agent saying it transferred to a human, but didn't call the tool" or "the user complaining that web search results were stale" @raindrop_ai signals will write the code, assemble the context, train a custom classification head for your task, and self-improve.

ben hylak 的头像
ben hylak1 个月前

rd-signal-2 also powers @raindrop_ai's real-time issue detection. many emerging failures are detected in under 5 minutes. because they share the same foundation, any raindrop issue can be instantly converted to a signal, compared in experiments, and more.

ben hylak 的头像
ben hylak1 个月前

rd-signal-2 already processes around 20 billion traces a month. members of the @raindrop_ai team: - invented fraud transformer models at robinhood - built the first recommender system at pinterest - have built agents that have millions of customers - and more dm me to join

ben hylak 的头像
ben hylak1 个月前

stop paying for judges. starting using signals. try the agent observability platform trusted by vercel, speak, and the fortune 100. read more at

eric provencher 的头像
eric provencher1 个月前

@raindrop_ai Damn your motion design is on point! Looks awesome

ben hylak 的头像
ben hylak1 个月前

@raindrop_ai thank you!

gian 的头像
gian1 个月前

@raindrop_ai sweeet, great job! adaptive online observability is basically table stakes at this point

ben hylak 的头像
ben hylak1 个月前

@raindrop_ai we agree.

Soleio 的头像
Soleio1 个月前

@raindrop_ai “Stop paying for judges. Start using signals.” :chefs_kiss:

Eduard Faus Gil 的头像
Eduard Faus Gil1 个月前

@raindrop_ai Typical raindrop W so glad I told @orchid_hq to use you guys

Logan Ford 的头像
Logan Ford1 个月前

@raindrop_ai very excited to try this out!

Dev 的头像
Dev1 个月前

@raindrop_ai hey @alialobai1 @hegargarcia 👀

ben hylak 的头像
ben hylak1 个月前

@raindrop_ai @alialobai1 @hegargarcia would love your feedback :)

bani 的头像
bani1 个月前

@raindrop_ai huge

ben hylak 的头像
ben hylak1 个月前

@raindrop_ai thanks bani! :)

Nizzy 的头像
Nizzy1 个月前

@raindrop_ai woahhh

ben hylak 的头像
ben hylak1 个月前

@raindrop_ai give it a try!

Sam Hogan 🇺🇸 的头像
Sam Hogan 🇺🇸1 个月前

@raindrop_ai very cool

Christian 的头像
Christian1 个月前

@raindrop_ai Very interesting 🤔🤔🤔

Eric 的头像
Eric1 个月前

@raindrop_ai well shit that sure is a lot cheaper

ben hylak 的头像
ben hylak1 个月前

@raindrop_ai indeed

Jackson Grove 的头像
Jackson Grove1 个月前

@raindrop_ai this is huge

ben hylak 的头像
ben hylak1 个月前

@raindrop_ai thank you - please give us any/all feedback :)

Jackson Grove 的头像
Jackson Grove1 个月前

@raindrop_ai will do excited for this :)

Rohanth Marem 的头像
Rohanth Marem1 个月前

@raindrop_ai interested how this is implemented, really cool

Jeff Barg 的头像
Jeff Barg1 个月前

@raindrop_ai Very cool!

ben hylak 的头像
ben hylak1 个月前

@raindrop_ai thanks jeff!

any 的头像
any1 个月前

@raindrop_ai thats crazy but nobody uses frontier models for classifiers anyway?

ben hylak 的头像
ben hylak1 个月前

@raindrop_ai yes, so then you get very bad precision/recall. you can also see a comparison to 5.6 Luna (which is a good classifier)

Alim 的头像
Alim1 个月前

@raindrop_ai dang this is good

ben hylak 的头像
ben hylak1 个月前

@raindrop_ai thanks friend!

ray arayilakath ‎.߆ 的头像
ray arayilakath ‎.߆1 个月前

@rsdgpt @raindrop_ai smart play!

Nick 的头像
Nick1 个月前

@raindrop_ai very impressive!

Jeremy 的头像
Jeremy1 个月前

@raindrop_ai Wait. This. FUCKING BANGS......

ben hylak 的头像
ben hylak1 个月前

@raindrop_ai thank you! which part resonated the most ooc

Aggroed Lighthacker- Peace, Prosperity, & Freedom 的头像
Aggroed Lighthacker- Peace, Prosperity, & Freedom1 个月前

@raindrop_ai @Teknium @NousResearch @yeahfortommy this seems promising

Adhyaay Karnwal 的头像
Adhyaay Karnwal1 个月前

@raindrop_ai sick dude

daksh 的头像
daksh1 个月前

@raindrop_ai whoa this wasn’t on my bingo card but so excited and happy for ben :)

jacob 的头像
jacob1 个月前

@raindrop_ai oo this is cool

ras 的头像
ras1 个月前

@raindrop_ai oh my god i have a very good use case for this if feasible

ben hylak 的头像
ben hylak1 个月前

@raindrop_ai dm me!

René Sultan 的头像
René Sultan1 个月前

@raindrop_ai oss?

ben hylak 的头像
ben hylak1 个月前

@raindrop_ai not this time! we do have plenty of other open source work though.

Prabal 的头像
Prabal1 个月前

@raindrop_ai ngl my main question is how did you create the video. great job - looks amazing.

Ty 的头像
Ty1 个月前

@sjwhitmore @raindrop_ai Fire

Jason Cameron 的头像
Jason Cameron1 个月前

@raindrop_ai Super cool! Thanks for releasing it publicly

Manveen Koticha 的头像
Manveen Koticha1 个月前

@raindrop_ai Cool stuff! @raindrop_ai team!!

toolshed 的头像
toolshed1 个月前

@raindrop_ai 1600x cheaper only matters if the error rate holds. A classifier two points worse costs more in human review time than it ever saves in tokens, at any price per call. What does the confusion matrix look like against Sol on the same eval set?

Reid Christian 的头像
Reid Christian1 个月前

@raindrop_ai Whoa

ben hylak 的头像
ben hylak1 个月前

@raindrop_ai agreed!

pidš 的头像
pidš1 个月前

@raindrop_ai I see that your Triage boy is free to try, saw someone made a token with royalities to your github. So you can claim the trading fees from it, if you want. GL chad

Clipur.com 的头像
Clipur.com1 个月前

@raindrop_ai cheap inference meta

Dan McAteer 的头像
Dan McAteer1 个月前

@raindrop_ai 1600x cheaper? Didn't even know a price could go that low.

ben hylak 的头像
ben hylak1 个月前

@raindrop_ai constraining the problem to binary classification helps re: cost a lot.

Dan McAteer 的头像
Dan McAteer1 个月前

@raindrop_ai Makes sense to me.

相关视频