Video wird geladen...

Video konnte nicht geladen werden

Beim Laden dieses Videos ist ein Problem aufgetreten. Dies könnte an einem vorübergehenden Netzwerkproblem liegen oder das Video ist möglicherweise nicht verfügbar.

SAMURAI vs. MetaAI's SAM 2! Traditional visual object tracking struggles in crowded, fast-moving, or self-occluded scenes, as does SAM2. Meet SAMURAI: a completely open-source adaptation of the Segment Anything Model for zero-shot visual tracking! Here's why it's a game-changer: 🚫 No need for retraining or finetuning 🎯 Boosts success... rate and precision 🤖 Motion-aware memory selection 💪 Zero-shot performance on diverse datasets But that's not all: 🔬 Refines mask selection 🔮 Predicts object motion effectively 📈 Gains: 7.1% AUC on LaSOT, 3.5% AO on GOT-10k 🏆 Competes with fully supervised methods without extra training Link to the GitHub repo in the next tweet! _____ Find me → Akshay 🚀 ✔️ For more insights & tutorials on AI and Machine Learning.show more

Akshay 🚀

283,503 subscribers

363,412 Aufrufe • vor 1 Jahr •via X (Twitter)

Wissenschaft & Technologie

Anya Rossi• Live Now

Private livecam show

9 Kommentare

Profilbild von Akshay 🚀

Akshay 🚀vor 1 Jahr

GitHub repo: _____ Interested in ML/AI Engineering? Sign up for our newsletter for in-depth lessons and get a FREE eBook with 150+ core DS/ML lessons:

Profilbild von BensenHsu

BensenHsuvor 1 Jahr

The paper focuses on adapting the Segment Anything Model 2 (SAM 2) for visual object tracking, which is a challenging task for the original model. SAM 2 has shown strong performance in object segmentation, but it faces difficulties in handling crowded scenes with fast-moving or self-occluding objects. The improvements in tracking accuracy are attributed to the incorporation of motion information and the enhanced memory selection mechanism. These advancements help SAMURAI better handle challenging scenarios, such as crowded scenes and occlusions, where the original SAM 2 model struggles. full paper:

Profilbild von TechPat

TechPatvor 1 Jahr

Very cool! Isn’t SAM 2 open source too?

Profilbild von Eswar RB

Eswar RBvor 1 Jahr

Perhaps combination of different colour models can fetch promising results. Seems this is only on RGB, as in when smokes covers Samurai fails to capture the subject.

Profilbild von Rohan gupta

Rohan guptavor 1 Jahr

Accuracy is so crazy

Profilbild von kaiban

kaibanvor 1 Jahr

Awesome simulation

Profilbild von Akshay 🚀

Akshay 🚀vor 1 Jahr

Great choice of the video to test it! Loved it!

Profilbild von FlameJack

FlameJackvor 1 Jahr

Now this is what AI should be used for, not generative AI that is using resources without any reason other than a lack of care to learn to make things the human way that gives things meaning .

Profilbild von Brandon Tyler

Brandon Tylervor 1 Jahr

I’m curious if you understand how ai tracking works for technologies like hudle and veo for basketball?

Ähnliche Videos

Microsoft has launched a powerful new data analysis tool! Introducing Data Formulator, a 100% open-source LLM-powered, no-code tool that transforms data in a snap and creates stunning visualizations. Key features include: 🤖 AI-powered data transformation 🖱️ Interactive drag-and-drop UI for visualizations 💬 Seamless blend of UI & natural language inputs But that’s not all: You can even create charts beyond your initial dataset. Data Formulator automatically identifies extra computation needs, generates fields for you, and outputs the final visualization. Find the GitHub repo in the next tweet! _____ Find me → Akshay 🚀 ✔️ For more insights and tutorials on AI and Machine Learning.

Microsoft has launched a powerful new data analysis tool! Introducing Data Formulator, a 100% open-source LLM-powered, no-code tool that transforms data in a snap and creates stunning visualizations. Key features include: 🤖 AI-powered data transformation 🖱️ Interactive drag-and-drop UI for visualizations 💬 Seamless blend of UI & natural language inputs But that’s not all: You can even create charts beyond your initial dataset. Data Formulator automatically identifies extra computation needs, generates fields for you, and outputs the final visualization. Find the GitHub repo in the next tweet! _____ Find me → Akshay 🚀 ✔️ For more insights and tutorials on AI and Machine Learning.

Akshay 🚀

280,449 Aufrufe • vor 1 Jahr

Everyone is sleeping on Meta's SAM 3 release. But it's actually a big deal. Here's why: Companies spend millions paying humans to label images and videos frame by frame. A single autonomous driving dataset? Months of work, hundreds of annotators, millions in cost. Without labeled data, you can't train custom models. Without custom models, you're stuck with generic solutions. This is why most companies never move past pilots. SAM 3 breaks this cycle. First let's look at the evolution: SAM 1 segmented objects when you clicked on them. Revolutionary, but one object at a time. SAM 2 added video tracking with memory. Game-changing, but you still manually prompted every object. SAM 3 changes everything with text prompts. Type "yellow school bus" and it finds ALL of them in your image or video. Not just one. Every instance across thousands of frames. Now here's where people get confused: "Can't I just use GPT-5 or Gemini for this?" No, and here's why that's a terrible approach. Large multimodal LLMs are great for reasoning, but they're slow and expensive for production visual tasks. You're paying API costs per image, waiting seconds for responses, getting inconsistent results. SAM 3 runs in 30 milliseconds on a single GPU for 100+ objects. That's 100x faster, and you own the infrastructure. More importantly, SAM 3 gives you precise pixel-level masks, not descriptions. Try asking an LLM to segment every defective part on a manufacturing line in real-time. It won't work. SAM 3 does this effortlessly. The real breakthrough is their data engine. Meta built an AI-human hybrid system that's 5x faster for complex annotations. They trained SAM 3 on 4 million unique visual concepts - 50x more than existing benchmarks like LVIS. SAM 3 is trained on 4 million unique visual concepts, it handles everything: - Text-based concept search - Interactive refinement with clicks - Video tracking across frames - Zero-shot detection of new concepts The model is open source. Weights, code, and benchmarks are on GitHub. If you're building computer vision applications, this is the foundation model to evaluate. The annotation time savings alone will pay for integration costs within weeks. Find the relevant links in the next tweet!

Everyone is sleeping on Meta's SAM 3 release. But it's actually a big deal. Here's why: Companies spend millions paying humans to label images and videos frame by frame. A single autonomous driving dataset? Months of work, hundreds of annotators, millions in cost. Without labeled data, you can't train custom models. Without custom models, you're stuck with generic solutions. This is why most companies never move past pilots. SAM 3 breaks this cycle. First let's look at the evolution: SAM 1 segmented objects when you clicked on them. Revolutionary, but one object at a time. SAM 2 added video tracking with memory. Game-changing, but you still manually prompted every object. SAM 3 changes everything with text prompts. Type "yellow school bus" and it finds ALL of them in your image or video. Not just one. Every instance across thousands of frames. Now here's where people get confused: "Can't I just use GPT-5 or Gemini for this?" No, and here's why that's a terrible approach. Large multimodal LLMs are great for reasoning, but they're slow and expensive for production visual tasks. You're paying API costs per image, waiting seconds for responses, getting inconsistent results. SAM 3 runs in 30 milliseconds on a single GPU for 100+ objects. That's 100x faster, and you own the infrastructure. More importantly, SAM 3 gives you precise pixel-level masks, not descriptions. Try asking an LLM to segment every defective part on a manufacturing line in real-time. It won't work. SAM 3 does this effortlessly. The real breakthrough is their data engine. Meta built an AI-human hybrid system that's 5x faster for complex annotations. They trained SAM 3 on 4 million unique visual concepts - 50x more than existing benchmarks like LVIS. SAM 3 is trained on 4 million unique visual concepts, it handles everything: - Text-based concept search - Interactive refinement with clicks - Video tracking across frames - Zero-shot detection of new concepts The model is open source. Weights, code, and benchmarks are on GitHub. If you're building computer vision applications, this is the foundation model to evaluate. The annotation time savings alone will pay for integration costs within weeks. Find the relevant links in the next tweet!

Akshay 🚀

46,421 Aufrufe • vor 8 Monaten

Just read a neat AI paper called SAMURAI -- it takes SAM 2 (Meta's "segment anything" model) and makes it way better at tracking objects in videos. Basic problem is SAM 2 gets confused when things move fast or there's a crowd of similar objects (big problem for VFX and video intelligence alike). The fix? They basically gave it a sense of motion and smarter memory selection using our old friend the Kalman filter -- helps it remember where things were going instead of just where they are. No retraining needed and runs in realtime. Elegant solution that matches (and even beats!) a lot of purpose built tracking systems. Nice example of how sometimes you don't need fancy new architectures -- just smart ways to use what the model already knows, plus some tried & tested classical methods. Every time someone's like "we solved tracking!" I check the methods section and... yep, there's our boy Kalman, still crushing it after all these years. Sometimes the old school stuff just works.

Just read a neat AI paper called SAMURAI -- it takes SAM 2 (Meta's "segment anything" model) and makes it way better at tracking objects in videos. Basic problem is SAM 2 gets confused when things move fast or there's a crowd of similar objects (big problem for VFX and video intelligence alike). The fix? They basically gave it a sense of motion and smarter memory selection using our old friend the Kalman filter -- helps it remember where things were going instead of just where they are. No retraining needed and runs in realtime. Elegant solution that matches (and even beats!) a lot of purpose built tracking systems. Nice example of how sometimes you don't need fancy new architectures -- just smart ways to use what the model already knows, plus some tried & tested classical methods. Every time someone's like "we solved tracking!" I check the methods section and... yep, there's our boy Kalman, still crushing it after all these years. Sometimes the old school stuff just works.

Bilawal Sidhu

47,246 Aufrufe • vor 1 Jahr

Star Wars cost billions to make. But today, you can create videos like this for just a few dollars. Filmmaking is going to be disrupted by AI sooner than we thought. If I were to take a shot at it, here's what I'd use: - o3-mini-high for writing the script - Imagen 3 or Flux Pro for images - Kling 1.6 for the video There are also excellent unified platforms like ChatLLM, where you can access all these resources in one place. For developers like us, there's more under the same offering: CodeLLM as a coding assistant, and RouteLLM automatically selects the best model for your chat queries. The company claims that they see thousands of humans becoming AI creators every day using their platform! Maybe it's a new role in the making! I have shared a link to ChatLLM in the next tweet! ______ Find me → Akshay 🚀 ✔️ For more insights and tutorials on AI and Machine Learning.

Star Wars cost billions to make. But today, you can create videos like this for just a few dollars. Filmmaking is going to be disrupted by AI sooner than we thought. If I were to take a shot at it, here's what I'd use: - o3-mini-high for writing the script - Imagen 3 or Flux Pro for images - Kling 1.6 for the video There are also excellent unified platforms like ChatLLM, where you can access all these resources in one place. For developers like us, there's more under the same offering: CodeLLM as a coding assistant, and RouteLLM automatically selects the best model for your chat queries. The company claims that they see thousands of humans becoming AI creators every day using their platform! Maybe it's a new role in the making! I have shared a link to ChatLLM in the next tweet! ______ Find me → Akshay 🚀 ✔️ For more insights and tutorials on AI and Machine Learning.

Akshay 🚀

52,457 Aufrufe • vor 1 Jahr

A team tested Pi0, Pi0 Fast, Gr00t, and ACT on real robot arms in manufacturing tasks. (🔖 Bookmark this for later!) The task was precise: place thin rectangular frames from a messy stack into a holder. The team fine-tuned each model on 100 real trajectories and compared training time, inference speed, motion quality, and success rates. ⬇️ Here’s a breakdown of what they found Pi0 (Original) ✅ Strongest overall performance in precise pick-and-place ✅ High success rate even in edge cases ✅ Longest training time (~11 hours, ~$30 per run) ✅ Inference time of 80 ms causes short pauses between actions Despite delays, it handles complex scenarios well… solid for high-precision tasks, but slow to train. Gr00t ✅ Trains fast (~2 hours, ~$5 per run) ✅ Performs almost as well as Pi0 on large-object tasks ✅ Struggles with fine precision; random movement in some trials ✅ More training didn’t fix jitter or random offsets Best suited for tasks where exact precision isn’t critical. Not ready for manufacturing-grade accuracy without more tuning. Pi0 Fast ✅ Promised faster training, but results were underwhelming ✅ Training at 6 hours still showed low success rates ✅ Inference was slower than expected ✅ Not reliable for generalizing even slightly new tasks Currently too unstable for real-world deployment. Doesn’t live up to the “Fast” name yet. ACT (Baseline) ✅ 200MB model—lightweight, but limited ✅ Struggles with stacked objects or ambiguous scenes ✅ Success rates around 70% in best-case setups ✅ Can’t match newer models on precision or generalization Still a solid baseline, but clearly a generation behind in robustness. 🚨 Extra Notes All newer models share a common issue: •Inference takes longer than a frame (80 ms vs 33 ms), so robots “pause” between chunks. •This results in jittery movements, but not a dealbreaker unless tasks are time-sensitive. Language-conditioned tasks also fell short: after training on two labeled tasks, the model couldn’t generalize to a third unseen combination using only text prompts. ✅ The good news? These models adapt well to new robot arms with quick fine-tuning. ❌ The bad news? There’s still no plug-and-play solution for improving performance after deployment. Reinforcement learning or DAgger-style data collection during real-world operation may be the next big step, something many teams in robotics are actively working on.

A team tested Pi0, Pi0 Fast, Gr00t, and ACT on real robot arms in manufacturing tasks. (🔖 Bookmark this for later!) The task was precise: place thin rectangular frames from a messy stack into a holder. The team fine-tuned each model on 100 real trajectories and compared training time, inference speed, motion quality, and success rates. ⬇️ Here’s a breakdown of what they found Pi0 (Original) ✅ Strongest overall performance in precise pick-and-place ✅ High success rate even in edge cases ✅ Longest training time (~11 hours, ~$30 per run) ✅ Inference time of 80 ms causes short pauses between actions Despite delays, it handles complex scenarios well… solid for high-precision tasks, but slow to train. Gr00t ✅ Trains fast (~2 hours, ~$5 per run) ✅ Performs almost as well as Pi0 on large-object tasks ✅ Struggles with fine precision; random movement in some trials ✅ More training didn’t fix jitter or random offsets Best suited for tasks where exact precision isn’t critical. Not ready for manufacturing-grade accuracy without more tuning. Pi0 Fast ✅ Promised faster training, but results were underwhelming ✅ Training at 6 hours still showed low success rates ✅ Inference was slower than expected ✅ Not reliable for generalizing even slightly new tasks Currently too unstable for real-world deployment. Doesn’t live up to the “Fast” name yet. ACT (Baseline) ✅ 200MB model—lightweight, but limited ✅ Struggles with stacked objects or ambiguous scenes ✅ Success rates around 70% in best-case setups ✅ Can’t match newer models on precision or generalization Still a solid baseline, but clearly a generation behind in robustness. 🚨 Extra Notes All newer models share a common issue: •Inference takes longer than a frame (80 ms vs 33 ms), so robots “pause” between chunks. •This results in jittery movements, but not a dealbreaker unless tasks are time-sensitive. Language-conditioned tasks also fell short: after training on two labeled tasks, the model couldn’t generalize to a third unseen combination using only text prompts. ✅ The good news? These models adapt well to new robot arms with quick fine-tuning. ❌ The bad news? There’s still no plug-and-play solution for improving performance after deployment. Reinforcement learning or DAgger-style data collection during real-world operation may be the next big step, something many teams in robotics are actively working on.

Ilir Aliu

21,844 Aufrufe • vor 1 Jahr

$Introducing LifeGPT, showing that LLMs can simulate complex, Turing-complete systems like Conway's Game of Life with near-perfect accuracy—no prior topology needed.🌐This unlocks new potential for AI in modeling self-organizing systems in biology, materials science, & beyond.🔬🤖 #AI #LifeGPT. Cellular Automata (CA), like Conway's Game of Life ("Life"), are computationally irreducible, meaning their evolution is difficult to predict without an a-priori understanding of the rules of the game, including the topology on which it is played. LifeGPT is a topology-agnostic generative model that learns the rules of Life without prior knowledge of its grid structure or boundary conditions, from only a tiny number of game states. The success in simulating Life suggests promising avenues for scientific discovery, particularly in bridging the gap between AI, artificial life, and real-world biological systems, for both forward and inverse problems. The potential for universal computation within generative AI, including LLMs, through approaches like LifeGPT, represents an exciting area for future research, especially when combined with reinforcement learning. Model Convergence: LifeGPT exhibits rapid convergence during training, achieving high accuracy in predicting next-game-states. We attribute the non-zero cross-entropy loss to the lack of causal relationships within randomly generated ICs. Accuracy & Temperature: LifeGPT achieves near-perfect accuracy, particularly at lower sampling temperatures, but can be continually tuned towards higher creativity to discover patterns that the original ruleset would not be able to produce. This finding highlights the trade-off between model creativity (higher temperature) and accuracy in deterministic predictions, with high relevance to model real-world dynamical systems for which no closed-form rulesets exist. Zero/Few-Shot Learning: Trained on a small fraction of possible initial conditions, LifeGPT demonstrates strong zero/few-shot learning, accurately simulating Life for unseen initial conditions. However, rare prediction errors highlight that LifeGPT approximates rather than perfectly replicates the Life algorithm. Autoregressive Autoregressor: A recursive implementation of LifeGPT demonstrates the model's ability to simulate Life over multiple timesteps. LifeGPT is topology-agnostic with respect to its training data and our results show that a GPT model is capable of capturing the deterministic rules of a Turing-complete system with near-perfect accuracy, given sufficiently diverse training data. The work showcases the possibility for future models to synthesize stochastic generative capabilities with deterministic computational capabilities. Link to code, paper, etc. below. Podcast generated using #NotebookLM. LAMM@MIT DMSE at MIT$

Introducing LifeGPT, showing that LLMs can simulate complex, Turing-complete systems like Conway's Game of Life with near-perfect accuracy—no prior topology needed.🌐This unlocks new potential for AI in modeling self-organizing systems in biology, materials science, & beyond.🔬🤖 #AI #LifeGPT. Cellular Automata (CA), like Conway's Game of Life ("Life"), are computationally irreducible, meaning their evolution is difficult to predict without an a-priori understanding of the rules of the game, including the topology on which it is played. LifeGPT is a topology-agnostic generative model that learns the rules of Life without prior knowledge of its grid structure or boundary conditions, from only a tiny number of game states. The success in simulating Life suggests promising avenues for scientific discovery, particularly in bridging the gap between AI, artificial life, and real-world biological systems, for both forward and inverse problems. The potential for universal computation within generative AI, including LLMs, through approaches like LifeGPT, represents an exciting area for future research, especially when combined with reinforcement learning. Model Convergence: LifeGPT exhibits rapid convergence during training, achieving high accuracy in predicting next-game-states. We attribute the non-zero cross-entropy loss to the lack of causal relationships within randomly generated ICs. Accuracy & Temperature: LifeGPT achieves near-perfect accuracy, particularly at lower sampling temperatures, but can be continually tuned towards higher creativity to discover patterns that the original ruleset would not be able to produce. This finding highlights the trade-off between model creativity (higher temperature) and accuracy in deterministic predictions, with high relevance to model real-world dynamical systems for which no closed-form rulesets exist. Zero/Few-Shot Learning: Trained on a small fraction of possible initial conditions, LifeGPT demonstrates strong zero/few-shot learning, accurately simulating Life for unseen initial conditions. However, rare prediction errors highlight that LifeGPT approximates rather than perfectly replicates the Life algorithm. Autoregressive Autoregressor: A recursive implementation of LifeGPT demonstrates the model's ability to simulate Life over multiple timesteps. LifeGPT is topology-agnostic with respect to its training data and our results show that a GPT model is capable of capturing the deterministic rules of a Turing-complete system with near-perfect accuracy, given sufficiently diverse training data. The work showcases the possibility for future models to synthesize stochastic generative capabilities with deterministic computational capabilities. Link to code, paper, etc. below. Podcast generated using #NotebookLM. LAMM@MIT DMSE at MIT

Markus J. Buehler

114,217 Aufrufe • vor 1 Jahr

🚀 The Segment Anything Model (SAM) has been upgraded to SAM2, featuring an efficient image encoder for segmenting images and videos. But does SAM2 outperform SAM1 in medical image and video segmentation? We're thrilled to present our paper "Segment Anything in Medical Images and Videos: Benchmark and Deployment"! We comprehensively benchmark SAM2 across 11 medical image modalities and videos. 📄 Paper: 💻 Code: **Highlights:** 1. SAM2 doesn’t always outperform SAM1 in 2D medical images, but excels in video segmentation, making it more accurate and efficient for 3D images, such as CT and MR scans. 2. MedSAM still outperforms SAM2 on most 2D modalities, but SAM2 surpasses MedSAM for 3D image segmentation in a slice-by-slice approach. 3. Segmentation performance varies with model size; sometimes the smallest model outperforms larger ones. 4. Fine-tuning SAM2 significantly boosts its performance for medical image segmentation. While SAM2 may struggle with challenging objects that have unclear boundaries or low contrast, it excels in generating good initial segmentation masks for common medical images and videos. However, the official interface doesn’t support medical data formats and has limitations on video length. To address this, we've developed a 3D Slicer Plugin and Gradio API for efficient 3D medical image and video segmentation. We invite you to try them out and provide feedback! 🔧 Deployment: - 3D Slicer Plugin: - Gradio API: (Note: Due to GPU limitations, the online API is available for only 12 hours and may be slow. We highly recommend deploying the Gradio API with your own computing resources: A big shoutout to Jun Ma (JunMa) who recently joined our UHN AI hub (UHN AI Hub) as Machine Learning Lead, and kudos to all co-authors: Sumin Kim, Feifei Li, Mohammed Baharoon (Mohammed Baharoon), Reza Asakereh, and Hongwei Lyu! This is true teamwork! Looking forward to collaborating with the community to advance 3D medical image and video segmentation foundation models! University Health Network U of T Department of Computer Science Department of Laboratory Medicine & Pathobiology Temerty Centre for AI in Medicine (T-CAIREM) Vector Institute #MedTech #AIinHealthcare #DeepLearning #MedicalImaging #SAM2 #MedSAM #AIResearch

🚀 The Segment Anything Model (SAM) has been upgraded to SAM2, featuring an efficient image encoder for segmenting images and videos. But does SAM2 outperform SAM1 in medical image and video segmentation? We're thrilled to present our paper "Segment Anything in Medical Images and Videos: Benchmark and Deployment"! We comprehensively benchmark SAM2 across 11 medical image modalities and videos. 📄 Paper: 💻 Code: Highlights: 1. SAM2 doesn’t always outperform SAM1 in 2D medical images, but excels in video segmentation, making it more accurate and efficient for 3D images, such as CT and MR scans. 2. MedSAM still outperforms SAM2 on most 2D modalities, but SAM2 surpasses MedSAM for 3D image segmentation in a slice-by-slice approach. 3. Segmentation performance varies with model size; sometimes the smallest model outperforms larger ones. 4. Fine-tuning SAM2 significantly boosts its performance for medical image segmentation. While SAM2 may struggle with challenging objects that have unclear boundaries or low contrast, it excels in generating good initial segmentation masks for common medical images and videos. However, the official interface doesn’t support medical data formats and has limitations on video length. To address this, we've developed a 3D Slicer Plugin and Gradio API for efficient 3D medical image and video segmentation. We invite you to try them out and provide feedback! 🔧 Deployment: - 3D Slicer Plugin: - Gradio API: (Note: Due to GPU limitations, the online API is available for only 12 hours and may be slow. We highly recommend deploying the Gradio API with your own computing resources: A big shoutout to Jun Ma (JunMa) who recently joined our UHN AI hub (UHN AI Hub) as Machine Learning Lead, and kudos to all co-authors: Sumin Kim, Feifei Li, Mohammed Baharoon (Mohammed Baharoon), Reza Asakereh, and Hongwei Lyu! This is true teamwork! Looking forward to collaborating with the community to advance 3D medical image and video segmentation foundation models! University Health Network U of T Department of Computer Science Department of Laboratory Medicine & Pathobiology Temerty Centre for AI in Medicine (T-CAIREM) Vector Institute #MedTech #AIinHealthcare #DeepLearning #MedicalImaging #SAM2 #MedSAM #AIResearch

Bo Wang

178,572 Aufrufe • vor 2 Jahren

NOBODY wants to send their data to Google or OpenAI. Yet here we are, shipping proprietary code, customer information, and sensitive business logic to closed-source APIs we don't control. While everyone's chasing the latest closed-source releases, open-source models are quietly becoming the practical choice for many production systems. Here's what everyone is missing: Open-source models are catching up fast, and they bring something the big labs can't: privacy, speed, and control. I built a playground to test this myself. Used CometML's Opik to evaluate models on real code generation tasks - testing correctness, readability, and best practices against actual GitHub repos. Here's what surprised me: OSS models like MiniMax-M2, Kimi k2 performed on par with the likes of Gemini 3 and Claude Sonnet 4.5 on most tasks. But practically MiniMax-M2 turns out to be a winner as it's twice as fast and 12x cheaper when you compare it to models like Sonnet 4.5. Well, this isn't just about saving money. When your model is smaller and faster, you can deploy it in places closed-source APIs can't reach: ↳ Real-time applications that need sub-second responses ↳ Edge devices where latency kills user experience ↳ On-premise systems where data never leaves your infrastructure MiniMax-M2 runs with only 10B activated parameters. That efficiency means lower latency, higher throughput, and the ability to handle interactive agents without breaking the bank. The intelligence-to-cost ratio here changes what's possible. You're not choosing between quality and affordability anymore. You're not sacrificing privacy for performance. The gap is closing, and in many cases, it's already closed. If you're building anything that needs to be fast, private, or deployed at scale, it's worth taking a look at what's now available. MiniMax-M2 is 100% open-source, free for developers right now. I have shared the link to their GitHub repo in the next tweet. You will also find the code for the playground and evaluations I've done.

NOBODY wants to send their data to Google or OpenAI. Yet here we are, shipping proprietary code, customer information, and sensitive business logic to closed-source APIs we don't control. While everyone's chasing the latest closed-source releases, open-source models are quietly becoming the practical choice for many production systems. Here's what everyone is missing: Open-source models are catching up fast, and they bring something the big labs can't: privacy, speed, and control. I built a playground to test this myself. Used CometML's Opik to evaluate models on real code generation tasks - testing correctness, readability, and best practices against actual GitHub repos. Here's what surprised me: OSS models like MiniMax-M2, Kimi k2 performed on par with the likes of Gemini 3 and Claude Sonnet 4.5 on most tasks. But practically MiniMax-M2 turns out to be a winner as it's twice as fast and 12x cheaper when you compare it to models like Sonnet 4.5. Well, this isn't just about saving money. When your model is smaller and faster, you can deploy it in places closed-source APIs can't reach: ↳ Real-time applications that need sub-second responses ↳ Edge devices where latency kills user experience ↳ On-premise systems where data never leaves your infrastructure MiniMax-M2 runs with only 10B activated parameters. That efficiency means lower latency, higher throughput, and the ability to handle interactive agents without breaking the bank. The intelligence-to-cost ratio here changes what's possible. You're not choosing between quality and affordability anymore. You're not sacrificing privacy for performance. The gap is closing, and in many cases, it's already closed. If you're building anything that needs to be fast, private, or deployed at scale, it's worth taking a look at what's now available. MiniMax-M2 is 100% open-source, free for developers right now. I have shared the link to their GitHub repo in the next tweet. You will also find the code for the playground and evaluations I've done.

Akshay 🚀

50,323 Aufrufe • vor 8 Monaten

🎉 The best way to start the week is to find out that our MedSAM is finally published today in Nature Communications! **Segment anything in medical images** Paper: arXiv: Data & Code: MedSAM is the first promotable foundation model for medical image segmentation. **Highlights**: ⭐ Before its formal publication, we have received 220 citations and 1400+ GitHub stars 🙏🙏❤️‍🔥❤️‍🔥❤️‍🔥 📊 We curated a large-scale medical image dataset with 1,570,263 image-mask pairs, covering 10 imaging modalities and over 30 cancer types. 🚀 Built on top of SAM (AI at Meta ) with transfer learning, we have significantly enhanced its segmentation performance of medical images. 📈 Comprehensive evaluations of 86 internal validation tasks and 60 external validation tasks demonstrate its better accuracy and robustness than modality-wise specialist models. **What is Next? --- Clinical Translation!!** 🍕Our next goal is to make the model deployable on laptops (CPUs) or other edge devices without reliance on GPUs. We have distilled a lightweight model, LiteMedSAM, offering a speed boost of 10x while maintaining accuracy. Plus, we have integrated it into the 3D Slicer plugin, providing an efficient tool for medical image segmentation. 🌐 To further promote developments in this field, we organize a competition on #CVPR2026: Segment Anything in Medical Images on Laptop! An out-of-the-box baseline has been released to reduce the entry barriers. Welcome to join us to push the boundary further: 🙏 Massive thanks to MetaAI AI at Meta for their open-source project SAM and many reviewers/users for their invaluable feedback. A huge shoutout to my postdoc Jun Ma (JunMa) for his leadership on this project!! UHN AI Hub Vector Institute Peter Munk Cardiac Centre AI Department of Laboratory Medicine & Pathobiology U of T Department of Computer Science University of Toronto University Health Network Brad Wouters 🇨🇦 Barry Rubin MD, PhD, FRCSC Shaf Keshavjee

🎉 The best way to start the week is to find out that our MedSAM is finally published today in Nature Communications! Segment anything in medical images Paper: arXiv: Data & Code: MedSAM is the first promotable foundation model for medical image segmentation. Highlights: ⭐ Before its formal publication, we have received 220 citations and 1400+ GitHub stars 🙏🙏❤️‍🔥❤️‍🔥❤️‍🔥 📊 We curated a large-scale medical image dataset with 1,570,263 image-mask pairs, covering 10 imaging modalities and over 30 cancer types. 🚀 Built on top of SAM (AI at Meta ) with transfer learning, we have significantly enhanced its segmentation performance of medical images. 📈 Comprehensive evaluations of 86 internal validation tasks and 60 external validation tasks demonstrate its better accuracy and robustness than modality-wise specialist models. What is Next? --- Clinical Translation!! 🍕Our next goal is to make the model deployable on laptops (CPUs) or other edge devices without reliance on GPUs. We have distilled a lightweight model, LiteMedSAM, offering a speed boost of 10x while maintaining accuracy. Plus, we have integrated it into the 3D Slicer plugin, providing an efficient tool for medical image segmentation. 🌐 To further promote developments in this field, we organize a competition on #CVPR2026: Segment Anything in Medical Images on Laptop! An out-of-the-box baseline has been released to reduce the entry barriers. Welcome to join us to push the boundary further: 🙏 Massive thanks to MetaAI AI at Meta for their open-source project SAM and many reviewers/users for their invaluable feedback. A huge shoutout to my postdoc Jun Ma (JunMa) for his leadership on this project!! UHN AI Hub Vector Institute Peter Munk Cardiac Centre AI Department of Laboratory Medicine & Pathobiology U of T Department of Computer Science University of Toronto University Health Network Brad Wouters 🇨🇦 Barry Rubin MD, PhD, FRCSC Shaf Keshavjee

Bo Wang

140,229 Aufrufe • vor 2 Jahren

DRONE VIDEOGRAPHERS CHARGE $10K FOR THIS SHOT. HE PULLS IT FROM GOOGLE EARTH AND A PROMPT You never buy a drone, book a pilot, or leave the house. You pick any city on Earth, trace the flight path you want, and let Gemini render it as real-looking FPV footage. Clients pay thousands for this shot. You make it from a screenshot Here is the exact process: 1. Open Google Earth. Find the city or building you want. Frame the angle you'd want a drone to start from and take a screenshot 2. Draw the path. On that screenshot, draw a red line showing exactly where the drone should fly through the scene. This line is what the AI follows 3. Open Gemini and drop in the screenshot. Use the video generation in the Gemini app, the part that animates a still image into motion. Nano Banana handles images, the video engine is what turns your shot into footage 4. Paste the prompt. Tell it to follow the red flight path through the city, fast smooth motion, banking around buildings, golden-hour light, motion blur, 9:16 vertical, real FPV drone look. Full prompt is in the comments 5. Generate and clean it up. One clip is a few seconds. Stitch a couple together for a full flythrough and you have a reel Set the prompt once and you can re-run it for any location on the planet Who pays for this: Real estate agents, hotels, restaurants and event venues all need aerial b-roll and almost none can afford a real drone shoot Pull listings or venues with flat, ground-level photos and zero aerial footage. Send a free sample flythrough of their own location, then charge per clip or a monthly rate for ongoing reels One agent with ten listings is a recurring client, fully online Full prompt in the comments Bookmark this

DRONE VIDEOGRAPHERS CHARGE $10K FOR THIS SHOT. HE PULLS IT FROM GOOGLE EARTH AND A PROMPT You never buy a drone, book a pilot, or leave the house. You pick any city on Earth, trace the flight path you want, and let Gemini render it as real-looking FPV footage. Clients pay thousands for this shot. You make it from a screenshot Here is the exact process: 1. Open Google Earth. Find the city or building you want. Frame the angle you'd want a drone to start from and take a screenshot 2. Draw the path. On that screenshot, draw a red line showing exactly where the drone should fly through the scene. This line is what the AI follows 3. Open Gemini and drop in the screenshot. Use the video generation in the Gemini app, the part that animates a still image into motion. Nano Banana handles images, the video engine is what turns your shot into footage 4. Paste the prompt. Tell it to follow the red flight path through the city, fast smooth motion, banking around buildings, golden-hour light, motion blur, 9:16 vertical, real FPV drone look. Full prompt is in the comments 5. Generate and clean it up. One clip is a few seconds. Stitch a couple together for a full flythrough and you have a reel Set the prompt once and you can re-run it for any location on the planet Who pays for this: Real estate agents, hotels, restaurants and event venues all need aerial b-roll and almost none can afford a real drone shoot Pull listings or venues with flat, ground-level photos and zero aerial footage. Send a free sample flythrough of their own location, then charge per clip or a monthly rate for ongoing reels One agent with ten listings is a recurring client, fully online Full prompt in the comments Bookmark this

Yarchi

53,401 Aufrufe • vor 2 Monaten

Mistral AI Releases Robostral Navigate: An 8B Model Enabling Robots to Navigate Complex Environments Hitting 76.6% on R2R-CE With One RGB Camera. No LiDAR. No depth sensor. No multi-camera rig. Here's how it works. 👇 1. Pointing, not metric commands The model predicts the pixel coordinates of the next target in the camera view, plus the arrival orientation. Working in pixel space keeps it robust to camera intrinsics and world scale. When the target leaves the frame, it falls back to local displacements ("2m forward, 1.5m left, turn 25°"). 2. Grounding-first No open-source VLM base. It starts from Mistral's grounding model (pointing, counting, localization). Navigation emerges once the model knows where things are. → ~400,000 trajectories across 6,000 simulated scenes 3. Prefix-caching for training A tree-based attention mask packs a full episode into one sequence — all time steps in a single forward pass. → 22× fewer training tokens; months of training done in days 4. Online RL on top After supervised training, CISPO adds trial-and-error learning to fight distribution shift from behavior cloning. → +3.2% success rate from RL alone 5. The numbers (R2R-CE, Matterport3D) → 76.6% success on validation unseen → +9.7 pts over best single-camera approach → +4.5 pts over best depth/multi-camera system The key takeaway: state-of-the-art continuous VLN without a sensor stack — grounding-init, pixel-space actions, prefix-cached SFT, and online RL, on one RGB camera. Full analysis: Technical details: Mistral AI Mistral AI for Developers

Mistral AI Releases Robostral Navigate: An 8B Model Enabling Robots to Navigate Complex Environments Hitting 76.6% on R2R-CE With One RGB Camera. No LiDAR. No depth sensor. No multi-camera rig. Here's how it works. 👇 1. Pointing, not metric commands The model predicts the pixel coordinates of the next target in the camera view, plus the arrival orientation. Working in pixel space keeps it robust to camera intrinsics and world scale. When the target leaves the frame, it falls back to local displacements ("2m forward, 1.5m left, turn 25°"). 2. Grounding-first No open-source VLM base. It starts from Mistral's grounding model (pointing, counting, localization). Navigation emerges once the model knows where things are. → ~400,000 trajectories across 6,000 simulated scenes 3. Prefix-caching for training A tree-based attention mask packs a full episode into one sequence — all time steps in a single forward pass. → 22× fewer training tokens; months of training done in days 4. Online RL on top After supervised training, CISPO adds trial-and-error learning to fight distribution shift from behavior cloning. → +3.2% success rate from RL alone 5. The numbers (R2R-CE, Matterport3D) → 76.6% success on validation unseen → +9.7 pts over best single-camera approach → +4.5 pts over best depth/multi-camera system The key takeaway: state-of-the-art continuous VLN without a sensor stack — grounding-init, pixel-space actions, prefix-cached SFT, and online RL, on one RGB camera. Full analysis: Technical details: Mistral AI Mistral AI for Developers

Marktechpost AI

39,955 Aufrufe • vor 28 Tagen

Remote Viewer Just Exposed a SECRET Military War That No One Realizes Is Going On. No it's not the Iran War. No It's not the Gaza War.... It's an INVISIBLE WAR. Remote Viewer, Edward Riordan, exposes what's actually going on, that we are not aware of: ."..Was some object here flying across? It was in motion. Flying, zipping, moving, moving motion accelerating. Very fast speeds like a rubber bullet. It reminded me of just very fast moving." "The object itself was manmade. The notions here are like space travel technology and or weapons of war, flight or missile built in and for the same thing. So these are like military space weapon kind of stuff. Military and contractor budget facility. That was this place, a military contractor budget facility. And with these kinds of crafts, spacecrafts here or flying..." "...It was, a sense of advanced technology, space flight and weapons. But for what, though, I wonder? Stealth machines... How does this relate to the tasking though I questioned... A secret budget operation is what it felt designed for." "But what is it developed for, though? I pondered protecting, protection from enemies. See, this is where I get interesting protection from enemies seen and not seen." "Stealth level, even cloaked, even in remote flying. Meaning the pilot is on the ground in a VR simulator while the plane flies the mission. Is this what the target is all about? So I wonder, the testing of a system, a flying system protecting an unseen entity. This is case where stuff got very interesting. Here we're protecting an unseen entity, protecting the unseen. But what does that even mean, though, I wonder? And defense system designed and built for the unseen. For those who you are not to see or cannot see." "is all of that unseen seen surface at all theoretical in theory, a threat from an an interdimensional force and to protect the unseen from here?" "...Very sci fi, heavy on the defense. To fight off, an invisible or cloaked enemy...." "he signal was like that. The dark night, satellite thinking. Wow, this is really pretty wild. Wow... getting this notion here that this thing, that this invisible thing is going to bleed over from that world into this world.... seen as a continuation war or hostility factor in the world that people will continue to quote, seems as a continuation war or hostility factor in the world. People will continue to question, is it a real, is it a simulation? Do I need to be concerned? All their questions, the effects of this hostility don't seem to do much for the average person living out their lives. It affects them, but not in ways they can or will register directly. The main element remains hidden and or secret..."

Remote Viewer Just Exposed a SECRET Military War That No One Realizes Is Going On. No it's not the Iran War. No It's not the Gaza War.... It's an INVISIBLE WAR. Remote Viewer, Edward Riordan, exposes what's actually going on, that we are not aware of: ."..Was some object here flying across? It was in motion. Flying, zipping, moving, moving motion accelerating. Very fast speeds like a rubber bullet. It reminded me of just very fast moving." "The object itself was manmade. The notions here are like space travel technology and or weapons of war, flight or missile built in and for the same thing. So these are like military space weapon kind of stuff. Military and contractor budget facility. That was this place, a military contractor budget facility. And with these kinds of crafts, spacecrafts here or flying..." "...It was, a sense of advanced technology, space flight and weapons. But for what, though, I wonder? Stealth machines... How does this relate to the tasking though I questioned... A secret budget operation is what it felt designed for." "But what is it developed for, though? I pondered protecting, protection from enemies. See, this is where I get interesting protection from enemies seen and not seen." "Stealth level, even cloaked, even in remote flying. Meaning the pilot is on the ground in a VR simulator while the plane flies the mission. Is this what the target is all about? So I wonder, the testing of a system, a flying system protecting an unseen entity. This is case where stuff got very interesting. Here we're protecting an unseen entity, protecting the unseen. But what does that even mean, though, I wonder? And defense system designed and built for the unseen. For those who you are not to see or cannot see." "is all of that unseen seen surface at all theoretical in theory, a threat from an an interdimensional force and to protect the unseen from here?" "...Very sci fi, heavy on the defense. To fight off, an invisible or cloaked enemy...." "he signal was like that. The dark night, satellite thinking. Wow, this is really pretty wild. Wow... getting this notion here that this thing, that this invisible thing is going to bleed over from that world into this world.... seen as a continuation war or hostility factor in the world that people will continue to quote, seems as a continuation war or hostility factor in the world. People will continue to question, is it a real, is it a simulation? Do I need to be concerned? All their questions, the effects of this hostility don't seem to do much for the average person living out their lives. It affects them, but not in ways they can or will register directly. The main element remains hidden and or secret..."

Future Forecasting Group

19,006 Aufrufe • vor 5 Monaten

Researchers built a new RAG approach that: - does not need a vector DB. - does not embed data. - involves no chunking. - performs no similarity search. And it hit 98.7% accuracy on a financial benchmark (SOTA). Here's the core problem with RAG that this new approach solves: Traditional RAG chunks documents, embeds them into vectors, and retrieves based on semantic similarity. But similarity ≠ relevance. When you ask "What were the debt trends in 2023?", a vector search returns chunks that look similar. But the actual answer might be buried in some Appendix, referenced on some page, in a section that shares zero semantic overlap with your query. Traditional RAG would likely never find it. PageIndex (open-source) solves this. Instead of chunking and embedding, PageIndex builds a hierarchical tree structure from your documents, like an intelligent table of contents. Then it uses reasoning to traverse that tree. For instance, the model doesn't ask: "What text looks similar to this query?" Instead, it asks: "Based on this document's structure, where would a human expert look for this answer?" That's a fundamentally different approach with: - No arbitrary chunking that breaks context. - No vector DB infrastructure to maintain. - Traceable retrieval to see exactly why it chose a specific section. - The ability to see in-document references ("see Table 5.3") the way a human would. But here's the deeper issue that it solves. Vector search treats every query as independent. But documents have structure and logic, like sections that reference other sections and context that builds across pages. PageIndex respects that structure instead of flattening it into embeddings. Do note that this approach may not make sense in every use case since traditional vector search is still fast, simple, and works well for many applications. But for professional documents that require domain expertise and multi-step reasoning, this tree-based, reasoning-first approach shines. For instance, PageIndex achieved 98.7% accuracy on FinanceBench, significantly outperforming traditional vector-based RAG systems on complex financial document analysis. Everything is fully open-source, so you can see the full implementation in GitHub and try it yourself. I have shared the GitHub repo in the replies!

Researchers built a new RAG approach that: - does not need a vector DB. - does not embed data. - involves no chunking. - performs no similarity search. And it hit 98.7% accuracy on a financial benchmark (SOTA). Here's the core problem with RAG that this new approach solves: Traditional RAG chunks documents, embeds them into vectors, and retrieves based on semantic similarity. But similarity ≠ relevance. When you ask "What were the debt trends in 2023?", a vector search returns chunks that look similar. But the actual answer might be buried in some Appendix, referenced on some page, in a section that shares zero semantic overlap with your query. Traditional RAG would likely never find it. PageIndex (open-source) solves this. Instead of chunking and embedding, PageIndex builds a hierarchical tree structure from your documents, like an intelligent table of contents. Then it uses reasoning to traverse that tree. For instance, the model doesn't ask: "What text looks similar to this query?" Instead, it asks: "Based on this document's structure, where would a human expert look for this answer?" That's a fundamentally different approach with: - No arbitrary chunking that breaks context. - No vector DB infrastructure to maintain. - Traceable retrieval to see exactly why it chose a specific section. - The ability to see in-document references ("see Table 5.3") the way a human would. But here's the deeper issue that it solves. Vector search treats every query as independent. But documents have structure and logic, like sections that reference other sections and context that builds across pages. PageIndex respects that structure instead of flattening it into embeddings. Do note that this approach may not make sense in every use case since traditional vector search is still fast, simple, and works well for many applications. But for professional documents that require domain expertise and multi-step reasoning, this tree-based, reasoning-first approach shines. For instance, PageIndex achieved 98.7% accuracy on FinanceBench, significantly outperforming traditional vector-based RAG systems on complex financial document analysis. Everything is fully open-source, so you can see the full implementation in GitHub and try it yourself. I have shared the GitHub repo in the replies!

Avi Chawla

972,905 Aufrufe • vor 6 Monaten

Reinforcement Learning from Human Feedback (RLHF) is gaining traction. This field aims to make AI more responsible by including human values and preferences. In this video, Nathan Lambert, a research scientist and RLHF team lead at Hugging Face explores its inner workings, applications and industry impact. RLHF has gained the spotlight in recent years. The growth of language models like Anthropic’s Claude and OpenAI's ChatGPT have increased interest in human-feedback integration. "There are some rumors that Open AI had two teams; one was doing RLHF and the other instruction fine-tuning. And the RLHF team kept getting more and more performance." Understanding RLHF The RLHF process has three main steps: Pre-training: Much like with GPT models, the journey starts with pre-training on a large corpus of data. This can range from text data, web scrapes, to specialized datasets. Reward Modeling: This is the RLHF counterpart of supervised fine-tuning in large language models. This stage involves creating a reward model that resonates with human values and preferences. RL Optimization: This stage parallels reward modeling and reinforcement learning in traditional AI models. The AI system fine-tunes itself based on the reward model, employing reinforcement learning algorithms for that extra layer of optimization. The Data Challenge Data collection and curation in RLHF closely resemble the challenges you'd encounter in large language model training. Datasets from organizations like OpenAI can serve as a useful foundation. However, the need for high-quality, task-specific data cannot be overstated. Implementing RLHF: A Practical Guide If you’re someone who loves getting hands-on with AI libraries like Hugging Face, implementing RLHF is right way to do. It’s essential to understand its limitations. Think about model stability, over-optimization, and exploration strategies, much like you would when prompt engineering. Ongoing Research and Next Steps While he suggests that some basics figured out, there are layers of complexity that still need to be unraveled: 1. New Benchmarks: How do we measure the effectiveness of RLHF? 2. Preference Modeling: How can the model be made to understand human preferences better? 3. Interpreting RLHF: Much like explainability in traditional models, how do we make RLHF more interpretable? 4. System-Wide Evaluation: Going beyond individual performance, how does RLHF affect an entire system? The Transformative Power of RLHF Whether you're an AI developer, a business analyst, or a marketer, RLHF promises to revolutionize your domain. Imagine customer service chatbots that understand human emotions better, or content generators that align more closely with human values. RLHF is an emerging field that focuses on enhancing machine learning models through human feedback. While it tackles important issues like bias and ethics, its broader goal is to improve system performance across various applications. Whether you're deeply invested in the ethics of AI or simply curious about advancements in machine learning, RLHF offers valuable insights. If you're interested in the next wave of AI development, this area is definitely one to watch.

Reinforcement Learning from Human Feedback (RLHF) is gaining traction. This field aims to make AI more responsible by including human values and preferences. In this video, Nathan Lambert, a research scientist and RLHF team lead at Hugging Face explores its inner workings, applications and industry impact. RLHF has gained the spotlight in recent years. The growth of language models like Anthropic’s Claude and OpenAI's ChatGPT have increased interest in human-feedback integration. "There are some rumors that Open AI had two teams; one was doing RLHF and the other instruction fine-tuning. And the RLHF team kept getting more and more performance." Understanding RLHF The RLHF process has three main steps: Pre-training: Much like with GPT models, the journey starts with pre-training on a large corpus of data. This can range from text data, web scrapes, to specialized datasets. Reward Modeling: This is the RLHF counterpart of supervised fine-tuning in large language models. This stage involves creating a reward model that resonates with human values and preferences. RL Optimization: This stage parallels reward modeling and reinforcement learning in traditional AI models. The AI system fine-tunes itself based on the reward model, employing reinforcement learning algorithms for that extra layer of optimization. The Data Challenge Data collection and curation in RLHF closely resemble the challenges you'd encounter in large language model training. Datasets from organizations like OpenAI can serve as a useful foundation. However, the need for high-quality, task-specific data cannot be overstated. Implementing RLHF: A Practical Guide If you’re someone who loves getting hands-on with AI libraries like Hugging Face, implementing RLHF is right way to do. It’s essential to understand its limitations. Think about model stability, over-optimization, and exploration strategies, much like you would when prompt engineering. Ongoing Research and Next Steps While he suggests that some basics figured out, there are layers of complexity that still need to be unraveled: 1. New Benchmarks: How do we measure the effectiveness of RLHF? 2. Preference Modeling: How can the model be made to understand human preferences better? 3. Interpreting RLHF: Much like explainability in traditional models, how do we make RLHF more interpretable? 4. System-Wide Evaluation: Going beyond individual performance, how does RLHF affect an entire system? The Transformative Power of RLHF Whether you're an AI developer, a business analyst, or a marketer, RLHF promises to revolutionize your domain. Imagine customer service chatbots that understand human emotions better, or content generators that align more closely with human values. RLHF is an emerging field that focuses on enhancing machine learning models through human feedback. While it tackles important issues like bias and ethics, its broader goal is to improve system performance across various applications. Whether you're deeply invested in the ethics of AI or simply curious about advancements in machine learning, RLHF offers valuable insights. If you're interested in the next wave of AI development, this area is definitely one to watch.

Muratcan Koylan

27,168 Aufrufe • vor 2 Jahren

Love OpenClaw but hate the token burn? 💸 Running a 24/7 agent on GPT-4/Claude is overkill. You don't need SOTA reasoning to handle a greeting or a simple lookup. LLMRouter 🩷 OpenClaw The first production-ready, agentic router designed to plug directly into OpenClaw. LLMRouter fully supports Multimodal, Memory-Equipped routing that adapts 100% to your needs—compatible with FREE open-source models. The Logic is Simple:🔹 Simple query → Cheap/Local model 🔹 Complex reasoning → SOTA model (GPT-4/Claude 3.5) 🔹 Multimodal input → Vision/Audio specialized model Why this isn't just a switch: 📉 30–50% drop in inference costs 🧠 Zero loss in response quality 🔓 100% compatible with OpenAI-style APIs 🚀 Deploy in Seconds General Usage: Get the library and serve any model: pip install llmrouter-lib llmrouter serve OpenClaw Native Integration: Want the full agent experience? LLMRouter built a dedicated integration for OpenClaw users: LLMRouter Resources: 🔗 Repo: 📦 PyPI: 🤝 Works with: Route smarter. Train your own. Pay less. More on LLMRouter: Most routers are static if/else. LLMRouter is an intelligent, learning system. 🤖 Agentic & Memory-Aware: Decisions aren't stateless. We use RAG-powered memory to route based on context and history. 👤 Fully Personalized: It learns from your usage patterns via RL feedback loops. 🔬 Research-Grade: Switch between 16+ routing strategies (KNN, SVM, BERT, Graph, RL) with a single flag.

Love OpenClaw but hate the token burn? 💸 Running a 24/7 agent on GPT-4/Claude is overkill. You don't need SOTA reasoning to handle a greeting or a simple lookup. LLMRouter 🩷 OpenClaw The first production-ready, agentic router designed to plug directly into OpenClaw. LLMRouter fully supports Multimodal, Memory-Equipped routing that adapts 100% to your needs—compatible with FREE open-source models. The Logic is Simple:🔹 Simple query → Cheap/Local model 🔹 Complex reasoning → SOTA model (GPT-4/Claude 3.5) 🔹 Multimodal input → Vision/Audio specialized model Why this isn't just a switch: 📉 30–50% drop in inference costs 🧠 Zero loss in response quality 🔓 100% compatible with OpenAI-style APIs 🚀 Deploy in Seconds General Usage: Get the library and serve any model: pip install llmrouter-lib llmrouter serve OpenClaw Native Integration: Want the full agent experience? LLMRouter built a dedicated integration for OpenClaw users: LLMRouter Resources: 🔗 Repo: 📦 PyPI: 🤝 Works with: Route smarter. Train your own. Pay less. More on LLMRouter: Most routers are static if/else. LLMRouter is an intelligent, learning system. 🤖 Agentic & Memory-Aware: Decisions aren't stateless. We use RAG-powered memory to route based on context and history. 👤 Fully Personalized: It learns from your usage patterns via RL feedback loops. 🔬 Research-Grade: Switch between 16+ routing strategies (KNN, SVM, BERT, Graph, RL) with a single flag.

Jiaxuan You

31,451 Aufrufe • vor 6 Monaten

🚀 We're thrilled to introduce Orthrus 🧬🐕—a groundbreaking mature RNA foundation model designed to push the boundaries of RNA property prediction! 🔬 What is Orthrus? Orthrus is a Mamba-based RNA foundation model, pre-trained using a novel self-supervised contrastive learning objective with biologically inspired augmentations. It optimizes the similarity between splicing isoforms and orthologous transcripts, capturing functional and evolutionary relationships to enhance mature RNA property prediction accuracy. 📑 Preprint: 💻 Code: 🌐 Project Page: 📦 Model Weights: 🧠 Why Orthrus? Decoding the RNA regulatory code is key to understanding biology, but traditional experimental approaches are slow and costly. Existing genomic foundation models rely on techniques like masked language modeling or next-token prediction, which aren't fully aligned with the complexities of genomic data—leading to suboptimal results. 🌟 Orthrus Highlights: - Biologically-Informed Contrastive Learning 🧪: A novel contrastive learning objective designed specifically for genomics, maximizing similarity between splicing isoforms and orthologous transcripts across species. - Extensive Pre-training 📊: Trained on splicing annotations from 10 species and orthologous alignments from 400+ mammalian species (Zoonomia Project), with a focus on sequences of high functional importance. - Superior Representations🏅: Orthrus outperforms existing genomic models on 5 mRNA property prediction tasks, often surpassing supervised methods with just a simple linear transformation. - Efficiency in Low-Data Settings📉: Orthrus excels in low-data regimes, achieving state-of-the-art results with as few as 45 labeled examples for fine-tuning on RNA half-life prediction. Shoutout to the amazing leading authors Phil (Phil Fradkin) and Ian (Ian Shi)! Also the work is impossible without an outstanding collaboration by Karina (Karin(a) Isaev), Brendan (Brendan Frey) , Quaid (Quaid Morris), Leo J. Lee! Vector Institute University Health Network U of T Department of Computer Science Temerty Centre for AI in Medicine (T-CAIREM) Department of Laboratory Medicine & Pathobiology

🚀 We're thrilled to introduce Orthrus 🧬🐕—a groundbreaking mature RNA foundation model designed to push the boundaries of RNA property prediction! 🔬 What is Orthrus? Orthrus is a Mamba-based RNA foundation model, pre-trained using a novel self-supervised contrastive learning objective with biologically inspired augmentations. It optimizes the similarity between splicing isoforms and orthologous transcripts, capturing functional and evolutionary relationships to enhance mature RNA property prediction accuracy. 📑 Preprint: 💻 Code: 🌐 Project Page: 📦 Model Weights: 🧠 Why Orthrus? Decoding the RNA regulatory code is key to understanding biology, but traditional experimental approaches are slow and costly. Existing genomic foundation models rely on techniques like masked language modeling or next-token prediction, which aren't fully aligned with the complexities of genomic data—leading to suboptimal results. 🌟 Orthrus Highlights: - Biologically-Informed Contrastive Learning 🧪: A novel contrastive learning objective designed specifically for genomics, maximizing similarity between splicing isoforms and orthologous transcripts across species. - Extensive Pre-training 📊: Trained on splicing annotations from 10 species and orthologous alignments from 400+ mammalian species (Zoonomia Project), with a focus on sequences of high functional importance. - Superior Representations🏅: Orthrus outperforms existing genomic models on 5 mRNA property prediction tasks, often surpassing supervised methods with just a simple linear transformation. - Efficiency in Low-Data Settings📉: Orthrus excels in low-data regimes, achieving state-of-the-art results with as few as 45 labeled examples for fine-tuning on RNA half-life prediction. Shoutout to the amazing leading authors Phil (Phil Fradkin) and Ian (Ian Shi)! Also the work is impossible without an outstanding collaboration by Karina (Karin(a) Isaev), Brendan (Brendan Frey) , Quaid (Quaid Morris), Leo J. Lee! Vector Institute University Health Network U of T Department of Computer Science Temerty Centre for AI in Medicine (T-CAIREM) Department of Laboratory Medicine & Pathobiology

Bo Wang

114,924 Aufrufe • vor 1 Jahr