How to prepare and solve problem when deploying LLMs in the real world. You've gotta watch this. It's a 50-minute MIT lecture from the Comet team. Can't miss if you are building production systems.

@Cometml

@Cometml Real-world LLM deployments can be tricky, but this lecture breaks it down perfectly. Don't miss it! 🎓✨

@Cometml This MIT lecture is a must-watch for anyone deploying LLMs in production; it's packed with practical insights.

@Cometml Is this available on YouTube? It has a better video player and curation features

@Cometml Here are their tools they use. May have to check it out.

@Cometml Sorry, but this video doesn't prepare you to solve real production problems. It's too basic information. And I see a lack of real cases and knowledge. Honestly if you are already building with AI, this video does not add anything to the real day2day of a production environment

@Cometml Not to be a downer, but a lot of what they said seemed to be just common sense within a tech context. In addition, he seemed to be downplaying hallucinations, which to me is a perfect word. And shadowing LLM with the production one is a no-brainer.

@Cometml For fraud detection,data is highly imbalanced, accuracy on the validation set is silly and model are carefully calibrated at a specific operating range, usually via ROC or PRC plots. that slide on fraud looks very off the mark

@Cometml The title is misleading. Half of the lecture gives examples about poor ML implementations and the other half presents common examples of how LLMs fail. It does not discuss how businesses use LLMs in the real world, or the challenges associated with deploying LLM solutions.

@Cometml Great information!