AI chat sample now streaming the chat replies across multiple browsers and can work with multiple replicas by publishing the chat stream via Redis (saving the full message to the db). Still very chatty (pun intended) and inefficient but fun to build! The video shows how aspire can amplify... with dev-ex by bringing together all of the tools required to make it easier to diagnose and interact with these sorts of applications. #dotnet #aspireshow more

Wonder if Orleans could be feasible for this? For example, to have a chat room grain that stores last xx messages in the grain state. Since grain, when activated stays in the memory for some time could be similar to Redis? For older messages, query the db through grain.

Probably overkill but try Orleans, no need for redis, simpler architecture.

Do you have this sample code in a repo?

Realtime apps are fun. My little side project is similar .NET 9/redis/SignalR live chat app, built loosely over OpenAI assistants API. How are you handling user sending 2nd message while current one is processing?

Before the latest change I was using a semaphore, but now that I moved to redis that’s not currently blocked. I need to reimplement mutual exclusion ( I also need to implement cancellation of the steam.

I used your repo on a brand new home pc. I had issues with podman, networking, but the Aspire part, getting started with another one's repo, worked like a charm.

There's no way to get a trace starting from the client app, right?

You can. I just didn’t do it.

Time to fine tune your model on Aspire deployment ?