Matthew Lam's banner

Matthew Lam

@mattlam_ • 3,734 subscribers

becoming an expert harness eng and sharing learnings https://t.co/OFoB1tETpy, https://t.co/4ozpoCTH3G, computer-use-mcp @codexreleases @pichangelog

Videos

Anya Rossi

sweetdream.ai

SweetDream.ai•Sponsored•Livecam

Watch Anya Live

Anya is streaming live right now! Join her private show and enjoy exclusive content.

Exclusive private shows

1.2k viewers online

Private Show

Join now for exclusive access

Free preview available • Premium content

got this in pi-gui but even cooler soon. Here's a sneak peek. Pi extensions will be able to create custom desktop views, so you can have your own fully customized Pi-GUI experience! You'll also be able to add custom cards that show up across the app, in the transcript, composer window, side bar, and others. Here's a demo extension, where you can trace the agent's last turn, tool calls, latency, and usage. Fully updated with Pi v1.0 Mario Zechner :)

got this in pi-gui but even cooler soon. Here's a sneak peek. Pi extensions will be able to create custom desktop views, so you can have your own fully customized Pi-GUI experience! You'll also be able to add custom cards that show up across the app, in the transcript, composer window, side bar, and others. Here's a demo extension, where you can trace the agent's last turn, tool calls, latency, and usage. Fully updated with Pi v1.0 Mario Zechner :)

33,191 views • 2 days ago

introducing OpenBench v1, an open framework for measuring AI performance and efficiency for your codebase and use case. Companies are realizing that you can't simply tokenmaxx, and are frantically looking for better ways to use and measure AI use and efficiency. One approach is evidenced by the rise in model routers: OpenRouter, Cloudflare, Databricks, Vercel, and now Ramp to name a few. But they'll also need the ability to evaluate how agents are performing in their actual use cases and codebase. A good example is DoorDash's recent evals on their codebase and prs. This will become even more important as companies explore different model + harness combinations. OpenBench will be built to this direction, focusing on the cross between correctness vs efficiency in both token use and latency. Starting off, OpenBench makes it easy for anyone to add to the task set, and run a variety of harness + models. The framework also makes it easy to add any custom harness variant, for example I've been testing codex variants with ablations against the stock codex harness. OpenBench will also have tooling to help with discovering/adding reliable tasks from your repo. Today I have integrated harnesses: codex, claude, cursor, devin, grok build, pi, and eval'ed them for different models like gpt 5.6-sol, GLM 5.2, Kimi K3, Grok 4.5, etc. measuring correctness %, token use (in/out/cache), and latency.

introducing OpenBench v1, an open framework for measuring AI performance and efficiency for your codebase and use case. Companies are realizing that you can't simply tokenmaxx, and are frantically looking for better ways to use and measure AI use and efficiency. One approach is evidenced by the rise in model routers: OpenRouter, Cloudflare, Databricks, Vercel, and now Ramp to name a few. But they'll also need the ability to evaluate how agents are performing in their actual use cases and codebase. A good example is DoorDash's recent evals on their codebase and prs. This will become even more important as companies explore different model + harness combinations. OpenBench will be built to this direction, focusing on the cross between correctness vs efficiency in both token use and latency. Starting off, OpenBench makes it easy for anyone to add to the task set, and run a variety of harness + models. The framework also makes it easy to add any custom harness variant, for example I've been testing codex variants with ablations against the stock codex harness. OpenBench will also have tooling to help with discovering/adding reliable tasks from your repo. Today I have integrated harnesses: codex, claude, cursor, devin, grok build, pi, and eval'ed them for different models like gpt 5.6-sol, GLM 5.2, Kimi K3, Grok 4.5, etc. measuring correctness %, token use (in/out/cache), and latency.

124,151 views • 2 months ago

Built out a yolo /remote-control in the Codex cli using /goal. - /remote-control starts a tiny server on laptop - generates fresh token and qr code - phone connects through webapp - full sync between phone and laptop codex - touch grass A glimpse into what OpenAI will cook up, we're just waiting on that Codex iOS now 👀 Also a demo of how powerful /goal is, especially prototyping, most of this was done overnight by Codex while I was sleeping lol

Built out a yolo /remote-control in the Codex cli using /goal. - /remote-control starts a tiny server on laptop - generates fresh token and qr code - phone connects through webapp - full sync between phone and laptop codex - touch grass A glimpse into what OpenAI will cook up, we're just waiting on that Codex iOS now 👀 Also a demo of how powerful /goal is, especially prototyping, most of this was done overnight by Codex while I was sleeping lol

106,802 views • 4 months ago

not enough people know about /side and it's underrated. Think claude's /btw but add: - multiple message followups allowed - you can spin up > 1 /side chats ask questions while the agent is working on your main session, or ask questions to explore other directions, and it won't affect your session's context!

not enough people know about /side and it's underrated. Think claude's /btw but add: - multiple message followups allowed - you can spin up > 1 /side chats ask questions while the agent is working on your main session, or ask questions to explore other directions, and it won't affect your session's context!

75,389 views • 4 months ago

Pi-GUI now has computer use and a build in integrated terminal! v0.1.0-beta.26 To showcase the power of Mario Zechner 's Pi and OSS, I saw a pi computer-use extension on X, asked my agent to audit the extension for security, then asked it to patch pi-gui to make it compatible. Crazy how easy it was to find an oss extension for a feature I wanted, then make pi-gui work with the feature I want. Links below

Pi-GUI now has computer use and a build in integrated terminal! v0.1.0-beta.26 To showcase the power of Mario Zechner 's Pi and OSS, I saw a pi computer-use extension on X, asked my agent to audit the extension for security, then asked it to patch pi-gui to make it compatible. Crazy how easy it was to find an oss extension for a feature I wanted, then make pi-gui work with the feature I want. Links below

65,535 views • 5 months ago

Building out a Pi GUI app that mimics the Codex app ux for Pi fans out there, happy to share a demo that now includes: - multi parallel sessions - worktrees - notification settings - slash commands - skills panel A core design principle I'll be working toward is maintaining Pi's extensibility, but hoping this looks exciting for Pi users out there. What else in Pi would you like to see included? Built on Mario Zechner pi-mono

Building out a Pi GUI app that mimics the Codex app ux for Pi fans out there, happy to share a demo that now includes: - multi parallel sessions - worktrees - notification settings - slash commands - skills panel A core design principle I'll be working toward is maintaining Pi's extensibility, but hoping this looks exciting for Pi users out there. What else in Pi would you like to see included? Built on Mario Zechner pi-mono

57,348 views • 6 months ago

This is crazy, I can talk to OpenAI's realtime API, and it'll respond back with voice, and then help call Codex to help me code! Got curious about Codex and its realtime integration, and found codex's feature flag `realtime_conversation` integrated for the codex app-server. How it works: - launch codex app-server with realtime - connect a client to realtime through WebRTC voice connection - Codex app-server is also attached on the side - talk to realtime - realtime decides whether coding is needed, and responds back with events - Codex app-server takes these events and runs codex agent turns - codex writes code like your codex cli exp Keep in mind OpenAI hasn't released this officially, they're still actively working on it, I just created a fork. This is a completely new UX than the usual chat interface for agentic coding. For example, when the agent responds back in voice, it's actually faster for me to read the response, and I don't actually find voice response that useful unless it does so in the background. But image realtime working in the background, you can talk to it, realtime keeps track of the different agents working on your tasks and your status. More to come on demo side, i'll make a more polished version and include voice, but thought this was cool to share. You can see in the responses the distinction between my voice input, realtime response, and realtime calling Codex to do work!

This is crazy, I can talk to OpenAI's realtime API, and it'll respond back with voice, and then help call Codex to help me code! Got curious about Codex and its realtime integration, and found codex's feature flag `realtime_conversation` integrated for the codex app-server. How it works: - launch codex app-server with realtime - connect a client to realtime through WebRTC voice connection - Codex app-server is also attached on the side - talk to realtime - realtime decides whether coding is needed, and responds back with events - Codex app-server takes these events and runs codex agent turns - codex writes code like your codex cli exp Keep in mind OpenAI hasn't released this officially, they're still actively working on it, I just created a fork. This is a completely new UX than the usual chat interface for agentic coding. For example, when the agent responds back in voice, it's actually faster for me to read the response, and I don't actually find voice response that useful unless it does so in the background. But image realtime working in the background, you can talk to it, realtime keeps track of the different agents working on your tasks and your status. More to come on demo side, i'll make a more polished version and include voice, but thought this was cool to share. You can see in the responses the distinction between my voice input, realtime response, and realtime calling Codex to do work!

17,674 views • 4 months ago

No more content to load