Matthew Lam
@mattlam_ • 3,044 subscribers
Builder, eng. https://t.co/OFoB1tETpy, https://t.co/4ozpoCTH3G, computer-use-mcp @codexreleases @pichangelog
Videos
introducing OpenBench v1, an open framework for measuring AI performance and efficiency for your codebase and use case. Companies are realizing that you can't simply tokenmaxx, and are frantically looking for better ways to use and measure AI use and efficiency. One approach is evidenced by the rise in model routers: OpenRouter, Cloudflare, Databricks, Vercel, and now Ramp to name a few. But they'll also need the ability to evaluate how agents are performing in their actual use cases and codebase. A good example is DoorDash's recent evals on their codebase and prs. This will become even more important as companies explore different model + harness combinations. OpenBench will be built to this direction, focusing on the cross between correctness vs efficiency in both token use and latency. Starting off, OpenBench makes it easy for anyone to add to the task set, and run a variety of harness + models. The framework also makes it easy to add any custom harness variant, for example I've been testing codex variants with ablations against the stock codex harness. OpenBench will also have tooling to help with discovering/adding reliable tasks from your repo. Today I have integrated harnesses: codex, claude, cursor, devin, grok build, pi, and eval'ed them for different models like gpt 5.6-sol, GLM 5.2, Kimi K3, Grok 4.5, etc. measuring correctness %, token use (in/out/cache), and latency.
Matthew Lam121,718 次观看 • 29 天前
Built out a yolo /remote-control in the Codex cli using /goal. - /remote-control starts a tiny server on laptop - generates fresh token and qr code - phone connects through webapp - full sync between phone and laptop codex - touch grass A glimpse into what OpenAI will cook up, we're just waiting on that Codex iOS now 👀 Also a demo of how powerful /goal is, especially prototyping, most of this was done overnight by Codex while I was sleeping lol
Matthew Lam105,741 次观看 • 3 个月前
not enough people know about /side and it's underrated. Think claude's /btw but add: - multiple message followups allowed - you can spin up > 1 /side chats ask questions while the agent is working on your main session, or ask questions to explore other directions, and it won't affect your session's context!
Matthew Lam75,389 次观看 • 3 个月前
Pi-GUI now has computer use and a build in integrated terminal! v0.1.0-beta.26 To showcase the power of Mario Zechner 's Pi and OSS, I saw a pi computer-use extension on X, asked my agent to audit the extension for security, then asked it to patch pi-gui to make it compatible. Crazy how easy it was to find an oss extension for a feature I wanted, then make pi-gui work with the feature I want. Links below
Matthew Lam65,535 次观看 • 3 个月前
Building out a Pi GUI app that mimics the Codex app ux for Pi fans out there, happy to share a demo that now includes: - multi parallel sessions - worktrees - notification settings - slash commands - skills panel A core design principle I'll be working toward is maintaining Pi's extensibility, but hoping this looks exciting for Pi users out there. What else in Pi would you like to see included? Built on Mario Zechner pi-mono
Matthew Lam57,348 次观看 • 4 个月前
This is crazy, I can talk to OpenAI's realtime API, and it'll respond back with voice, and then help call Codex to help me code! Got curious about Codex and its realtime integration, and found codex's feature flag `realtime_conversation` integrated for the codex app-server. How it works: - launch codex app-server with realtime - connect a client to realtime through WebRTC voice connection - Codex app-server is also attached on the side - talk to realtime - realtime decides whether coding is needed, and responds back with events - Codex app-server takes these events and runs codex agent turns - codex writes code like your codex cli exp Keep in mind OpenAI hasn't released this officially, they're still actively working on it, I just created a fork. This is a completely new UX than the usual chat interface for agentic coding. For example, when the agent responds back in voice, it's actually faster for me to read the response, and I don't actually find voice response that useful unless it does so in the background. But image realtime working in the background, you can talk to it, realtime keeps track of the different agents working on your tasks and your status. More to come on demo side, i'll make a more polished version and include voice, but thought this was cool to share. You can see in the responses the distinction between my voice input, realtime response, and realtime calling Codex to do work!
Matthew Lam17,674 次观看 • 2 个月前
没有更多内容可加载