Inside Vercel's Agent Desktop, Plus Decision Models: AI Collective Webinar (w/ The AI Collective)

AI agents are getting good at code. The next frontier is the rest of your computer. Part 1: Inside Agent Desktop Agent Desktop is an open-source CLI from Vercel Labs that lets AI agents see and operate real desktop apps on macOS, Windows and Linux. It reads the operating system's accessibility tree, not screenshots, and gives your agent a reference for every button and field it can act on. I'll show how it works today, as Vercel built it. Part 2: Adding decision models Then I'll share my own experiment: adding decision models on top. Decision models answer typed questions with calibrated probabilities instead of writing text. That can mean confidence-gated actions, faster and cheaper next-step choices, and an agent that knows when to ask a human. I'll walk through Jev from TypeSafe AI, and test it head to head against Nace.AI's Drex. This layer is my work, not part of Agent Desktop. What you'll learn How Agent Desktop works: snapshot, act on a ref, verify What decision models are and how they differ from LLMs How Jev and Drex compare on real desktop tasks How to gate agent actions on confidence, so risky steps wait for a human Where this setup works today, and where it still breaks Who it's for Builders, engineers and operators automating work across desktop apps. Format A live demo of both parts, then open Q&A. Streaming live on LinkedIn, YouTube and X. The join link is coming soon. Hosted by AJ Green, President & Executive Director of The AI Collective.