Kevin Feng

📢 New paper, published by @knightcolumbia.org We often talk about AI agents augmenting vs. automating work, but how exactly can different configurations of human-agent interaction look like? We introduce a 5-level framework for AI agent autonomy to unpack this. 🧵👇

Finally, autonomy levels can help with agentic safety evaluations and setting thresholds for autonomy risks in safety frameworks. How do we know what level an agent is operating at, and what appropriate risk mitigations should be put in place? Read our paper for more!

📢 New paper, published by @knightcolumbia.org We often talk about AI agents augmenting vs. automating work, but how exactly can different configurations of human-agent interaction look like? We introduce a 5-level framework for AI agent autonomy to unpack this. 🧵👇

Why have this framework at all? We argue that agent autonomy can be a deliberate design decision, independent of the agent's capability or environment. This framework offers a practical guide for designing human-agent & agent-agent collaboration for real-world deployments.

📢 New paper, published by @knightcolumbia.org We often talk about AI agents augmenting vs. automating work, but how exactly can different configurations of human-agent interaction look like? We introduce a 5-level framework for AI agent autonomy to unpack this. 🧵👇

Level 4: user as an approver. The agent only requests user involvement when it needs approval for a high-risk action (e.g., writing to a database) or when it fails and needs user assistance. Example: most coding agents (Cursor, Devin, GH Copilot Agent, etc.).

📢 New paper, published by @knightcolumbia.org We often talk about AI agents augmenting vs. automating work, but how exactly can different configurations of human-agent interaction look like? We introduce a 5-level framework for AI agent autonomy to unpack this. 🧵👇

Level 2: user as a collaborator. The user and the agent collaboratively plan and execute tasks, handing off information to each other and leveraging shared environments and representations to create common ground. Example: Cocoa (arxiv.org/abs/2412.10999).

📢 New paper, published by @knightcolumbia.org We often talk about AI agents augmenting vs. automating work, but how exactly can different configurations of human-agent interaction look like? We introduce a 5-level framework for AI agent autonomy to unpack this. 🧵👇

Web: knightcolumbia.org/content/leve.... arXiv: arxiv.org/abs/2506.12469. Co-authored w/ David McDonald and @axz.bsky.social . Summary thread below.

🤖LLMs are being integrated everywhere, but how do we ensure they're delivering meaningful user experiences? In our #chi2025 paper, we empower designers to think about this via 🎨designerly adaptation🎨 of LLMs and built a Figma widget to help! 📜 arxiv.org/abs/2401.09051 🧵👇

Thanks to my fantastic collaborators @qveraliao.bsky.social @ziangxiao.bsky.social @jennwv.bsky.social @axz.bsky.social and David McDonald. This work was done during a summer internship with the FATE team at Microsoft Research.

🤖LLMs are being integrated everywhere, but how do we ensure they're delivering meaningful user experiences? In our #chi2025 paper, we empower designers to think about this via 🎨designerly adaptation🎨 of LLMs and built a Figma widget to help! 📜 arxiv.org/abs/2401.09051 🧵👇

Finally, Canvil's seamless integration into Figma allowed designers to use their existing collaborative workflows to work with design and non-design teammates on model adaptatation. We believe lowering the barrier to adaptation can catalyze more thoughtfully designed LLM UX.

🤖LLMs are being integrated everywhere, but how do we ensure they're delivering meaningful user experiences? In our #chi2025 paper, we empower designers to think about this via 🎨designerly adaptation🎨 of LLMs and built a Figma widget to help! 📜 arxiv.org/abs/2401.09051 🧵👇

We used Canvil as a probe to better understand how designers would make use of designerly adaptation in practice. We found that using Canvil, designers identified clear opportunities to tweak their designs based on observed model behaviors to enhance user interaction with LLMs.

🤖LLMs are being integrated everywhere, but how do we ensure they're delivering meaningful user experiences? In our #chi2025 paper, we empower designers to think about this via 🎨designerly adaptation🎨 of LLMs and built a Figma widget to help! 📜 arxiv.org/abs/2401.09051 🧵👇

We interviewed 12 designers at Microsoft to learn how they design LLM-powered UX. We found that designers often thought deeply about how to adapt model behavior to improve UX, but lacked the means to actually experiment with model adaptation within their workflows.