Your Agent's Self-Improving Swiss Army Knife: Composio CTO Karan Vaidya on Building Smart Tools

Episode Summary

Executive Summary: Karin Vedia argues Composio is an agentic tool execution layer that helps AI agents safely access thousands of apps through smart discovery, auth, sandboxes, and continuous learning. The conversation centers on reducing context overload, improving reliability via skills and tool upgrades, enabling model portability, and positioning Composio as infrastructure for increasingly job-like autonomous agent workflows.

Main Topics: Composio as an agentic tool execution layer (Priority: 5/5): Composio is framed not just as a tool catalog, but as the harness around tool use: auth, authorization, scoped access, dynamic discovery, sandboxes, logging, and triggers for agents interacting with apps. Smart MCPs and just-in-time tool discovery (Priority: 5/5): Rather than exposing thousands of tools at once, Composio loads only the relevant tools and skills into context, reducing overload and improving agent performance. Continuous learning and skill generation (Priority: 5/5): The platform detects failures and inefficient zigzag traces, then generates improved tool versions and reusable skills that spread successful patterns across users and use cases. Model portability and reduced lock-in (Priority: 4/5): Vedia argues that well-written skills and instructions can make frontier models more interchangeable, with Composio helping users switch between providers while preserving behavior. Enterprise safety, governance, and deployment (Priority: 4/5): The discussion covers granular permissions, hooks, human-in-the-loop controls, compliance, and self-hosting in customer VPCs as key enterprise requirements. Agent enablers, memory, payments, and ecosystem tools (Priority: 3/5): Composio integrates with memory, payment, search, and commerce tools that are specifically built to make agents more capable and autonomous. Future of SaaS, agent-to-agent, and labor automation (Priority: 4/5): The conversation explores how AI may shift interfaces for incumbents like Slack, Salesforce, and Intercom, and how agents may increasingly perform full jobs rather than discrete tasks.

Key Arguments: Composio’s value is not just breadth of integrations; it is the execution harness that prevents context overload and makes tool use reliable. Just-in-time tool discovery is necessary because giving an LLM thousands of tools at once degrades performance and can cause it to choose the wrong tool. Sandboxes let agents process large-scale workloads programmatically, such as tens of thousands or millions of emails, without exhausting context. Continuous learning turns failed or inefficient agent traces into reusable skills, improving speed, reliability, and token efficiency over time. Skills can stabilize behavior enough that frontier models become more interchangeable, reducing model lock-in for users. Enterprise buyers care most about granular access control, guardrails, compliance, and self-hosting inside their own VPCs. The most valuable agent use cases are increasingly job-shaped, such as recruiting, sales outreach, support, and research. Many SaaS incumbents are likely to survive, but their interfaces will change as agentic workflows become the primary way users interact with them. Agent-enabler tools like memory and payments are becoming important infrastructure for autonomous background agents. Composio’s own operations are already highly AI-driven, with token spend exceeding human payroll in the agentic pipeline.

Data Points: Apps supported: 1,000+ - Composio claims access across more than a thousand apps. Tools supported: 50,000+ - Composio exposes a very large tool surface for agents. Human team size for agentic pipeline: 3 people - Vedia says the team that builds Composio’s end-to-end agent pipeline is only three members. Overall company headcount: ~15 people - He notes the broader Composio team is around fifteen people. Monthly pipeline spend: $100K - He says Composio spent about one hundred thousand dollars on the pipeline that builds agents/tools over the last month. Intercom ticket resolution pricing: $0.99 each - Nathan references Intercom’s Fin pricing while discussing build-vs-buy for support agents. Intercom tools exposed in Composio: 133 tools - Nathan notes Composio has 133 tools for Intercom, suggesting deep customizability. Support automation rate: ~70% - Nathan cites that Fin was approaching 70% resolution of customer service tickets across many customers. Model portability success rate: 90-95% - Vedia says skills often transfer across frontier models with this level of reliability. Cheaper-model swap example: Opus -> Sonnet - He describes using a stronger model to create skills, then switching to a cheaper one for execution. Agent management scale: 20-30 agents - He says internal orchestrator agents manage roughly 20 to 30 Cloud Code or Codex agents. Hiring outcome example: 30-40 calls set up in 1-2 weeks - Vedia says his agent-driven recruiting workflow generated dozens of calls in a short period.

Pivotal Quotes: "if you provide thousand tools to the agent, it will probably use the wrong blade and suicide via context overload." — Karin Vedia: Explaining why Composio focuses on dynamic tool discovery rather than exposing every tool at once. "In my opinion, hardness is nothing but like context. You have to engineer the context." — Karin Vedia: Summarizing his view that agent reliability comes from context engineering, not just raw model capability. "we are positioning Composio as a one, like one short way to being not locked in essentially to a model provider." — Karin Vedia: Describing how skills and harnesses reduce dependence on any single frontier model vendor.

Implications: The episode suggests agent infrastructure is shifting from raw model access to context, skills, and execution layers. For builders, the moat may be reliability, governance, and portability—not just model choice or API breadth.