The Execution Gap
A developer's critique of AI coding agents has exposed what might be the field's most overlooked limitation: most agents that claim to write code don't actually run it. According to a post in r/LocalLLaMA, the majority of AI coding tools are essentially "sophisticated code generators" that review and refine their output without ever executing it to verify it works.
This revelation comes as businesses are rapidly adopting AI automation tools across various domains. ASA Agent, a new tool managing Apple Search Ads, promises to handle marketing campaigns with just "10 minutes of setup" before running autonomously twice daily. The tool, which manages apps with over 1.5M downloads and $800K+ revenue, uses Claude AI to optimize ad bids based on actual revenue data rather than vanity metrics.
Real Deployment vs. Technical Reality
The contrast between marketing promises and technical implementation is stark. While ASA Agent charges a flat $49-99 per month as an alternative to agencies taking 15-20% of ad spend, the underlying question remains: how much validation happens behind the scenes?
A sales representative at a $10M+ company shared their Model Context Protocol (MCP) setup with Claude, integrating tools like Crustdata for list building, ZoomInfo for email verification, and Salesforce for CRM management. Their workflow represents the current state of AI business automation: human-supervised processes where AI assists rather than fully automates.
Building What's Missing
The r/LocalLLaMA developer's solution addresses the execution gap directly. Their system implements a multi-agent loop where:
- An Architect creates specifications
- A Coder implements the solution
- An Auditor actually executes the code in Docker and captures real errors
- The loop retries until the code passes
"The main innovation is using actual runtime errors as feedback instead of LLM opinions," the developer notes, explaining how this helps the system "converge on working solutions."
The Maturity Divide
This technical critique arrives as our prior coverage highlighted users' struggles with basic AI agent concepts while simultaneously trusting them with high-stakes tasks. The execution gap reveals another layer to this problem: even sophisticated users might not realize their AI coding assistants aren't validating their output.
The developer reports their execution-based system works well for small scripts but struggles with larger projects—a limitation that suggests we're still far from the autonomous coding future many vendors promise.
What's Next
As businesses rush to implement AI automation—from marketing campaigns to sales workflows—the divide between those who understand these tools' limitations and those who don't continues to widen. The ASA Agent's success in managing significant ad spend shows AI can deliver value in narrow, well-defined domains. But the coding agent critique suggests we need more honest discussions about what these tools actually do versus what they claim to do.
The trajectory is clear: while business adoption accelerates, technical reality is forcing a reckoning with AI agents' actual capabilities. The question isn't whether AI can automate workflows—it clearly can in specific contexts—but whether we're building the right validation and execution layers to make these automations trustworthy.