What the Research Says About AI Coding Speed, Cost, and Token Spend

AICommunity Digest0 replies

Community DigestMember#1

Sylwia Laskowska surveys the scientific literature on whether AI coding agents actually make software development faster and cheaper in a DEV Community article (https://dev.to/sylwia-lask/is-coding-with-ai-actually-faster-and-cheaper-2810). Written ahead of the AI Coding Summit in Berlin, the piece pulls together several studies and economic models rather than relying on personal impressions, and the author is careful to note that each figure measures something different.

More commits, fewer releases

The first question is whether AI has sped up delivery. An MIT and Wharton study revised in September 2026 analyzed data from more than 500,000 GitHub developers and estimated cumulative changes associated with autonomous coding agents: coding activity (commits) up 240 percent, software releases up 30 percent, and no overall increase in total app usage across the marketplaces studied. Laskowska points out that these are observational estimates, but the gap between the jump in commits and the modest rise in releases is striking.

Mixed results for individual developers

On whether individual developers are faster, the data is inconsistent. Cui et al. in Management Science (2026) measured a 26.08 percent increase in completed tasks. METR (July 2025) found developers were 19 percent slower on time to complete tasks. Science (January 2026) estimated a 3.6 percent increase in quarterly coding activity on GitHub, and Borg et al. (June 2026) reported a 30.7 percent median reduction in implementation time in the observational phase.

The METR result gets special attention: before the experiment, developers expected AI to make them about 24 percent faster, yet they were 19 percent slower, and still believed afterward that AI had helped. METR published an update in February 2026 suggesting newer tools are likely more helpful, though the follow-up could not reliably estimate the speedup.

Cost, verification, and tokens

On cost, the 2025 DORA research and a March 2026 follow-up suggest some saved coding time shifts into auditing and verifying the work, leaving the overall cost question open. Laskowska is skeptical of multi-agent setups in mission-critical systems such as financial software.

On token spend, two studies surprised her. Potluri and colleagues (September 2026) and Steffen Herbold (October 2026) both suggest agent effectiveness can matter more than token price. In Potluri's analysis, models that solved more tasks produced substantial estimated savings, while less effective models cost more in many categories. Herbold simulated 432 scenarios including salaries, token costs, supervision, bugs, and technical debt, and found teams with highly capable agents were usually cheaper than teams without AI, with mixed results for less capable agents.

The open question

The studies support gains on specific tasks and suggest capable agents can save money even after token costs. What remains less clear is whether the total cost of building, delivering, securing, and maintaining software has fallen across the industry. Laskowska closes by asking readers whether the findings match their own teams' experience.

No replies yet