iTechGuides is reader-supported. When you buy through links on our site, we may earn an affiliate commission. As an Amazon Associate I earn from qualifying purchases. Learn more
AI can make an individual coding task faster and still fail to improve—and sometimes appear to worsen—team productivity. The reason is that “productivity” changes with the task, codebase, developer, tool version and metric. A controlled Copilot experiment found a bounded JavaScript task completed 55.8% faster, while a 2025 randomized trial of experienced open-source maintainers working in repositories they knew found that allowing contemporary AI tools made tasks take 19% longer. Those results are not mathematically contradictory: they measure different work in different conditions.
What the evidence actually shows
There is no single, stable “AI productivity effect.” The strongest findings in the available studies are specific to their settings:
| Study | Participants and setting | AI and task | Measured result | What it does not establish |
|---|---|---|---|---|
| METR randomized trial (2025) | 16 experienced open-source developers, 246 tasks, mature projects they maintained; average of five years’ project experience | AI use randomly allowed or disallowed; tools available February–June 2025, primarily Cursor Pro with Claude 3.5/3.7 Sonnet | Completion took 19% longer when AI was allowed | A universal claim that AI is harmful or that it causes the same slowdown in every team |
| Microsoft Research controlled experiment (2023) | Developers assigned a bounded JavaScript HTTP-server implementation task | GitHub Copilot versus a control condition | Copilot participants completed the task 55.8% faster | End-to-end delivery speed, maintenance work or whole-team productivity |
| GitHub task experiment (2022; report updated 2024) | 95 developers completing a defined programming task | Copilot versus control | Average completion time was 1 hour 11 minutes versus 2 hours 41 minutes, described by GitHub as 55% faster | An independent, cross-industry estimate or a guarantee for production repositories |
| GitHub–Accenture enterprise study (2024) | Organization telemetry, adoption analysis and user surveys | Copilot adoption in enterprise teams | 8.69% more pull requests per developer and a 15% higher pull-request merge rate in the reported analysis | Faster lead time, higher reliability, lower coordination cost or greater customer value |
The METR authors—Joel Becker, Nate Rush, Beth Barnes and David Rein—summarize their result plainly: “Surprisingly, we find that allowing AI actually increases completion time by 19%—AI tooling slowed developers down.” Their participants had predicted a 24% reduction before starting and later estimated a 20% reduction, illustrating how expectations can diverge from observed performance.
Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallCrashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteWhy the two speed results are not a contradiction
Bounded implementation versus real repository work
The Copilot experiments used a self-contained programming exercise with a clearly defined finish line: implement a JavaScript HTTP server. Such a task concentrates effort on writing and completing code. METR’s developers instead handled real issues in established projects, where understanding local conventions, tests, dependencies, history and edge cases is part of the job. A suggestion that is syntactically correct can still be wrong for a mature codebase.
#1 Best Overall
Familiarity changes where the work happens
METR deliberately studied maintainers in repositories they knew well—participants averaged about five years of experience with those projects. Familiarity can make unaided navigation and design decisions efficient, leaving an assistant with fewer obvious wins. In an unfamiliar codebase, generated explanations, search and scaffolding may save substantial orientation time. The studies do not provide a head-to-head ranking of tools or a universal familiarity adjustment; they show why context must be recorded.
Tool generation and model behavior matter
The Copilot experiment represents an earlier tool and model era than METR’s February–June 2025 environment, which primarily used Cursor Pro with Claude 3.5 or 3.7 Sonnet. Completion speed from one generation should not be transferred unchanged to another. Latency, context-window behavior, codebase indexing, autonomous edits and error rates all alter the amount of supervision a developer must provide.
Typing time is only one component
An assistant can reduce keystrokes while adding prompt writing, context selection, waiting, diff inspection, test runs and correction. That combination is a plausible explanation for a slowdown in complex maintenance work, but the METR result alone does not identify which mechanism caused it. Treat it as a hypothesis to test in your own workflow, not as a settled law.
How an individual speedup can disappear at team level
Teams deliver a chain of work, not isolated code snippets. A faster implementation matters only if it survives review, integration, testing, release and operation.
- Review load: More generated code can increase the number of lines, assumptions and security questions reviewers must verify.
- Integration conflicts: Parallel AI-assisted changes may touch the same interfaces or files, creating rework rather than throughput.
- Quality and reliability: A pull request that merges quickly but introduces defects can lengthen the true delivery cycle through rollback and incident work.
- Coordination: Teams still need design decisions, ownership, documentation and communication. Faster local edits do not remove those dependencies.
- Onboarding and skill effects: Novices may gain scaffolding, while experienced developers may spend more time correcting output that conflicts with project conventions.
- Review bottlenecks: If reviewer capacity is fixed, increasing author output can enlarge a queue and make cycle time worse.
That is why the enterprise figures—8.69% more pull requests per developer and a 15% higher merge rate—are signals, not proof of faster end-to-end delivery. Pull-request activity and merge rate do not measure lead time, escaped defects, operational reliability or customer value.
What “team productivity” should include
GitHub’s discussion of the SPACE framework groups productivity into several dimensions rather than one activity count: satisfaction and well-being, performance, activity, communication and collaboration, and efficiency and flow. Use that broader lens when evaluating an assistant.
| Dimension | Useful questions | Examples of measures |
|---|---|---|
| Efficiency and flow | Does work move from start to production with less waiting? | Lead time, active time, blocked time, review turnaround |
| Performance | Does the delivered software meet its intended outcome? | Defect rate, change-failure rate, reliability, acceptance results |
| Activity | What work is being produced, and is the volume meaningful? | Commits, pull requests, review events—interpreted as activity, not value |
| Communication and collaboration | Are decisions and handoffs clearer? | Review participation, unresolved handoffs, design rework, documentation completeness |
| Satisfaction and well-being | Is the workflow sustainable for developers? | Short recurring surveys, interruption burden, perceived autonomy and frustration |
No single metric should be used to rank developers. Compare teams or time periods with the same definitions, account for project mix, and protect individual privacy.
Recommended Free Tools
A practical way to measure an AI coding rollout
1. Define the delivery outcome first
Choose an outcome your organization actually values, such as time from first commit to production, change-failure rate, or validated feature completion. Write the definition before enabling the tool so the metric cannot be changed to fit a favorable result.
2. Establish a baseline
Collect several comparable delivery cycles without changing the normal workflow. Record repository, language, task type, developer experience, review policy and release process. A sprint containing mostly incident fixes is not a fair baseline for a sprint containing new product work.
3. Log the work stages
For each sampled change, capture time in implementation, waiting for CI, waiting for review, review and rework, merge, deployment and post-release correction. This separates a coding-time change from a queue or quality change.
4. Compare like with like
Stratify results by task complexity, codebase familiarity, developer experience, AI familiarity and tool/model version. Keep an AI-allowed and comparison group when feasible, or use a phased rollout with predeclared analysis. Do not average a small toy task with a large migration and call the result “AI productivity.”
5. Track quality and human cost
Pair speed measures with escaped defects, rollback or incident rates, test effectiveness, review hours and a short satisfaction survey. Include time spent fixing generated code; otherwise the apparent gain is only displaced work.
Best Value
- New
- Mint Condition
- Dispatch same day for order received before 12 noon
- Guaranteed packaging
- No quibbles returns
6. Set a review date and a stop rule
After a defined period, ask whether delivery outcomes improved without unacceptable quality or well-being costs. Pause or narrow the rollout if defects, review queues or operational load rise, even when activity metrics look positive.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.When AI is most likely to help—and when to be cautious
Higher-probability use cases
- Well-specified, bounded tasks with an objective test or acceptance check.
- Boilerplate, repetitive transformations and familiar language patterns.
- Drafting tests, documentation or example code that a knowledgeable developer can verify quickly.
- Explaining an unfamiliar API when the answer can be checked against authoritative documentation.
Higher-risk use cases
- Cross-cutting changes in mature repositories with implicit conventions.
- Security-, privacy- or safety-sensitive code where plausible output needs specialist review.
- Large refactors with ambiguous requirements and many integration points.
- Changes whose cost is dominated by review, deployment or incident response rather than typing.
These are decision heuristics, not claims that a particular tool always succeeds or fails. Pilot them with the measurement plan above.
Common measurement mistakes
- Counting output as value: More commits or pull requests can mean smaller changes, duplicated work or extra review burden.
- Using forecasts as results: METR participants expected a 24% time reduction and later estimated 20%, yet observed a 19% increase in completion time in that setting.
- Mixing experiments and telemetry: A randomized task experiment supports a different inference from an observational enterprise report.
- Ignoring time horizon: Onboarding benefits may appear early, while maintenance and review costs emerge later.
- Comparing unlike tools: The Copilot and METR studies used different generations, interfaces and model configurations; their percentages cannot be averaged.
What the evidence can—and cannot—answer
Does AI make programmers faster? Sometimes, for a defined task and population; the 55.8% Copilot result is a concrete example. Can it make experienced developers slower? Yes, the METR trial observed 19% longer completion times under its 2025 repository conditions. Does AI generally make teams slower? The available evidence here does not establish that causal, universal conclusion. It does establish that individual task speed and pull-request activity are insufficient proxies for team delivery.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Scan for outdated or missing drivers - takes under a minute3Repair Windows errors before they cause bigger problemsThe defensible conclusion is conditional: evaluate an assistant as a change to an entire socio-technical system. Measure the work your team must finish— including review, integration, quality and operations—rather than assuming faster code generation is equivalent to faster delivery.
Quick Recap
Sources and further reading
- METR, “Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity” (2025)
- Microsoft Research, “The Impact of AI on Developer Productivity: Evidence from GitHub Copilot” (2023)
- GitHub, “Quantifying GitHub Copilot’s impact on developer productivity and happiness” (2022; updated 2024)
- GitHub, “Quantifying GitHub Copilot’s impact in the enterprise with Accenture” (2024)
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
