I Used AI Writing Tools for 3 Years.
Here’s the Ugly Truth.
$1,847 spent. 500+ hours logged. One hallucinated statistic that cost me a client’s trust. This isn’t a tools roundup — it’s a confession with receipts.
- AI tools cut first-draft time by 40–50%. They rarely cut total work time — they usually fill it with more tasks.
- Freelancers with agency over when and how they use AI report better outcomes than employees who have it imposed on them. This split is the most important finding in the research.
- The hidden costs — trust erosion from hallucinations, context-switching fatigue, wrist strain from more typing — don’t appear in any productivity benchmark.
- I cannot prove my $1,847 paid for itself. I stopped tracking when the answer started to look uncomfortable.
The Research I Used (And Three Studies I Rejected)
I read over 20 papers. Four made the cut — not because they confirmed what I believed, but because they challenged it. I’ll also tell you which ones I dismissed and why I might have been wrong to do so.
Aruna Ranganathan and Xingqi Maggie Ye embedded for 8 months inside a 200-person tech company. What they found: product managers writing code with no pay increases, “one last prompt” sessions bleeding into 8 pm, and mental exhaustion from constant AI-human toggling.
The quote that made me stop and stare at my wall for five minutes: “You had thought that maybe, oh, because you could be more productive with AI, you would save some time and work less. But no, you’re just doing more stuff.”
The freelancer/employee split is the most important finding I’ve encountered. Freelancers using AI: 90% faster skill acquisition, 40% higher rates, better self-reported well-being. Employees using AI: productivity gains plus record burnout (88% of high AI users in 2025).
Randomized trial, 16 experienced developers, 246 tasks. Result: 19% slower with AI tools. One developer with 50+ hours of Cursor experience showed a 38% speedup. The tools were early 2025 versions. Rejection rate: 44% of tasks.
I initially read “19% slower” as “AI doesn’t work,” then read the full paper and realized that was too simple. Novices showed the largest slowdowns. The tasks involved complex legacy codebases (1M+ lines). Whether my work is more or less complex than that — I genuinely don’t know.
55.8% faster on simple coding tasks with Copilot. I include this because it contradicts my narrative and I need you to know I know about it.
I almost left it out with “industry-funded, old, simple tasks only.” These are valid concerns. They’re also convenient excuses to protect my skepticism. The truth: I don’t know whether “simple tasks” means “most of what developers do” or edge cases. I could have researched this more. I didn’t.
Three Studies I Rejected (And Might Be Wrong To)
Claimed 60–70% productivity gains across 63 use cases. I rejected it for heavy reliance on expert interviews rather than measured outcomes. But 63 use cases is more than my sample of 4. My dismissal might be motivated reasoning — I didn’t want gains that high to be true because they don’t match my experience. I should have included it with heavier caveats.
Showed 35% productivity gains for customer service workers. I rejected it partly because customer service is highly structured. I also only read the abstract. That’s a methodological failure on my part, not the study’s.
Three Experiments (With Specific Failures)
Real experiment logs. Unedited. Including the parts where I failed and didn’t notice until writing this.
Experiment 1: AI-Assisted vs. Manual Writing (14 days)
Two weeks alternating AI-assisted (Claude) and manual days for client reports. Here’s the raw data I actually tracked:
| Date | Method | First Draft | Final Submit | Energy (1-10) | Client Rating | Note |
|---|---|---|---|---|---|---|
| Mar 15 | AI | 40 min | 130 min | 3 | 4/5 | Client asked “Did you write this?” |
| Mar 16 | Manual | 95 min | 105 min | 6 | 5/5 | Used AI for structure only |
| Mar 18 | AI | 50 min | 125 min | 4 | 4/5 | Tired from previous day |
| Mar 19 | Manual | 85 min | 110 min | 5 | 4.5/5 | Also tired |
| Mar 22 | AI | 45 min | 140 min | 3 | 3/5 | ⚠ Major failure — hallucinated stat |
| Mar 23 | Manual | 100 min | 105 min | 7 | 5/5 | Relief |
I was writing a market analysis for a fintech client. Claude suggested: “According to SEC filings, Fintech X grew 340% in Q3 2023.” I didn’t verify. I sent it.
The client emailed: “This number doesn’t appear in any SEC filing I can find. Please clarify the source.”
I spent 45 minutes panicking and searching before confirming Claude had synthesized multiple sources into a number that didn’t exist. The client didn’t fire me, but I lost the “trusted advisor” status I’d built over two years. The 140-minute total included rewriting the entire section manually, plus an apology call. That 45-minute panic? Not in the table.
Current status: I still use AI for first drafts. The 40–50 minute dopamine hit is hard to quit, even knowing the net time is similar or worse, even knowing I might hallucinate again. That’s not a productivity insight. That’s an addiction pattern worth naming.
Experiment 2: Notification Elimination (That I Abandoned)
Day 1: Turned off all AI notifications. Felt anxious by 10 am. Checked email 40 times (normally 15). Day 3: Scheduled AI use in blocks — 9–10 am, 2–3 pm. Actually felt more focused between sessions. Day 5: A client needed a “quick turnaround.” I turned notifications back on “temporarily.”
That was 8 months ago. The temporary became permanent.
Experiment 3: The “One Tool” Month
Commitment: Only Claude for everything. Abandoned on Day 23. Here’s what broke it: Day 5, I needed to transcribe a 90-minute interview. Claude doesn’t do transcription. Broke the rule, used Otter, felt guilty. Day 12, German grammar check — Claude’s German is worse than DeepL’s. Day 19, I calculated 4–5 hours wasted that week forcing Claude to do tasks it wasn’t built for.
What I actually learned: I don’t have enough control over my work environment to run proper experiments. That’s data about freelancing constraints, not about AI tools.
The Hidden Costs Nobody Benchmarks Speculative
Every productivity benchmark measures task time. None of them measure what I’ve actually experienced:
The agency split from the Upwork data starts to make sense here. When you control the tool — when you choose when, how, and whether — these costs are manageable. When the tool is imposed and targets get raised around it, you absorb all the costs with none of the choice.
What I Actually Think (Not “It Depends”)
“It depends on use case” is true. It’s also useless. Here’s my actual working model, labeled honestly:
The tool isn’t the variable — agency is. Probable Freelancers with control over when and how they use AI report better outcomes across every metric I’ve seen. Employees with AI imposed on them and targets raised around it report worse outcomes. This is the most actionable finding in all the research.
“Productivity” is underspecified in almost every benchmark. Established Task time? Output volume? Quality? Well-being? Long-term skill development? Each study picks different metrics. I pick different metrics depending on what I want to believe that day. This isn’t a flaw in the research — it’s a fundamental problem with how we think about this.
The efficiency gains are real and so are the rebound effects. Probable AI saves 40–50% on first drafts. That time doesn’t vanish into rest or leisure. It fills with more tasks, more prompting, more editing. The net change in total work is roughly zero, sometimes negative.
Tool quality evolves faster than research completion cycles. Speculative By the time we understand the costs of 2025 tools, 2027 tools will be different. The research will always lag. That means we’re flying partly blind, and the honest thing to do is say so.
What Could Be Wrong With This Article
- My experiment sample is n=1 (me). Freelance writer. Mixed technical/creative work. Pre-existing tendency toward overwork. Any of these factors could dominate the conclusions.
- I selected studies that surprised me. I explicitly filtered for papers that challenged my beliefs — which is also a bias. A truly balanced review would weight all evidence proportionally, not by how uncomfortable it made me feel.
- The fatigue correlation (r=0.4, n=180 days) is self-reported and not controlled. I do not know if AI causes my fatigue or if I use AI more when I’m already tired. Causation is unknowable from this data.
- Everything above describes freelance writing in 2024–2026. This almost certainly doesn’t generalize to software developers, customer service roles, structured data work, or anyone with meaningful organizational support for working boundaries.
- I rejected the McKinsey 60–70% gains data too easily. That rejection was convenient for my narrative. I’ve flagged it, but flagging bias doesn’t remove it.
Questions I Actually Get Asked
Sources
- Ranganathan & Ye — “AI and the Expanding Workload” — UC Berkeley / HBR, Feb 2026. Qualitative, n=1 company, tech sector. I haven’t verified raw interview data.
- Upwork Research: “AI’s Impact on Work” — 2024 & 2025 waves. Self-reported, industry-funded, metric inconsistency between waves.
- METR — “Evaluating Real-World Coding Agents” — July 2025. RCT, n=16, complex legacy codebases, early-2025 tools.
- GitHub / Microsoft — “Quantifying Copilot’s Impact” — 2023. Industry-funded, simple tasks only, “simple” undefined.