We Tried Replacing 1000 Human Jobs with AI

The results were shocking—and not for the reasons you’d think. Here's what happened when we tried replacing 1000 freelancers with AI...

How close are we to Economic AGI? Despite all the recent talk and hype around AGI, nobody has a clue or benchmark.

I previously built a $1B+/yr marketplace, and so I wanted to know: What % of human jobs on UpWork & Freelancer .com could be solved using AI today? That’s our proxy for an Economic AGI benchmark.

Ryan Brandt and I scraped over 1,000 latest job postings and used the latest AI models (o1, Claude, Gemini) and agents/tools (Windsurf, Axiom, etc.) to apply for jobs and attempt to complete tasks.

The results? AI could solve ~15% of tasks…but we made exactly $0. Here’s what we learned—and what this means for the future of work:👇

1. ~5% of jobs: AI could solve these in 1 shot (e.g., logo design, content writing, simple scripts) by simply pasting the request into ChatGPT
Example: Someone offered $750 to update a simple logo. Another paid $20/hr to convert text PDFs to Word. Many clients were simply unaware of any AI tools 🤯

2. ~10% of jobs: AI could solve these with agents/tools (e.g., storefronts, web scraping, browser automation).
🛠️ But the agent/tooling space is messy & unreliable. People just wanted to pay for working solutions.

3. ~5% of jobs were ironically about clients delegating AI tasks to humans. (eg, use AI voice generation tools to make a voiceover)
Clients want humans to "deal with it" rather than wrestling with agents themselves.

4. Most job descriptions themselves were detailed and well-written prompts, which we could directly just paste into ChatGPT!

So Why Did We Earn $0?
Even with AI’s power:
- Pay-to-play: Workers must pay to apply to jobs. Each bid alone cost $1+ just to apply
- Crowded market: Jobs attract 20+ bids, often from workers with thousands of 5 star reviews and decades of project experience
- Broken UX: Platforms aren’t built for AI-driven work

We applied to ~30 jobs (max limit on our plan), priced in the bottom 10th percentile, and shared full AI solutions upfront in 50% of bids.
Results:
- Less than 1/2 of bids were even opened
- Only 6 clients replied
- After multiple of back-and-forth clarifications and rework, we never got paid
- Net loss: $100 in credits + API fees

Lessons Learned
1. AI is here—but adoption is slow. People are stuck in old ways.
2. The AI tools/agents market is a mess. People want solutions, not more tools.
3. Traditional marketplaces aren't built for the AI economy. UpWork/Freelancer have absolute terrible UX for both sides:

I’ve built a $1B+ marketplace at Super.com and am deeply passionate about this space.
If you’re building in AI, Agents, or thinking about the future economic engine and AGI, I’d love to chat, help ideate and angel invest.