Henry’s Best Hits

Henry’s Best Hits

Lean AI at Scale: The Official Playbook for Scale-Ups and Growth Stage Companies

Based on proven strategies from Super.com's AI transformation to $200M+ annualized revenue

Henry Shi's avatar
Henry Shi
Jul 26, 2025
∙ Paid
33
4
Share

Table of Contents

  1. The Three-Pillar Framework

  2. Product Integration Strategies

  3. Building Internal AI Capabilities

  4. Productivity 2.0 Implementation

  5. Measurement and Results

  6. Future Strategic Initiatives (Super.com’s next AI bets)

  7. Key Lessons and Best Practices

A little back story….

Most AI success stories are marketing fluff. Generic case studies and surface-level insights that tell you what happened, but never how it actually worked.

This playbook is different.

For the first time, we are pulling back the curtain on a real AI transformation at scale.

For some background, I am a co-founder (now a board member) at Super.com.

This gives me unprecedented access to share what no outsider has ever seen: the complete, unfiltered playbook of how a $200M+ annualized revenue company achieved ~$1 million revenue per employee through systematic AI adoption.

Super.com has become the first scale-up to join the Lean AI Leaderboard and demonstrate what truly lean AI operations look like at this scale.

Since then, I have received countless messages from scale-ups and growth-stage founders asking for our playbook.

So, I am sharing our operational playbook (the one we use internally to guide our AI transformation) with complete transparency, no gatekeeping.

I'm giving you the exact roadmap we used, and everything you are about to read comes from our boardroom discussions and team implementations.

Shout-out to our CEO, Hussein, and our CTO, Ryan, for helping me bring this playbook to my readers.

Our Model: Proven Results at Scale

Super.com wasn't built as an AI company.

We started 9 years ago in the pre-AI era, grinding through the traditional playbook of hiring more people to solve more problems (although we started as a chatbot and won an IEEE award).

Like most companies, we scaled by adding headcount instead of getting more done with the resources we had

Then everything changed as we adopted AI.

Our transformation shows how growth-stage companies and scale-ups can systematically implement these principles to drive measurable business results.

With $200M+ annualized revenue and just over 200 employees, we have achieved remarkable outcomes through our structured approach to AI adoption.

This playbook distills our systematic approach into actionable strategies for similar organizations, providing a proven path from AI principles to measurable business transformation.

Our Three-Pillar Framework

We organize our AI strategy into three mutually reinforcing pillars:

Pillar 1: Product Integrations

Goal: Leverage AI in products to drive business and customer value

Our Implementations:

  • 40% of customer support chat contacts resolved by AI

  • Millions of incremental revenue using AI-optimized pricing

  • Hundreds of thousands saved annually on SEM keyword optimization

  • Low-code AI personalization - no two users will see the same app

  • Triaging and resolving customer-reported issues

  • AI-driven inventory consolidation, showing a 100% improvement in early testing

Pillar 2: Internal Expertise, Capabilities, Guidelines

Goal: Build company AI familiarity to leverage AI across use cases (technical and non-technical)

Our Structure:

  • Two AI guilds: Technical Guild and Adoption Guild

  • AI Lead role for internal strategy ownership

  • Internal AI Platform to make it simple to experiment with new AI integrations

  • Published internal AI policy and guidelines

Pillar 3: Productivity 2.0

Goal: Expand AI for internal productivity and cost efficiency

Our Results:

  • 93.5% of developers use AI tools; 86% use Cursor

  • 25% of Cursor sessions are non-engineers (codebase Q&A, writing scripts, small PR fixes)

  • Built an internal MCP server to get more done within Cursor

  • AI code reviews, unit test generation, end-to-end test generation, and security analysis

  • Recruitment and deep research automation

  • Product development lifecycle AI agents (status updates, PRDs, story pointing)

  • Automating internal Ops workflows, saving many hours per week

Product Integration Strategies

Customer Support Automation

Our Implementation with Ada:

  • Previous Platform: Gladly Sidekick

  • Current Platform: Ada for leading AI self-serve customer support

  • Results: 49.8% relative decrease in customers who speak to an agent

  • Current Performance: 40% of customer chat contacts are now resolved by AI

  • Efficiency Gain: Resolution time decreases by 90% when resolved by AI vs. human

  • Cost savings: $100,000+ in cost savings from reduced agent volume

Implementation Strategy:

  1. Platform Migration

    • Evaluate AI-native customer support platforms

    • Focus on self-serve capabilities

    • Ensure integration with existing workflows

  2. Gradual Rollout

    • Start with common query types

    • Monitor resolution rates and customer satisfaction

    • Expand to more complex workflows over time

Future Expansion Plans:

  • Shifting phone support to chat

  • More complex automated workflows

  • Personalized AI responses

  • Integration with Loris.AI for agent quality monitoring

SEM Cost Optimization

Our Results:

  • Annual Savings: $290K from keyword optimization (anticipated savings of $500K by 2026)

  • Methodology: AI-driven keyword pruning to eliminate low-performing spend

  • Scope: Excluded 4.35% of search terms, representing 1.2% of total spend

  • Implementation Time: Approximately 1 quarter with 1 engineer to achieve full savings

Technical Implementation:

  • AI Model: ChatGPT-4o LLM via OpenRouter API

  • Integration Platform: Superblocks for low-code UI

  • Process: Automated semantic similarity analysis between search terms and matched keywords

System Architecture:

  1. Prompt Management: Amplitude stores prompts for consistent analysis and enables A/B testing

  2. Similarity Scoring: AI evaluates the semantic similarity between search terms and keywords

  3. Results Processing: System returns similarity scores for optimization decisions, and a DAG actions accordingly

Revenue Optimization through Multi-Armed Bandits

Our Pricing Strategy:

  • Approach: Reinforcement learning style pricing strategy with feedback loops

  • Key Insight: There are different optimal take rates depending on itinerary context, so we used different feedback loops per “context”

  • Testing Method: Fluctuating take rate based on user behavior and context

  • Results: Reward metric (proxy for Revenue Per Search) increased by >50%, corresponding to millions in incremental revenue

Personalization with Predictive Cohorts

Our Amplitude Integration:

  • Platform: Amplitude Predictive Cohorts for low-code personalization

  • Use Case: Predicting the likelihood of users earning rewards to become Super+ members

  • Data Source: Leverages existing analytics data sent to Amplitude

Conversion Results by Predicted Likelihood:

Planned Applications:

  • Craft optimal experiences for each user based on app behavior

  • Customize what users see, notifications sent, and app layout

  • Experiments for homepage component ordering

Future Vision: No two users will see the same app - AI personalizes layouts, offerings, and notifications for each individual to maximize customer and business value.

Building Internal AI Capabilities

AI Guilds Structure

Our Two-Guild System:

AI Technical Guild:

  • Focus: Technical aspects of AI implementation

  • Members: Engineers and technical team members

  • Responsibilities: Evaluate AI tools, develop technical best practices, and support complex integrations

AI Adoption Guild:

  • Focus: Business adoption and use case development

  • Members: Cross-functional representation from all departments

  • Responsibilities: Identify AI opportunities, share success stories, and drive adoption

Meeting Structure: Monthly meetings with company-wide representation to build AI integration expertise and capabilities across both technical and non-technical team members.

Internal AI Platform

Our Implementation:

  • Purpose: Low-code LLM integrations and prompt refinement via A/B testing

  • Current Applications: SEM keyword optimization and inventory consolidation projects

  • Architecture: Centralized service for consistent AI implementation across departments

Service Capabilities:

  • Prompt management and optimization

  • Integration with business tools and workflows

  • Standardized AI model access and usage

  • Cost tracking and performance monitoring

AI Lead Role

Our Hiring Strategy:

  • Responsibilities: AI integration expert for the organization

  • Sample Projects: AI Inventory Consolidation, AI Fraud Detection

  • Focus: Strategic AI implementation across departments

Role Requirements:

  • AI integration expertise

  • Cross-functional leadership capabilities

  • Technical background with business acumen

  • Experience scaling AI/ML in enterprise environments

AI Guidelines and Governance

Our Policy Framework:

  • Published Policy: Internal guidelines covering AI Assistants and responsible AI usage

  • Scope: AI Assistants, Coding Copilots, Product Integrations, and Notetakers

  • Focus: Responsible, transparent, and secure AI usage across the organization

Policy Areas:

  1. Approved Use Cases: Clear guidelines on where and how AI can be used

  2. Quality Standards: Requirements for accuracy, human oversight, and performance monitoring

  3. Security Guidelines: Data protection and privacy considerations

  4. Compliance Requirements: Regulatory and legal considerations for AI usage

Productivity 2.0 Implementation

Developer Productivity

Our Developer AI Adoption:

  • Overall Adoption: 93.5% of developers using AI tools

  • Primary Tool: 86% of AI-using developers use Cursor

  • Productivity Impact: Based on developer survey, 86.7% agree AI improves productivity, 10% neutral, 3% disagree

  • Output Metrics: 50% year-over-year increase in lines of code with approximately same headcount

Implementation Areas:

  1. Core Development: Feature implementation, codebase refactors (e.g., TypeScript migration)

  2. Code Quality: Asking questions about the codebase, writing one-time scripts, and debugging errors

  3. Advanced Features: AI Unit Testing, AI Code Reviews, and AI E2E testing

Telemetry and Measurement:

  • Built out IDE telemetry to observe how developers use AI tools

  • Process of tying usage to metrics like lines of code (LOC) and cycle time

  • Regular surveys to track productivity improvements and satisfaction

Non-Engineering AI Adoption

Our Cross-Functional Usage:

  • Adoption Rate: 25% of Cursor sessions are from non-engineers

  • Teams Using AI: Data analytics, product design, operations, corporate support, QA

  • Primary Use Cases: Codebase Q&A, small code fixes, writing one-time scripts

  • Onboarding Success: Non-SWE can onboard to Cursor with access to our remote execution environments in under 10 minutes

Specific Applications by Role:

  • Product Managers: Codebase exploration, understanding system architecture

  • Designers: Small UI fixes, component adjustments

  • Operations: Data scripts, process automation

  • QA: Bug fixes, test script creation

  • Support: Technical issue debugging

Engineering Sessions:

Non-Engineering Sessions:

Non-Engineering Sessions By Department

Advanced Security and Quality Assurance

AI-Powered Security:

  • Vulnerability Detection: Successfully leveraging AI to identify security vulnerabilities in code.

  • Code Review Enhancement: Rolling out AI review of code changes for quality assurance

  • Security Review Automation: AI identifies what types of work require a security review.

Testing and Compliance:

  • Automated Test Writing: Using AI for writing code tests to comply with test coverage goals

  • Quality Assurance: AI tools help maintain coding standards and best practices

Specialized Productivity Applications

Deep Research:

  • Applications: Data monetization research, direct deposit landscape analysis

  • Approach: AI-assisted comprehensive research and analysis

  • Output: Strategic insights for business decision-making

Recruitment Enhancement:

  • Tools: Job descriptions, pre-screen questions, talent sourcing with Gem

  • AI Integration: Enhanced candidate sourcing and evaluation processes

  • Efficiency: Streamlined hiring workflows

Experimental Use Cases:

  • Hotel Supplier Connectivity: AI assistance for building new supplier API connections

    • Tool Used: Cline for building new hotel supplier API connections

    • Results: 30% speed improvement reported by developers

    • Application: Accelerated integration development with third-party suppliers

    • Legacy Integration: AI analyzes existing supplier integration patterns to accelerate new supplier onboarding

    • Benefit: Faster expansion of the supplier network without rebuilding integration frameworks

  • Operations Process Automation: Eliminated 10-15 hours of manual effort per week, using Cursor for automating Ops processes

  • Brand Animations: AI-powered creative content generation

  • Automated Status Updates: AI-generated project and progress updates

  • Design Doc Creation: AI-assisted documentation and product design refinement

Measurement and Results

ROI and Business Impact

Overall Investment Approach:

  • Budget Strategy: No fixed AI budget, primarily focused on identifying opportunities

  • ROI Assessment: Once the opportunities are identified, the ROI becomes clear

  • Growth Attribution: ~10% of overall growth, ~15% of travel business growth attributed to AI initiatives

2025 AI Strategy Targets:

  • Financial Goal: Drive ≥ $2.5MM annual incremental contribution profit from AI initiatives

  • Project Volume: Complete 10 AI OKR projects combining problem-solving and new experiences

  • Adoption Target: ≥90% of software developers leveraging AI for coding

Change Management and Cultural Transformation

Communication and Evangelization:

  • Weekly Success Stories: Every weekly business review meeting includes AI success story sharing

  • Transparent Communication: Regular communication about AI outcomes and achievements across the company

  • Continuous Reinforcement: Repeated evangelization is required for a cultural shift to an AI-first mindset

OKR Integration for AI-First Mindset:

  • Systematic Approach: When forming OKRs, teams always ask: "How can AI help you achieve your goal?"

  • Cultural Reinforcement: Continuous reminding of AI-first thinking in goal-setting processes

  • Mindset Shift: Recognition that cultural transformation takes time and repeated reinforcement

Implementation Philosophy:

  • People Challenge Recognition: It's as much of a people problem or people challenge as it is a technical challenge

  • Adoption Curve Strategy: Start with early adopters, expand gradually across organization

  • Executive Leadership: Top-down executive commitment is essential for successful transformation

  • Time Investment: Cultural shift doesn't happen overnight, requires sustained effort

Organizational Buy-In Strategy:

  • Whole Company Approach: AI adoption can’t be solved by only bringing on folks or teams to work on AI

  • Functional Leader Buy-In: All functional leaders must be committed to AI adoption

  • Distributed Implementation: AI ambitions cannot be centralized to one team

  • Support Structure: AI specialists provide consulting and support, not centralized execution

Future Strategic Initiatives (Super.com’s next AI bets)

Upcoming Bet #1: AI-First Incubation Hub

Vision:

We are launching an AI-first incubation hub designed to reimagine product development from the ground up. This initiative goes beyond traditional product development by integrating AI throughout the entire product development lifecycle, from engineering and design to quality assurance and product management.

Strategic Approach:

  • AI-first approach to product building across the entire PDLC (engineering, product, design, QA, etc.)

  • Focus on new PMMCs (Product-Market-Model-Channel fits) and new AI-first approaches to product building

Implementation Philosophy:

  • Like new PMMCs, new AI-first approaches and tools will not all be successful. But the idea is to learn and iterate, and once we find what works well, we’ll evangelize in other teams.

  • Experimental approach with systematic learning and scaling

Upcoming Bet #2: Expanding AI Development Tools Beyond Engineering

Current Success Demonstrations:

We have demonstrated significant success in enabling non-engineering team members to interact directly with code through AI tools.

Recent company presentations have showcased successful implementations across multiple use cases including

  • Codebase Q&A

  • Writing Utility Scripts

  • Creating new sites

Recent weekly business reviews have showcased non-SWEs successfully utilizing AI tooling to interact with code:

Role-Specific Use Cases:

  • Product Manager: Eliminates frequent engineering interruptions by enabling self-service codebase exploration and architecture understanding.

  • Designer: Empowers immediate resolution of UI issues through direct code modifications rather than creating low-priority tickets

  • Operations Team Member: Enables internal tool modifications for edge cases without engineering dependencies

  • QA Team Member: Facilitates immediate bug fixes rather than routing through engineering workflows

  • Technical Account Management: Provides debugging capabilities for faster customer issue resolution

DevOps Mission Evolution:

Updated mission: To build and manage infrastructure, improve developer tooling and experience, resolve incidents, and enhance all our employees' ability to leverage AI technologies.

Technology Strategy:

  • Starting with Cursor but maintaining vendor agnosticism

  • Although Cursor is the preferred AI IDE, this could change

  • Initiative not tied to one specific vendor

Engineering Impact and Staffing:

  • Continued Growth: We will continue to grow our engineering headcount in line with ambitions, launching new teams and products.

  • Role Clarity: Non-SWE with AI IDE is not equivalent to SWE. Our amazing engineers will continue to be a core strength as a company.

  • Engineer Benefits: This initiative will mean more time for high-skilled, deep-focus work and less time answering questions and handling smaller issues.

  • Career Development:The goal is to re-highlight the importance of eagerly embracing AI-first development as engineers. This is important for the company success and productivity, but it's also very important for engineering career growth in today's world.

Key Lessons and Best Practices

This post is for paid subscribers

Already a paid subscriber? Sign in
© 2025 Henry Shi
Privacy ∙ Terms ∙ Collection notice
Start your SubstackGet the app
Substack is the home for great culture