A new engineering role is emerging between ML research and software engineering, focused on building products with foundation models via APIs.
Source: Latent Space | Author: Swyx ([email protected]) | Date: June 2023
Core Thesis
Foundation models have created a “shift right” in applied AI. A new professional role, the AI Engineer, is emerging as distinct from traditional Machine Learning Engineers. AI Engineers build products by orchestrating foundation models through APIs rather than training models from scratch.
Tasks requiring 5 years and research teams in 2013 now take afternoons in 2023.
This follows the pattern of previous role innovations: Site Reliability Engineering, DevOps, Data Engineering, Analytics Engineering. Each emerged when a technology shift created enough surface area for a dedicated discipline.
AI Engineer vs ML Engineer
| Dimension | ML Engineer | AI Engineer |
|---|---|---|
| Background | PhD, research, PyTorch | Software engineering, API fluency |
| Core skill | Training models | Orchestrating foundation models |
| Starting point | Data collection, model architecture | API docs, prompt engineering |
| Validation speed | Weeks to months | Hours to days |
| Language | Python-centric | Python + JavaScript |
| Product type | Classifiers, recommenders, fraud detection | Generative apps, agents, copilots |
The author predicts a “flippening” in the job market. Currently ML Engineer roles outnumber AI Engineer roles 10:1 on job boards. Within five years, this ratio inverts based on growth trajectory analysis.
Five Drivers of the AI Engineer Role
1. Foundation Models as Few-Shot Learners
Models exhibit emergent capabilities beyond their creators’ original intentions. Practitioners discover novel applications through experimentation, not training. The skill is knowing how to prompt and orchestrate, not how to build from scratch.
2. Concentrated Research Talent
~5,000 LLM researchers globally vs ~50 million software engineers. APIs serve as “AI Research as a Service.” The bottleneck is not model capability but product engineering on top of models.
3. GPU Economics
Hardware concentration forces startups to use APIs rather than train models. Billion-dollar funding rounds (Inflection $1.3B, Mistral $113M) emphasize GPU ownership. Most companies will consume AI, not produce it.
4. Rapid Prototyping (“Fire, Ready, Aim”)
LLM-based prototyping enables 10-100x faster product validation compared to traditional ML workflows. No data collection phase, no model training, no infrastructure setup. Prototype first, validate, then optimize.
5. Language Expansion
Tools now support JavaScript alongside Python. This doubles the potential developer audience and expands the total addressable market for AI tooling.
Software 1.0, 2.0, 3.0
Building on Andrej Karpathy’s “Software 2.0” concept:
| Era | Definition | Example |
|---|---|---|
| Software 1.0 | Hand-coded programming logic | Traditional applications |
| Software 2.0 | Neural networks approximating logic | Trained ML models |
| Software 3.0 | Human-written code orchestrating LLM power | LangChain apps, AI agents, copilots |
Software 3.0 is where AI Engineers operate. The moat is in orchestration code, not raw LLM access. This explains valuations like LangChain’s $200M+.
The “Code Core, LLM Shell” Pattern
Successful AI products use human code as the core with LLM capabilities wrapped around it. This pattern offers:
- Defensibility against competitors who can access the same models
- Reliability through deterministic code paths where possible
- Security against prompt injection via code-level guardrails
This aligns directly with compound engineering principles: “Most ‘AI agents’ in production aren’t pure agentic systems. They’re predominantly deterministic code with targeted LLM decision-making.”
Notable AI Engineers
The article highlights practitioners building successful products without traditional ML credentials:
- Simon Willison – Tools and exploration with LLMs
- Riley Goodside (Scale AI) – Prompt engineering as a discipline
- Pieter Levels – Photo AI, Interior AI (indie products)
- Teams at Notion, Figma (via Diagram acquisition), Vercel
Compensation range: $300k-$900k at major labs, demonstrating market validation of the role.
Connection to Compound Engineering
The AI Engineer role maps directly to the compound engineering stack:
- Foundation model APIs replace model training (the “AI Research as a Service” layer)
- Orchestration code is the harness (deterministic code wrapping LLM decisions)
- Prompt engineering is context engineering (structuring information for models)
- Rapid prototyping aligns with the RALPH Loop (fast iteration with fresh context)
The AI Engineer is the practitioner. Compound engineering is the methodology. Context engineering is the core skill.
Key Insight
“This will likely be the highest-demand engineering job of the decade.”
The gap between ~5,000 LLM researchers and ~50 million software engineers means the leverage is in building on top of models, not in building the models themselves. Demand-and-supply economics will drive this role to dominance regardless of perception from traditional ML practitioners.
Related
- Six Waves of AI Coding – Evolution from completions to agent fleets
- The Meta-Engineer Identity – Building systems that build systems
- Building the Harness – The four-layer harness around Claude Code
- Skill Atrophy – What skills to maintain vs. delegate
- Highest Leverage: Plans & Validation – Where humans add most value
- Thought Leaders – People to follow in compound engineering
