Skip to content
Back to Blog
Agentic Systems

The Software 1.0 → 3.0 Shift: What It Actually Means for Engineers

6 min read1108 words
#software-paradigm#ai-engineering#agentic-systems#career#llm-engineering

In 2017, Andrej Karpathy wrote a post called "Software 2.0" that described a shift in how we write software. Instead of writing explicit instructions (Software 1.0), we were increasingly specifying desired behaviors and letting neural networks learn the implementation (Software 2.0).

At the time it felt like an academic framing. In 2026, it's the job description.

But the framing has evolved. Software 3.0 which some are now calling the agentic paradigm is where LLMs are not just the model, they're the runtime. The program is the prompt. The logic is the agent loop.

This post is my practical take on what this shift means for engineers who aren't AI researchers, but who need to decide where to invest their skills.

A Quick Taxonomy

Software 1.0 is the traditional stack. You write explicit logic in a programming language. The computer executes exactly what you write. Deterministic, debuggable, versioned.

Software 2.0 is trained models. You specify inputs and desired outputs, and a neural network learns the mapping. The "code" is the weights. Non-deterministic, harder to debug, requires training infrastructure.

Software 3.0 is LLM-orchestrated systems. You write prompts and agent architectures. An LLM performs reasoning, planning, and decision-making at runtime. The "code" is the system prompt and the orchestration logic around it.

These aren't replacing each other they're stacking. Most production systems in 2026 use all three:

  • Software 1.0 for deterministic business logic, data transformation, validation
  • Software 2.0 for specialized ML tasks (classification, recommendation, computer vision)
  • Software 3.0 for orchestration, reasoning, and tasks requiring language understanding

What Changes for Engineers

1. Debugging Changes Fundamentally

In Software 1.0, debugging means tracing execution, reading stack traces, adding log statements. The program did exactly what you wrote, so you find the wrong line and fix it.

In Software 3.0, the "program" is a conversation between your orchestration layer and an LLM. A wrong answer doesn't have a stack trace. You need to trace the conversation: what was in the context window? What did the model reason? Which tool call produced bad data?

This is why observability isn't optional in LLM systems it's the new debugger. The trace IS the stack trace.

2. Testing Changes Fundamentally

Unit tests make assumptions about determinism. Call function X with input Y, expect output Z. This works for Software 1.0. For LLM-powered systems, the same input can produce different outputs, and "correctness" is often semantic rather than binary.

Testing LLM systems requires:

  • Behavioral testing: does the agent achieve the goal across a diverse set of inputs?
  • Robustness testing: does the agent fail gracefully on edge cases and adversarial inputs?
  • Regression testing: after prompt changes, does behavior remain consistent on known cases?
  • Cost testing: does the agent stay within expected token/cost budgets?

None of these fit neatly into existing test frameworks. They require new tooling and new thinking about what "passing tests" means when the system is non-deterministic.

3. Prompt Is Code

In Software 3.0 systems, the system prompt is production code. It has the same requirements as any other production code: version control, code review, staging environments, rollback procedures.

Changing a system prompt without testing it on representative inputs is equivalent to deploying untested code. The blast radius can be larger a changed system prompt affects every user of the agent, not just the code path you tested.

// Prompts should live in version-controlled files, not hardcoded strings
const AGENT_SYSTEM_PROMPT = await fs.readFile(
  `./prompts/agent-v${process.env.PROMPT_VERSION}.md`,
  'utf-8'
)
 
// A/B test prompt changes before full rollout
const promptVersion = userId.hashCode() % 100 < 10 ? 'v2' : 'v1'

Prompt management at scale is a real engineering discipline one that barely existed 18 months ago.

4. Context Window Is Your Architecture Constraint

In Software 1.0, your architecture is constrained by compute, memory, and network. In Software 3.0, the context window is a fundamental constraint that shapes every architectural decision.

  • How much history can you include?
  • How do you summarize long documents for LLM processing?
  • How do you retrieve only the relevant information rather than loading everything?
  • How do you structure multi-step workflows to stay within context limits at each step?

Context window management is the new memory management. It requires conscious design, not afterthought optimization.

💡200k Tokens Does Not Mean Unlimited Context

Large context windows reduce some constraints but don't eliminate them. Cost scales with context size. Quality can degrade at the edges of long contexts (the "lost in the middle" problem). Retrieval-based approaches often outperform stuffing everything into context, even when the window is large enough to fit it.

Where to Invest Right Now

If you're a Software 1.0 engineer deciding how to upskill for this paradigm, here's where I think the leverage is highest:

Highest leverage:

  • LLM API integration OpenAI, Anthropic, direct API usage, structured outputs, tool calling. This is the base layer that everything else builds on.
  • Agent orchestration patterns the patterns in this blog: memory systems, multi-agent coordination, approval gates. These are engineering problems, not ML research problems.
  • Observability and evaluation if you can instrument and evaluate LLM systems, you can improve them. This skill is scarcer than the ability to write prompts.

Medium leverage:

  • RAG and vector databases retrieval-augmented generation is a standard pattern now, and knowing how to build and tune retrieval pipelines is valuable.
  • Prompt engineering understanding what makes prompts reliable, not just creative. Testing and iterating on prompts systematically.

Lower leverage (for most engineers):

  • Model training and fine-tuning unless you're doing specialized ML work, the frontier models are good enough for most tasks and improving faster than your fine-tunes.
  • LangChain/framework expertise frameworks come and go. The underlying API knowledge and architectural patterns are more durable.

The Honest Assessment

Software 3.0 is genuinely exciting. The capability jumps in the last two years have been real. Agents that could only run toy tasks in 2023 are now doing meaningful production work.

But it's also genuinely immature. The tooling is still catching up to the problems. Debugging is hard. Testing is hard. Cost unpredictability is real. Production failures have unfamiliar shapes.

This is actually a good moment to engage with it deeply the tooling is good enough to build real things, but immature enough that genuine engineering skills (debugging, system design, reliability engineering) are highly differentiated. The people who can make LLM systems reliable, not just impressive, are rare and valuable.

That gap will narrow. It won't stay this wide forever. But right now, the engineers who understand both the Software 1.0 fundamentals and the Software 3.0 patterns are well-positioned in a way that's genuinely unusual.