The Bottleneck Behind the Bottleneck: Why a DIY AI PDLC Is Harder Than It Looks

Madrona's 2026 Builders Summit survey landed on a very interesting finding: code generation is essentially solved, and the new bottlenecks are review, validation, and the clarity of the spec itself. (Read it here.) While that diagnosis is correct, we'd argue there is another bottleneck behind it as well.

Most teams trying to fix the code validation problem are reaching for the frontier model itself, specifically a better prompt, a newer Claude, or a custom GPT. The answer is not there. The answer is in the layer wrapped around the model. The frontier model is the easy part. The harness, the scaffolding, and the context are the difficult pieces, and most teams trying to stand up an AI-driven product development lifecycle on their own underestimate all three.

Model, Harness, Scaffolding. A Quick Vocabulary Check

Let’s start with a quick run through on some of the terminology commonly in use in AI today. It’s important to understand the nuance between these technical terms in order to understand what pieces of an agentic AI approach to product development can be upgraded and at what costs.

‍

‍

• The model is the frontier LLM (Claude, GPT, Gemini). It is stateless, brilliant inside a single turn, and forgets between turns.
• The harness is everything wrapped around the model in production. Prompt orchestration, tool use, memory, retrieval, evaluators, guardrails, retries, routing across models, and handoff between agents. 
• The scaffolding is the workflow shape. Roles, review steps, checkpoints, the artifacts agents produce and consume, the points at which a human approves. 

To ground the discussion in an analogy: the model is a car’s engine, the harness is the drivetrain, the scaffolding is the road system. A powerful engine is ineffective without a chassis and infrastructure suited to the terrain. This matters because it illustrates that the surrounding system design, rather than solely the core model capability, determines the ultimate success of a product.

A real product development lifecycle has dozens of handoffs, multiple personas, brownfield code, edge cases, dependencies, and political context. None of it lives in the model. All of it has to be built into the harness and the scaffolding.

The DIY On Claude Trap

The trap in the belief that an AI-driven PDLC can be a DIY project is that Claude is genuinely good. The prototype of a "spec writer" or "PR reviewer" agent works on a Friday afternoon. The PM shows it to the leadership team on Monday, and they say, "do that for the whole PDLC."

Then the wheels start to come off. Allstacks CEO Hersh Tapadia put it well in a recent piece: "The agent starts sharp. A few turns in, it's tracking the task and making good calls. Thirty turns in, it's lost the thread: forgetting earlier decisions, contradicting itself, fixating on something irrelevant." ([link to Hersh's piece — TK]) (edited) 

This happens because a PDLC is not a single conversation. It is a multi-week, multi-agent, multi-human workflow. A singular context window is the wrong abstraction for the entirety of the effort. Memory has to be external, structured, and curated. The agent has to know what was decided two sprints ago, who decided it, what changed since, and which of those changes should override its instinct in the current turn. None of this is possible with only an engine.

The DIY failure mode is predictable. The first 70% of the harness takes a quarter of time to build. The next 20% takes a year. The last 10% is where the real edge cases live: the legacy service nobody owns, the architectural decision that lives in a Slack thread, the team that ships differently from the rest of the org. At each layer, the people who built it have to keep building it instead of shipping product. The continued cost of building, maintaining, curating context, and improving the agent becomes untenable.

The demo is easy. The system becomes the work.

‍Context Is the Actual Moat. It Has to be Earned Over Time

The model only knows what is in its context window. Everything useful about your business lives outside that window, the codebase, the delivery history, the team capacity, the customer voice, the design system, the strategy decks, the meeting notes where a tradeoff was actually decided.

Getting that into the model is not a one-time ETL job. There are four ongoing jobs.

Build it. Connectors to every tool of record, normalized into a shape an agent can reason over. Jira, GitHub, Figma, Gong, Productboard, the wiki, the data warehouse. Each integration is small. The sum is not.

Maintain it. Code drifts daily. Org charts change. Last quarter's architectural decision is this quarter's tech debt. Stale context is worse than no context, because the agent will confidently apply it.

Curate it. Not all context is equal. The right slice for a spec review is different from the right slice for an incident postmortem. Retrieval and ranking have to be tuned, evaluated, and re-tuned every time the model or the workflow changes.

Apply it. Different agents need different views into the same underlying graph. A security reviewer reads the codebase differently than a feasibility assessor reading roadmap commitments. The same graph has to serve both without one corrupting the other.

Be honest about the surface area. There are at least six classes of context to integrate and keep current: code and architecture, work and team, customer voice, design, product usage, and strategy and decisions. Each one is a small product on its own and just like no human can keep it all in one place, no single agent can keep the context in a singular window long term.

If your plan for the AI-driven PDLC is "we'll feed it the right docs," your plan is the part you have not built yet.

The Harness itself is a Moving Target

Even if you nail the context today, the harness around the model is not a stable target. The state of the art is moving in quarters, not years. Change is the only thing that’s guaranteed and your plans most consider the ongoing cost of this change.

In the last twelve months alone, the landscape of AI development has shifted rapidly. Sub-agent and multi-agent patterns have become the default for any non-trivial workflow, while evaluator agents have evolved from an afterthought to a necessity for assessing model performance. Simultaneously, long-running, background agents have become viable, fundamentally altering how teams design checkpoints and human approvals. These advancements, combined with constantly shifting tool-use protocols and standards, make the harness around any AI model a rapidly moving target.

The cost of falling behind is not just missing a feature. The assumptions in your harness gets baked into a workflow your team has been using for six months. Ripping them out is an organizational change, not a code change. A DIY harness is a product team you did not plan to staff, building against a roadmap that someone else is writing.

What a Serious AI-driven PDLC Harness Has to Do

Product leaders must consider the entire system architecture rather than isolated tools. This begins with a living product definition that remains synchronized with your codebase, work trackers, customer feedback, and architectural decisions, moving beyond simple prompt-time snapshots that fail to scale. Alongside this, you need a set of adversarial AI reviewers that evaluate specifications from specialized perspectives—such as security, QA, architecture, and feasibility—effectively challenging the work before it even reaches the engineering team.

Beyond these definition and review components, operational success requires a continuous background loop that actively monitors for discrepancies like drift between specifications and tickets, duplicate efforts across teams, and evolving delivery risks between human interactions. Complementing this, a robust scoring layer acts as a gatekeeper, verifying that work is genuinely prepared for execution before it leaves the planning phase. Together, these elements form the necessary infrastructure for a sophisticated, AI-driven product development lifecycle.

This is the shape of the system Allstacks has built inside Product Studio. The point of this post is not to sell that. The point is to be honest about the size of the problem if you are building it yourself so you don’t waste time running into brick walls.

The Work in Front of Us

Madrona's frame is the right one. The bottleneck has shifted from code generation to post-creation validation and the quality of specifications prior to development. The next bottleneck after that is the system that decides what the model sees, when, and from where. That system is what an AI-driven product development lifecycle actually is.

Frontier models will keep getting smarter. The teams that get the most out of them will be the ones who stop treating the model as the product, and start treating the context, the harness, and the scaffolding as the real work.

‍