Taking Aila from a mega prompt to an agentic system

Aila is Oak National Academy's AI lesson planning assistant, designed to help teachers create structured lesson plans and additional materials grounded in Oak's pedagogical approach. You can try it at labs.thenational.academy. It has been well received, with two-thirds of surveyed teachers using Aila saying it reduced their workload or helped them reclaim time for other important aspects of their role.

Like a lot of useful AI products, it began as one large, carefully engineered prompt.

That prompt had to do a lot. It needed to interpret a teacher’s request, work out where they were in the lesson planning process, apply Oak’s pedagogical constraints, pull in relevant examples, and return something structured enough for the product to use. For an early system, that was a sensible design, it was fast to build, fairly easy to reason about, and good enough to get a real product into teachers’ hands.

For a while, it worked well. In some ways, it still explains a lot about why prompt engineering remains hard to replace with clean diagrams. A monolithic prompt can accumulate nuance from the interaction between many constraints at once. The whole can be more than the sum of its parts.

There is a tendency in AI engineering to talk as though the first version of a system was obviously naive and the newer version is obviously correct. That is not how this felt. The “mega prompt” era of Aila was productive. But success at one stage of a product does not mean the architecture keeps scaling with you.

Why the mega prompt started to hurt

The core problem with a monolithic prompt is that all the reasoning is implicit.

In the original setup, a lot of Aila's behaviour lived inside one composite instruction set. In code, that showed up through AilaLessonPromptBuilder, which assembled the system prompt by combining the current lesson state, retrieval results, schema constraints, formatting rules, and other pieces of runtime context into a single message for the model.

That gave us one place to shape behaviour, but it also meant that small changes could have wide effects. If the model produced something odd, it was often hard to tell whether the issue came from planning, retrieval, section-specific generation, or just the interactions between too many instructions living side by side.

It also limited how we could evaluate the system. We could look at the final lesson plan and decide whether it seemed good or bad, but that only tells you so much. If you want to improve an AI system in a disciplined way, you need to inspect the steps that led to the output, not just the output itself.

That was fine for a proof of concept. It was less fine for a product we wanted to observe, test and improve systematically.

What we mean by agentic here

“Agentic” is one of those words that becomes less useful the more confidently people use it.

In this case, I do not mean a fully autonomous system with open-ended tool use, recursive self-critique, or long-running feedback loops deciding for themselves when to stop. We have explored some of that territory separately, including spikes with LangGraph, and there are interesting ideas there. But that is not the system we are trying to ship today.

What I mean here is something narrower and more practical: a system where the reasoning is decomposed into explicit steps, each step has a defined role, and the orchestration layer decides which specialist should act next.

So yes, it is more agentic than a single monolithic prompt. But it is still deliberately constrained. There are no fully general recursive loops here. That is partly a product decision and partly an engineering one. For the kind of work Aila does, reliability, inspectability and control matter more than making the system feel maximally autonomous.

What we moved towards instead

The replacement was not “let the agents loose and hope for the best”. That version of agentic AI makes for a good demo, but it is not a particularly good fit for a product that needs to be reliable, structured and safe.

What we moved towards is better described as an orchestrated pipeline with specialised reasoning steps.

At the start of a turn, a planning agent decides what should happen next. Not by writing lesson content itself, but by producing a plan: which sections need work, in what order, and whether the user actually needs content generation at all. In some cases the right result is to exit and ask for clarification rather than blindly generate.

From there, section-specific agents handle the actual work. One agent might generate or edit a quiz section. Another might work on prior knowledge or key learning points. Each step has a clearer contract: what input it receives, what part of the lesson it owns, and what output it is expected to return.

That is the important shift. We did not just split one prompt into several smaller prompts. We made the reasoning structure explicit. It is also where the trade-off sits: smaller agents are not automatically better on day one, because a strong monolithic prompt can carry shared context and nuance that is easy to lose if you split it too mechanically.

A simplified view of the change

The old setup looked something like this:

One large prompt could carry a lot of context at once, which is part of why it worked. But it also made the reasoning hard to inspect.

The newer flow is closer to this:

That diagram leaves out a lot, but it captures the main architectural difference. The interesting part is not just that there are more boxes. It is that the handoffs between them are explicit.

What changed in practice

The new lesson-generation business logic now lives in the agentic-system_directory. First, Aila runs a planning phase. If planning succeeds and content generation makes sense, it executes the planned steps one by one. Each step is routed to the relevant section agent. The result of each step is validated, applied to the in-memory lesson document, and streamed back into the rest of the product.

A useful way to think about these agents is that each one is essentially an inference spec: the schema for the response, the messages that define the task, and the model parameters used to run it.

export type GenericPromptAgent<ResponseType> = {
  responseSchema: z.ZodType<ResponseType>;
  input: (
    | { role: "developer"; content: string }
    | { role: "user"; content: string }
  )[];
  modelParams: Omit<
    ResponseCreateParamsNonStreaming,
    "input" | "text" | "stream"
  >;
};

It’s also worth a note about how we released this change.

We released the agentic system last week (22 July 2026), after keeping it behind a feature flag while we tested it. Some newer features had started to depend on the agentic architecture, while other parts of Aila still needed to work with the older mega-prompt flow. That meant testing a bundle of changes together, rather than isolating every change as neatly as we’d like.

That isn’t ideal, but it’s a normal part of evolving a live AI feature without enforcing a code-freeze on other aspects of the product. The important thing was to keep the release path deliberate: feature-flagged, tested across the full lesson flow, and reviewed from both a technical and educational point of view.

Why this is a better fit for the next stage of Aila

The biggest improvement is observability.

Once planning is explicit and section generation is separated, we can ask better diagnostic questions. Did the planner choose the right sections? Did a section agent preserve the lesson context and follow its output schema? Did retrieval bring in useful Oak examples, or add noise? Those are more useful engineering questions than simply asking whether the final lesson plan felt right.

It also makes experimentation more realistic. If each stage has a clearer boundary, you can change one component without rewriting the whole system prompt. You can compare strategies, test narrower hypotheses, and build evaluations around specific behaviours. In the Aila codebase, that is already visible in the scoring harness around the agentic flow.

The same structure helps with release testing. The point is not that the agentic version magically solves lesson quality. It is that it gives us clearer units to test: planner behaviour, section generation, user-requested edits, and the familiar end-to-end lesson journey. Our current test plan reflects that. It checks the new section-by-section flow, then runs the standard Aila smoke tests, with review from both a technical and educational perspective. It also calls out the awkward but important risks: RAG and Oak lesson matching, moderation and threat detection, sharing and exports, feature flag behaviour, and the current state of end-to-end test coverage.

There are safety benefits too. In a monolithic design, a lot of the burden sits on one prompt getting everything right. In a staged system, you have more chances to constrain behaviour, validate outputs, and stop bad states from flowing further downstream.

What comes next

The architectural change is only half the story. The harder and more interesting question is what it delivers in practice, and our improved ability to evaluate it more robustly.

Does the new system produce better lesson plans? Is it more consistent? Is it easier to recover from mistakes? What happens to cost and latency? Which parts of the pipeline are genuinely pulling their weight, and which are just neat abstractions?

Those are evaluation questions, not architecture questions, and they deserve their own treatment. We are not yet claiming a quality victory. We are claiming a better architecture for observing, testing and improving quality deliberately.

We will share more on how we evaluate Aila’s outputs in a follow-up post, including an early model comparison on the new agentic flow, where the results are encouraging enough to be worth a write-up of their own.

They also point towards a broader shift in how we want to do prompt engineering. One reason this architecture is appealing is that it lends itself much better to data-driven approaches. If the steps are explicit, they are easier to measure, compare and tune. That is part of the direction we are exploring now, including with DSPy, which deserves a separate post of its own.

But getting to that point required a change in shape first. We needed a system where reasoning steps were visible, testable and replaceable. Aila started as one well-crafted prompt. To keep improving it, we needed something more like a reasoning pipeline.

That is what this shift has really been about: turning implicit behaviour into explicit structure, so the system can be improved on purpose.

There is a broader reason to share this work too. AI lesson planning is still a young space, and many of the difficult questions are not just model questions. They are questions about product design, evaluation, safety, pedagogy and release confidence. Where we can, we want to make those trade-offs visible so others working in education technology can learn from them, challenge them, and build on them.

You can see the current implementation in the agentic-system directory. If you're building on this, we'd like to hear how - the comments are open.

Building on Oak's open content? Click here to get developer updates.