Agentic AI: Part 2–Evolution of AI Agents
In Part 1, we explored what Agentic AI is, how it differs from traditional AI, and why it matters more than ever. If you haven’t read it yet, I highly recommend starting there — it lays the foundation for what comes next.
In this post, we will focus on the evolution of AI Agents.
Ever wonder how we moved from chatbots that simply respond to AI agents that can reason, plan, and act on their own?
That’s what we’ll unpack here.
What began as static, one-off responses has rapidly progressed into intelligent, goal-driven agents capable of retrieving information, making decisions, adapting over time, and taking action with minimal human input.
We’ll walk through the six stages that define this evolution, and explore what they mean for the future of intelligent automation particularly in industries like insurance, where context, compliance, and coordination are critical.
Because for modern enterprises, this isn’t just about smarter tools, it’s about reimagining how work gets done.
Stage 1: Basic LLM (Input → Output)
Let’s start with the earliest stage, when AI models were purely reactive and limited to what they had learned during training.
Key Components:
- Input: Text
- LLM: Large Language Model
- Output: Text
This is the simplest form of AI interaction.
You give it a prompt, and it gives you a response based on patterns it learned during training. Think of ChatGPT answering “What is underwriting?” with a textbook definition.
At this stage, the model is powerful but static, no awareness of your specific needs, no access to external data, no memory of past conversations. It’s like asking a well-read intern with no access to your company files.
Let’s look at an example:
A customer asks, “What does my travel insurance cover?”
The LLM responds with a generic answer like, “Typically, trip cancellations and medical emergencies are covered.”
But here’s the catch, it doesn’t actually check the customer’s specific policy or access any real-time data. It’s a best guess based on its training, not a personalised response.
This was impressive in 2022. But enterprises quickly realised its limits especially in regulated industries like insurance, where context is everything.
Naturally, the next step was to give the model more to work and expanding with what the model could understand. Instead of relying only on what the model had seen during training, what if we allowed users to upload documents?
That’s exactly what Stage 2 enabled, multimodal input that opened the door to richer interactions and document understanding.
Stage 2: LLM + Input Variety (Text + Files)
As businesses needed deeper understanding, AI began to handle a wider range of inputs ushering in the era of document-aware models.
Key Components:
- Input: Text + PDFs/Images
- LLM: Still pre-trained
- Output: Text
Here, we unlock multimodal input.
Instead of just asking questions in text, users can upload documents like scanned claims or policy PDFs and ask the model to interpret them.
The LLM can now read attachments and extract key insights, but it still can’t retrieve data from external sources like databases or systems of record.
For Example:
An underwriter uploads a scanned medical report and asks, “Does this diagnosis qualify under our policy exclusions?”
The LLM processes the PDF, extracts the relevant text using its built-in document understanding capabilities, and uses its training knowledge to interpret the medical terms. It then attempts to answer the question based on general policy knowledge it learned during pretraining.
This opened doors to document intelligence, but the responses were still based on static knowledge, not linked to real-time or policy-specific company data.
To move beyond static answers, one-size-fits-all responses, the next stage focused on grounding the model’s answers in real-time, enterprise-specific information making interactions more relevant, accurate, and useful for practical decision-making.
Stage 3: LLM + Tool Use (RAG)
As expectations grew for more accurate and context-aware responses, the next evolution empowered AI models to tap into an organisation’s existing knowledge bridging the gap between static intelligence and dynamic understanding.
Key Components:
- Input: Text + Files
- LLM
- Tool Use: Retrieval-Augmented Generation (RAG)
- External Vector DB for retrieval
This stage adds tools specifically retrieval mechanisms via RAG that allow the model to search external documents or knowledge bases (via a vector database), pull in relevant chunks, and use them to generate more accurate and grounded responses.
It’s like giving your AI intern access to the company SharePoint.
Example:
A claims manager asks, “Where do I find the process for fast-tracking a flood claim?” The agent searches internal SOPs and provides the exact steps, complete with links. Retrieval changes the game. Now, the LLM is grounded in your organisation’s knowledge not just what it learned during pretraining.
But while this stage brought in greater accuracy, it still lacked one critical ingredient: memory. Without the ability to retain and build on prior interactions, every query existed in isolation.
The next step in the evolution aimed to fix that by making conversations more continuous, contextual, and intelligent.
Stage 4: LLM + Tool Use + Memory
With grounding in real-time information in place, the next evolution focused on making interactions feel more natural by allowing AI to remember and build on past conversations.
Key Components:
- Tool Use
- Memory (short-term context tracking)
- Vector DB
- Output grounded in history
This is where things get smarter.
The agent retains context across interactions using memory. Ask a follow-up question and it remembers what you talked about before. Combined with RAG, this creates fluid, contextual conversations.
A quick example:
A customer contacts their insurer and says, “I was in an accident and need help filing a claim.”
The AI agent gathers the necessary details and guides them through the process.
Later, the customer returns and says, “I’ve received the repair estimate, can I upload it here?”
Because the agent remembers the earlier conversation, it immediately understands the context, links the estimate to the ongoing claim, and continues the process without asking the customer to repeat any details.
Memory makes AI feel human. It creates continuity, a big leap toward automation that feels personal.
But remembering recent conversations is only part of the equation. The next leap forward focuses on enabling the AI to take more initiative responding not just with awareness, but with thoughtful actions, deeper reasoning, and the ability to manage more complex, multi-step interactions across time
Stage 5: LLM + Decision-Making + Multi-Type Memory
As agents evolved to remember and contextualise conversations, the next step was to give them the ability to reason and decide, to move from reactive responders to intelligent collaborators capable of handling complex tasks independently.
Key Components:
- Tool Use + Memory
- Decision Logic
- Short-Term + Long-Term + Episodic Memory
- Vector + Semantic Databases
This stage marks the rise of true agents not just chatbots. They can now make informed decisions by combining various types of memory:
- Short-term: Active conversation context
- Long-term: Past interactions and user preferences
- Episodic: Sequences of events or tasks the agent has handled before
Layered with a semantic database, essentially a structured knowledge base that understands relationships, the agent doesn’t just retrieve facts; it connects the dots, reasons across them, and takes action.
Example:
An AI claims agent reviews a customer’s previous claims, considers current underwriting rules, and suggests whether a new claim should be approved, factoring in things like evolving policy conditions, geographical risk patterns, and historical fraud indicators.
We’re not just automating workflows, we’re enabling systems to reason, personalise, and make decisions in complex environments.
This is a major step toward goal-driven intelligence.
But making good decisions is only part of the journey. To operate at scale in real-world enterprises, these agents must now work together, coordinating tasks, managing roles, and integrating seamlessly across systems. That’s where the future architecture begins to take shape.
Stage 6: Future Architecture of AI Agents
The final stage is all about scaling intelligence.
This is where individual agents become part of a coordinated system, an Agentic AI System that can plan, act, learn, and evolve across the enterprise.
This diagram shows how advanced AI agents will work together like a team of digital assistants, each with a role talking to each other, making decisions, improving over time, and helping humans in smarter ways.
Input Layer: “What’s coming in?”
Think of this as the AI agent’s senses.
It takes in all kinds of information, such as: PDFs (like a scanned accident report), Messages or emails (like “I want to lodge a claim”), Images (like photos of vehicle damage), Real-time system data (e.g., weather data, traffic reports, Customer feedback (like survey results or chatbot ratings)
Example: A customer sends an email with photos of their damaged car and a claim form. The AI agent sees this as input.
Data + Feedback Layer: “What info do I already know or need to keep learning from?”
This is where all your data and feedback go to help the agent personalise responses and improve, such as Historical data: Past claims, customer profiles, previous decisions, Real-time updates: Has the policy changed recently? Is the claim location under a flood warning? and User feedback: If customers often get confused by a form, the agent learns to explain it better next time.
Example: The AI sees that this customer has had 2 prior claims and lives in a high-risk flood area. That adds useful context.
Data Storage/Retrieval Layer: “Where do I find the answers?”
This is the agent’s memory bank where it looks up information to make informed decisions such as Structured/Unstructured Data: Policy terms, claim forms, support emails, Vector Stores: Helps the agent search across documents by meaning, not just keywords and Knowledge Graphs: Understands relationships — like which policies belong to which products or customer segments.
Example: The agent looks up the latest policy rules, finds relevant exclusions, and pulls up previous claims to compare.
AI Agents: “What should I do now?”
This is where the magic happens, each AI agent can: Plan steps to complete a task (e.g., start a claim, notify underwriting, update CRM), Reflect on how well it’s working (e.g., “Was this claim rejected too often?”), Use tools like APIs, calculators, or translation engines and Learn on the go, improving through a self-learning loop.
Each agent can run a model, and there may be multiple models (Model 1, 2, 3…) working together.
Example:
- Agent A handles document extraction
- Agent B checks fraud risk
- Agent C recommends approval
They all talk to each other behind the scenes.
Agent Orchestration Layer: “How do we work together as a team?”
This is the brain of the operation, managing the agents like a team manager: Dynamic task allocation: Assigns tasks to the right agent, Inter-agent communication: Agents talk and share insights and Monitoring & Observability: Tracks performance, what worked, what didn’t
Example: One agent finishes processing the claim and automatically alerts another to update the claims portal, without any human needing to step in.
Output Layer: “What do I show the user?”
This is where all the AI’s effort becomes visible such as Custom outputs: Chat messages, emails, dashboards, reports, Knowledge updates: Adds new learnings to the system and Synthetic data: Generates scenarios or test data for simulations
Example: The customer gets a detailed response saying:
“Your claim has been received. Based on your policy and damage details, it qualifies for fast-track approval.”
Service Layer: “How do I deliver this?”
Think of this as the delivery engine, how and where the AI’s output is shared via Multi-channel delivery: Web chat, mobile apps, call center tools, emails and Automated insights: Summary reports for claims managers or compliance teams
Example: The claims team receives a dashboard view of flagged cases, while the customer gets an SMS update.
Governance Layer: “Is this safe and compliant?”
This is your trust and control system essential in industries like insurance such as Safety & Bias Control, Regulatory Compliance, Versioning & Audit Logs and Human-AI Collaboration
Example: A supervisor can step in to override an AI decision, or regulators can audit how a decision was made — ensuring transparency.
Why This Evolution Matters (And What to Do Now)
This journey from static LLMs to orchestrated, autonomous agents isn’t just a technical upgrade.
It’s a paradigm shift.
Enterprises that only focus on LLM features miss the real transformation.
Agentic AI is about reimagining how decisions are made, tasks are completed, and intelligence is embedded into systems.
Key Takeaways:
Rethink your AI roadmap: Are you still building chatbots? Start designing agent ecosystems.
Invest in AI-ready data infrastructure: You can’t scale agents without governed, retrievable, compliant data for both structured and unstructured.
Shift from tools to orchestration: Build your agent frameworks with modular components (e.g., RAG, memory, feedback loops, APIs) that can grow and evolve.
Design for explainability and control: Agentic AI must be trustworthy. Build in governance from the start not as an afterthought.
Start small, but build with scale in mind: Pick high-impact use cases (e.g., triage, document processing, claims analysis), but implement them on an architecture that supports expansion.
We’re entering an era where AI doesn’t just support decisions it makes them. Not in a vacuum, but with accountability, context, and control.
As leaders, we must ask: Are we building tools, or are we building intelligence? The answer will define the next decade of enterprise innovation.
Let’s lead with intention. Let’s build agents that serve smartly, safely, and at scale.
Acknowledgement
This blog was inspired by the brilliant visual created by @rakeshgohel01, illustrating the Evolution of AI Agents. I’ve built on that framework to explore how each stage translates into real-world insurance use cases helping industry leaders understand how to navigate the shift from static models to intelligent agentic systems.
