Building AI Agents in Java with Spring Boot: A Comprehensive Guide

Building AI Agents in Java with Spring Boot: A Comprehensive Guide

Welcome, aspiring AI agent builder! This document is your complete guide to understanding and creating intelligent AI agents using the powerful combination of Java and Spring Boot. Whether you’re entirely new to AI or looking to leverage your Java skills in this exciting field, this guide will take you from the very basics to building sophisticated agentic systems.

We’ll focus on practical, real-world examples using leading Java AI frameworks like Spring AI and Google’s Agent Development Kit (ADK) for Java. By the end, you’ll not only grasp the theory but also have hands-on experience in building agents that can reason, plan, and interact with the world.


1. Introduction to AI Agents

What are AI Agents?

At their core, AI agents are computer systems designed to autonomously perceive their environment, make decisions, and execute actions to achieve specific goals. Think of them as intelligent software entities that don’t just respond to direct commands but can also understand intent, break down complex problems, and utilize tools to accomplish tasks.

Unlike a simple chatbot that gives predefined answers, an AI agent often exhibits characteristics such as:

  • Autonomy: They can act without constant human supervision.
  • Proactivity: They initiate actions based on goals rather than just reacting to inputs.
  • Reactivity: They respond to changes in their environment.
  • Social Ability: They can communicate and collaborate with other agents or humans.
  • Goal-Oriented: They are designed to achieve specific objectives.

In the context of Large Language Models (LLMs), an AI agent typically leverages an LLM for its “brain” – its reasoning and natural language understanding capabilities. However, an LLM alone isn’t an agent. The “agentic” part comes from the surrounding logic that enables the LLM to:

  • Plan: Break down a complex user request into smaller, executable steps.
  • Reason: Decide which tools to use, when, and with what arguments.
  • Observe: Evaluate the results of its actions.
  • Act: Execute tools to interact with external systems or data.
  • Iterate: Refine its approach based on feedback or new information.

Why Learn AI Agents with Java and Spring Boot?

You might associate AI primarily with Python, but Java and Spring Boot offer compelling advantages for building AI agents, especially in enterprise environments:

  • Enterprise Dominance: Java and Spring Boot are the backbone of countless enterprise applications. Building AI agents in this ecosystem allows for seamless integration into existing infrastructure, data sources, and business processes.
  • Scalability & Performance: The Java Virtual Machine (JVM) is renowned for its robust performance, scalability, and mature concurrency models, making it ideal for high-throughput, real-time AI applications.
  • Structured & Maintainable Code: Java’s strong typing and Spring’s opinionated, modular architecture encourage well-structured, maintainable, and testable codebases, crucial for complex AI systems.
  • Growing Ecosystem: Frameworks like Spring AI and Google’s ADK for Java are rapidly evolving, providing high-level abstractions and integrations that simplify AI development for Java developers.
  • Security & Governance: Spring Security offers mature solutions for securing AI agent endpoints and data, a critical concern in enterprise settings.
  • Cloud-Native Readiness: Spring Boot applications are inherently cloud-native friendly, easily deployable to platforms like Azure Container Apps, Google Cloud Run, or Kubernetes.

A High-Level Overview: How an AI Agent Works

Let’s demystify the internal workings of an AI agent, particularly how it “chooses” tools and uses prompts.

  1. Human Message / User Input: The process starts with a user interacting with the agent in natural language, e.g., “Find me flights from New York to London for next Tuesday and book the cheapest one.”

  2. The LLM as the Brain (Initial Prompting): The user’s input is sent to a Large Language Model (LLM). Crucially, the LLM isn’t just given the user’s message. It’s also provided with:

    • System Prompt/Instructions: A set of directives telling the LLM its role, personality, constraints, and overall goals (e.g., “You are a helpful travel assistant. Always confirm details before booking. If a tool fails, try a different approach.”).
    • Available Tools (Function Definitions): A description of all the external “tools” (functions) the agent can use, along with their purpose and required parameters. This is often provided in a structured format like JSON Schema.
  3. LLM’s Reasoning & Tool Selection (The “Illusion” of Function Calling):

    • The LLM analyzes the user’s request, the system prompt, and the available tool definitions.
    • It doesn’t actually call the function itself. Instead, the LLM generates structured text (often JSON) that describes the function call it believes is necessary to fulfill the user’s request. For our flight example, it might generate:
      {
        "tool_name": "searchFlights",
        "parameters": {
          "origin": "New York",
          "destination": "London",
          "date": "next Tuesday"
        }
      }
      
    • This is the “function generation” part. The LLM’s output is text, but it’s structured in a way that an external “orchestration layer” can understand.
  4. Orchestration Layer (The “Real” Action):

    • The Java Spring Boot application (the orchestration layer) receives the LLM’s generated function call.
    • It parses this structured text.
    • It then executes the actual Java method (the “tool”) corresponding to searchFlights with the extracted parameters. This tool might make an API call to an airline booking service.
  5. Tool Execution & Result:

    • The searchFlights tool runs and returns its output (e.g., a list of available flights with prices).
  6. Feedback Loop & Further Reasoning:

    • The original user query, the LLM’s initial “thought process,” and the result from the searchFlights tool are all fed back into the LLM as part of the conversation context.
    • The LLM then continues its reasoning. It might identify the cheapest flight and decide it needs to use a bookFlight tool. Or, if no flights were found, it might ask the user for alternative dates.
  7. Iteration and Final Response: This cycle (reasoning -> tool generation -> tool execution -> feedback) continues until the agent believes it has successfully achieved the user’s goal or needs more information. Finally, the LLM generates a natural language response to the user, incorporating all the information gathered and actions taken.

This iterative process, guided by the LLM’s reasoning and the execution of external tools, is what gives AI agents their power and flexibility.

Setting up Your Development Environment

To follow along with the examples in this guide, you’ll need the following:

Prerequisites:

  • Java Development Kit (JDK) 17 or later: Download and install from Oracle or use an OpenJDK distribution like Adoptium Temurin.
  • Maven 3.6+ or Gradle: (Maven is used in examples) Build automation tool.
  • An Integrated Development Environment (IDE):
    • IntelliJ IDEA (Ultimate Edition recommended): Excellent Spring Boot and AI development support.
    • Visual Studio Code with Java Extension Pack: A lighter-weight but powerful option.
  • Git: For version control.
  • An API Key for an LLM Provider:
    • OpenAI: Widely used, easy to get started. Requires an API key.
    • Google AI Studio (Gemini models): Free tier available, requires a Google API key.
    • Local LLM (e.g., Ollama): For privacy or reduced cost, you can run models locally. This adds complexity to setup. We will use OpenAI or Google Gemini for simplicity in this guide.

Steps to Set Up:

  1. Install Java JDK: Download and install JDK 17 (or newer) from your preferred source. Verify the installation:

    java -version
    javac -version
    
  2. Install Maven (if not already present): Follow instructions on the Apache Maven website. Verify installation:

    mvn -version
    
  3. Install your IDE: Download and install IntelliJ IDEA Community/Ultimate or VS Code.

  4. Get an LLM API Key:

    • OpenAI: Go to platform.openai.com, create an account, and generate a new API key. Keep it secure!
    • Google AI Studio: Go to aistudio.google.com, sign in with your Google account, and generate an API key.
  5. Configure API Key as Environment Variable: It’s crucial not to hardcode your API keys in your code. Set them as environment variables.

    • macOS/Linux (add to ~/.bashrc, ~/.zshrc, or ~/.profile):
      export OPENAI_API_KEY="YOUR_OPENAI_API_KEY_HERE"
      # Or for Google Gemini:
      export GOOGLE_API_KEY="YOUR_GOOGLE_API_KEY_HERE"
      
      Remember to source ~/.bashrc (or your relevant file) after adding.
    • Windows: Go to System Properties -> Environment Variables, then add a new system variable OPENAI_API_KEY (or GOOGLE_API_KEY) with your key value.

Now your development environment is ready to start building!


2. Core Concepts and Fundamentals of AI Agents in Java

In this section, we’ll dive into the fundamental building blocks for creating AI agents in Java with Spring Boot, primarily using Spring AI, which provides a streamlined and opinionated way to interact with various LLMs and build agentic workflows.

2.1 The ChatClient - Your Gateway to LLMs

The ChatClient is the central interface in Spring AI for interacting with chat models (LLMs). It provides a fluent API, making it easy to send prompts and receive responses.

Detailed Explanation:

The ChatClient abstracts away the specifics of different LLM providers (OpenAI, Google Gemini, Azure OpenAI, etc.). You configure which model to use via properties, and Spring AI handles the underlying API calls. It’s similar in concept to Spring’s WebClient or JdbcTemplate – a high-level abstraction for a common task.

Code Examples:

Let’s start with a “Hello World” equivalent for an LLM: a simple chatbot that echoes and enhances your message.

1. Project Setup (using Spring Initializr):

Go to start.spring.io and create a new Maven project with the following dependencies:

  • Spring Web
  • Spring AI OpenAI Starter (if using OpenAI)
  • Spring AI Gemini Starter (if using Google Gemini)

Example pom.xml snippet for OpenAI:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>3.2.5</version> <!-- Use a recent stable version -->
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.example.ai</groupId>
    <artifactId>simple-ai-agent</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>simple-ai-agent</name>
    <description>Demo project for Spring AI Agent</description>
    <properties>
        <java.version>17</java.version>
        <spring-ai.version>0.8.1</spring-ai.version> <!-- Check for latest stable Spring AI version -->
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <!-- For OpenAI -->
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-openai-spring-boot-starter</artifactId>
        </dependency>
        <!-- For Google Gemini (if you prefer, comment out OpenAI and add this) -->
        <!--
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-vertexai-gemini-spring-boot-starter</artifactId>
        </dependency>
        -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.ai</groupId>
                <artifactId>spring-ai-bom</artifactId>
                <version>${spring-ai.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
</project>

2. Configure application.properties:

In src/main/resources/application.properties, configure your API key and chosen model. For OpenAI:

spring.ai.openai.api-key=${OPENAI_API_KEY}
spring.ai.openai.chat.options.model=gpt-4o-mini # Or gpt-3.5-turbo, gpt-4, etc.

For Google Gemini:

spring.ai.vertexai.gemini.api-key=${GOOGLE_API_KEY}
spring.ai.vertexai.gemini.chat.options.model=gemini-pro # Or gemini-1.5-flash, etc.

Make sure the environment variable (OPENAI_API_KEY or GOOGLE_API_KEY) is set as described in the setup section.

3. Create a Simple Chat Controller:

package com.example.ai.simple_ai_agent;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class SimpleChatController {

    private final ChatClient chatClient;

    public SimpleChatController(ChatClient.Builder chatClientBuilder) {
        this.chatClient = chatClientBuilder.build();
    }

    @GetMapping("/chat")
    public String chat(@RequestParam(value = "message", defaultValue = "Tell me a joke!") String message) {
        // Send a simple user message to the LLM
        return chatClient.prompt()
                .user(message)
                .call()
                .content();
    }

    @GetMapping("/chat/enhanced")
    public String enhancedChat(@RequestParam(value = "message", defaultValue = "Explain quantum physics briefly.") String message) {
        // Provide a system message to guide the LLM's behavior
        String systemPrompt = "You are a witty, helpful assistant. Respond concisely and with a touch of humor.";

        return chatClient.prompt()
                .system(systemPrompt)
                .user(message)
                .call()
                .content();
    }
}

Running and Testing:

  1. Run your Spring Boot application (e.g., mvn spring-boot:run or from your IDE).
  2. Open your browser or use curl:
    • http://localhost:8080/chat
    • http://localhost:8080/chat?message=What is Spring Boot?
    • http://localhost:8080/chat/enhanced
    • http://localhost:8080/chat/enhanced?message=What is the meaning of life?"

Expected Output for /chat/enhanced?message=What is the meaning of life? (example, LLM responses vary):

Well, the meaning of life is a bit like a rubber chicken: squishy, unexpected, and often leaves you wondering why you're holding it. But it's probably best experienced with a good cup of coffee and some friendly company!

Exercises/Mini-Challenges:

  1. Modify the enhancedChat endpoint to change the system prompt to make the AI a “very serious and analytical professor.” Observe the change in tone.
  2. Add a new endpoint /chat/french that translates the user’s message into French. You’ll need to instruct the LLM to act as a French translator.

2.2 System Messages and User Messages

As seen in the enhancedChat example, a system message is crucial for defining the AI’s persona, role, and constraints.

Detailed Explanation:

  • System Message: This acts as the “prime directive” for the AI. It sets the context for the entire interaction, guiding the LLM’s behavior, tone, and knowledge focus. It’s often the first message in a conversation. Good system messages are clear, concise, and direct.
  • User Message: This is the actual query or instruction from the human user.
  • AI Message: The response generated by the AI.

These different “roles” (system, user, AI) are fundamental to how conversational models maintain context and follow instructions.

Code Examples:

Let’s expand on the concept of roles.

package com.example.ai.simple_ai_agent;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.messages.UserMessage;
import org.springframework.ai.chat.model.ChatResponse;
import org.springframework.ai.chat.prompt.Prompt;
import org.springframework.ai.chat.prompt.SystemPromptTemplate;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

import java.util.Map;

@RestController
public class PromptController {

    private final ChatClient chatClient;

    public PromptController(ChatClient.Builder chatClientBuilder) {
        this.chatClient = chatClientBuilder.build();
    }

    @GetMapping("/role-play")
    public String rolePlayChat(
            @RequestParam(value = "role", defaultValue = "historian") String role,
            @RequestParam(value = "topic", defaultValue = "World War II") String topic) {

        // Define a system prompt template with a placeholder for the role
        SystemPromptTemplate systemPromptTemplate = new SystemPromptTemplate(
                "You are a professional and knowledgeable {role}. " +
                "Your task is to provide insightful information about the given topic. " +
                "Keep your responses concise and informative."
        );

        // Fill the placeholder in the system prompt
        String systemPrompt = systemPromptTemplate.create(Map.of("role", role)).getContents();

        // Create the user message
        UserMessage userMessage = new UserMessage("Tell me something interesting about " + topic + ".");

        // Construct the full prompt with both system and user messages
        Prompt prompt = new Prompt(userMessage, systemPrompt);

        // Call the chat client
        ChatResponse chatResponse = chatClient.prompt(prompt).call().chatResponse();

        return chatResponse.getResult().getOutput().getContent();
    }
}

Running and Testing:

  1. Run your Spring Boot application.
  2. Test with different roles and topics:
    • http://localhost:8080/role-play?role=scientist&topic=black holes
    • http://localhost:8080/role-play?role=poet&topic=rainy days
    • http://localhost:8080/role-play?role=chef&topic=pasta

Expected Output for /role-play?role=scientist&topic=black holes:

As a scientist, I find black holes fascinating regions of spacetime where gravity is so strong that nothing, not even light, can escape. They form from the remnants of massive stars and can range from stellar-mass to supermassive sizes at the centers of galaxies. Their event horizon marks the point of no return.

Exercises/Mini-Challenges:

  1. Add another @RequestParam called tone (e.g., sarcastic, optimistic). Modify the SystemPromptTemplate to include this tone and observe how the AI’s response changes.
  2. Implement a scenario where the AI is a “travel agent.” The system prompt defines its role. The user message provides a destination and budget. The AI should generate a suggestion for an activity within that budget.

2.3 Conversation Memory

For an AI agent to have a coherent conversation, it needs to remember past interactions. Spring AI provides ChatMemory for this.

Detailed Explanation:

ChatMemory stores the history of messages (user input and AI responses) within a specific conversation. When you make a new request, this history is sent along with the new user message, giving the LLM context. Without memory, each interaction would be like talking to the AI for the very first time.

Spring AI offers different ChatMemory implementations, such as MessageWindowChatMemory (remembers the last N messages) or token-based memory.

Code Examples:

Let’s create an endpoint that maintains conversation memory.

1. Update SimpleAiAgentApplication.java (Main Class) to provide ChatMemoryProvider:

package com.example.ai.simple_ai_agent;

import org.springframework.ai.chat.memory.ChatMemory;
import org.springframework.ai.chat.memory.MessageWindowChatMemory;
import org.springframework.context.annotation.Bean;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class SimpleAiAgentApplication {

    public static void main(String[] args) {
        SpringApplication.run(SimpleAiAgentApplication.class, args);
    }

    // Configure a global ChatMemoryProvider for the application
    @Bean
    public ChatMemory chatMemory() {
        // This remembers the last 10 messages across the entire application.
        // For a real agent, you'd want per-user or per-session memory,
        // which we'll cover in intermediate topics.
        return new MessageWindowChatMemory(); // Default constructor has a reasonable window
    }
}

Correction: The chatMemory() bean defined above provides a singleton ChatMemory instance, meaning all users would share the same conversation history. For a proper agent, we need per-session memory. Spring AI allows configuring this directly in the ChatClient.Builder.

Let’s adjust the SimpleChatController to use a ChatClient configured with per-conversation memory.

package com.example.ai.simple_ai_agent;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.memory.ChatMemory;
import org.springframework.ai.chat.memory.MessageWindowChatMemory;
import org.springframework.ai.chat.client.advisor.MessageChatMemoryAdvisor; // Import this
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class StatefulChatController {

    private final ChatClient chatClient;
    // We'll use a simple in-memory map for demonstration.
    // In a real application, this would be backed by a persistent store.
    private final java.util.Map<String, ChatMemory> conversationMemories = new java.util.concurrent.ConcurrentHashMap<>();

    public StatefulChatController(ChatClient.Builder chatClientBuilder) {
        // The chatClient will be built for each request with a specific advisor
        this.chatClient = chatClientBuilder.build();
    }

    @GetMapping("/chat/stateful")
    public String statefulChat(
            @RequestParam(value = "message") String message,
            @RequestParam(value = "sessionId", defaultValue = "defaultUser") String sessionId) {

        // Get or create ChatMemory for the specific session
        ChatMemory sessionMemory = conversationMemories.computeIfAbsent(sessionId, id ->
                MessageWindowChatMemory.builder().maxMessages(10).build()); // Retain last 10 messages

        // Apply the MessageChatMemoryAdvisor to the current ChatClient prompt
        // This advisor will manage adding messages to and retrieving messages from the sessionMemory
        return chatClient.prompt()
                .advisors(new MessageChatMemoryAdvisor(sessionMemory))
                .user(message)
                .call()
                .content();
    }

    // Optional: An endpoint to clear memory for a session
    @GetMapping("/chat/clear-memory")
    public String clearMemory(@RequestParam(value = "sessionId", defaultValue = "defaultUser") String sessionId) {
        conversationMemories.remove(sessionId);
        return "Memory cleared for session: " + sessionId;
    }
}

Running and Testing:

  1. Run your Spring Boot application.
  2. Interact with the stateful chat:
    • http://localhost:8080/chat/stateful?message=My favorite color is blue. (Use sessionId=user1 for instance)
    • http://localhost:8080/chat/stateful?message=What is it again? (Same sessionId=user1)

Expected Output (example for sessionId=user1):

  • First call: /chat/stateful?message=My favorite color is blue.&sessionId=user1
    That's a lovely choice! Blue is often associated with calm and peace. How can I help you today?
    
  • Second call: /chat/stateful?message=What is it again?&sessionId=user1
    You mentioned your favorite color is blue.
    

Exercises/Mini-Challenges:

  1. Experiment with the maxMessages parameter in MessageWindowChatMemory. What happens if you set it to 1? Test by asking multiple questions and then asking a question that relies on context beyond the last message.
  2. Implement an endpoint /chat/summarize-memory?sessionId=... that asks the LLM to summarize the entire conversation history for a given sessionId. You’ll need to fetch the ChatMemory and construct a prompt asking for a summary of its contents.

3. Intermediate Topics: Building Smarter Agents

Now that we have the fundamentals, let’s explore how to make our AI agents more powerful by enabling them to use external tools and leverage external knowledge.

3.1 Tool Calling (Function Calling)

This is a cornerstone of agentic AI: giving the LLM the ability to “use” external functions or APIs.

Detailed Explanation:

As discussed in the introduction, LLMs don’t execute code. Instead, when you provide an LLM with descriptions of functions it can use (often in a schema format), it can generate structured output suggesting a function call and its arguments. The Spring Boot application then intercepts this suggestion, executes the actual Java method, and feeds the result back to the LLM.

This allows agents to:

  • Access real-time information (e.g., current weather, stock prices).
  • Perform actions in external systems (e.g., book a flight, create a calendar event).
  • Query databases or internal APIs.

Code Examples:

Let’s create a simple weather agent that can tell you the current weather for a city.

1. Define a “Tool” (a simple Java method):

package com.example.ai.simple_ai_agent;

import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.stereotype.Service;
import org.springframework.ai.tool.ToolFunction; // Import this

import java.util.function.Function;

@Configuration
public class ToolConfiguration {

    // Define a record for the tool's input
    public record WeatherRequest(String city) {}
    // Define a record for the tool's output
    public record WeatherResponse(String city, String temperature, String conditions) {}

    // This method simulates fetching weather data
    // In a real app, this would call a weather API
    @Service("weatherService") // Give it a name for the LLM to refer to
    public static class WeatherService {
        public WeatherResponse getCurrentWeather(WeatherRequest request) {
            String city = request.city();
            // Simulate different weather for different cities
            if ("London".equalsIgnoreCase(city)) {
                return new WeatherResponse(city, "15°C", "Partly cloudy");
            } else if ("New York".equalsIgnoreCase(city)) {
                return new WeatherResponse(city, "22°C", "Sunny");
            } else if ("Tokyo".equalsIgnoreCase(city)) {
                return new WeatherResponse(city, "20°C", "Rainy");
            } else {
                return new WeatherResponse(city, "N/A", "Unknown");
            }
        }
    }

    // Expose the tool as a Spring Bean using ToolFunction
    @Bean
    public Function<WeatherRequest, WeatherResponse> getCurrentWeatherTool(
            WeatherService weatherService) {
        return weatherService::getCurrentWeather;
    }
}

2. Update SimpleChatController to use the tool:

package com.example.ai.simple_ai_agent;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class ToolCallingChatController {

    private final ChatClient chatClient;

    // Inject the ChatClient.Builder. Spring AI will automatically discover the ToolFunction beans.
    public ToolCallingChatController(ChatClient.Builder chatClientBuilder) {
        this.chatClient = chatClientBuilder.build();
    }

    @GetMapping("/chat/weather-agent")
    public String weatherAgent(
            @RequestParam(value = "message", defaultValue = "What's the weather like in New York?") String message) {

        // When the user asks about weather, the LLM will "see" the `getCurrentWeatherTool`
        // and infer it needs to call it.
        return chatClient.prompt()
                .user(message)
                .call()
                .content();
    }

    @GetMapping("/chat/mixed-agent")
    public String mixedAgent(
            @RequestParam(value = "message", defaultValue = "What is the weather in London and then tell me a joke?") String message) {
        // The agent can call tools and then continue generating text
        String systemPrompt = "You are a helpful assistant. Use tools when necessary and always be friendly.";
        return chatClient.prompt()
                .system(systemPrompt)
                .user(message)
                .call()
                .content();
    }
}

Running and Testing:

  1. Run your Spring Boot application.
  2. Test the weather agent:
    • http://localhost:8080/chat/weather-agent?message=What is the weather in London?
    • http://localhost:8080/chat/weather-agent?message=How about Tokyo?
    • http://localhost:8080/chat/weather-agent?message=What is the capital of France? (The AI won’t use the tool here, as it’s not relevant)
    • http://localhost:8080/chat/mixed-agent

Expected Output for /chat/weather-agent?message=What is the weather in New York?:

The current weather in New York is 22°C and sunny.

Expected Output for /chat/mixed-agent:

In London, it's currently 15°C and partly cloudy.

Why don't scientists trust atoms?
Because they make up everything! 😂

Exercises/Mini-Challenges:

  1. Create another tool called StockPriceService that takes a stock symbol (e.g., AAPL, GOOG) and returns a simulated stock price. Integrate it into a new endpoint.
  2. Modify the weatherAgent to include a system prompt that explicitly tells the AI to use the getCurrentWeather tool if the user asks about weather. What difference does this make?

3.2 Retrieval Augmented Generation (RAG)

LLMs have broad knowledge, but they are limited by their training data. RAG allows agents to retrieve specific, up-to-date, or proprietary information from an external knowledge base (like your company’s documents) and “augment” the LLM’s prompt with this retrieved context before generating a response.

Detailed Explanation:

The core idea of RAG is:

  1. Ingestion: Your documents (text, PDFs, internal wikis) are broken into smaller chunks (e.g., paragraphs).
  2. Embedding: Each chunk is converted into a numerical vector (an “embedding”) representing its semantic meaning.
  3. Vector Store: These embeddings are stored in a special database called a “vector store” (e.g., Chroma, Pinecone, Azure Cosmos DB for PostgreSQL with pgvector).
  4. Retrieval: When a user asks a question, their question is also converted into an embedding. The vector store then finds document chunks whose embeddings are “most similar” (semantically relevant) to the query.
  5. Augmentation: These retrieved chunks of text are then added to the prompt that’s sent to the LLM.
  6. Generation: The LLM generates its answer using both its general knowledge and the specific context provided by the retrieved documents. This helps reduce hallucinations and grounds the AI’s response in factual, relevant information.

Spring AI provides excellent abstractions for EmbeddingModel (to create embeddings) and VectorStore (to store and retrieve them).

Code Examples:

Let’s build a simple RAG application that answers questions about a hypothetical company’s policy document.

1. Add Dependencies: In pom.xml, add starters for an embedding model (e.g., OpenAI) and a vector store (e.g., spring-ai-vectorstore-simple for in-memory, or spring-ai-chroma-spring-boot-starter for Chroma DB).

        <!-- For Embedding Model (if using OpenAI) -->
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-openai-spring-boot-starter</artifactId>
        </dependency>
        <!-- For in-memory Vector Store (simple for demo) -->
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-vector-store-simple</artifactId>
        </dependency>
        <!-- For actual Vector DB like Chroma (uncomment and configure if needed) -->
        <!--
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-chroma-spring-boot-starter</artifactId>
        </dependency>
        -->

2. Create a VectorStore Configuration:

package com.example.ai.simple_ai_agent;

import org.springframework.ai.document.Document;
import org.springframework.ai.embedding.EmbeddingClient;
import org.springframework.ai.vectorstore.SimpleVectorStore;
import org.springframework.ai.vectorstore.VectorStore;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.Resource;
import org.springframework.boot.ApplicationRunner; // To load data on startup

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.util.List;

@Configuration
public class RagConfiguration {

    // Inject a dummy policy document from resources
    @Value("classpath:/data/company-policy.txt")
    private Resource companyPolicy;

    @Bean
    public VectorStore vectorStore(EmbeddingClient embeddingClient) {
        // For demonstration, SimpleVectorStore is in-memory.
        // For production, you'd use a persistent vector database.
        return new SimpleVectorStore(embeddingClient);
    }

    // This ApplicationRunner will load our policy document into the vector store on startup
    @Bean
    public ApplicationRunner initVectorStore(VectorStore vectorStore) {
        return args -> {
            // Read the policy document
            String policyContent = new String(companyPolicy.getInputStream().readAllBytes(), StandardCharsets.UTF_8);

            // Create a Document from the content. Spring AI will handle splitting and embedding.
            Document policyDocument = new Document(policyContent);

            // Add the document to the vector store
            vectorStore.add(List.of(policyDocument));
            System.out.println("Company policy loaded into vector store.");
        };
    }
}

3. Create src/main/resources/data/company-policy.txt:

## Company Vacation Policy

Employees are eligible for 20 days of paid vacation per calendar year. New employees accrue vacation days at a rate of 1.67 days per month for the first year. After the first year, all 20 days are available at the beginning of the calendar year.

Vacation requests must be submitted through the HR portal at least two weeks in advance. All requests are subject to manager approval based on team staffing needs. Unused vacation days can be carried over for a maximum of 5 days into the next calendar year. Any days exceeding this limit will be forfeited.

## Remote Work Policy

Our company supports remote work arrangements. Employees must maintain a suitable home office environment and ensure reliable internet connectivity. Regular check-ins with managers are required. For international remote work, specific country regulations and tax implications must be discussed with HR prior to any arrangement. Failure to comply may result in termination.

## Expense Reimbursement Policy

Employees can claim reimbursement for approved business expenses, including travel, meals, and professional development courses. Receipts are mandatory for all claims. Expenses exceeding $50 require prior manager approval. Reimbursement requests must be submitted within 30 days of the expense incurrence.

4. Create a RAG Controller:

package com.example.ai.simple_ai_agent;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.client.advisor.RetrievalAdvisor; // Import for RAG
import org.springframework.ai.vectorstore.VectorStore;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class RagChatController {

    private final ChatClient chatClient;
    private final VectorStore vectorStore;

    public RagChatController(ChatClient.Builder chatClientBuilder, VectorStore vectorStore) {
        this.vectorStore = vectorStore;
        // Configure ChatClient with a RetrievalAdvisor to enable RAG
        this.chatClient = chatClientBuilder
                .defaultAdvisors(new RetrievalAdvisor(vectorStore)) // Use the vector store for retrieval
                .build();
    }

    @GetMapping("/chat/policy-agent")
    public String policyAgent(
            @RequestParam(value = "query", defaultValue = "How many vacation days can I carry over?") String query) {

        String systemPrompt = "You are a helpful HR assistant. Answer questions based on the provided policy documents only. If you cannot find the answer, politely state that the information is not available in the documents.";

        return chatClient.prompt()
                .system(systemPrompt)
                .user(query)
                .call()
                .content();
    }
}

Running and Testing:

  1. Run your Spring Boot application. Observe the “Company policy loaded into vector store.” message.
  2. Query the policy agent:
    • http://localhost:8080/chat/policy-agent?query=How many vacation days can I carry over?
    • http://localhost:8080/chat/policy-agent?query=What is the remote work policy?
    • http://localhost:8080/chat/policy-agent?query=What is the company dress code? (This information is NOT in the document)

Expected Output for /chat/policy-agent?query=How many vacation days can I carry over?:

You can carry over a maximum of 5 unused vacation days into the next calendar year. Any days exceeding this limit will be forfeited.

Expected Output for /chat/policy-agent?query=What is the company dress code?:

I apologize, but the company dress code policy is not available in the documents I have access to.

Exercises/Mini-Challenges:

  1. Add more content to company-policy.txt (e.g., a section on “IT Support Policy”). Reload the application and ask questions about the new policy.
  2. Modify the policyAgent to also include chat memory, so you can have a conversation about the policy. Ensure the memory and RAG work together. (Hint: Add MessageChatMemoryAdvisor to the defaultAdvisors along with RetrievalAdvisor).

4. Advanced Topics and Best Practices

As your AI agents become more complex, you’ll encounter patterns for managing multi-step tasks, combining capabilities, and ensuring reliability.

4.1 Agentic Workflows (Spring AI Patterns)

Anthropic’s research, and Spring AI’s implementation, highlight several patterns for building effective agentic systems. These distinguish between Workflows (predefined paths) and Agents (LLMs dynamically directing themselves).

Detailed Explanation:

Instead of relying on a single, massive prompt, agentic workflows break down complex problems into smaller, manageable steps. This improves accuracy, makes debugging easier, and allows for programmatic checks and human intervention at various stages.

Spring AI implements several of these patterns:

  • Chain Workflow: Decomposes a task into a sequence of steps, where each LLM call processes the output of the previous one.
  • Parallelization Workflow: Runs multiple LLM calls simultaneously on independent subtasks and aggregates the results.
  • Routing Workflow: Uses an LLM to analyze input and route it to a specialized prompt or handler based on its classification.
  • Orchestrator-Workers: A central LLM orchestrates task decomposition, and specialized worker LLMs handle subtasks.
  • Evaluator-Optimizer: One LLM generates a response, and another LLM evaluates it, providing feedback for iterative refinement.

Let’s illustrate the Chain Workflow and Routing Workflow with examples.

4.1.1 Chain Workflow

Detailed Explanation:

Imagine you need to process text through several transformations: extract data, format it, then summarize. A Chain Workflow handles this by linking sequential LLM calls, where the output of one step becomes the input for the next.

Code Example: Data Transformation Chain

Let’s say we have a raw performance report and want to:

  1. Extract key metrics.
  2. Convert them to a standardized format.
  3. Summarize them into a brief executive overview.

1. Create the ChainWorkflow class:

package com.example.ai.simple_ai_agent;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.stereotype.Service;

@Service
public class ChainWorkflow {

    private final ChatClient chatClient;
    private final String[] systemPrompts;

    public ChainWorkflow(ChatClient.Builder chatClientBuilder) {
        this.chatClient = chatClientBuilder.build();
        this.systemPrompts = new String[] {
            """
            STEP 1: You are an expert data extractor.
            From the given text, identify and extract all numerical performance metrics and their corresponding descriptions.
            Present them as a list, one per line, in the format: "Metric Name: Value".
            Example: "Customer Satisfaction: 92 points"
            """,
            """
            STEP 2: You are an expert data formatter.
            Take the list of metrics provided. For each metric, convert numerical values to percentages where appropriate (e.g., "92 points" becomes "92%"). If a value is already a percentage or a currency, retain its format.
            Maintain the format: "Metric Name: Value".
            Example: "Customer Satisfaction: 92%"
            """,
            """
            STEP 3: You are an executive summarizer.
            Take the formatted list of metrics and write a concise, one-paragraph executive summary (max 3 sentences) highlighting the most significant points.
            Focus on key achievements or areas needing attention.
            """
        };
    }

    public String processReport(String rawReport) {
        String currentOutput = rawReport;
        System.out.println("--- Starting Chain Workflow ---");
        System.out.println("Initial Input:\n" + rawReport);

        for (int i = 0; i < systemPrompts.length; i++) {
            System.out.println("\n--- Step " + (i + 1) + " ---");
            // Combine the system prompt for the current step with the output from the previous step
            String stepInput = currentOutput; // Input to the LLM for this step is the previous output
            currentOutput = chatClient.prompt()
                    .system(systemPrompts[i]) // Apply the specific system prompt for this step
                    .user(stepInput)
                    .call()
                    .content();
            System.out.println("Output:\n" + currentOutput);
        }
        System.out.println("\n--- Chain Workflow Finished ---");
        return currentOutput;
    }
}

2. Create a Controller for the Chain Workflow:

package com.example.ai.simple_ai_agent;

import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class WorkflowController {

    private final ChainWorkflow chainWorkflow;

    public WorkflowController(ChainWorkflow chainWorkflow) {
        this.chainWorkflow = chainWorkflow;
    }

    @PostMapping("/workflow/chain-report")
    public String runChainWorkflow(@RequestBody String reportContent) {
        return chainWorkflow.processReport(reportContent);
    }
}

Running and Testing:

  1. Run your Spring Boot application.
  2. Send a POST request to http://localhost:8080/workflow/chain-report with a raw report body. Example Request Body:
    Q4 Performance Summary: Our customer satisfaction score reached 92 points this quarter, a significant increase from 80. Revenue grew by 12% compared to last year, totaling $1.2M. Market share is now at 25% in our primary market, up from 20%. Customer churn decreased to 3% from 7%. New user acquisition cost is $50 per user. Product adoption rate increased to 85%. Employee satisfaction is at 88 points. Operating margin improved to 30%.
    

Expected Output (console and HTTP response, will vary slightly):

--- Starting Chain Workflow ---
Initial Input:
Q4 Performance Summary: Our customer satisfaction score reached 92 points this quarter, a significant increase from 80. Revenue grew by 12% compared to last year, totaling $1.2M. Market share is now at 25% in our primary market, up from 20%. Customer churn decreased to 3% from 7%. New user acquisition cost is $50 per user. Product adoption rate increased to 85%. Employee satisfaction is at 88 points. Operating margin improved to 30%.

--- Step 1 ---
Output:
Customer Satisfaction: 92 points
Revenue Growth: 12%
Market Share: 25%
Customer Churn: 3%
Previous Customer Churn: 7%
New User Acquisition Cost: $50
Product Adoption Rate: 85%
Employee Satisfaction: 88 points
Operating Margin: 30%

--- Step 2 ---
Output:
Customer Satisfaction: 92%
Revenue Growth: 12%
Market Share: 25%
Customer Churn: 3%
Previous Customer Churn: 7%
New User Acquisition Cost: $50
Product Adoption Rate: 85%
Employee Satisfaction: 88%
Operating Margin: 30%

--- Step 3 ---
Output:
This quarter saw strong performance with customer satisfaction reaching 92% and product adoption at 85%. Revenue grew by 12% with a healthy 30% operating margin, indicating efficient operations. Customer churn was significantly reduced to 3%, highlighting improved retention.

--- Chain Workflow Finished ---

4.1.2 Routing Workflow

Detailed Explanation:

A Routing Workflow is useful when your agent needs to handle different types of user requests with specialized logic. An initial LLM call acts as a “router,” classifying the intent and then directing the query to a specific, specialized prompt or tool.

Code Example: Customer Support Router

Let’s create an agent that can route customer queries to either a billing specialist or a technical support agent based on the query’s content.

1. Create the RoutingWorkflow class:

package com.example.ai.simple_ai_agent;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.prompt.PromptTemplate;
import org.springframework.stereotype.Service;

import java.util.Map;

@Service
public class RoutingWorkflow {

    private final ChatClient chatClient;

    public RoutingWorkflow(ChatClient.Builder chatClientBuilder) {
        this.chatClient = chatClientBuilder.build();
    }

    // A tool/function to simulate specialized handlers
    public String handleBilling(String query) {
        return "Billing Specialist: We are looking into your billing query: '" + query + "'. Please provide your account number.";
    }

    public String handleTechnical(String query) {
        return "Technical Support: I understand you have a technical issue with: '" + query + "'. Can you please restart your device?";
    }

    public String handleGeneral(String query) {
        return "Customer Service: Thank you for your general inquiry: '" + query + "'. How else can I assist?";
    }

    public String routeAndRespond(String userQuery) {
        // The LLM acts as the router to decide which handler is most appropriate
        String routingPrompt = """
            Given the user query: "{query}", classify its intent as either 'billing', 'technical', or 'general'.
            Respond with *only* the chosen category.
            Example:
            User: "My internet is not working."
            Output: technical
            User: "I was charged twice."
            Output: billing
            User: "I want to know your operating hours."
            Output: general
            """;

        String category = chatClient.prompt()
                .system(routingPrompt)
                .user(userQuery)
                .call()
                .content()
                .trim()
                .toLowerCase();

        System.out.println("Detected category: " + category);

        // Based on the category, invoke the specialized handler
        switch (category) {
            case "billing":
                return handleBilling(userQuery);
            case "technical":
                return handleTechnical(userQuery);
            case "general":
                return handleGeneral(userQuery);
            default:
                return "I couldn't classify your request. Please try rephrasing.";
        }
    }
}

2. Create a Controller for the Routing Workflow:

package com.example.ai.simple_ai_agent;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class RoutingController {

    private final RoutingWorkflow routingWorkflow;

    public RoutingController(RoutingWorkflow routingWorkflow) {
        this.routingWorkflow = routingWorkflow;
    }

    @GetMapping("/workflow/route-query")
    public String routeCustomerQuery(@RequestParam(value = "query") String query) {
        return routingWorkflow.routeAndRespond(query);
    }
}

Running and Testing:

  1. Run your Spring Boot application.
  2. Test with different queries:
    • http://localhost:8080/workflow/route-query?query=My last bill was incorrect.
    • http://localhost:8080/workflow/route-query?query=My software is crashing constantly.
    • http://localhost:8080/workflow/route-query?query=When do you close on Fridays?
    • http://localhost:8080/workflow/route-query?query=What is the capital of Sweden? (This might fall to ‘general’ or fail classification)

Expected Output for /workflow/route-query?query=My last bill was incorrect.:

Detected category: billing
Billing Specialist: We are looking into your billing query: 'My last bill was incorrect.'. Please provide your account number.

Exercises/Mini-Challenges:

  1. Add a new category, e.g., “sales,” and implement a handleSales method. Update the routing prompt to include this new category.
  2. Instead of returning simple strings, modify handleBilling, handleTechnical, and handleGeneral to make a further LLM call, providing a more detailed, category-specific response. For example, handleBilling could use another ChatClient prompt like “As a billing specialist, politely ask the user for their account number and explain why it’s needed for the query: ‘{query}’.”

4.2 Multi-Agent Systems (Conceptual)

While Spring AI is excellent for building individual agents and orchestrating workflows, true “multi-agent systems” (MAS) involve multiple, often autonomous agents collaborating to solve a larger problem.

Detailed Explanation:

In a MAS, different agents might have specialized roles (e.g., a “Researcher Agent,” a “Summarizer Agent,” a “Planner Agent”). They communicate and delegate tasks to each other. This is a more advanced architectural pattern often seen in frameworks like Google ADK or the underlying concepts of Microsoft’s AutoGen.

Key aspects include:

  • Agent Encapsulation: Each agent has a clear role, set of capabilities (tools), and instructions.
  • Orchestration/Coordination: A central orchestrator or a peer-to-peer communication mechanism directs which agent handles which part of a task.
  • Communication: Agents need ways to exchange information and task assignments.
  • Shared Memory/State: To maintain context across agents, a shared memory (e.g., a database) is crucial.

Best Practices for AI Agent Development:

  1. Start Simple, Iterate: Begin with the simplest possible agent. Add complexity (tools, RAG, memory) incrementally as needed.
  2. Clear System Prompts: Invest time in crafting precise, unambiguous system prompts. Define the agent’s role, constraints, and desired output format.
  3. Define Tools Well: Provide clear names, descriptions, and JSON schemas for your tools. The LLM relies on these descriptions to decide when and how to use them.
  4. Error Handling and Guardrails: Agents can “hallucinate” or fail. Implement robust error handling for tool calls, and consider “guardrails” (e.g., explicit rules checked programmatically before actions are taken) to ensure safety and reliability.
  5. Observability: Implement logging, tracing, and monitoring. It’s crucial to see the LLM’s thought process (the intermediate prompts and responses) and tool invocations to debug effectively.
  6. Manage Context: Efficiently use conversation memory and RAG. Be mindful of token limits and costs.
  7. Cost Awareness: LLM API calls can incur costs. Optimize prompts, use smaller models where appropriate, and leverage caching mechanisms.
  8. Security: Treat LLM inputs and outputs as untrusted. Sanitize inputs, validate outputs, and manage API keys securely.

5. Guided Projects

Let’s put everything together with two guided projects.

Project 1: Intelligent Customer Service Assistant with RAG and Tool Calling

Objective: Build a Spring Boot AI agent that acts as a customer service representative. It should be able to:

  1. Answer questions based on an internal knowledge base (RAG).
  2. Look up customer order status using a mock API (tool calling).
  3. Maintain conversation memory.

Problem Statement: A company, “TechGadget Inc.,” receives many customer inquiries. They need an AI assistant to handle common questions about their products and policies (from a knowledge base) and retrieve real-time order status when requested by a customer.

Step-by-Step Implementation:

Prerequisites:

  • A fresh Spring Boot project (similar to the initial setup).
  • Dependencies: spring-ai-openai-spring-boot-starter, spring-ai-vector-store-simple, spring-boot-starter-web.
  • An OpenAI API key (or Google Gemini configured).

Step 1: Setup and Dependencies

Ensure your pom.xml includes:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>3.2.5</version> <!-- Use a recent stable version -->
        <relativePath/>
    </parent>
    <groupId>com.example.techgadget</groupId>
    <artifactId>customer-service-agent</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>customer-service-agent</name>
    <description>Customer Service AI Agent for TechGadget Inc.</description>
    <properties>
        <java.version>17</java.version>
        <spring-ai.version>0.8.1</spring-ai.version> <!-- Check for latest stable Spring AI version -->
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-openai-spring-boot-starter</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.ai</groupId>
            <artifactId>spring-ai-vector-store-simple</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.ai</groupId>
                <artifactId>spring-ai-bom</artifactId>
                <version>${spring-ai.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>
    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>
</project>

And application.properties:

spring.ai.openai.api-key=${OPENAI_API_KEY}
spring.ai.openai.chat.options.model=gpt-4o-mini

Step 2: Knowledge Base (RAG)

Create src/main/resources/data/techgadget-kb.txt:

## TechGadget Product Warranty
All TechGadget products come with a one-year limited warranty covering manufacturing defects. This warranty does not cover accidental damage, misuse, or normal wear and tear. To claim warranty, customers must provide proof of purchase.

## TechGadget Return Policy
Customers can return products within 30 days of purchase for a full refund, provided the product is in its original condition and packaging. A 15% restocking fee may apply for opened products. Custom-ordered items are non-returnable.

## TechGadget Support Channels
For technical support, please visit our website's support page or call us at 1-800-TECH-SUP. For sales inquiries, email sales@techgadget.com.

## TechGadget Product Line
We offer a range of smart home devices, including smart thermostats, intelligent lighting systems, and security cameras. Our flagship product is the "GlowOrb Smart Light."

Configure the VectorStore to load this knowledge base:

// src/main/java/com/example/techgadget/RagConfig.java
package com.example.techgadget;

import org.springframework.ai.document.Document;
import org.springframework.ai.embedding.EmbeddingClient;
import org.springframework.ai.vectorstore.SimpleVectorStore;
import org.springframework.ai.vectorstore.VectorStore;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.ApplicationRunner;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.Resource;

import java.nio.charset.StandardCharsets;
import java.util.List;

@Configuration
public class RagConfig {

    @Value("classpath:/data/techgadget-kb.txt")
    private Resource techGadgetKb;

    @Bean
    public VectorStore vectorStore(EmbeddingClient embeddingClient) {
        return new SimpleVectorStore(embeddingClient);
    }

    @Bean
    public ApplicationRunner initVectorStore(VectorStore vectorStore) {
        return args -> {
            String kbContent = new String(techGadgetKb.getInputStream().readAllBytes(), StandardCharsets.UTF_8);
            vectorStore.add(List.of(new Document(kbContent)));
            System.out.println("TechGadget knowledge base loaded into vector store.");
        };
    }
}

Step 3: Order Status Tool

Define a mock OrderService and expose it as a ToolFunction.

// src/main/java/com/example/techgadget/OrderService.java
package com.example.techgadget;

import org.springframework.ai.tool.ToolFunction;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.stereotype.Service;

import java.util.function.Function;

@Configuration
public class OrderServiceConfig {

    public record OrderStatusRequest(String orderId) {}
    public record OrderStatusResponse(String orderId, String status, String estimatedDelivery) {}

    @Service("orderService")
    public static class OrderService {
        public OrderStatusResponse getOrderStatus(OrderStatusRequest request) {
            String orderId = request.orderId();
            // Simulate looking up order status based on orderId
            switch (orderId) {
                case "TG12345":
                    return new OrderStatusResponse(orderId, "Shipped", "October 5, 2025");
                case "TG67890":
                    return new OrderStatusResponse(orderId, "Processing", "October 10, 2025");
                default:
                    return new OrderStatusResponse(orderId, "Not Found", "N/A");
            }
        }
    }

    @Bean
    public Function<OrderStatusRequest, OrderStatusResponse> getOrderStatusTool(
            OrderService orderService) {
        return orderService::getOrderStatus;
    }
}

Step 4: Customer Service Agent Controller

Combine RAG, Tool Calling, and Conversation Memory.

// src/main/java/com/example/techgadget/CustomerServiceController.java
package com.example.techgadget;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.client.advisor.MessageChatMemoryAdvisor;
import org.springframework.ai.chat.client.advisor.RetrievalAdvisor;
import org.springframework.ai.chat.memory.ChatMemory;
import org.springframework.ai.chat.memory.MessageWindowChatMemory;
import org.springframework.ai.vectorstore.VectorStore;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;

@RestController
public class CustomerServiceController {

    private final ChatClient chatClient;
    private final Map<String, ChatMemory> conversationMemories = new ConcurrentHashMap<>();

    public CustomerServiceController(ChatClient.Builder chatClientBuilder, VectorStore vectorStore) {
        // Configure ChatClient with both RetrievalAdvisor and default MessageChatMemoryAdvisor
        this.chatClient = chatClientBuilder
                .defaultAdvisors(
                        new RetrievalAdvisor(vectorStore), // For RAG
                        new MessageChatMemoryAdvisor(new MessageWindowChatMemory()) // Default memory if not overridden per session
                )
                .build();
    }

    @GetMapping("/api/customer-service")
    public String serveCustomer(
            @RequestParam(value = "message") String message,
            @RequestParam(value = "customerId", defaultValue = "guest") String customerId) {

        ChatMemory sessionMemory = conversationMemories.computeIfAbsent(customerId, id ->
                MessageWindowChatMemory.builder().maxMessages(10).build());

        String systemPrompt = """
            You are "TechGadget Support Assistant", a friendly and helpful AI agent for TechGadget Inc.
            Your primary goal is to assist customers with product information, company policies, and order inquiries.
            You have access to internal knowledge base documents for product and policy details.
            You can also check order statuses using the 'getOrderStatusTool'.
            If a customer asks for an order status, you MUST ask for their order ID if not provided.
            Always maintain a polite and professional tone.
            """;

        return chatClient.prompt()
                .system(systemPrompt)
                .advisors(new MessageChatMemoryAdvisor(sessionMemory)) // Override or add specific session memory
                .user(message)
                .call()
                .content();
    }

    @GetMapping("/api/customer-service/clear-session")
    public String clearCustomerSession(@RequestParam(value = "customerId", defaultValue = "guest") String customerId) {
        conversationMemories.remove(customerId);
        return "Customer session cleared for " + customerId;
    }
}

Step 5: Run and Test

  1. Run the application (mvn spring-boot:run).

  2. Open your browser or Postman/curl and try these interactions (using customerId=john_doe):

    • http://localhost:8080/api/customer-service?message=What is the warranty for your products?&customerId=john_doe Expected: Should retrieve info from KB about warranty.

    • http://localhost:8080/api/customer-service?message=What is your return policy?&customerId=john_doe Expected: Should retrieve info from KB about return policy.

    • http://localhost:8080/api/customer-service?message=What is the status of my order?&customerId=john_doe Expected: Should ask for the order ID.

    • http://localhost:8080/api/customer-service?message=My order ID is TG12345.&customerId=john_doe Expected: Should use the getOrderStatusTool and report “Shipped”.

    • http://localhost:8080/api/customer-service?message=And for TG67890?&customerId=john_doe (demonstrates memory) Expected: Should remember the context and use the tool for the new ID.

    • http://localhost:8080/api/customer-service?message=Tell me about the "GlowOrb Smart Light".&customerId=john_doe Expected: Should retrieve product info from KB.

    • http://localhost:8080/api/customer-service?message=What are your operating hours?&customerId=john_doe Expected: Should politely state it cannot find the info (since it’s not in the KB and no tool for it).

Encourage Independent Problem-Solving:

  • How would you add another tool to OrderService that allows the agent to cancel an order? What safety mechanisms (like confirmation) would you implement in the system prompt?
  • Implement a mechanism to store ChatMemory in a database (e.g., H2 in-memory DB or an actual PostgreSQL). This would involve creating a ChatMemoryRepository and a custom ChatMemoryProvider bean.

Project 2: Multi-Step Recipe Planner

Objective: Create a multi-step agent that can plan a recipe based on user input, ensuring the plan includes ingredient gathering, preparation, and cooking steps.

Problem Statement: A user wants to cook a specific dish but needs a detailed, step-by-step plan that goes beyond just ingredients or just instructions. The agent should be able to generate a coherent plan.

Step-by-Step Implementation:

Prerequisites:

  • A fresh Spring Boot project.
  • Dependencies: spring-ai-openai-spring-boot-starter, spring-boot-starter-web.
  • An OpenAI API key (or Google Gemini configured).

Step 1: Setup and Dependencies

Similar pom.xml and application.properties as Project 1, but without vector-store-simple.

Step 2: Define Recipe Step Output Structure

We’ll use Spring AI’s structured output capabilities to ensure the LLM returns recipe steps in a predictable format.

// src/main/java/com/example/recipeplanner/RecipeSteps.java
package com.example.recipeplanner;

import java.util.List;

public record RecipeSteps(
        String recipeName,
        List<String> ingredients,
        List<Step> preparationSteps,
        List<Step> cookingSteps) {

    public record Step(int stepNumber, String description) {}
}

Step 3: Implement the Recipe Planning Workflow

This will use a more sophisticated system prompt to guide the LLM to generate the desired structured output.

// src/main/java/com/example/recipeplanner/RecipePlannerService.java
package com.example.recipeplanner;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.prompt.PromptTemplate;
import org.springframework.stereotype.Service;

import java.util.Map;

@Service
public class RecipePlannerService {

    private final ChatClient chatClient;

    public RecipePlannerService(ChatClient.Builder chatClientBuilder) {
        this.chatClient = chatClientBuilder.build();
    }

    public RecipeSteps planRecipe(String dish) {
        String systemPrompt = """
            You are a master chef and culinary planner.
            Your task is to create a detailed, step-by-step recipe plan for "{dish}".
            The plan must include:
            1. The exact name of the recipe.
            2. A list of all necessary ingredients.
            3. Detailed preparation steps.
            4. Detailed cooking steps.

            Generate the output as a JSON object matching the `RecipeSteps` Java record structure.
            Do not include any other text or explanation.
            RecipeSteps Java record:
            public record RecipeSteps(
                String recipeName,
                List<String> ingredients,
                List<Step> preparationSteps,
                List<Step> cookingSteps) {
                public record Step(int stepNumber, String description) {}
            }
            """;

        // We use .entity() to map the LLM's JSON output directly to our Java record
        return chatClient.prompt()
                .system(new PromptTemplate(systemPrompt).create(Map.of("dish", dish)).getContents())
                .user("Plan the recipe for " + dish)
                .call()
                .entity(RecipeSteps.class);
    }
}

Step 4: Create the Controller

// src/main/java/com/example/recipeplanner/RecipePlannerController.java
package com.example.recipeplanner;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class RecipePlannerController {

    private final RecipePlannerService recipePlannerService;

    public RecipePlannerController(RecipePlannerService recipePlannerService) {
        this.recipePlannerService = recipePlannerService;
    }

    @GetMapping("/api/plan-recipe")
    public RecipeSteps getRecipePlan(@RequestParam(value = "dish", defaultValue = "Spaghetti Carbonara") String dish) {
        return recipePlannerService.planRecipe(dish);
    }
}

Step 5: Run and Test

  1. Run the application (mvn spring-boot:run).
  2. Access the endpoint:
    • http://localhost:8080/api/plan-recipe?dish=Spaghetti Carbonara
    • http://localhost:8080/api/plan-recipe?dish=Chicken Tikka Masala

Expected Output (example for Spaghetti Carbonara, formatted JSON):

{
  "recipeName": "Spaghetti Carbonara",
  "ingredients": [
    "200g spaghetti",
    "100g guanciale (or pancetta)",
    "2 large egg yolks",
    "50g Pecorino Romano cheese, grated",
    "Black pepper, freshly ground",
    "Salt, to taste"
  ],
  "preparationSteps": [
    {
      "stepNumber": 1,
      "description": "Bring a large pot of salted water to a boil for the spaghetti."
    },
    {
      "stepNumber": 2,
      "description": "Cut the guanciale (or pancetta) into small strips or cubes."
    },
    {
      "stepNumber": 3,
      "description": "In a bowl, whisk together the egg yolks, grated Pecorino Romano, and a generous amount of freshly ground black pepper."
    }
  ],
  "cookingSteps": [
    {
      "stepNumber": 1,
      "description": "Cook the spaghetti according to package directions until al dente."
    },
    {
      "stepNumber": 2,
      "description": "While the pasta cooks, heat a large non-stick pan over medium heat. Add the guanciale and cook until crispy and golden. Remove the guanciale with a slotted spoon, leaving the rendered fat in the pan."
    },
    {
      "stepNumber": 3,
      "description": "Before draining the spaghetti, reserve about 1 cup of the pasta cooking water."
    },
    {
      "stepNumber": 4,
      "description": "Drain the spaghetti and immediately add it to the pan with the guanciale fat. Toss well."
    },
    {
      "stepNumber": 5,
      "description": "Quickly add the egg mixture to the pasta, tossing vigorously to coat. Add a splash of reserved pasta water if needed to create a creamy sauce. The heat from the pasta will cook the egg, but prevent it from scrambling."
    },
    {
      "stepNumber": 6,
      "description": "Stir in the crispy guanciale. Season with additional black pepper and salt if necessary. Serve immediately."
    }
  ]
}

Encourage Independent Problem-Solving:

  • How would you add a “difficulty” rating (Easy, Medium, Hard) to the RecipeSteps record and update the prompt to include this?
  • Implement a “nutritional info” tool that the agent could call after planning the recipe to get approximate nutritional values for the ingredients. (This would involve creating another ToolFunction and integrating it with a multi-step prompt).

6. Bonus Section: Further Learning and Resources

Congratulations on making it this far! Building AI agents is a dynamic and evolving field. Here are resources to help you continue your journey.

Official Documentation:

Blogs and Articles:

  • Spring Blog: Look for articles tagged “Spring AI” or “AI” on the official Spring blog (https://spring.io/blog). Christian Tzolov and Josh Long often publish excellent content.
  • Medium: Many developers share their experiences with Spring AI, LangChain4j, and Google ADK. Search for “Spring AI agents,” “Java AI agents,” etc. (e.g., articles by Arfat Bin Kileb, Bayram EKER).
  • Microsoft DevBlogs: (https://devblogs.microsoft.com/) Search for AI + Java content.

YouTube Channels:

  • Spring Developers (VMware Tanzu): Often features talks and tutorials on Spring AI.
  • Google Cloud Tech: Covers Google’s AI offerings, including Gemini and ADK.
  • Various independent tech channels: Search for “Spring AI tutorial” or “Java AI agent” to find practical walkthroughs.

Community Forums/Groups:

  • Stack Overflow: For specific coding questions, tag with spring-ai, java, spring-boot, artificial-intelligence.
  • Spring AI GitHub Discussions: The official Spring AI repository often has active discussions for features and issues.
  • LinkedIn Groups: Join groups focused on Java, Spring Boot, and AI/ML.

Next Steps/Advanced Topics:

  • Advanced Prompt Engineering: Learn about few-shot prompting, chain-of-thought, and tree-of-thought prompting for more complex reasoning.
  • Custom Tool Orchestration: Beyond simple function calling, explore how to dynamically choose, chain, and re-evaluate tools.
  • Multi-Modal Agents: Integrate models that can process and generate images, audio, or video.
  • Advanced Vector Stores: Explore different vector database solutions (Pinecone, Weaviate, Milvus, Qdrant) and their integration with Spring AI.
  • Deployment and MLOps: Learn about deploying and managing AI agents in production environments (Kubernetes, Azure Container Apps, Google Cloud Run) with proper monitoring, logging, and continuous integration/delivery (CI/CD).
  • Evaluator-Optimizer Agents: Delve into building agents that can iteratively refine their outputs based on feedback from another LLM or programmatic checks.
  • Human-in-the-Loop (HITL): Design systems where human oversight and approval are integrated into the agent’s workflow, especially for critical actions.
  • Security and Compliance for AI: Understand the unique security and ethical challenges of AI systems and how to address them in Java.

Keep building, experimenting, and learning! The field of AI agents is rapidly evolving, and your Java skills provide a strong foundation to contribute to this exciting future.