Spring AI Provides Integration with OpenAI and Azure OpenAI

The experimental Spring AI project was introduced during the SpringOne conference and allows the creation of AI applications by using common concepts of Spring. Currently the project integrates Azure OpenAI and OpenAI as AI backends. Use cases like content generation, code generation, semantic search and summarization are supported by the project.

Historically, Python was often used to provide fast access to AI algorithms written in languages such as C and C++. Generative AI, used by solutions such as OpenAI's ChatGPT, makes it possible to access pre-trained models via HTTP. This makes it easier for languages such as Java to interact efficiently with AI algorithms.

The abbreviation ChatGPT stands for Chat Generative Pre-Trained Transformer as it uses pre-trained models. This makes it easier for developers to use AI, similar to using a database. AI no longer requires data scientists to gather data and train a model.

However, the various Java Client APIs for AI solutions such as OpenAI and Azure OpenAI differ, which makes it harder to switch between those solutions. Spring AI provides an abstraction layer on top of those client APIs inspired by the Python libraries LangChain and LlamaIndex. Those libraries are based on similar design values as Spring projects such as modularity, extensibility and integration with various data sources.

Spring AI offers several features such as integration with AI models through a common API. Prompts, optionally using templates comparable to views in Spring MVC, are used to interact with the AI model. Chaining calls to the model is supported, for more difficult questions, in order to break down the problem and solve it piece by piece. Output parsing allows converting the String output to, for example, CSV or JSON. The model might use custom data, such as a project's FAQ, and is able to learn a specific conversation style. Evaluating the answers through testing may be used to maintain the quality of the project.

Spring AI can be used after adding the following repository, as it's an experimental project with only snapshot releases:

<repositories>
    <repository>
        <id>spring-snapshots</id>
        <name>Spring Snapshots</name>
        <url>https://repo.spring.io/snapshot</url>
        <releases>
            <enabled>false</enabled>
        </releases>
    </repository>
</repositories>

When using OpenAI the following dependency may be used:

<dependency>
    <groupId>org.springframework.experimental.ai</groupId>
    <artifactId>spring-ai-openai-spring-boot-starter</artifactId>
    <version>0.2.0-SNAPSHOT</version>
</dependency>

In order to use Azure OpenAI instead of OpenAI, the following dependency may be used:

<dependency>
    <groupId>org.springframework.experimental.ai</groupId>
    <artifactId>spring-ai-azure-openai-spring-boot-starter</artifactId>
    <version>0.2.0-SNAPSHOT</version>
</dependency>

Alternatively, the Spring CLI might be used to create a new project. Spring CLI provides binaries for various operating systems and more information may be found in the documentation. The following command may be used to create a new project for OpenAI:

spring boot new ai

Otherwise, the following command may be used to create a new project for Azure OpenAI:

spring boot new ai-azure

The Spring CLI Getting Started guide offers more information about creating a project with Spring AI.

The API key should be provided, before the project can use OpenAI or Azure OpenAI. For OpenAI a token may be generated on the API Keys page. Then the token should be available for the spring.ai.openai.api-key property, for example, by exporting the environment variable:

export SPRING_AI_OPENAI_API_KEY=<INSERT KEY HERE>

For Azure OpenAI, the endpoint and api-key may be retrieved from the Azure OpenAI Service section. Then the tokens should be made available for the properties such as spring.ai.azure.openai.api-key, for example by exporting the environment variables:

export SPRING_AI_AZURE_OPENAI_API_KEY=<INSERT KEY HERE>
export SPRING_AI_AZURE_OPENAI_ENDPOINT=<INSERT ENDPOINT URL HERE>

Now the configuration may be used to ask a question to the Azure OpenAI service:

AiClient aiClient = new AzureOpenAiClient();
String response = aiClient.generate("What's the answer to the Ultimate Question of Life, the Universe, and Everything?");

Actually, the Spring Boot Starter will create the AiClient, which makes it possible to interact with an AI model in just one line of code.

Instead of creating hard coded questions, it's also possible to use prompt templates. First a file with type .st should be created, for example your-prompt.st, which contains a sentence with a placeholder called topic:

Tell me a fun fact about {topic}

Next, the prompt template can be used when calling the AI model, for example by providing the value developers to the topic template:

@Bean
ApplicationRunner applicationRunner (@Value("classpath:/your-prompt.st) 
        Resource resource, AiClient aiClient) { return args -> {

    var promptTemplate = new PromptTemplate(resource);
    var prompt = promptTemplate.create(Map.of("topic", "developers");

    var response = aiClient.generate(prompt);
    System.out.println(response.getGeneration());
    };
}

The project allows users to provide custom information or data to the model, for example company internal information such as FAQs, or new data which wasn't available when the model was created. A resource may be defined for the relevant information:

@Value("classpath:/myCustomInformation.txt) Resource myCustomInformation;

Then the context may be added as a prompt template:

var prompt = "... {context}...";

Lastly, the context is filled with the provided information:

var promptTemplate = new PromptTemplate( prompt);
promptTemplate.create(Map.of("context", myCustomInformation)));

The prompt input is first verified before sending it to the AI model, to reduce the number of requests and thereby the cost.

Josh Long, Spring developer advocate at VMware, conducted an interview with Dr Mark Pollack, the leader of the Spring AI project, where they discussed the various possibilities and created several code examples.

More information can be found in the reference documentation and a workshop with instructions for Azure OpenAI is available. The workshop example may be converted to support OpenAI instead of Azure OpenAI by using the correct dependency.