LLM Integration in Enterprise Software: 4 Safe Patterns

Large language models (LLMs) live two different lives in enterprise software: pilots sitting in a corner as a "chat box", and integrations embedded in a business process, measured and audited. Moving to the second is less about model choice than about architectural discipline: limiting where the model is used, binding its output to a schema, feeding it company data, restricting the actions it can take, and measuring everything.

This article covers the four patterns that serve enterprise Java teams best — structured output, RAG, tool calling and guardrails — with Spring AI examples, the LLM gateway as a single exit point, model choice under privacy constraints, and measuring with an evaluation set.

First decision: where to use an LLM, and where not to

LLMs are strong at four kinds of work: classification (which department does this e-mail belong to?), extraction (date, amount, supplier from an invoice), summarization and transformation (meeting notes → task list) and question answering over documents. Where they are weak is equally clear: exact arithmetic, deterministic business rules ("is the credit limit exceeded?"), decisions with legal weight. The rule is simple: if there is a rule, write code; if there is ambiguity, use the model; if the two mix, let the model propose and the code decide.

Pattern 1: Structured output — the model returns JSON, the code validates

A model returning free text cannot be wired into a business process. Binding the output to a schema is the first and most important step of LLM integration. With Spring AI:

public record InvoiceSummary(
    String supplierName, String invoiceNumber, LocalDate date,
    BigDecimal totalAmount, String currency, List<Line> lines) {}

@Service
public class InvoiceExtractionService {
    private final ChatClient chat;

    public InvoiceExtractionService(ChatClient.Builder builder) {
        this.chat = builder.defaultSystem("""
            You are an accounting assistant. Use only information in the given document;
            leave a field null when unsure, never invent.""").build();
    }

    public InvoiceSummary extract(String invoiceText) {
        InvoiceSummary summary = chat.prompt()
            .user(u -> u.text("Extract the fields from the invoice text below:\n{text}").param("text", invoiceText))
            .call()
            .entity(InvoiceSummary.class);       // JSON schema + parse; exception if invalid
        validate(summary);                        // total = sum of lines? plausible date?
        return summary;
    }
}

Two rules: keep the schema narrow (the fewer fields the model must fill, the fewer errors) and keep validation in code. A total that doesn't match the line sum, an invalid date, an unknown currency — these are entrusted to deterministic code, not to the model; inconsistent output is rejected or routed to a human.

Pattern 2: RAG — answers from company data

The model's training data doesn't know your procedures, contracts or product documentation. RAG (retrieval-augmented generation) finds the document chunks most relevant to the question in a vector store and hands them to the model as context:

// Split documents and write them to the vector store (once, and on change)
List<Document> chunks = new TokenTextSplitter(400, 50, 5, 10000, true).apply(reader.get());
vectorStore.add(chunks);

// Question answering: relevant chunks are added to the prompt automatically
String answer = chat.prompt()
    .advisors(QuestionAnswerAdvisor.builder(vectorStore)
        .searchRequest(SearchRequest.builder().topK(5).similarityThreshold(0.7).build())
        .build())
    .user(question)
    .call()
    .content();

RAG quality comes from the data pipeline more than from the model: chunk size, document freshness, filtering by metadata (department, date, confidentiality class) and permission awareness — a user must not get an answer from a document they cannot see. We cover this in detail in a separate article; the rule here: an answer without a cited source is not fit for enterprise use.

Pattern 3: Tool calling — the model asks, the code acts

With tool calling, the model requests calls to functions you define when it needs them; your code makes the call and returns the result. It is the most powerful and the riskiest way to connect a model to your system:

@Component
class OrderTools {

    @Tool(description = "Returns the current status and estimated delivery date of the given order number")
    OrderStatus orderStatus(String orderNumber) {
        return orderService.status(OrderNumber.of(orderNumber));   // read-only: safe
    }

    @Tool(description = "Creates an approval request to cancel the order; does not cancel it directly")
    ApprovalRequest requestCancellation(String orderNumber, String reason) {
        return approvalService.create(orderNumber, reason);         // write: goes to human approval
    }
}

String answer = chat.prompt()
    .tools(orderTools)
    .user("Where is my order O-2026-118, and if it's late I want to cancel it")
    .call().content();

Design rules: read tools are free, write tools require approval. Rather than cancelling an order, making a payment or deleting a record directly, the model produces an "approval request"; every tool call has a timeout, an idempotency key and an audit record; the data a tool returns to the model is minimized (no personal data, no whole tables). Tool descriptions are part of the prompt: a vague description means the wrong tool gets picked.

Pattern 4: Guardrails — input, output and policy

Architecture: every call goes through an LLM gateway

Ten different places in the application don't call the model API directly. One internal service — the LLM gateway — does the following centrally: model routing (small/large model by task type), key and quota management, PII masking, caching (same question, same answer), cost measurement (tokens per feature), timeouts and retries, and an audit log (who asked what, which model, what it answered). Thanks to this layer, switching models — from a cloud provider to an in-house open-weight model, for instance — happens in one place.

Model choice and privacy: for content containing personal data or trade secrets, the country where data is processed, the provider's commitment not to train on it and the retention period must be settled by contract. For sensitive data classes, an open-weight model running in-house (with servers such as Ollama or vLLM) can sit behind the gateway as a second route; Spring AI drives both with the same code.

Don't ship without measuring

An LLM feature ships not on a "seems to work" feeling but with an evaluation set: 100–300 real examples (invoices, e-mails, questions), each with an expected answer; the set runs automatically on every prompt or model change. What gets measured: per-field accuracy in extraction, groundedness and the "I don't know" rate in RAG, the confusion matrix in classification, latency (p95) and cost per request. When this set is part of CI, a prompt change is tested like a code change.

Roadmap

1. Pick one low-risk, high-volume job: incoming document classification, support-ticket summarization; a human does the final check.

2. Build the gateway and the evaluation set from day one; retrofitting them as features multiply is expensive.

3. Start with structured output, add RAG second and tool calling third.

4. Report cost per feature; don't send tasks a small model handles to a large one.

Frequently asked questions

Which model should we choose?

More than one, by task type and data class: a small, cheap model is usually enough for extraction and classification; a large model for complex reasoning; an in-house model for sensitive data. The gateway does this routing; don't tie yourself to a single model.

Can hallucinations be prevented entirely?

No; but their impact can be bounded: schema-bound output, validation in code, mandatory sources, refusal at low confidence and human approval for critical actions. The design goal is not "no errors" but "errors get caught before they harm the system".

Are there alternatives to Spring AI?

LangChain4j is a mature alternative on the Java side; the Python ecosystem (LangChain, LlamaIndex) can be integrated as a separate service. For enterprise Java teams, Spring AI's advantage is direct fit with the existing Spring Boot infrastructure, observability and security layers.

Conclusion

Enterprise LLM integration is not connecting to a model API; it is building an architecture that binds output to a schema, feeds the model company data, limits its actions, checks input and output, and measures every call. These four patterns and the gateway layer are the difference between a pilot and production.

To add an LLM feature to your existing application, or bring a pilot up to production quality, see our enterprise application development and integration and automation services. The discovery call and first estimate are free.