LLMs are not just for coding: how real businesses use AI
How real companies are using language models and agentic AI across customer service, finance, manufacturing, healthcare, legal work, and industrial operations
When software engineers talk about large language models, the conversation often collapses into one topic: coding assistants.
That is understandable. Coding is where many of us first felt the change personally. We saw autocomplete become pair programming. We saw boilerplate disappear. We saw a model explain a stack trace, generate a unit test, or turn a vague requirement into a working function. For developers, this was the obvious entry point.
But inside real enterprises, the more interesting transformation is happening elsewhere.
LLMs are not just helping people write code. They are becoming a new interface to institutional knowledge, customer operations, clinical workflows, legal processes, manufacturing data, industrial assets, sales teams, supply chains, and executive decision-making.
The important enterprise question is no longer:
“Can AI generate text?”
It is:
“Can AI safely understand the context of our business, take action through our systems, and help people make better decisions faster?”
That is where agentic AI enters the picture.
Not in the science-fiction sense of autonomous digital employees replacing everyone overnight, but in a more practical form: AI systems that can interpret intent, retrieve the right information, call tools, follow workflows, escalate to humans, cite sources, and perform bounded tasks under governance.
In other words, enterprise AI is moving from “chat with a model” to “coordinate work.”
And that changes what AI-focused software engineers need to understand.
Before I talk about real-world use cases for LLMs and agentic AI, I’ll mention that I created a short introductory course on Pluralsight that teaches you the core, technology-agnostic principles of making enterprise-grade agentic systems robust. Many people found this course useful, and it’s not in the top 2% of the most popular courses on the platform.
The course is available here:
Now, back to our subject.
The real enterprise problem: decision latency
Most large organizations do not suffer from a lack of software. They suffer from too much disconnected software.
A bank has research PDFs, compliance manuals, CRM records, call notes, emails, market commentary, and policy documents. A manufacturer has SCADA systems, historians, MES data, maintenance logs, engineering drawings, operator notes, asset models, and spreadsheets. A hospital has patient records, dictations, appointment histories, clinical guidelines, and administrative workflows. A retailer has supply chain systems, employee apps, merchandising data, product catalogs, customer interactions, and HR portals.
The problem is not that data does not exist. The problem is that the right person cannot get the right answer at the right time without knowing where to look, what to search for, which system owns the truth, and how to interpret the result.
LLMs are useful because they compress the distance between question and answer.
Agentic AI is useful because it compresses the distance between answer and action.
That is why serious enterprises are building AI systems around specific workflows: customer support routing, contract review, clinical documentation, advisor knowledge retrieval, industrial troubleshooting, procurement, marketing operations, and field service.
Here are some use cases of how real enterprises that I worked with or spoke to deal with this problem
Use case 1: enterprise knowledge retrieval
Every large company has internal knowledge, but most employees experience it as a maze. Documents live in SharePoint, Confluence, Teams, Salesforce, ServiceNow, Box, Google Drive, ERP systems, wikis, PDFs, email threads, and people’s heads.
Traditional search assumes the user knows the right keyword. LLM-based retrieval assumes the user knows the question.
That is a major shift.
A financial advisor, for example, should not need to remember which investment policy document contains a particular rule, which research note mentions a specific market scenario, or where the latest client communication guidelines live. They should be able to ask a natural-language question and receive a grounded answer with references.
Morgan Stanley is one of the clearest examples. Its wealth-management AI assistant helps financial advisors retrieve internal knowledge and client-relevant information faster. The interesting part is not merely that the assistant can answer questions. The interesting part is that the company invested heavily in reliability, evaluation, permissioning, and adoption. In a regulated industry, an answer is only useful if people trust where it came from.
That pattern is becoming common: LLMs sit on top of enterprise knowledge, but the enterprise value comes from grounding, access control, citations, evaluation, and workflow fit.
For software engineers, this means retrieval-augmented generation is becoming one of the core patterns of enterprise software.
A good enterprise knowledge assistant needs to answer questions such as:
What data sources are authoritative?
Which users are allowed to see which content?
How fresh is the content?
Can the assistant cite its sources?
Can the organization measure answer quality?
What happens when the model is uncertain?
How does feedback improve the system?
How does the assistant fit into the employee’s existing workflow?
That is why enterprise AI is less about “prompt engineering” and more about information architecture.
Use case 2: customer service and support
Customer service is one of the most obvious places to apply LLMs, but it is also one of the easiest places to get wrong.
A bad customer-service bot is worse than no bot at all. It creates the illusion of progress while trapping customers in a loop. But a well-designed AI support system can reduce resolution time, improve routing, summarize context, handle repetitive tasks, and give human agents better information.
Klarna became one of the most cited examples when its AI assistant handled a large share of customer-service conversations, supported many languages, reduced repeat inquiries, and shortened average resolution time. But the more important lesson is not “replace support agents with a bot.” The more durable lesson is that customer service should be split into different types of work.
Of course, at one point, Klarna got it wrong, too. Following the initial success of their AI-based support assistance, they decided to lay off many human employees.
This backfired massively, as the quality of technical support and customer satisfaction degraded over time. This led the company to start rehiring people again.
This gave the birth to the term Klarna effect. It describes a situation when an overconfident business decides to lay people off because they are, supposedly, can be replaced by AI, only to find later that it’s not the case at all.
But nevertheless, while Klarna’s case showed that AI cannot replace professionals, their AI-enabled customer service system is still useful. They found the hard way that humans still need to be in the loop. However, this doesn’t mean that AI was useless.
On the contrary, AI reduced many significant bottlenecks in the process. AI showed real ROI. That’s precisely what made leadership believe that humans were no longer needed.
Some work is routine: refund status, return instructions, account updates, password help, order tracking, policy explanations.
Some work is emotional or ambiguous: complaints, financial stress, edge cases, vulnerable customers, complex disputes.
AI is well-suited to the first category. Humans are often essential for the second.
That is why the best customer-service AI architectures are hybrid. They use LLMs and agents to classify intent, retrieve policy, summarize history, propose replies, automate repetitive actions, and escalate when confidence is low or emotion is high.
Cisco provides another useful pattern. Rather than simply using generative AI to summarize case handoffs, the more valuable transformation was intelligent routing: getting the customer to the right expert earlier. This is a subtle but important lesson. AI should not just make broken workflows faster. It should help redesign the workflow.
Use case 3: everyday knowledge work
A large amount of enterprise work is not glamorous. It is reading, writing, summarizing, comparing, preparing, analyzing, planning, and communicating.
This is where tools like Microsoft 365 Copilot, ChatGPT Enterprise, Gemini for Workspace, and similar enterprise assistants are being adopted.
Employees use LLMs to summarize meetings, draft emails, rewrite documents, analyze spreadsheets, prepare presentations, produce first drafts, compare long documents, extract action items, and brainstorm options.
This may sound mundane, but mundane work consumes enormous organizational time.
Accenture’s large-scale Copilot rollout is a good example of enterprise productivity at scale. The key point is not that one employee saves a few minutes writing an email. The point is that, across hundreds of thousands of employees, small improvements in routine tasks can compound into significant organizational capacity.
However, there is a trap here. Giving every employee an AI assistant does not automatically transform the enterprise.
The companies that gain the most do not just distribute licenses. They redesign workflows, train employees, measure outcomes, and identify where AI belongs inside repeatable processes.
The difference is this:
Weak adoption: “Everyone now has access to an AI chatbot.”
Strong adoption: “Here is how sales prepares account plans now.”
Weak adoption: “Use AI to save time.”
Strong adoption: “Use AI to produce the first draft of the client brief, grounded in approved source documents, then route it through human review.”
Weak adoption: “Summarize meetings.”
Strong adoption: “Turn meeting summaries into CRM updates, follow-up tasks, risk flags, and project-plan changes.”
The enterprise value comes when AI moves from individual productivity to process productivity.
Use case 4: scientific, clinical, and regulated work
Healthcare and life sciences are high-friction environments. They involve regulation, specialist language, complex documentation, and high stakes.
That makes them both difficult and attractive for LLM adoption.
Moderna is a strong example of a company using generative AI beyond software development. Its employees created hundreds of custom GPTs across the organization, including assistants for clinical development, data analysis, legal work, research, manufacturing, and commercial operations. One pilot, Dose ID, was designed to help clinical study teams analyze large datasets, visualize results, and support dose-selection reasoning.
This does not mean the model replaces clinical judgment. It means the model becomes an analytical partner that can accelerate exploration, summarization, and evidence preparation.
Healthcare documentation is another major use case. Clinicians spend large amounts of time writing notes, summarizing encounters, and updating electronic health records. Ambient AI tools can listen to patient conversations, draft clinical notes, and surface relevant information. Microsoft’s Dragon Copilot is an example of this direction: voice AI, ambient documentation, healthcare-specific safeguards, and workflow automation combined into a clinical assistant.
In regulated domains, the question is not “Can the model produce a plausible answer?” The question is:
Can the answer be verified?
Can the source be traced?
Can the output be reviewed by a qualified person?
Can the system preserve privacy?
Can the organization audit what happened?
Can the AI be prevented from acting outside its authority?
Use case 5: legal, compliance, and contract workflows
Legal work is another area where LLMs are useful because the raw material is language.
Contracts, clauses, policies, regulations, emails, filings, memos, and precedents are text-heavy and expensive to process manually.
A&O Shearman’s work with Harvey shows the direction of travel. The firm uses legal AI to assist with drafting, summarization, clause comparison, precedent search, legal analysis, translation, and contract review. It also developed ContractMatrix with Harvey and Microsoft for contract drafting, review, and negotiation workflows.
AI is embedded into specific legal workflows where the task is bounded, and the output can be reviewed by lawyers.
This is exactly how many enterprise AI systems succeed: they start as assistive systems in high-volume, high-friction tasks.
A legal AI system might:
summarize a long agreement;
compare two versions of a contract;
identify non-standard clauses;
generate a first draft based on a template;
retrieve relevant precedents;
translate legal language into business language;
highlight missing terms;
prepare a negotiation checklist.
But it should not silently make legal decisions without accountability.
Use case 6: retail and employee operations
Retailers have massive operational complexity: stores, supply chains, associates, merchants, customer service, logistics, inventory, promotions, HR, and merchandising.
Walmart has been building AI tools for employees, including MyAssistant for corporate associates and AI assistants for merchants. This is an important category because it shows AI being used internally, not just as a customer-facing chatbot.
In a retailer, an internal AI assistant can help employees draft communications, summarize operational updates, analyze merchandising data, generate product descriptions, answer policy questions, support training, and reduce administrative burden.
But the deeper value comes when AI is connected to enterprise context.
A generic model can write a product description. A retailer’s internal assistant can write a product description using approved brand voice, category rules, compliance constraints, product attributes, pricing context, and marketplace requirements.
That distinction matters.
Enterprise AI becomes valuable when it knows the business.
Use case 7: industrial operations
The most under-discussed enterprise AI use cases are in industrial environments.
Factories, energy assets, utilities, mines, chemical plants, wind farms, water systems, and manufacturing lines produce huge volumes of operational data. But that data is often fragmented across control systems, historians, maintenance systems, engineering tools, spreadsheets, dashboards, and documents.
This is where AVEVA CONNECT is especially interesting.
AVEVA CONNECT is an industrial intelligence platform designed to bring industrial data together in one place. It can combine on-premises and cloud data, help users access and visualize industrial information, and support engineers, operators, and data scientists across the industrial lifecycle.
The important idea is that CONNECT provides the data foundation. The Industrial AI Assistant then gives users a natural-language interface over that foundation.
This is a very different enterprise AI pattern from a generic chatbot.
A plant operator does not want a model to hallucinate a maintenance procedure. A reliability engineer does not want a poetic explanation of vibration data. A production supervisor does not want generic manufacturing advice. They want trusted answers based on their actual assets, tags, events, dashboards, documents, and operational history.
AVEVA’s Industrial AI Assistant is built for this industrial context. It allows users to ask natural-language questions about the site or plant operations. It can find and summarize information from industrial data sources, assets, events, documents, saved visualizations, and engineering or operational content. It uses retrieval-augmented generation and hybrid semantic search to locate relevant information even when the user does not know the exact tag or asset identifier.
That last detail is important.
Industrial data is messy. Tags are not always named intuitively. Assets may have aliases. Data streams may be understandable only to experienced engineers. Much operational knowledge lives as “tribal knowledge” in the heads of senior people.
An AI assistant over industrial data can reduce the cost of finding and interpreting that knowledge.
For example, a user might ask:
“What were the bearing and nacelle temperatures for turbine GE07 over the last 24 hours?”
The assistant needs to understand the intent, identify GE07 as an asset, map the requested temperatures to relevant real-time data streams, retrieve the values from CONNECT, and return the answer in a useful format.
Another user might ask:
“Do we have visualizations related to wind turbine utilization?”
The assistant needs to search saved visualizations, infer semantic relevance, rank the results, and return useful links or summaries.
In manufacturing, the same pattern applies to downtime, throughput, quality, maintenance, process variance, and energy usage.
There are also important safety and governance implications. AVEVA’s Industrial AI Assistant is designed around permissioned data access and does not train on customer data. It returns information based on what the current user is allowed to see. It also provides citations so users can verify responses.
This matters because industrial AI is not a place for casual autonomy.
In a plant environment, the first valuable step is often not “let the agent operate the machine.” It is:
find the right data faster;
summarize the maintenance history;
detect process variance;
compare trends;
identify likely causes;
generate a visualization;
help the engineer decide what to inspect next;
recommend a safe next step;
escalate when confidence is low.
That is practical agentic AI.
The agent is not replacing the operator. It is reducing the cognitive load required to understand what is happening.
What makes AI “agentic” in the enterprise?
The word “agent” is now used everywhere, often carelessly. I previously wrote what an agent is. In practical enterprise terms, an agentic AI system has some combination of these capabilities:
It understands a user’s goal.
It breaks the goal into steps.
It retrieves context from enterprise systems.
It calls tools or APIs.
It maintains state across a workflow.
It evaluates whether the task is complete.
It asks for clarification when needed.
It escalates to a human when risk is high.
It logs its reasoning, actions, and sources.
It operates within permissions and policy boundaries.
While a simple chatbot answers a question, an enterprise agent helps complete a business process.
For example:
A support agent classifies a ticket, retrieves customer history, checks known issues, drafts a response, and routes the case.
A finance agent gathers data from ERP, reconciles anomalies, drafts a variance explanation, and asks a controller to approve.
A legal agent compares contract clauses, highlights deviations, drafts fallback language, and routes risky clauses to counsel.
An industrial agent retrieves sensor trends, checks maintenance logs, compares recent events, generates a chart, and recommends inspection steps.
A sales agent reviews CRM notes, summarizes account risks, drafts a meeting brief, and creates follow-up tasks.
Notice that in serious enterprise deployments, the agent rarely has unlimited freedom. The power comes from bounded autonomy.
The agent may retrieve data autonomously. It may draft autonomously. It may classify autonomously. It may recommend autonomously.
But sending an external email, changing a production setting, approving a payment, denying a claim, or modifying a legal agreement usually requires human approval.
The architecture behind real enterprise AI
The companies getting value from LLMs are not simply buying access to a model. They are building systems around the model.
A real enterprise AI architecture usually includes several layers.
1. The model layer
This is the LLM itself: OpenAI, Azure OpenAI, Anthropic, Gemini, open-source models, or domain-specific models.
But the model is only one component. In many enterprise systems, the model is replaceable. The surrounding architecture is where the business value lives.
2. The context layer
This includes vector databases, semantic indexes, document stores, enterprise search, data catalogs, knowledge graphs, metadata, permissions, and data pipelines.
For many companies, the context layer is the hardest part. Their data is fragmented, duplicated, stale, poorly governed, or locked inside legacy systems.
This is why platforms like AVEVA CONNECT matter in industrial settings. The assistant is valuable because it sits on top of a curated operational context.
3. The tool layer
Agents become useful when they can do more than generate text.
They need tools:
search documents;
query databases;
retrieve telemetry;
create tickets;
update CRM records;
generate charts;
call ERP APIs;
read calendars;
trigger workflows;
open pull requests;
prepare reports;
classify requests;
route approvals.
This is where software engineers become critical. The model may reason in natural language, but enterprise value requires integration with real systems.
4. The workflow layer
This is the orchestration logic: what steps happen, in what order, under what conditions, with what approval gates.
Some workflows are deterministic. Some are agentic. Many are hybrid.
For example, a customer-support flow may always classify the issue first, then retrieve policy, then search past cases, then draft an answer, then either respond automatically or escalate to a human depending on confidence, customer sentiment, or account value.
5. The governance layer
This includes identity, access control, audit logs, data-loss prevention, model monitoring, evaluation, human approval, policy enforcement, and rollback.
Without governance, agentic AI is a liability.
With governance, agentic AI becomes a scalable business capability.
6. The evaluation layer
Enterprise AI systems need continuous evaluation. It is not enough to ask, “Did the model answer?” You need to ask:
Was the answer correct?
Was it grounded in approved sources?
Did it respect permissions?
Did it hallucinate?
Did it route the task correctly?
Did it save time?
Did it improve customer outcomes?
Did users trust it?
Did it reduce rework?
Did it create new risks?
This is one of the biggest differences between demos and production systems.
A demo needs to impress someone once.
An enterprise AI system needs to behave reliably thousands or millions of times.
What enterprises have learned so far
A few lessons are emerging across industries.
Lesson 1: AI succeeds when it is attached to a workflow
Generic AI access is useful, but workflow-specific AI is more valuable.
A general assistant can summarize a document. A workflow assistant can summarize the document, compare it with policy, identify missing fields, route it for approval, update the system of record, and prepare the next action.
Lesson 2: the data layer matters more than the prompt
Many organizations begin with prompt templates. They soon discover that the real blocker is data quality, data access, and data governance.
The model cannot answer from data it cannot retrieve. It cannot cite sources that are not indexed. It cannot respect permissions if permissions are not represented. It cannot reason about assets, customers, or contracts if the enterprise has not modeled them properly.
Lesson 3: human-in-the-loop is not a temporary compromise
In many business processes, human review is the correct design.
The goal is not to remove humans from every workflow. The goal is to remove unnecessary friction so humans spend more time on judgment, empathy, strategy, creativity, and accountability.
Lesson 4: agentic AI requires stronger product management
Building agents is not just a technical project. It requires product thinking.
You need to define:
the user;
the job to be done;
the allowed actions;
the risk boundaries;
the fallback path;
the approval model;
the success metrics;
the failure modes;
the operating model.
Without this, agents become impressive prototypes that nobody trusts.
Lesson 5: the best use cases are often boring
The highest-value AI use cases are not always the most futuristic ones.
They are often:
summarizing support cases;
finding maintenance history;
drafting first versions of documents;
extracting contract clauses;
preparing meeting briefs;
routing tickets;
generating reports;
comparing data across systems;
explaining anomalies;
translating expert language into business language.
Boring work at enterprise scale is not boring economically.
What this means for AI-focused software engineers
If you are a software engineer trying to move deeper into AI engineering, the lesson is simple:
Do not stop at model APIs.
The future belongs to engineers who can connect models to business context and production workflows.
You need to understand:
retrieval-augmented generation;
vector search and hybrid search;
semantic indexing;
structured and unstructured data integration;
tool calling;
agent orchestration;
identity and access control;
evaluation frameworks;
observability;
prompt and response logging;
privacy and compliance;
human approval workflows;
domain-specific UX;
cost and latency trade-offs;
model selection and fallback strategies.
The best AI engineers will not be the ones who know the trendiest model name. They will be the ones who can answer:
“How do we make this useful, safe, measurable, and maintainable inside a real organization?”
That is a very different skill from writing a chatbot demo.
It is closer to enterprise architecture, distributed systems, product engineering, data engineering, security engineering, and domain modeling.
In fact, agentic AI may bring enterprise software engineering back to its roots. The hard part is still understanding the business process. The hard part is still integrating systems. The hard part is still permissions, reliability, observability, and change management.
The model changes the interface, but it does not remove the need for engineering discipline.
The future: AI as the operating layer of the enterprise
The first wave of enterprise AI was experimentation. Employees used chatbots to draft emails, summarize documents, brainstorm ideas, and write code.
The second wave is integration. AI is being embedded into CRM, ERP, productivity tools, support platforms, legal workflows, clinical systems, and industrial data platforms.
The third wave is orchestration. AI agents will coordinate work across systems, people, and processes.
But this future will not arrive evenly.
Some companies will remain stuck in pilot mode because they treat AI as a tool rollout. Others will redesign workflows and build AI into the operating model of the business.
The difference will come down to leadership, data readiness, governance, engineering capability, and willingness to change how work gets done.
The most important thing to understand is that LLMs are not just a better user interface for developers. They are becoming a new user interface for the enterprise itself.
A financial advisor asks a question and gets the right policy.
A customer gets an answer in minutes instead of waiting in a queue.
A clinician gets a draft note instead of spending the evening documenting.
A lawyer gets a contract comparison in seconds.
A retailer gives employees an assistant that understands the company context.
A plant operator asks about an asset and gets relevant operational data without hunting through five systems.
That is the real enterprise AI story.