A Chatbot Is Not an AI Strategy

Bolting a chat window onto your product is the least valuable thing AI can do for it. Where LLMs actually move business metrics — extraction, routing, generation inside existing workflows — with examples by industry.

Every AI roadmap I've seen in the last two years opens with the same slide: a chat window bolted to the corner of the product. It demos well. It ships fast. And it's almost always the weakest thing an LLM can do for the business.

The reason is simple. A chat box hands all the work back to the user — they have to know what to ask, phrase it well, and then read the answer. The places where language models actually move a P&L are the opposite: quiet, embedded, and running on data the user never types.

Why chat became the default

A chat interface is the easiest AI feature to imagine and the easiest to sell. Everyone has used ChatGPT, so a text box feels like proof that "we have AI now." It's also cheap to prototype — no schema design, no evaluation harness, no integration with the systems that actually hold your data.

But an interface is not a strategy. Chat pushes the cognitive load onto the person you're supposed to be helping, and it makes output quality hard to measure. When the answer is a free-form paragraph, "was it right?" has no clean definition. That vagueness feels comfortable in a demo and gets expensive in production.

Extraction: turning documents into rows

The highest-ROI use of an LLM is usually the least glamorous one: reading unstructured text and returning structured data. Invoices, contracts, resumes, support emails, scanned PDFs, intake forms — anything a person currently retypes into a system by hand.

With structured outputs — a JSON schema the model is forced to fill — you get typed fields you can validate, store, and route. Teams that automate this commonly cut manual data entry by 50-80% and turn a multi-day back-office queue into minutes.

• Vendor invoices → line items, totals, tax IDs, due dates
• Inbound contracts → parties, renewal dates, liability caps, governing law
• Support tickets → product area, severity, affected account
• Job applications → skills, years of experience, location

Routing: the decision you make a thousand times a day

Classification is where LLMs quietly replace brittle rule engines. Instead of a regex tree that breaks every time the wording shifts, you describe the categories and let the model assign one. It handles synonyms, typos, and mixed languages that keyword rules never survive.

The trick is to constrain the output to a fixed set of labels and log every decision, so you can measure accuracy against a human-labeled set and catch drift before it costs you.

• Route incoming email to the right team
• Tag tickets by intent and urgency
• Flag transactions for fraud review
• Sort user feedback into product themes

Generation, but as a first draft

Generation is real, but its value shows up when a human stays in the loop. The model writes the draft reply, the summary, the product description, the code scaffold — and a person edits and approves. You're not replacing judgment; you're removing the blank page.

This is where the "80% done in seconds" framing actually holds. A support agent who edits a suggested response works two to three times faster than one starting cold, and the customer still gets a human's final call.

The economics change when the AI is invisible

Embedded AI is measurable in a way chat never is. Each call has one job, one expected shape of output, and a ground truth you can check against. That unlocks the boring engineering that makes AI dependable: automated evals, caching, batch processing, and a cost per task you can actually forecast.

You also control the input. Instead of hoping a user phrases their question well, you feed the model clean, structured context from your own systems — often with RAG over a vector store — and you decide exactly what "good" looks like before a single request goes out.

When a chatbot is the right answer

Sometimes chat genuinely fits. Open-ended search over internal knowledge, first-line deflection against a large documentation base, or a tool where the user's questions are truly unpredictable — those are real jobs for a conversational interface.

The test is whether conversation is the point or just the packaging. If the user's goal is a specific outcome you could trigger from a button or a form, chat is friction. If exploration itself is the value, chat earns its place.

When not to use AI at all

Honesty is cheaper than a failed pilot. Some problems don't want a model, and forcing one in makes them worse.

• Deterministic rules already work — a tax calculation or a validation check doesn't need an LLM, and shouldn't have one.
• Errors are unacceptable and volume is too high to review — if you can't afford a human check and can't afford a mistake, the workflow isn't ready.
• Volume is too low to justify the eval and monitoring work — a handful of documents a week is a job for a person, not a pipeline.
• The data to ground the model doesn't exist — no amount of prompting fixes missing source information.

A test before you build

Before you add AI to anything, ask where the language actually lives in the workflow — and whether a person has to type to reach it. If the value is buried in documents, emails, and tickets your users never want to read, put the model there, invisibly, and measure it against a labeled set. If you're reaching for a chat window first, you're probably starting at the least valuable end. Pick the task, define what correct means, and only then choose the interface.