What is an LLM?

The short answer

An LLM is a neural network trained on a large body of text to predict what token comes next. That single capability, applied at scale with billions of parameters, produces systems that can answer questions, summarize documents, write code, translate languages, and reason through problems. The name is literal: large refers to the number of parameters, language refers to text as the primary medium, and model refers to the statistical structure learned during training.

The term gets used loosely to describe everything from the underlying model weights to the full product experience layered on top. For the purposes of building with them, what matters is the underlying mechanics, because those mechanics determine what you can rely on and what you cannot.

How training works

Training exposes the model to enormous amounts of text and repeatedly adjusts internal weights to improve next-token prediction. The process runs on clusters of specialized hardware for weeks or months, consuming energy and compute that most organizations cannot replicate. What comes out is a file of weights, essentially a compressed representation of statistical relationships across the training corpus. That file is what you download or call via API when you use Claude, GPT-4, or Gemini.

The training corpus matters more than most teams realize. A model that saw vast amounts of code will be better at coding tasks. A model trained heavily on formal English will handle informal or multilingual inputs less gracefully. Understanding where a model was trained helps predict where it will fail, which is useful knowledge before you commit to one for a production system.

Tokenization

LLMs do not read text the way humans do. They read tokens, which are fragments of text roughly equivalent to three or four characters on average. The word "production" might be a single token. The word "antidisestablishmentarianism" might be split into several. Tokenization is the first step in every request, and it matters for two practical reasons: you are billed by token in most APIs, and the context window limit is measured in tokens, not words or characters. A 100,000-token context window holds roughly 75,000 words of English text.

Tokenization also explains some otherwise mysterious model behaviors. Arithmetic errors, unusual handling of rare names, and unexpected splits in hyphenated terms all trace back to how the tokenizer carved up the input. When a model behaves strangely on a specific input, the first thing worth checking is whether the tokenization produced something unexpected.

Parameters and scale

Parameters are the numerical weights that encode what the model learned. More parameters generally means better performance up to a point, but size is not everything. A smaller model trained on higher-quality data often outperforms a larger model trained carelessly. What matters more than raw size is whether the model was trained on data relevant to your domain and whether it was fine-tuned or aligned for the kind of task you need. For enterprise applications, the hosted frontier models, Claude, GPT-4o, Gemini 1.5 Pro, outperform open-weight models on most business tasks without requiring you to manage infrastructure.

The practical implication is that picking a model based on parameter count alone is the wrong frame. Evaluation on your actual tasks, with your actual data, is the only reliable signal. A model that scores well on public benchmarks can still fall apart on the specific document format or domain vocabulary your system depends on.

Context window

Every LLM has a context window: the maximum amount of text it can process in a single request. Anything outside that window is invisible to the model. Early models had windows of 4,000 tokens. Current frontier models support 100,000 to 1,000,000 tokens. A larger window does not guarantee better results, because models lose focus on material buried in the middle of very long contexts. Effective use of the context window is its own discipline, covered in context engineering.

For enterprise systems, the context window is a shared resource that gets consumed by system instructions, conversation history, retrieved documents, tool definitions, and the actual user message. Managing that budget deliberately, rather than filling it opportunistically, is one of the most consistent levers teams have for improving both quality and cost.

Inference vs training

Training is what creates the model. Inference is what happens when you use it: you send a request, the model generates a response, and you receive output. Training happens once, costs enormously, and is done by the lab. Inference happens every time a user sends a message, costs per token, and is where most enterprise costs live. Optimizing inference cost is its own problem, covered in LLM cost optimization.

The distinction also matters for latency. Training is done offline and its duration is irrelevant to users. Inference latency is experienced directly, and it varies with model size, prompt length, output length, and infrastructure load. For interactive applications, latency is a product constraint, not just an engineering concern.

Hallucination

LLMs generate plausible text, not verified facts. A model can state something confidently and be wrong, a phenomenon called hallucination. The root cause is architectural: the model produces the token sequence that fits the statistical patterns it learned, and that sequence can include incorrect information. Hallucination rates vary by model, task, and prompt design. For production systems, treating model output as requiring verification rather than ground truth is essential. We covered the production implications in hallucination in production.

Hallucination is not a bug that will be patched away. It is an inherent property of how these systems work. The engineering response is not to wish it away but to design systems that catch it, through evals, grounding, guardrails, and human review where stakes are high.

What enterprise teams actually need to know

You almost certainly do not need to train your own LLM. The cost is prohibitive, the expertise required is scarce, and the frontier models available via API already outperform anything a typical organization could build. What you do need to decide: which model for which task, how to structure requests so the model has the right information, how to evaluate whether the output meets your quality bar, and how to instrument the system so you can catch failures. Those decisions are where production AI work actually lives, and getting them right is the difference between a demo that works and a system that holds up.

The organizations that succeed with LLMs in production are not the ones with the best model selection instinct. They are the ones that build disciplined evaluation practices, invest in context quality, and treat the model as one component in a system rather than the whole answer. That orientation is what separates AI features that scale from ones that get quietly retired after the pilot.