Large Language Model (LLM)

A Large Language Model (LLM) is an AI system trained on massive amounts of text to predict the next token in a sequence. The prediction capability scales into broader abilities (reasoning, code generation, analysis, conversation, translation, summarization) as models grow in size and training data. Modern frontier LLMs range from 70 billion to 1+ trillion parameters and are the technology underlying ChatGPT, Claude, Gemini, Llama, and other generative AI products that have transformed software since 2022. It's the specific type of foundation model that handles text.

What LLMs actually do (the mechanics):

Tokens, not words: LLMs break text into tokens (sub-word units). "Tokenization" of a sentence might produce 10-20 tokens for what looks like 5-10 words.

Next-token prediction: given a sequence of tokens, the model predicts the probability distribution of the next token. This is the fundamental operation.

Generation as repeated prediction: to generate text, the model predicts the next token, adds it to the sequence, predicts the next, and so on. Repeated next-token prediction produces coherent text.

Emergent capabilities: at sufficient scale, simple next-token prediction produces remarkable capabilities, reasoning, code, analysis, translation, etc. These weren't explicitly trained for; they emerged from scale and training data.

The scaling laws:

Empirical observation: model capability scales predictably with three factors:

1. Model size (parameters): 7B → 70B → 700B parameters.
2. Training data (tokens): 1T → 10T → 100T tokens.
3. Compute (training FLOPs): more compute = better model.

Capability roughly scales as a power law in these three variables, which is why labs keep training bigger models on more data with more compute.

The training process:

Pre-training: largest phase. Model trained on massive text corpus (web, books, code, etc.) to predict next tokens. Takes weeks-months on thousands of GPUs.

Supervised fine-tuning (SFT): model fine-tuned on human-written demonstrations of desired behavior.

Reinforcement learning from human feedback (RLHF): model fine-tuned with human preferences to refine output quality.

Constitutional AI (Anthropic's approach): model self-refines using a constitution of principles, reducing need for human labeling.

The current state-of-the-art (as of mid-2026):

Model	Lab	Approximate parameters	Notes
GPT-5.5 / o3	OpenAI	Likely 1T+	Reasoning model; leads math benchmarks
Claude Opus 4.6 / 4.7	Anthropic	Undisclosed	Leads SWE-bench Pro; coding + reasoning
Gemini 3.1 Pro	Google	Undisclosed	Long context, multimodal, strong price/perf
Llama 4 / Scout	Meta	Open weights	Largest open model; Scout has 10M token context
DeepSeek V3	DeepSeek	671B (open)	Strong open model

What LLMs can do (with appropriate context and prompting):

• Conversation: ChatGPT-style dialog.
• Code generation: write, debug, explain code.
• Analysis: extract structure from unstructured text.
• Summarization: condense long documents.
• Translation: between languages.
• Reasoning: multi-step problem solving.
• Following complex instructions: agentic workflows.
• Tool use: calling external APIs based on context.

What LLMs struggle with:

• Real-time information: training data has a cutoff; can't know recent events without external data.
• Mathematical precision: better than 2022 but still error-prone without tools.
• Hallucination: sometimes confidently generates incorrect information.
• Long-horizon planning: gets weaker as task length grows.
• Specialized domain knowledge: may lack depth in niche areas without fine-tuning or RAG.
• Consistency across long contexts: middle-of-context degradation.

The startup implications:

Build on top, don't compete: use existing LLMs via API.

Pick by use case: Claude for reasoning/coding, GPT for general, Gemini for multimodal, Llama for self-host.

Plan for capability improvement: LLMs get dramatically better every 6-12 months.

Watch the inference cost curve: per-token costs dropping 10x every 12-18 months.

Augment with RAG and tools: most production LLM applications use retrieval and tool-calling, not just the raw model.

What founders get wrong: Building LLM applications without understanding how LLMs actually work (just calling APIs and hoping). The right discipline: understand tokenization, context windows, prompting techniques, RAG patterns, tool use; build with patterns that scale with capability improvement.

Related: Foundation Model · Generative AI · Transformer Architecture · Training Data · Context Window · Inference Cost

FAQ

What is a Large Language Model (LLM)?
An AI system trained on massive amounts of text to predict the next token in a sequence. The prediction capability scales into broader abilities (reasoning, code, analysis, conversation) as models grow. Underlies ChatGPT, Claude, Gemini, Llama.

How do LLMs actually work?
At core: next-token prediction. Trained to predict the most likely next token given previous tokens. To generate text, model repeatedly predicts next tokens. At sufficient scale, simple next-token prediction produces capabilities like reasoning, coding, and analysis.

How big are modern LLMs?
Frontier LLMs range from 70B to 1+ trillion parameters. Open models (Llama 4, DeepSeek V3) at 400B-700B parameters. Closed frontier models (GPT-5, Claude 4, Gemini 2) likely 500B-1T+ parameters (most labs don't disclose).

What can LLMs do?
Conversation, code generation, analysis, summarization, translation, reasoning, complex instruction-following, tool use. Struggle with real-time information, mathematical precision, hallucination, long-horizon planning, and very specialized domain knowledge.