How AI Actually Works
Understanding the basics changes how you use AI — the mental model that transforms your results.
INFO
- Time: ~15 minutes
- Difficulty: Beginner
- What you'll learn: The mental model that changes how you use AI
This Page Covers
- The Mental Model - AI predicts words based on patterns, not "thinking"
- What Are Models - Understanding GPT-4, Claude, Gemini and what new releases mean
- Hallucinations - Why AI is confident even when wrong
- Context Windows - AI's limited "working memory"
- AI is Stateless - Why AI doesn't "learn" from your conversations
- Training vs Search - The difference between what AI "knows" and what it looks up
- Why AI Stops Mid-Response - Understanding output limits
- Practical Implications - How this knowledge improves your results
The Mental Model
When you type something into ChatGPT or Claude, here's what's actually happening:
AI predicts the next word. That's it. It looks at everything you've written and predicts what word should come next, then repeats this process thousands of times to form a response.
This isn't "thinking" in any human sense. It's extremely sophisticated pattern matching. The AI was trained on massive amounts of text from the internet - books, articles, code, conversations - and learned patterns about how words follow other words.
Think of it like the world's most advanced autocomplete. Your phone suggests "you" after you type "thank" because it's seen that pattern millions of times. AI does the same thing, just at a scale and complexity that produces surprisingly coherent responses.
Why This Matters
Understanding this changes how you use AI:
- AI doesn't "know" things - It recognizes patterns that look like knowledge
- It can't fact-check itself - It has no way to verify if what it's saying is true
- Confidence doesn't mean accuracy - It predicts confident-sounding text because that's what good responses look like
Pattern Matching in Action
AI is excellent at tasks where patterns are abundant in its training data:
| Task | Why AI is Good at It |
|---|---|
| Writing emails | Seen millions of emails |
| Explaining concepts | Seen countless explanations |
| Writing code | Trained on vast code repositories |
| Summarizing text | Seen many summary examples |
AI struggles with:
- Novel problems it hasn't seen patterns for
- Very recent events (training data has a cutoff)
- Niche topics with little training data
- Tasks requiring real-world verification
Never the Same Twice
Run the exact same prompt twice and you'll get different responses. This isn't a bug - it's by design.
When predicting the next word, AI doesn't always pick the single most likely option. Instead, it samples from the top candidates with some randomness. This is controlled by a setting called temperature:
- Low temperature → More predictable, focused responses
- High temperature → More creative, varied responses
This means:
- Don't expect identical outputs - Even copy-pasting the same prompt produces variations
- Regenerate if unsatisfied - The next attempt might be better
- Slight differences are normal - Tone, word choice, and structure will vary
Note on Temperature
You cannot adjust temperature in ChatGPT or Claude's consumer apps - this is an API-only setting used by developers. When you use ChatGPT.com or Claude.ai, the AI provider chooses the temperature for you. For everyday use, just embrace the variation - it often surfaces better ideas.
What Are Models?
You'll hear about "GPT-4," "Claude Sonnet," "Gemini Pro" - these are all models. A model is the trained neural network that does the actual predicting. Think of it as the "brain" that powers the AI tool.
Why Different Models Exist
Companies train different models with different trade-offs:
| Trade-off | Example |
|---|---|
| Speed vs. capability | Smaller models respond faster but handle less complexity |
| Cost vs. quality | Larger models cost more to run but produce better results |
| General vs. specialized | Some models excel at code, others at conversation |
This is why you see model names like "Claude Sonnet" (balanced) vs "Claude Haiku" (fast and cheap) vs "Claude Opus" (most capable).
What "New Model" Means
When a company announces a new model, they've typically:
- Trained on more or better data
- Improved the underlying architecture
- Expanded capabilities (longer context, better reasoning, new features)
Headlines will claim the new model is "X% better" - but better at what?
Benchmarks Don't Tell the Whole Story
AI companies publish benchmark scores showing how models perform on standardized tests. The problem:
- Benchmarks test specific tasks - Your work probably isn't a standardized test
- Models can be optimized for benchmarks - High scores don't guarantee real-world performance
- Your use case is unique - A model that's "best" at coding may not be best for your marketing copy
Evaluate for Yourself
When a new model launches, test it on your actual tasks - not hypothetical ones. Run the same prompts you use daily and compare results. Your experience matters more than any benchmark.
Hallucinations: Confident but Wrong
AI "hallucinations" are when the model generates information that sounds authoritative but is completely made up. This happens because:
- AI optimizes for plausible-sounding text - If asked for a citation, it generates something that looks like a citation
- It can't distinguish fact from fiction - Both real and made-up facts are just patterns to match
- It doesn't know what it doesn't know - There's no internal "I'm not sure" signal
Common Hallucination Examples
- Fake citations - Academic papers that don't exist, with convincing-sounding authors and journals
- Invented statistics - "Studies show that 73% of..." with no real source
- Confident technical errors - Code that looks right but has subtle bugs
- Fictional events - Detailed descriptions of things that never happened
Important
AI will state made-up facts with the same confidence as real ones. Always verify important information from primary sources.
Context Windows: AI's Working Memory
AI has a limited "context window" - the amount of text it can consider at once. Think of it as working memory.
Claude: ~100,000+ tokens (roughly 75,000 words) ChatGPT: Varies by model (8k to 128k tokens)
What This Means in Practice
- Long conversations degrade - As the context fills up, AI may "forget" earlier parts
- Upload large documents - AI can lose track of details in very long files
- Starting fresh helps - When responses get weird, a new conversation often works better
Signs You're Hitting Context Limits
- AI contradicts something it said earlier
- It forgets details you mentioned
- Responses become less coherent
- It starts repeating itself
Solution: Start a new conversation and provide key context upfront.
AI is Stateless: It Doesn't "Learn" From You
Here's a common misconception: AI does not learn from your conversations.
When you chat with ChatGPT or Claude, it might feel like the AI is getting to know you. It remembers what you said earlier, refers back to your context, and builds on previous answers. This creates an illusion of learning - but it's just that, an illusion.
How Conversations Actually Work
Behind the scenes, every time you send a message:
- Everything gets pasted together - Your new message is concatenated with the entire conversation history
- AI receives it all at once - It's like receiving one giant document, not a flowing conversation
- AI predicts the response - Based on this combined text, with no actual memory
When you start a new conversation, the AI has zero recollection of previous chats. It's a completely fresh start. The "context window" isn't memory - it's just the current conversation being fed in repeatedly.
Why This Matters
- AI doesn't improve by talking to you - It won't get better at your specific tasks over time
- Each session is independent - Yesterday's breakthroughs don't carry forward
- You can't "train" it through use - No matter how many times you explain your preferences
The Exception: Projects and Memory Features
Both ChatGPT and Claude now offer ways to persist context:
- Projects let you save instructions and documents that persist across conversations
- ChatGPT Memory stores specific facts you tell it to remember
These features don't change how the underlying AI works - they just automatically include saved context at the start of each conversation. More on this in Lesson 3.
Training vs Search: What AI "Knows" vs. What It Looks Up
AI knowledge comes from two very different sources, and understanding this matters:
Trained Knowledge (Built-In)
When an AI model is created, it's trained on massive amounts of text. This knowledge is "baked in":
- Has a cutoff date - GPT-4 might know nothing after January 2024
- Cannot be updated - Once trained, the model's knowledge is fixed
- May contain outdated information - Facts change, but the model doesn't
This is what AI "knows" by default when you ask it a question.
Search/Retrieval (Real-Time)
Modern AI tools can also search the web or access uploaded documents:
- Current information - Can find today's news, latest prices, recent events
- Explicitly retrieved - AI looks it up, rather than generating from memory
- More reliable for facts - Actually checks sources rather than pattern-matching
How to Tell the Difference
| If you ask about... | AI likely uses... | Reliability |
|---|---|---|
| General concepts (photosynthesis, WW2) | Trained knowledge | Good for established facts |
| Recent events (today's stock price) | Search (if enabled) | Good if search is on |
| Obscure or niche topics | Trained knowledge | May hallucinate |
| Your uploaded documents | Retrieval | Good - it's reading the actual file |
Watch Out
When search is disabled or unavailable, AI will still confidently answer questions about recent events - using outdated or made-up information. Always check if your AI tool has web access enabled when asking about current topics.
Why AI Sometimes Stops Mid-Response
Have you noticed AI sometimes stops in the middle of a response? It's not random - there are specific reasons:
Output Token Limits
Just like there's a limit on how much AI can read (context window), there's a limit on how much it can write in one response:
- ChatGPT: Typically 4,000-8,000 tokens per response
- Claude: Typically 4,000-8,000 tokens per response
When the AI hits this limit, it simply stops - often mid-sentence.
How to Continue
If AI stops mid-response:
- Say "continue" - The AI will pick up where it left off
- Ask for the rest - "Please continue from where you stopped"
- Request chunked output - "Give me this in parts, starting with section 1"
Other Reasons AI Might Stop
| Symptom | Likely Cause | Solution |
|---|---|---|
| Stops mid-sentence | Output token limit | Say "continue" |
| Stops after a long conversation | Context window full | Start new conversation with key context |
| Stops with an error | Server/rate limits | Wait and retry |
| Gives very short answer | Misunderstood scope | Clarify what you need in more detail |
Long Content Strategy
For long content (like detailed reports or documentation), ask AI to break it into sections upfront. This prevents awkward cutoffs and gives you natural stopping points.
What This Means for You
Now that you understand how AI works, here's how to use it better:
Be Specific
Vague inputs produce vague outputs. The AI fills gaps with assumptions.
| Instead of... | Try... |
|---|---|
| "Write an email" | "Write a 3-paragraph email to a client explaining the project delay" |
| "Help me with code" | "Debug this Python function that should return the sum of a list" |
| "Give me ideas" | "Give me 5 marketing taglines for a sustainable coffee brand targeting millennials" |
Provide Context
AI can't read your mind. Tell it:
- Who you are (your role, expertise level)
- What you're trying to accomplish
- Any constraints or requirements
- Examples of what you want
Verify Important Facts
For anything consequential:
- Check statistics against primary sources
- Verify citations actually exist
- Test code before using it
- Cross-reference with authoritative sources
Start Fresh When Needed
If AI responses are getting strange:
- Start a new conversation
- Provide essential context upfront
- Don't try to "fix" a degraded conversation
Key Takeaways
- AI predicts words, it doesn't think - It's sophisticated pattern matching, not reasoning
- Models are the trained "brains" - Different models have different trade-offs; evaluate new ones yourself
- Confidence ≠ accuracy - AI sounds confident even when completely wrong
- Hallucinations are normal - Always verify important facts from primary sources
- Context is limited - Start fresh when conversations degrade
- AI is stateless - It doesn't learn from your conversations; each session starts fresh
- Training vs search - AI "knows" what it was trained on; it "looks up" current information
- Output limits exist - AI may stop mid-response; just ask it to continue
- Specificity wins - Clear inputs produce better outputs