Nguyen Le PhongNguyen Le Phong

What AI Cannot Know From the Prompt Alone

A practical reflection on why prompts are not enough: AI needs source context, runtime evidence, domain constraints, and human judgment to become useful engineering help.

The prompt looked clear: make the import flow safer. It named the file, described the symptom, and asked for tests. A model could produce a reasonable patch from that sentence. But the important facts were not inside the sentence. They were in the old retry job, the support runbook, the tenant boundary, the strange CSV one customer still sends, and the alert that only fires after midnight.

An engineer studies a laptop with abstract code blocks while blank cards and a system sketch sit on the desk beside it.
A prompt can ask for an answer, but the surrounding evidence decides whether the answer belongs in the system.

This is the first limit worth respecting. A prompt is not the system. It is a request made from one point of view. It may include the visible problem, but it often misses the constraints that make the solution safe. AI can be very helpful inside the frame it receives. It is much weaker at knowing which frame is incomplete.

Source code changes this. The real imports, call sites, tests, feature flags, schemas, and error handling paths tell a story that the prompt cannot carry alone. Sometimes the requested fix already exists in a helper. Sometimes the right change belongs in a usecase, not a handler. Sometimes the user-visible bug is caused by data shape, not the function named in the prompt. Without source context, the model is guessing from a shadow.

Runtime evidence matters too. Logs, traces, failed jobs, database rows, browser screenshots, and exact error text keep the answer honest. A generated explanation can sound calm and still be wrong about what happened. The more serious the change, the more the work should touch reality before it touches production.

Domain rules are another missing layer. AI can infer common patterns, but it does not automatically know which invariant your team treats as sacred. It does not know that money must be append-only, that tenant scoping must happen in the repository query, that a mobile client still depends on a deprecated field, or that an operational kill switch exists because a past incident hurt real users.

This is why context engineering is not just writing longer prompts. Longer prompts can help, but the deeper discipline is deciding what evidence the assistant is allowed to rely on. Give it the spec. Give it the failing test. Give it the relevant files. Give it the logs. Then ask it to explain uncertainty instead of filling gaps with confidence.

Human judgment remains load-bearing because someone has to decide whether the generated answer fits the system we actually own. That judgment is not a ceremonial approval at the end. It shapes the question, chooses the sources, rejects plausible but unsafe shortcuts, and verifies the result. The human does not need to type every line. The human does need to keep responsibility close.

A useful habit is to make the assistant cite its grounding in local evidence. Which file supports this? Which test would fail without it? Which log line proves the symptom? Which domain rule protects the edge case? If the answer cannot point back to evidence, treat it as a draft hypothesis, not an implementation plan.

AI becomes more useful when we stop asking the prompt to carry more than it can hold. The prompt starts the conversation. The system finishes it. Before accepting an AI answer, what piece of evidence outside the prompt would make you more confident that the answer belongs?

What did you think?