Epistemology in Software Engineering: Knowledge and Certainty

• Introduction: Do We Really "Know"?
• Chapter 1: Core Concepts of Epistemology and Software Engineering
  • What Is Epistemology?
  • What "Knowing" Means in Software Engineering
• Chapter 2: Three Epistemic Illusions
  • Illusion 1: I Read the Code = I Understood It (The Readability Illusion)
  • Illusion 2: The Tests Passed = It Is Correct (The Limits of Verification)
  • Illusion 3: It Works in Production = It Is Correct (The Problem of Induction)
• Chapter 3: Cartesian Methodical Doubt and Code Review
  • What Is Methodical Doubt?
  • Applying Methodical Doubt to Code Review
  • A Practical Checklist for Methodical Doubt
• Chapter 4: Karl Popper's Falsificationism and Test-Driven Development
  • The Core of Falsificationism
  • TDD Is Falsificationism in Practice
  • Writing Falsifiable Requirements
  • Rereading Dijkstra's Warning
• Chapter 5: Comparison Tables - Software Verification Methods Seen Epistemically
  • Overall Comparison: Verification Methods and Epistemic Strength
  • The Epistemic Hierarchy of Verification Methods
• Chapter 6: The Dunning-Kruger Effect and Technical Humility
  • The Nature of the Dunning-Kruger Effect
  • Dunning-Kruger in Software Engineering
  • The Value of "I Don't Know"
• Chapter 7: Bayesian Reasoning and Technical Decision-Making
  • The Bayesian Mindset
  • Bayesian Thinking in Incident Root-Cause Analysis
  • Bayesian Thinking in Everyday Technical Decisions
• Chapter 8: Nassim Taleb's Antifragility and System Design
  • Fragile, Robust, Antifragile
  • Applying This to Software Systems
  • Design That Exploits Uncertainty vs Design That Refuses It
• Chapter 9: Collective Intelligence and the Limits of Individual Knowledge
  • The View from Social Epistemology
  • The Epistemic Basis of Code Review
  • The Epistemic Role of ADRs and RFCs
• Chapter 10: Practical Implications - Working Epistemic Humility into Engineering Culture
  • Building a Culture Where You Can Say "I Don't Know"
  • The Sprint as a Hypothesis-Testing Cycle
  • Writing Technical Documents That Admit Uncertainty
• Chapter 11: Checklists - Habits of an Engineer Who Practices Epistemic Humility
  • Daily Habits
  • Weekly Habits
  • Decision-Making Checklist
  • Code Review Checklist
  • Incident Response Checklist
• Conclusion: Socrates' Wisdom, the Engineer's Humility
• References

Introduction: Do We Really "Know"?

Software engineers work every day on top of judgments that begin with "I know." I know what this code does. I know how this system behaves. I know why this architecture is right. But is that "knowing" really knowledge?

Twenty-five hundred years ago, Socrates said, "I know that I know nothing." That humble confession became the starting point of epistemology in Western philosophy. Epistemology is the branch of philosophy that investigates the nature, scope, and limits of knowledge, asking fundamental questions: "What can be known?", "What is knowledge?", "Are our beliefs justified?"

Remarkably, these ancient questions map precisely onto the central problems of modern software engineering.

"Program testing can be used to show the presence of bugs, but never to show their absence." - Edsger Dijkstra, "On the Cruelty of Really Teaching Computing Science" (1988)

Dijkstra's famous warning is, at bottom, an epistemological claim: our knowledge of software correctness is fundamentally incomplete.

This article applies the major concepts of epistemology to software engineering, analyzing the traps of technical certainty and exploring why a culture that admits "I don't know" produces better software.

Chapter 1: Core Concepts of Epistemology and Software Engineering

What Is Epistemology?

Epistemology is a compound of the Greek words episteme (knowledge) and logos (study) -- the study of knowledge. Plato defined knowledge as "justified true belief." Under this definition, saying that you "know" something requires three conditions to hold.

1. Belief: the subject must believe the proposition
2. Truth: the proposition must actually be true
3. Justification: the belief must be supported by rational grounds

Substituting software engineering into these three conditions yields an interesting insight.

What "Knowing" Means in Software Engineering

In Thinking, Fast and Slow (2011), Daniel Kahneman divided human thought into System 1 (fast, intuitive) and System 2 (slow, analytical). When engineers judge that "this code is right," System 1 is usually the one doing the work: skimming the code, recognizing a pattern, and arriving intuitively at "this is correct." But that intuitive judgment is vulnerable to a long list of cognitive biases.

Chapter 2: Three Epistemic Illusions

Illusion 1: I Read the Code = I Understood It (The Readability Illusion)

The most common trap in code review is the illusion that "I read it, therefore I understood it." Gerald Weinberg identified this problem as early as The Psychology of Computer Programming (1971).

"The most dangerous moment in programming is the very moment you become convinced you understand the program." - Gerald Weinberg

This is what epistemology calls the "illusion of understanding." When we read text, we get the feeling of having understood it without actually understanding it deeply. Code is no different.

Mechanisms that produce the readability illusion:

• Pattern matching: on seeing a familiar code pattern, we skip the analysis and move on with "ah, it's one of those"
• Naming bias: when variable and function names reflect intent well, we judge by the name alone without verifying the actual implementation
• Confirmation bias: when the PR description says "bug fix," only the part of the code where the bug was fixed catches the eye
• Anchoring effect: the author's explanation pins the reviewer's judgment in place and blocks independent analysis

Countermeasures:

1. Do not read the PR description before reading the code (prevents anchoring)
2. Execute the core logic by hand in your head (manual tracing)
3. Ask first, "what input would make this code fail?" (a falsificationist approach)
4. The moment you feel you have understood, doubt yourself once more

Illusion 2: The Tests Passed = It Is Correct (The Limits of Verification)

When the whole test suite is green, we want to believe the code is correct. But this runs straight into the fundamental limits of induction.

The eighteenth-century philosopher David Hume stated the problem of induction plainly: past experience does not guarantee the future. Observing a thousand swans, all of them white, does not license the conclusion that "all swans are white" -- not until a black swan is found in Australia.

Software testing is the same.

Epistemic limits by verification method:

• Unit tests: verify the inputs and outputs of individual functions, but miss emergent bugs that arise from interactions between functions
• Integration tests: verify interactions between components, but cannot reproduce every variable of the real production environment
• E2E tests: verify user scenarios, but cannot cover every scenario given runtime and cost constraints
• Formal verification: the closest thing to a mathematical proof, yet it cannot verify whether the specification itself faithfully reflects reality

Illusion 3: It Works in Production = It Is Correct (The Problem of Induction)

"It has been running fine in production for two years." To an engineer, that sentence feels like the strongest justification available. But it is a textbook error of inductive reasoning.

In The Black Swan (2007), Nassim Taleb offers the "turkey problem." A turkey fed every day builds up, over a thousand consecutive days of being fed, the conviction that "the farmer is the being who feeds me." On day 1,001, at Thanksgiving, the turkey's neck is wrung. A thousand days of empirical evidence is invalidated in an instant.

The same pattern repeats in software systems.

• Date-handling logic that was trouble-free for two years fails on a leap year
• A system stable for ten years cascades into failure the moment traffic crosses a particular threshold
• Concurrency code that "never had a problem" corrupts data under one specific timing

The difference between "it works" and "it is correct":

Chapter 3: Cartesian Methodical Doubt and Code Review

What Is Methodical Doubt?

In Discourse on the Method (1637), René Descartes (1596-1650) put forward methodical doubt. His approach is simple but radical: doubt everything that can be doubted, and keep only what cannot be doubted.

Descartes doubted the senses, memory, and even logical reasoning, and finally arrived at the conclusion that the one thing he could not doubt was "the existence of the self that is doing the doubting" -- the famous "Cogito, ergo sum" (I think, therefore I am).

Applying Methodical Doubt to Code Review

Applying Descartes' methodical doubt to code review produces the following framework of questions.

Step 1: Doubt the premises

• Does the problem this code is meant to solve actually exist?
• Are the requirements themselves accurate?
• Does the user behavior we are assuming match reality?

Step 2: Doubt the implementation

• Does this algorithm work in every edge case?
• Does the error handling cover every failure path?
• Are there concurrency problems, resource leaks, or integer overflows?

Step 3: Doubt the verification

• Are the tests actually verifying something meaningful?
• Are there bugs in the tests themselves?
• Does the test environment reflect production closely enough?

Step 4: Doubt yourself

• Is my confidence that I understood this code justified?
• Are there biases influencing my judgment?
• Did I invest enough time and concentration in this review?

A Practical Checklist for Methodical Doubt

Chapter 4: Karl Popper's Falsificationism and Test-Driven Development

The Core of Falsificationism

Karl Popper is one of the most influential philosophers of science of the twentieth century. The falsificationism he set out in his major work The Logic of Scientific Discovery (1934) supplies a criterion for distinguishing science from non-science.

For Popper, a scientific theory must be falsifiable: there must exist some observation or experiment capable of showing the theory to be wrong. "All swans are white" is a scientific claim, because a single black swan can refute it. By contrast, "God exists" or "everything happens for a reason" cannot be falsified and therefore are not scientific claims.

TDD Is Falsificationism in Practice

Test-driven development resembles Popper's falsificationism to a striking degree.

Popper said that a theory can never be proved; it merely remains un-refuted. In the same way, as Dijkstra put it, tests cannot prove the correctness of code. They only maintain a state of not yet having been refuted.

Writing Falsifiable Requirements

Popper's falsificationism applies to writing requirements as well. A falsifiable requirement carries a clear test criterion inside it.

Unfalsifiable (bad) requirements:

• "The system must be fast" -- there is no standard for how fast
• "The user experience must be good" -- the criterion for "good" is undefined
• "It must operate reliably" -- there is no definition of "reliable"

Falsifiable (good) requirements:

• "API response time must be 200ms or less at p99" -- refuted if it exceeds 200ms
• "The login process must complete within three steps" -- refuted at four steps or more
• "Service availability must be 99.9% or higher per month" -- refuted below 99.9%

Rereading Dijkstra's Warning

In his 1988 paper "On the Cruelty of Really Teaching Computing Science," Dijkstra explained the fundamental limits of software testing this way.

"Program testing can be a very effective way to show the presence of bugs, but it is hopelessly inadequate for showing their absence."

This reflects Popper's falsificationism exactly. When a test fails, the presence of a bug is confirmed irrefutably. But when a test passes, the absence of bugs is not proved. The best we can do is raise confidence in the code incrementally through more attempts at refutation -- that is, more tests.

Chapter 5: Comparison Tables - Software Verification Methods Seen Epistemically

Overall Comparison: Verification Methods and Epistemic Strength

The Epistemic Hierarchy of Verification Methods

Arranging verification methods in order of epistemic strength produces a pyramid.

Level 1 - Belief: the author's own conviction. "I believe this code is right." Epistemically the weakest.

Level 2 - Social consensus: agreement from peers through code review. "Our team reviewed this code and approved it." Multiple perspectives are reflected, but it is vulnerable to groupthink.

Level 3 - Empirical evidence: passing tests and production operating experience. "This code passed 1,000 tests and ran in production for six months." Strong inductive evidence, but it cannot guarantee anything about unobserved territory.

Level 4 - Logical proof: formal verification and mathematical proof. "We proved mathematically that this algorithm is correct." The strongest, but limited in applicability and very expensive.

The key insight is that most software development happens at levels 2 and 3. We work in the domain of "sufficient confidence" rather than absolute certainty, and admitting this is the starting point of epistemic humility.

Chapter 6: The Dunning-Kruger Effect and Technical Humility

The Nature of the Dunning-Kruger Effect

In 1999 the psychologists David Dunning and Justin Kruger published the results of an experiment conducted at Cornell University. Titled "Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments," the paper became one of the most widely cited studies in the history of cognitive bias research.

The core of the Dunning-Kruger effect is as follows.

• People of low ability overestimate their ability: because they do not know what they do not know
• People of high ability underestimate their ability: because they assume what they know is obvious
• Self-assessment grows more accurate as ability improves: because metacognition -- cognition about one's own cognition -- develops

Dunning-Kruger in Software Engineering

The effect is observed with unusual clarity in software engineering.

This is why a junior developer says "this architecture is the best" in a voice full of conviction while a senior developer says, carefully, "well, in our situation there are these trade-offs..." The senior's caution is not incompetence; it is developed metacognition.

The Value of "I Don't Know"

In software engineering culture, "I don't know" is often taken as a weakness. For a senior engineer or a lead in particular, admitting ignorance can feel like a loss of authority. Epistemically, though, "I don't know" is an honest statement of the state of one's knowledge.

Why "I don't know" has value:

1. It starts an inquiry: the certainty of "I know" halts investigation, while admitting "I don't know" begins it
2. It activates collective intelligence: once one person admits ignorance, others begin sharing what they know
3. It prevents bad decisions: a careful decision that acknowledges uncertainty is safer than a confident decision built on incomplete knowledge
4. It builds a learning culture: in an environment where "I don't know" is safe, questions and learning flourish

How to say "I don't know" productively:

• "I'm not certain about this part, so I'll check and share what I find"
• "I need to confirm whether my understanding is right. Could we look at it together?"
• "This area isn't my specialty -- is there someone on the team who knows it better?"
• "My instinct says A, but I can't be confident until I see the data"

Chapter 7: Bayesian Reasoning and Technical Decision-Making

The Bayesian Mindset

Bayesian reasoning, named after Thomas Bayes (1701-1761), is a method for updating the probability of a belief on the basis of new evidence. The core formula is as follows.

P(A|B) = P(B|A) x P(A) / P(B)

Here P(A) is the prior probability, P(A|B) is the posterior probability after observing the new evidence B, and P(B|A) is the likelihood.

In Thinking, Fast and Slow, Daniel Kahneman noted that humans do not perform Bayesian updating naturally. We tend to either overreact or underreact to new evidence.

Bayesian Thinking in Incident Root-Cause Analysis

When a production incident occurs, Bayesian reasoning supplies a powerful framework for root-cause analysis.

Scenario: API response time suddenly increased tenfold.

Step 1: Set the priors (based on experience and statistics)

Step 2: Gather evidence and update the probabilities

Evidence 1: "There was a deployment within the last 30 minutes" -- the probability of a code deployment issue rises (25% -> 45%)

Evidence 2: "DB query response times are normal" -- the probability of database load falls (35% -> 5%)

Evidence 3: "Only one specific API endpoint is slow" -- the probability of a code deployment issue rises further (45% -> 70%)

Step 3: Act on the updated probabilities

Bayesian Thinking in Everyday Technical Decisions

Bayesian thinking applies not only to incident analysis but to everyday technical decisions.

Technology selection:

• Prior: "the probability that this framework fits our project is 60%" (based on community reputation and documentation quality)
• New evidence: a published case study of a similarly sized team using it successfully
• Posterior: the probability of a good fit rises from 60% to 75%
• New evidence: compatibility testing against our team's stack surfaces problems
• Posterior: the probability of a good fit falls from 75% to 40%

The point is to update judgments flexibly as evidence arrives. Not fixating on the first judgment, and instead adjusting probabilities every time new information comes in, leads to better technical decisions.

Chapter 8: Nassim Taleb's Antifragility and System Design

Fragile, Robust, Antifragile

In Antifragile: Things That Gain from Disorder (2012), Nassim Taleb classified systems into three categories.

Applying This to Software Systems

Characteristics of a fragile system:

• A single point of failure exists
• Error handling is absent or minimal
• Changes produce unpredictable side effects
• There is no mechanism for recovering from failure
• One failure paralyzes the entire system

Characteristics of a robust system:

• Redundancy is implemented
• Errors can be detected and isolated
• Fallback mechanisms activate on failure
• It withstands shocks up to a particular magnitude
• But it still fails once the shock exceeds the expected range

Characteristics of an antifragile system:

• Learns from failures and improves automatically
• Deliberately injects failure through chaos engineering to strengthen the system
• Failed parts are isolated automatically and the system reconfigures itself
• Scales out automatically as load increases
• Learns from past failure patterns to improve defenses against similar future failures

Design That Exploits Uncertainty vs Design That Refuses It

Netflix's chaos engineering is the canonical example of antifragile design. Through Chaos Monkey, Netflix deliberately takes services down in the production environment. That "stress" makes the system sturdier. Rather than trying to avoid failure, they use failure to strengthen the system.

Chapter 9: Collective Intelligence and the Limits of Individual Knowledge

The View from Social Epistemology

Traditional epistemology focused on individual knowledge, but social epistemology attends to the social dimension of knowledge. Knowledge does not exist only inside an individual head; it is produced and validated through the interactions of a group.

This perspective matters enormously in software engineering. Modern software systems are too complex for one person to understand in full. So we rely on collective knowledge mechanisms.

The Epistemic Basis of Code Review

Code review is not merely a quality-control procedure. Epistemically, code review is a process of knowledge validation by multiple knowing subjects.

When one person writes code, their knowledge of that code is personal and subjective. When code review adds another person's perspective, the knowledge becomes intersubjective. That provides a stronger justification than individual belief does.

The epistemic value code review provides:

The Epistemic Role of ADRs and RFCs

Architecture Decision Records (ADRs) and the Request for Comments (RFC) process are mechanisms for raising the epistemic quality of decisions.

The epistemic functions of an ADR:

1. Explicit justification: recording "why we made this decision" supplies the grounds of justification to your future self and your colleagues
2. Context preservation: recording the constraints and background at the time of the decision prevents knowledge from being severed from its context
3. Recording alternatives: recording alternatives that were considered but not adopted leaves a trace of the attempts at refutation
4. Reversibility: it establishes the basis for reversing the decision if it turns out to be wrong

The epistemic functions of an RFC process:

1. Criticism up front: validating the design collectively before implementation surfaces errors at the stage where they are cheap
2. Securing diversity: gathering opinions from varied backgrounds reduces bias
3. Evidence-based discussion: requiring grounds for claims promotes justified assertions rather than mere opinions

Chapter 10: Practical Implications - Working Epistemic Humility into Engineering Culture

Building a Culture Where You Can Say "I Don't Know"

For "I don't know" to be a safe thing to say in an organizational culture, structural mechanisms are needed.

1. Leaders model it first

When a technical leader or manager says "I'm not certain about this part either," psychological safety rises across the whole team. Google's Project Aristotle research found that psychological safety is the single most important factor in high-performing teams.

2. Establish practices for recording uncertainty

• Mark a "confidence level" in technical documents: HIGH for certain content, MEDIUM for content resting on assumptions, LOW for guesses
• Add an "unknown unknowns" section to ADRs
• Mark uncertain parts in code comments: "whether this optimization actually helps needs to be confirmed by benchmark"

3. Build mechanisms that encourage questions

• Go beyond "there are no stupid questions" to a system that rewards good questions
• A culture that encourages "why?" in code review and answers such questions in good faith
• Require new hires to write a "list of questions" during onboarding

The Sprint as a Hypothesis-Testing Cycle

An agile sprint can be reinterpreted through the lens of Popper's falsificationism.

Writing Technical Documents That Admit Uncertainty

Here is a framework for handling uncertainty in technical documentation.

A confidence notation system:

Applying this notation consistently across technical documents lets readers grasp the certainty level of each piece of information immediately.

Chapter 11: Checklists - Habits of an Engineer Who Practices Epistemic Humility

Daily Habits

• When writing code, think first about "the cases in which this code could fail"
• Ask at least one "why?" question in every code review
• Instead of "I'm certain," say "based on the information available so far, this is my judgment"
• Once a day, look back on "what I admitted not knowing today"

Weekly Habits

• Share "an assumption we got wrong this week" in the team meeting
• Update the confidence notation in technical documents
• Look for "patterns that differ from expectations" on the monitoring dashboards
• Review and update the "things we don't know" list

Decision-Making Checklist

• What data underlies this decision?
• In what circumstances could this data be wrong?
• Have I looked at this decision from the perspective of someone who disagrees?
• Can this decision be reversed if it turns out wrong? (reversible vs irreversible)
• What biases of mine could be influencing this decision?
• Is there any part where the honest answer is "I don't know"?

Code Review Checklist

• Was my judgment pinned in place by the PR description before I read the code? (checking for anchoring)
• Did I execute the core logic in my head? (manual tracing)
• Did I ask, "what input would make this code fail?" (falsificationist thinking)
• Are the tests verifying something meaningful? (test quality check)
• Have I fallen into assuming "the author knows this code better than I do"? (checking for authority bias)
• Did I really understand this code, or do I merely feel that I did? (checking for the illusion of understanding)

Incident Response Checklist

• Have I fixated on the first hypothesis? (anchoring effect)
• Am I updating the probability of each cause as evidence arrives? (Bayesian thinking)
• Am I leaving open the possibility that "this may not be the cause"? (falsificationist thinking)
• Are past similar incidents distorting my current judgment? (availability bias)
• Am I bringing together the varied perspectives of my teammates? (social epistemology)

Conclusion: Socrates' Wisdom, the Engineer's Humility

Socrates' confession 2,500 years ago -- "I know that I know nothing" -- is wisdom more urgently needed in software engineering today than ever.

The systems we build grow steadily more complex, more interconnected, and less predictable. In the face of that complexity, the certainty that "I understand this completely" is the most dangerous form of ignorance.

The core lessons epistemology offers software engineering are these.

1. Knowledge is not absolute: the correctness of code can be claimed only through the absence of refutation, never through proof (Popper)
2. Certainty conceals bias: our judgments are distorted by a range of cognitive biases (Kahneman)
3. Uncertainty is not the enemy but information: exploit uncertainty and the system grows stronger (Taleb)
4. The wisdom of the group beats the certainty of the individual: code review, ADRs, and RFCs are not luxuries but epistemic necessities

As Gerald Weinberg said fifty years ago, programming is ultimately a psychological activity. And epistemic humility is the most fundamental psychological quality for building better software.

"True knowledge is knowing your own ignorance." - Socrates

Technical humility is not a weakness. It is the most powerful engineering principle for surviving in a world of complexity and uncertainty.

References

1. Karl Popper, The Logic of Scientific Discovery (1934) - the basic principles of falsificationism and scientific methodology
2. Edsger Dijkstra, "On the Cruelty of Really Teaching Computing Science" (1988) - the essential limits of software testing
3. Daniel Kahneman, Thinking, Fast and Slow (2011) - cognitive bias and the psychology of decision-making
4. Nassim Taleb, Antifragile: Things That Gain from Disorder (2012) - designing systems that exploit uncertainty
5. David Dunning and Justin Kruger, "Unskilled and Unaware of It" (1999) - the original paper on the Dunning-Kruger effect
6. Gerald Weinberg, The Psychology of Computer Programming (1971) - the psychological side of programming
7. Nassim Taleb, The Black Swan (2007) - extreme uncertainty and the limits of prediction
8. Google Research, "Project Aristotle" - psychological safety as the key factor in high-performing teams