4. Jnana
You have met the engineer who can draw the whole system from memory. Every service, every queue, the retry policy on each edge, the failure modes named and diagrammed. They are lucid in the design review, and they are right. Then the thing breaks at scale on a quiet Sunday, and they are standing at the same wall as everyone else, reading the same dashboards, as lost as the new hire beside them. The knowledge was real, and it was the wrong kind.
The Vivekachudamani, the “Crest-Jewel of Discrimination,” is a long poem on liberation attributed to Shankara. Partway through, he stops to draw a line between two kinds of knowledge, and he draws it hard. One kind is verbal. You can expound the scripture, win the debate, hold the entire doctrine in your head. Verse 58 grants that this is real, and even pleasant, and then sets it aside. Eloquent speech, “the skill in expounding the Scriptures, and likewise erudition”: these, it says, “merely bring on a little personal enjoyment to the scholar, but are no good for Liberation.” The knowledge that frees is the other kind. The tradition names it aparoksha-anubhava, direct and unmediated realization. Not paroksha, not the thing held at one remove through words and report, but the thing known first-hand.
He has a name for the pile of words, too. Verse 60 calls scripture a shabda-jala, a net of words, “a dense forest which merely causes the mind to ramble.” And verse 62 gives the mechanism in an image no engineer will misread: “A disease does not leave off if one simply utter the name of the medicine, without taking it.” You can know the name of the cure and stay sick. Naming is not taking.
We have built our whole idea of knowledge transfer on the opposite premise. Documentation is treated as knowledge in transit: write the runbook, draw the diagram, keep the wiki current, and the knowing moves from the person who has it to the person who reads it. We onboard by pointing a new hire at the docs. We call a well-documented system a well-understood one, as though those were the same claim.
They are not the same claim. A document carries information. It does not carry the knowing. The runbook line “the service degrades gracefully under load” is a sentence, and what the sentence is about, the actual behavior of the actual system when the load arrives, is not inside the sentence. It is inside the system, and the only road to it runs through the system. You can read what the moon is, but its nature, said verse 54, “is to be known with one’s own eyes.” No one sees it for you.
Line the two kinds of knowledge up against the work. The docs, all of them, the runbooks and diagrams and design records and the wiki nobody can hold in their head, are shabda-jala. They are paroksha, knowledge at one remove. The knowing is what remains after you have operated the system under real load, real time, real data, and real failure. That is aparoksha, and on-call is the discipline that produces it.
The architect from the opening had the docs perfectly. What they lacked was the hours of watching the thing behave. The runbook step “restart the service and it recovers” was true, and it was also only words, until the night a restart did not recover, because every instance came back at once, reconnected in a herd, and put the database on the floor. You knew the sentence, not the thing it pointed at. The gap between the two is the whole distance from paroksha to aparoksha, and it closes only when the system teaches you directly, never a moment before.
You don’t know a system until you’ve operated it under load and failure. Docs are knowledge about it; on-call is knowledge of it.
The runbook is a note from someone who knows, to someone who does not know yet. It can point, but it cannot hand the knowing over.
The source over-claims here, and we should say so. Shankara’s line is absolute. Verse 59 holds that scripture is fruitless before realization and equally fruitless after: “The study of the Scriptures is useless so long as the highest Truth is unknown, and it is equally useless when the highest Truth has already been known.” That is true for his end, which is moksha, and false for ours. Docs are not useless before, and they are not useless after. They are the map, and you want the map. It was drawn by people who made the trip, it will save you hours, and it is the first thing to reach for. The narrow claim is the true one and the only one worth carrying across: the map is a different kind of thing than the ground, and reading it is a different kind of knowing than walking it. Take that, and leave the absolutism where it belongs, in a tradition aimed at a goal software does not have.
There is a second limit. Not everything has to be suffered to be known. A pure function, a typed contract, an idempotent endpoint: you can read those and know them, and you should. The knowing that only failure teaches is the operational kind, the emergent behavior that surfaces when the system meets scale and concurrency and real time and the parts of reality no fixture holds. That is what on-call has and the docs do not. Keep the claim there.
And there is a way to read the principle that turns it into something ugly. “You don’t know it until you’ve seen it fail” is one bad step from a hazing ritual: real engineers earn it in the incident, the pager is a rite of passage, and a team with good docs and quiet nights is somehow soft. Read that way, it rewards hero culture, and it quietly rewards keeping the system undocumented so the knowing stays scarce and prestigious. That is the opposite of the point. The knowing is the goal, not the suffering, and the mature move is not to wait for the disaster. It is to manufacture the direct experience on purpose, safely, before production forces it on you.
That discipline already exists, and it is aparoksha-anubhava with a maintenance window. Jesse Robbins ran availability at Amazon under a title his manager approved, “Master of Disaster.” He had trained as a firefighter, and GameDay ran on a firefighter’s premise. You do not learn fire by reading about it. You build the intuition by facing controlled fire, on purpose, again and again, until the real thing arrives and finds you already knowing. GameDay broke Amazon’s systems deliberately and on a schedule, so that the people and the software met the failure before the failure counted.
Netflix industrialized the same idea. Chaos Monkey, built in 2010, roams the production fleet and kills instances at random during business hours, so that “the service survives an instance dying” stops being a line in a design doc and becomes something the team has watched happen and handled a hundred times. The Simian Army that followed in 2011 added the larger failures, whole availability zones. None of it runs once. The system keeps changing, the knowing expires with it, and the failures get met again on a schedule rather than at a rite of passage. The Principles of Chaos Engineering states the epistemology with none of the Sanskrit: it is “the discipline of experimenting on a system in order to build confidence in the system’s capability to withstand turbulent conditions in production.” The confidence comes from the experiment, not from the argument. You do not reason your way to trusting a system; you break it where breaking is cheap and watch how it behaves.
It has a limit of its own. You only get to rehearse the failures you thought to name, and the one that finally takes you down is often the one no game day imagined.
There is one more turn, and it is the honest one. The Vivekachudamani is a text whose entire argument is that texts do not free you. It makes that case in verse, for close to six hundred stanzas, and then hands you to a teacher and to your own realization. The Principles of Chaos Engineering is a document whose whole content is that reasoning from documents is not enough. Each is a pointer that names its own limit out loud.
So is this. Nothing here transfers the knowing. You can finish the essay agreeing with all of it and still not know your system, because the words were only ever about the thing. Like the runbook, and like the poem, this can point. The knowing is on the other side of the pager.
Further Reading
The Vivekachudamani, attributed to Shankara, is most accessible in Swami Madhavananda’s translation (Advaita Ashrama), public domain and the one quoted here; the verses on book-learning and direct realization run from 54 to 63, with the moon known with one’s own eyes at 54, the net-of-words image at 60, and the disease-and-medicine verse at 62. Swami Chinmayananda’s commentary works through the poem line by line for a modern reader. John Grimes’s translation and study (Ashgate, 2004) is the one to reach for on the scholarly question of whether Shankara wrote it at all, since the attribution is debated. On the software side, the Principles of Chaos Engineering are at principlesofchaos.org; Jesse Robbins and his co-authors set out GameDay and its firefighter lineage in “Resilience Engineering: Learning to Embrace Failure” (ACM Queue, 2012); and the Netflix Technology Blog documents Chaos Monkey and the Simian Army. For the runtime and the code as one thing, the essay on Vivarta takes the theme further.