Learning from First Principles

A Troublesome Student

I was a poor student in elementary school. I was unable to focus for more than a few minutes at a time. Bored and restless I treated the classroom as a kind of one-man show to entertain myself and earn the admiration of my peers. I remember bringing our family’s fancy silverware to school one morning in 4th grade. During math class I distributed the cutlery to each student at each desk as if we were about to enjoy a meal. But there was no meal and my mother and teachers were not amused.

In this day and age, I would have been diagnosed with a “learning problem.” Most likely Attention Deficit Disorder, the infamous ADD, and perhaps medicated. Luckily, I grew up in the dark ages of the late 1960s. The diagnosis was “troublemaker” and the solution was a desk in the hallway outside of the classroom. I was given only math worksheets and a pencil for entertainment. The educational system had given up on me.

Sitting alone and isolated was one of the best things that ever happened to me as a child.

With no audience and no windows in which to daydream I opened the workbook to page one and started doing math problems. There was nothing else to do and no instructor to rebel against. 

It was slow going at first but by the end of the month I was in love with math and several chapters ahead of the class. I moved up and out from 4th grade into 5th grade math. I was rehabilitated into the inside of the classroom. Eventually, I even started to pay attention to the math teachers. 

I had learned 4th grade math from first principles.

I should note that I’m not a great mathematician. Because of my loner learning style, I don’t do math like classroom-trained people do. For example, I never bothered to memorize my times tables. Instead I write out a table of three number lines, natural, even, and odd. I can then derive any integer product, dividend, or remainder by lookup and simple arithmetic. I had created a tool to avoid route memorization.

More importantly this was my rough introduction into learning how to learn.

Learning without Guidance

Learning from first principles means learning without a guide and building up from what you already know. The most famous example of first principles are the Elements of Euclid. I’m sure you’ve heard of this ancient Greek geometer. He started with a small set of fundamental observations about the behavior of points and lines on a plane and derived hundreds of conclusions that formed the basis of geometry as we practice it today. 

During the early part of my career we had to be a bit like Euclid to figure out how to develop software, deliver applications, and build teams from first principles. Back in the 1980s and 1990s we didn’t have best practices and opinionated programming languages. We had to solve problems with code without guidance.

At Apple we did object-oriented programming before there were object-oriented programming languages. We learned from first principles that function pointers and pointers to pointers (handles) could be used to encapsulate properties and methods into objects. We didn’t invent C++ or Objective-C but when these advances came around, they felt like old friends.

At DoubleClick in the 2000s we did cloud computing without containers or virtual machines. We learned from first principles how to configure hardware and software, so that every server was an exact clone. This enabled ad serving workloads to be automatically propagated through our network so that our QoS was easily maintained in the face of failures. We didn’t invent AWS but when Amazon started marketing their excess capacity, we had a similar business plan on the table. 

The power of learning from first principles is that it empowers you to figure out the future on your own with the tools you have at hand. The best practices and opinionated programming languages if today are the first principle hacks of yesterday.

A Short Guide to Guideless Learning

So, how does one learn this way? How does one start down the path of first principles?

It starts with observation. Start like Euclid did with a set of simple, self-evident facts that a non-expert can understand. Euclid did this by observing how two points and lines interact to create angles and shapes on a flat surface. 

It helps, in my opinion, to be a non-expert in order to make these observations. I fight hard to ignore what I “know” so I can see what is really going on.

Restrict your observations to the abstract. Euclid only recorded observations are universal regardless of the environment. He focused on how points and lines interact on an ideal surface. Euclid didn’t get caught up in specific details.

Abstraction is a superpower all of us share. Abstraction is the ability to discount the non-essential and to focus on the properties that are shared by seemingly unique entities. It’s been helpful to me to think about effects – not causes – when I’m thinking abstractly.

Build up a foundation of true definitions. Euclid recorded his observations as a handful of definitions upon which he built the empire of two-dimensional geometry. These definitions were Euclid’s building blocks in the same way that the definitions of legal terms build up a contract or the line items of expenses and credits build up a budget.

I find it helpful to record each observation about the system I’m studying on an index card. These cards become my working dictionary for the problem space. As a bonus index card are cheap, easy to order and reorder, and restrict my verbosity.

Apply simple logic to the definitions to make predictions. Euclid used his definitions to build up hundreds of propositions. These propositions were not simple and not obvious, and yet he derived them simply and obviously. When Euclid was done, he could predict the number of degrees in an angle, the length of lines, and the diameter of circles.

It helps to express predictions in diagrams and to show how the logic works in with arrows and annotation. I used to start with the diagram. But if the underlying observations, abstractions, and definitions are not rigorous then the diagram is worthless. I’ve seen these worthless diagrams on whiteboards and in business docs during my entire career.

Be a Lazy 9-Year-Old

The lazy 9-year-old version of me accidently discovered learning from first principles out of a fascination with patterns and a lack of fascination with listening to others. I’ve matured and now deeply value what others, even elementary school math teachers, have to say.

A great place to discover learning from first principles is in the classroom. The student has to be receptive. The teacher has to be empowered. Parents have to be patient. Society has to be supportive.

I still use learning from first principles to this day. It’s how I figure out how to deliver more value with less budget or scale software development with hundreds of engineers. In full disclosure I do read and value all the books and blog post on best practices and prior experiences. But I view this prior art as theory and not law. There is still a lazy 9-year-old deep inside me who isn’t content with being told. I need to find out how things work for myself.