Forsake thy cage,
Thy rope of sands,
— George Herbert, The Collar
I want to train a neural network.
So first I must learn the theory—read a textbook, ankify it. But to understand the math I must first learn probability and statistics in depth. Frequentism or Bayesianism? I heard really good things about Jaynes. Do I have the mathematical maturity? Maybe I should revise, say, set theory. The proofs in this textbook are very hard to read, so I rewrite them and make them more formal and refactor them into lemmas small enough to put into spaced repetition. They’re starting to look like Fitch notation. Maybe I can write my proofs in first-order logic? Learning logic might give me a stronger foundation for doing proofs. But what is logic, really?
Somewhere in another Everett branch, a more sensible version of me is doing a PyTorch tutorial while I read Quine on what is or isn’t an ontological commitment in a first-order theory. I have six SEP tabs open, and I still don’t know what a ReLU is.
I call it depth-first procrastination. You start with a high-level goal, and rather than taking the optimal route, a combination of intellectual curiosity and anxiety about standing on shaky foundations leads you to spawn subgoal after subgoal after subgoal, until you hit rock bottom and you find yourself doing some task so distantly related to the original goal you’re in another universe entirely. See also: yak shaving, OCD.
I do this all the time: I learned to program because I wanted to write a program to write novels in. And while I never did do that, I did invent my own programming language. In college I spent more time writing note-taking software than taking notes.
The upside of this is you learn self-reliance, and you learn foundational knowledge others might not. The downside is by the time you pop the stack back up to the top you’re old and dead.
And it’s always bullshit. Labouring in obscurity on some grandiose scheme for years is overrated. Tighten the feedback loop: trim the fat, ship, react, iterate. Do or die.