Learning.

One deep pain I feel, that I suspect is quite common to the human experience, is an appreciation for learning only after I left the education system. When the whole world is in front of us as children and into our teenage years, I think we become overwhelmingly excited by the experiences we haven't had and how much there is to explore in life. Not to say this isn't a form of learning, but is exceptionally biased toward action.

As example - my younger years were characterized by a strong motivation to accomplish certain things, such as a level of financial freedom to realize those experiences. This isn't to say that I was hyper-focused on any particular goal, but more simply that I found myself always in pursuit of something – whether it be short term rewards for the enjoy the day or a longer-term objective I didn't let go of across multiple years.

As we mature, we increasingly appreciate these pursuits less and less for the outcomes, and more more for the process. Whether it be taste of a glass of wine or the skills shaped by carrying out a project, the inebriated mental state is no more the goal than the fleeting joy of a launch announcement. More intriguing is the next sip of competence, trusting that each iteration will to metabolize into a more integrated self. Distilling this fully, I've come to find that the purest form of process is learning.

This makes the advent we live in all the more enthralling. AI, for all it's shortcomings, has an obsession with how systems learn. As much as it is about the underlying mathematical principles, it is so strongly shaped by the philosophy of learning. The frontier AI researchers are grappling with the very definition of intelligence. I am ecstatic to see the decreased emphasis on by rote knowledge, a push beyond entity relationships, and this deeper pursuit of processes. Processes not only of how we teach machines, but of how machines learn to improve themselves.

There's a lovely interplay between Richard Sutton, father of reinforcement learning, and Dwarkesh, a podcaster and LLM-intelligence advocate. In truth, this iceberg of a podcast is easiest to explore from multiple angles as tackling it head-on is a bit jarring. Literally, the podcast is a bit hard to listen to at moments – and I personally recommend interleaving Karpathy's take or Dwarkesh's reflection on the matter. These characters really get at this crux of what intelligence is, what it means to learn, and the practical aspects of what we can achieve.

Ironically, this is why I find it so hard to focus on precise goals at the moment, because I find such a deep connection to this philosophical problem. Learning, it seems to me, is at the core of what it means to be. That makes it all the more exciting that

Of course this prompts me to want to learn how to use LLMs a tool to solve problems in my own life, it also feels so much more real that we're getting sufficiently close to having enough fundamentals to build something cohesively "intelligent".

Reinforcement learning, or perhaps more precisely Sutton's "Bitter Lesson" of pure resolve of transforming experience into learning sets a framework of patterns and anti-patterns. Transformer architectures and back propagation seem to have unlocked practical mediums and methods for us to transfer the opus of raw information into something that starts to actually look intelligent. And at present - I can't help but wonder if mechanistic interpretability can shed more light on increasingly higher order concepts: knowledge, relationships, causality, and perhaps ultimately, self-reflection. I truly hope we do not get lost in the local optima of excessive human intervention – research remains to answer fundamental and beautiful questions.

Each day I find myself falling deeper in love with learning. It surrounds our lives in all elements – from popping in headphones and tuning into a podcast to entrenching ourselves deep into a problem we have to dig ourselves out of with nothing but our mind. Our daily challenges can be converted into opportunities to practice our craft. We improve the experience itself, and we improve at our ability to take on new experiences. That is why I've come to appreciate this intrinsic love of learning as so naturally intertwined with a fulfilling life.

—

P.S. My overuse of em-dashes predates LLMs, and I find it annoyingly ironic that to remove them from my craft would be inauthentic, despite their ties to in-authenticity!