Some thoughts on self-learning
Context: I recently had a discussion with a friend about the process of self-learning, which is very essential for scientific and engineering disciplines. During which, the point of maintaining consistency or flow state came up. I have struggled with this myself, and have had a disorganized pile of thoughts and conjectures about this for a long time. This post aims to organize them a bit.
Note: I am writing this late at night, so expecting many grammatical and structural mistakes, but I don’t care about perfection here.
Learning $\equiv$ Detective work
There is an inherent beauty in acquiring knowledge (ज्ञान). Several great human minds have talked about the importance of this, some notable and my favourite examples being:
The goal of mankind is knowledge . . . What man “learns” is really what he discovers by taking the cover off his own soul, which is a mine of infinite knowledge.
Swami Vivekanand
Spend each day trying to be a little wiser than you were when you woke up. Day by day, and at the end of the day-if you live long enough-like most people, you will get out of life what you deserve.”
Charlie Munger
Along with this, I believe there is an inherent flaw of not realizing this beauty and trying to gain knowledge purely to achieve some materialistic desire, for example. - learning system design for the purpose of passing a job interview, learning ML/AI because of the hype, and reading books because everyone seems to be doing it. It is okay for such desires to spark your initial interest, but the fuel for the remaining journey should be pure curiosity. There is a reason why reading fiction (specifically detective novels) or reading about some new controversy theory on the internet is comparatively easier. There is an inherently compelling story in such cases that raises many questions within us - Why the hell did he commit the murder? What is the government hiding? …
You will observe this to be true for your favourite professors or teachers. All of them began the class with “why” to take this course, and for each new topic or lecture, they started with an enquiry: Why are we studying this new technique? What was the flaw with the last topic we studied? What problem was the original inventor/discoverer trying to solve?…. Some will even let you experience the same journey through assignments or experiments. Some will let you devise your solution first, then point out its flaws - eventually leading you to think, “How the hell can this be even solved?”.
During self-learning, the responsibility of asking such queries is on the learner itself, and forming this inquisitive mindset is achievable once we are aware of it. This also helps in pruning and traversing the knowledge graph of what to study. For example, there are a plethora of articles/papers and videos on Machine Learning or just LLMs, and going with a mindset to understand it all is the perfect way to provide more business to the companies producing blood-pressure medicines. Instead, a better way would be to enter with queries -
- What the hell is a LLM?
- This may lead you to read some introductory and high level articles by amazing people like Andrej Karpathy or Sebastian Raschka’s substack.
- How did they (companies like OpenAI / Anthropic / Google) made it?
- Now if you are new to deep learning, this will lead to somewhat deeper dive. But if you have a decent understanding of machine learning and deep learning, you may ask - what changed in the recent times? Was just more data sufficient for this?
- This will lead you to realizing that the new architecture of transformers was particularly important to achieve this. So you read more about this …
Similarly for books, reading a book because many people read it is not a good approach. Spend some time skimming the intro of the book. Does it raise enough curiosity in you that you want to delve deeper? If not, move on, maybe it is not the right time and circumstances for you to read this now.
This investigative and divergent approach of learning will require more effort compared to the linear learning that we are used to in schools, but this doesn’t meaning taking structured-courses is wrong, even if you are taking a course - try to formulate a query or a question for every new thing.
Improving the system
You do not rise to the level of your goals. You fall to the level of your systems.
James Clear, author of Atomic Habits
Even if we have acquired a good mindset to learn something new, there are some ways in which the learning system can be improved for better consistency.
Iterative improvement after reflection
Just like any good system, premature optimization is bad for the learning system. This also means that we need to iteratively improve our system after carefully reflecting on what is not working, this is a purely subjective process.
Pain + Reflection = progress
Ray Dalio
To create an analogy to the Deep Learning techniques, we need to tune many hyperparameters for the learning process. An example being the learning rate of the gradient descent algorithm. Similarly, while starting a book, it is tempting to read a lot on the first few days, but it might consume a lot of effort, thus leading to lack of motivation in the next days - and the momentum collapses. A better solution would be to start small maybe just two pages a day, let the momentum build up, get used to the system and let consistency produce better results than one-shot efforts.
Also, there will some days when progress may not be possible, but consistency is not about a lot of progress every-day, but mainly about some progress for the majority of days.
Boredom is good for you
Learning is not just done while you are focusing on it while sitting at the table. It requires connecting the dots with the existing information while you are lost in some thoughts. There are several articles and reports on why boredom is good for us. I found this video by Veritasium and this article by Farnam Street particularly helpful. This also means we need to resist the temptation of checking phones / social media while we are bored.
Make sure to keep the balance
Medicine, law, business, engineering, these are noble pursuits and necessary to sustain life. But poetry, beauty, romance, love, these are what we stay alive for.
Robin Williams
While being on this journey, you may feel giving up because life is more than just acquiring knowledge and working on some projects. But satisfactions is not found in the extremes, it is better to maintain balance.
The weights of this balance can be subjective, for some, it may be
0.4*work + 0.2*family/friends + 0.2*exercise.... The weights will also change with time, and not just the weights, the dimensions of this equation itself might change. I doubt that anyone will be able to sustain the extreme case of one-hot vector - either completely focused on work or something else.
A good thing about this vector is that the dimensions are not independent, some effort in one of them will also be effective for the others. For example, I love to play football, anytime and anywhere, and have observed that I am able to better work if I have played a game in the recent days (btw it has been more than 3 months since I last played and my work productivity is failing rapidly because of this).
Life is made of star-stuff. We are the means through which the cosmos aims to understand itself.
Carl Sagan
