`
Hello :) Today is Day 366! The last day of this blog!
On January 2nd, 2024, I started with zero AI knowledge and self-studied it for 365 days.
My Background
I come from accounting and data analysis. Before the above date, my AI-related knowledge was pretty much non-existent - I knew about linear regression, R-squared, the power rule in differential calculus, and working experience with Python and SQL for data manipulation.
Books and Courses
Here is a list of books and courses I completed during these 365 days and my personal favourites: My Learning List.
I summarised book chapters, talks, videos, shared my handwritten notes from online lectures, talked about projects, tried to stream and made some videos. I tried to make this final post concise while also discussing specific topics that might interest people who found this blog through social media and this is the first post they have seen from my blog.
Original plan
Originally, my plan for this learning journey was for it to be 50 days - during my winter break from university. At day 50 I decided to continue until day 100. Days before day 100 I, once again, knew that it was not over, so I continued and after a few days post day 100 I decided to make it a full year of learning and blogging. I never thought about the time constraint - I knew I had plenty of time on my hands as I was just a 2nd semester Master student (out of 4) so I didn’t set a deadline to ‘learn AI’ by some deadline. Will I have learned more if I was doing an AI degree at university, or I had someone as a guide? I cannot know. However, I think I liked my journey as is. Self-studying can be more challenging in the sense that I am more free to look around, but also can get lost in the ocean of materials to study. I believe I managed this ocean pretty well and learned a lot.
Naming the Journey
When I started I said ‘learn AI’ because the whole blog needed some kind of name. However soon after I realised how ambiguous that name is and that AI contains many many things inside of it. After some time I changed the whole journey to ‘Learn Stats, Math, Data, AI’ as I realised that to understand ‘AI’ I need statistics, and mathematics, and for good AI I need good data. I will revisit these points about stats, math and data below.
One word: Serendipity
I think a good word to describe my year/learning journey is serendipity. I enjoy difficult topics, I enjoy the process of learning - being a beginner and improving. I started with a random course on ML from caltech by Professor Abu-Mostafa from 2012. I cannot recall how I found it but it seemed like a place to start. I wrote this quote on day 1 of the blog from the 1st lecture from Prof. Abu-Mostafa:
If you look at linear models, you can think of them as an economic car. They get you where you want to go and they don’t consume a lot of gas. You may not be very proud of them, but they actually do the job. It is remarkable how often they succeed on their own and they are sufficient to get you the learning performance that you want. Try to use them first and see if they achieve what you want.
It is amazing how relevant that quote is as throughout my journey I saw different incarnations of it again and again. I mentioned serendipity because I was doing something I liked - studying something different, something new to what I have studied so far in my life - and courses, books, resources kept ‘appearing’ randomly. I joined the reddit communities for ML/DL, followed a bunch of people who had AI/ML in their profile descriptions, and suddenly I was flooded with new resources that I can study. So many different resources, how could I know which ones to go for first? I was without guidance, I was diving into this whole journey by myself without any friends or close people who are in the field giving me advice on where to go next. But I was not scared. I knew I had time, and I gave myself time to study, and repeat. The inflow of many resources never stopped, but I was determined to study them - which brings me to my next point - how I wanted to run this blog.
Philosophy for the Blog
I wanted to write this blog as if someone randomly opened it and could read the post and just learn something from it. I wanted to write proper (longer) posts about the topic/book/course I did on a day. For this, I imagined I had an annoying version of myself on my shoulder asking me ‘what did you study today?’, ‘are you descriptive enough in today’s post?’, ‘don’t just post that you read something - describe it, tell me about it’, etc. Thanks to this ‘philosophy’ I managed to go over different books and courses that many times talked about similar topics. It’s no surprise there are many books on AI, and many more keep coming out. There are many good and many not as good books - you can find my personal top books in the list. But what I want to share is that - the thing that remains constant in every book, paper or course is the fundamentals which leads me to my next point.
Embracing the Fundamentals
Ohh the fundamentals - just like many others I see online (not sure about reality), in my 1st days I jumped directly into traditional machine learning and deep learning models and I could definitely make models, make notebooks that do cool predictions, but as soon as I started to ask the question ‘why’, ‘why does this happen’, ‘why does this work like that’ I felt helpless. Thankfully I am not scared of maths. As mentioned, you can run code, you can make a text generation model, but as soon as you start asking why it works, it will all fall apart. Well this led me to actually learn the math behind AI and also revisit my stats knowledge. I never avoided math or stats, I just started from a random point (like a model 😀) and after some time (epochs 😀) I learned where I needed to go and I started heading there. ‘There’ being math and stats for ML - again the books/courses for this can be found in my list. The idea of focusing on the fundamentals became a central point of my learning - try to not focus on specific technologies, rather understand the idea, the need, and the problem being solved. Tech changes, but fundamentals stay the same. I did not ignore tech of course, I did focus on the tech for practical knowledge and I learned a lot about tools for storing, transforming, loading data, model experiment tracking, orchestration, serving, monitoring, testing; and of course in projects I used specific tools, which can be seen on my blog’s Project tab. I intentionally don’t mention specific names again because tech changes, but the need behind tools stays the same. A recurring theme was studying so my knowledge is tool-agnostic.
Applying Knowledge
I realised I like to apply knowledge. When I was looking over my blog posts for what books/courses I read I saw time and time again that after I learned about some architecture/model, in the next few days I tried to make a notebook on Kaggle, or make a project that involves it - I like to apply knowledge. Further to applying knowledge, I wanted to learn to explain it simply.
I will bring in two quotes.
Einstein: ‘If you can’t explain it simply, you don’t understand it well enough.’
Aristotle (allegedly): ‘Teaching is the highest form of understanding.’
Maybe it’s something inherent to me, or something I picked up on my 1st job at Lloyds Banking Group - I like to transfer my knowledge onto others. I like to learn in such a way that I ask myself silly questions, run into different errors, edge cases - so that I can teach it to a beginner. It is a bit tricky - thinking about teaching a beginner when I am a beginner myself, but my idea is to question ideas from multiple perspectives and remember what struggles, what questions I asked when learning something - so that after I have learned it I can say it simply to another person. My idea is to help each other and learn together. I respect and value teachers immensely - that is why I look up to people who are experts in their field and also can explain difficult ideas in simple terms for beginners - this (in my opinion) is a skill, and, as Aristotle allegedly said, is the highest form of understanding. One such time when I felt that I understood something was with multicollinearity. Many books/videos say it is bad and just end there. But I started asking a lot of why questions and ended up digging in and finding that eigenvalues/eigenvectors are related and I made a notebook on Kaggle about it - The Full Story behind Multicollinearity. Maybe such a dive is not so valuable per se, but this dive helped me recall and see an actual application of eigenvalues/eigenvectors - math concepts that I had only seen in textbooks up until that point - and it showed me that things go full circle and reinforced my previous belief that if we start to question why and we don’t know the fundamentals - our knowledge ‘falls apart’. As mentioned, it is dangerous trying to teach when still learning, so I joined a lot of online student communities and just observed the type of questions being asked and made sure I can answer them, and not only I tried to answer them (confirming my own knowledge), but also ask myself why is the answer this and not that - in a way questioning myself to make sure I know the answer but also why the answer is the answer.
My background’s effect on my learning journey
I want to write a few sentences of how my background in accounting, banking and data analysis might have impacted my studies and approach to certain topics. In accounting data cannot be wrong. Incorrect data can result in inaccurate numbers in reports, suboptimal financial decision making, incorrect financial statements, deceiving stakeholders, etc. etc. Data has to be correct and that is why maybe I got interested more in data when it comes to AI. Yes there are many architectures but to make a good model we need good data (data quality is crucial). Also, I come from banking, and in both accounting and banking - regulations are king. Documentation (processes, decisions, data, code) is part of both data engineering and MLOps. Not only that but also fairness, explainability - crucial, as we need to explain in front of regulators that our data, code, models, whatever follow certain guidelines. So when I was learning about those things in relation to AI, I made the associations to my previous fields of interest/work.
Data is King
Another fundamental of AI is data. I think data as a fundamental of AI is not that popular like math and stats, but it is. ‘Garbage in, garbage out’ as the saying goes. That is why I was naturally drawn towards data and data engineering - good data will result in good models, and bad data in bad models. Besides data, I saw many times and learned the idea of anyone can make a model, focus on the business case and the business value - I think this led me to be interested in MLOps - understanding business needs, taking models to production, distributed computing (which is a recurring theme), maintaining them and creating a loop along with data. Before getting into MLOps I was just a notebook guy. Sometimes I wondered what happens after a model is developed in a notebook - well I realised there is a whole different world out there - the world of MLOps - and for the books/courses that taught me - you can find them in my list. Also a skill that’s common not just for these two but in general - communication. I learned you are not a good data engineer if you cannot talk with your stakeholders and understand their needs, their perspectives. When building a model - it’s not so much the model, it’s the business need we are trying to cover and value we are trying to bring, and to understand this value we need to communicate with stakeholders and translate those values into models and architectures to deliver those models and also the data for these models. In DE, there are also repeatable (analytical/design) patterns that happen, data structures that appear again and again - I wanted to focus on these recurring themes as they can be applied to any tool and project.
Surrounding context
Next, I want to talk about my surrounding context - I am a Master’s student, gave myself plenty of time, studied stats, math, data, AI alongside my university degree in Management Information Systems. I believe I had the privilege of time and the ability to not concentrate on specific tech specifically because of my situation - being a student and having time. While I value learning the fundamentals and the ideas behind tools, I can understand if someone who has a job is trying to learn and they don’t want to waste time going through different books/courses and they want to learn a specific tool so that they can apply to a specific job. As a student, I believe I have the privilege of concentrating on the fundamentals so that when the time comes I will translate my knowledge onto the specific tool that will be used on a job. And to be honest, I learned plenty of the popular tools, and saw the options that different tools (orchestrators) offer and the ideas are very similar, just the implementation is a bit different.
Free University Lectures
Next, I’d like to mention a bit about university lectures on math/stats/data/AI. It is amazing how many of the world’s biggest universities share their lectures online - there is so much free education out there, people just need to commit to learning it. I will specifically mention Stanford University - they have so many full lecture courses + homeworks for free online - anyone can find them, study the materials and even do the homeworks. I think the youtube algorithm was great to me - once I started watching one free lecture then on my home tab many more lectures and courses appeared. To be honest, that was the case with LinkedIn’s, X’s and reddit’s algorithm - once you are in the sphere the posts and content keeps coming in and that’s how I found many good courses and many good books recommended by known people (professors/professionals) in the industry all the time. Again, check my list for all the books/courses that I covered and my personal top.
Tools I Used extensively
Okay ~ Many might be interested in the tools I used. I did not want to write this final post without at all mentioning specific tools I learned/used extensively along the way. I tried to group similar ones but it is not perfect as many tools are used for multiple purposes. Here are the tools I feel confident using at the moment, and also feel confident having a conversation about them and talking about them to beginners.
- Scikit-learn, TensorFlow (stopped using after learning PyTorch), PyTorch, torch-geometric, HuggingFace, Unsloth, Ollama, LangChain
- dbt, Mage, Prefect, Apache Kafka, Apache Flink, Apache Iceberg, Apache Spark, chispa
- MLflow, EvidentlyAI, FastAPI, Looker, Grafana
- Docker (+ some K8s), Terraform, Google Cloud Platform: GCS, BigQuery, Compute Engine
- Neo4j, PostgreSQL, sqlite
Blog Tags and Metrics
I have tags on each blog. Tags like NLP, theory, GNN - just a base guide for what I did that day. Here is are 2 pie charts about 1)theory vs practice, and 2) learning breakdown by topic
Here is a more detailed description of the tags:
- Practice: doing exercises, or working on projects/notebooks
- Theory: consuming theoretical content (books, papers, lectures, videos)
- Traditional ML: means non deep learning modes like linear models, trees, ensembles of models, clustering models, dimensionality reduction, etc; this tag is accompanied by a theory and/or practice tag
- Conv NN: anything related to vision models - starting from basic theory, to learning about CNN architectures like LeNet, AlexNet, VGG Net, ResNet, Transformer-based; image generation with GANs, RNN-based architectures; object detection and localization; this tag is accompanied by a theory and/or practice tag
- NLP: anything related to text - starting from basic theory to learning about word2vec, GloVe, RNN-based, Attention, BERT, Transformer-based; also RAG apps; chat models; LoRA tuning, prompt engineering; this tag is accompanied by a theory and/or practice tag
- Graph NN: anything related to graph neural networks - starting from basic theory to node2vec, graph conv. networks, graph attention networks, graphormer; using torch-geometric; this tag is accompanied by a theory and/or practice tag
- Audio & Speech AI - this was mostly related to working on the Glaswegian voice assistant project
- MLOps: usually accompanied by theory and/or practice tag; starting from theoretical knowledge on the need for MLOps, a model’s life from data engineering, model engineering, evaluation, experiment tracking, orchestration, deployment, monitoring, maintenance - read material and/or did practice on the whole AI lifecycle from birth to development to production, also using cloud tools, containerisation
- Math for ML: linear algebra, matrix decomposition, calculus; this tag is accompanied by theory (covering videos/lectures/ppt slides) and/or practice - examples are solving problems, doing the math for neural network’s epoch by hand, or covering backpropagation
- Data Eng: usually accompanied by theory and/or practice tag; starting from beginner data eng theoretical knowledge (i.e. Joe Reis’ book) and the need for DE to building pipelines, learning about databases, batch, streaming, table formats, pipelines, data quality, data modelling, using cloud tools, containerisation
Projects/cool things I worked on:
- MLOps Architecture for Real-Time Fraud Detection - https://github.com/divakaivan/kb_project
- Transaction Stream Data Engineering Pipeline - https://github.com/divakaivan/transaction-stream-data-pipeline
- Your Personal Finance Voice Assistant - https://github.com/divakaivan/personal_finance_assistant
- Glaswegian Audio Dataset and ASR model - https://huggingface.co/datasets/divakaivan/glaswegian_audio
- MLOps Architecture for Insurance Fraud Detection - https://github.com/divakaivan/insurance-fraud-mlops-pipeline
- Diving into the full story behind multicollinearity - https://www.kaggle.com/code/divakaivan12/the-full-story-behind-multicollinearity
- Doing a neural network epoch with pen and paper - https://www.kaggle.com/code/divakaivan12/neural-network-epoch-by-hand
- PDF RAG from scratch - https://github.com/divakaivan/pdf-rag-from-scratch
- Using video generation models for predicting taxi demand matrices - https://github.com/divakaivan/taxi-demand-video-models-paper
- Youtube channel with 100 streams + videos + 150 subs: https://www.youtube.com/@ilearnthings123
- Became a Notebook Expert on Kaggle: https://www.kaggle.com/divakaivan12
- Full list on my blog’s Project tab: https://ivanstudyblog.github.io/projects/
I wanted to make 1 final big project (or work on improving the real-time transaction fraud detection project) that encapsulates a lot of what I learned but given that I got a free spot in Zach Wilson’s Jan bootcamp, and got as a present a spot at Maria Vechtomova’s MLOps course I decided to postpone it a little to gain even more knowledge.
Going Forward
Going forward, the studying doesn’t end. Just the blogging concludes. From Jan 6th I have Zach Wilson’s Data Eng bootcamp, and from Jan 27th I have Maria Vechtomova and Basak Eskili’s MLOps courses. For the past few months while studying for uni and this blog, I have also been applying for jobs in the UK but I haven’t gotten to any interview stage yet.. Even though I have unlimited right to stay and work in the UK, it is a bit harder as I am applying from abroad but now I can dedicate even more time to applying for jobs, doing pre-recorded interviews, tests, and hopefully I get an invite to an interview where I might be able to even talk about this blog/study journey. Now that I do not have to cover some new source or different every day I can invest more time into re-reading/re-watching content that I wanted to re-watch. I kind of did that some days but I hardly ever talked about it on the blog as I the ‘annoying/evil Ivan’ whom I had on one of my shoulders (as mentioned earlier 😀) kept telling me that writing ‘today I re-read XYZ is not good blog content and that I shouldn’t re-summarise old content’. This might be seen as a bit weird but that’s just how I wanted to run this blog - even if two books, for example, cover the same topic - I will read and summarise the two books rather than read the book and summarise it on day X, and then re-summarise it on some day X+N as I felt that would be kind of cheating 😀but also it gave me the opportunity to see what common patterns appear between books, what is considered important across the board and also be able to say for myself - ohh this book is better than that book because of abc. In addition, this method kept me accountable that I studied on a given day and also a consistent record if what I studied and I can use these posts for reference when needed because I covered a wide range of topics. What’s more, every post’s history can be seen on github and the commit dates (which is one of the main reasons I moved away from blogger(https://50daysml.blogspot.com/) to a github docs-based blog) so this can be proof that I posted every day. Anyway ~ I can now re-read, re-watch, and study some of my favourite content to prepare for interviews going forward :)
Also I think it is a good idea to stop now as the blog spans over a full year - 365 days worth of posts talking about different topics, and also when I start Zach Wilson’s and Maria Vechtemova’s DE and MLOps camps, respectively, I cannot share anything about them on public spaces so these posts wouldn’t be up to the standard I tried to follow during those 365 days.
Thank you.
And for the last time:
That is all for today, and this blog series!
P.S. I studied today and it felt weird as I kept thinking how to write/explain that thing in the blog however I should not ~ there will be some ‘transition back to normal’ period I guess 😄