Hello :) Today is Day 323!
A quick summary of today:
- new lecture on Additive dimensions and Graph Modelling
- prompt tuning is taking soo long
- trying to make a good model for predicting subway passengers
Additive dimensions and Graph Modelling
Today Lecture 3 from Zach Wilson’s camp was released, and here are my notes.
Prompt tuning
The LLM I left training last night … well it did not succeed. I wonder if the training epochs are too little - reason being the loss goes down consistently but is nowhere close to what I have seen with other models. But I am running 10 train epochs and its taking 9 hours… Am I going to train a model for 24 hours if I increase the epochs ? 😆 I left a model training tonight as well and we’ll see in the morning but again I suspect the train epochs won’t be enough. Of course the learning rate could be adjusted, and many other params but given my limited GPU resources… I have to make some educated guesses based on what I’m seeing.
Predicting subway demand
Today I saw that my university shared some small competition about predicting the passenger for every hour of a set of stations. I cannot join as the reward is a scholarship, and I am not eligible for a scholarship award (given I am already receiving a 100% one 😆). But I still wanted to check out the dataset and see how good of a model I can make. I spent most of the time preparing the data, hyperparam tuning, and cross validation with different split methods. Here is a link to a colab notebook. I uploaded it so I can share it. However, at the moment I think the mean absolute error (the metric they say they will evaluate the results by) is a bit high ~ around 100, but there is some inconsistency because of the splitter I was using. I am writing this a bit early today (early meaning midnight haha) and I will continue running some code after I post this so hopefully I get something good tomorrow.
Talking about time-series, given that subway passenger prediction is a great time-series prediction problem - this week’s probabl stream will be about time-series predictions and Vincent had asked if we had any questions/suggestions on what to show on stream so I asked to see how he would approach the problem using the library he will show. Then he requested if possible to find and put a dataset he could use (and try to make some kind of a showcase out of it) - so here it is. I found it from Seoul’s official subway webpage and just translated some of the Korean to English and uploaded it as parquet as the csv sizes were >25mb.
That is all for today!
See you tomorrow :)