Hello :) Today is Day 325!
A quick summary of today:
- started watching the paid content on dataexpert.io (but cannot share as it’s against the rules)
- another failed prompt tuning attempt
Dimension modelling lectures + labs on the dataexpert.io platform
I started watching the pre-recorded videos offered on the platform to start preparing for the Jan bootcamp.
As this is all behind paywall, nothing is available publicaly I can just share the topic and that I took notes. I think going forward I should do something alongside covering this content so that these posts are not just a few sentences long :/
The platform is great ~ even for the exercises there is a built-in sql editor.
I am yet to read the infamous Designing Data-Intensive Applications book so I think reading it before Jan would be extra helpful, and I can also share summaries of what I read while also covering the content on dataexpert.io.
24 hour prompt tuning
The 24hr company reviewer prompt tuning model … failed :/ I checked it at around 6pm when it finished and it started at loss ~ 13, and flat-lined around 5-6 … And this is a recurring thing I see with other failing training sessions - the end result is always around loss 5-6. So I am just starting to really wonder what to do. We have time, and having a successful prompt tuned model is not essential as the LoRA fine-tuned gpt4o-mini and llama3.2 are satisfactory, but for the sake of wider exploration I think it’s worth doing some more exploration. I adjusted the learning rate and some other hyperparams and put another version to train which will take ~26 hours. (note that the GPU we have is a 48GB A6000)
I guess ~ that is all for today!
See you tomorrow :)