(Day 72) Carnegie Mellon University - Advanced NLP Spring 2024 - assignment 1
March 13, 2024
(Day 71) Backprop, GELU, Tricking ChatGPT, and Stealing part of an LLM
March 12, 2024
Hello :) Today is Day 71!
A quick summary of today:
- Finished up the manual backprop code to make it more clear
- Read some research papers
- Gaussian Error Linear Units (GELUs)
- Using Hallucinations to Bypass RLHF Filters
- Stealing Part of a Production Language Model
(Day 70) Testing my backprop knowledge
March 11, 2024
Hello :) Today is Day 70!
(Day 69) Training an LLM to generate Harry Potter text
March 10, 2024
Hello :) Today is Day 69!
A quick summary of today:
- tried to build upon the built LLM from the book from the days before, and write a training loop in the hopes of generating some Harry Potter text (kaggle notebook)
(Day 68) Build a LLM from scratch chapter 4 - making the GPT-2 architecture
March 9, 2024
Hello :) Today is Day 68!
A quick summary of today:
- Covered chapter 4 of Build a LLM from scratch by Sebastian Raschka
(Day 67) Build a LLM from scratch chapter 3 - self-attention from scratch
March 8, 2024
Hello :) Today is Day 67!
A quick summary of today:
- Covered chapter 3 of Build a LLM from scratch by Sebastian Raschka