(Day 76) Finishing the Retrieval-based LM talk, and learning about distillation, quantization and pruning

March 17, 2024

Hello :) Today is Day 76!

A quick summary of today:

Finished section 6 and 7 about multilingual retrieval-based LMs and retrieval-based LMs’ challenges and opportunities (ACL 2023)
Covered lecture 11 of CMU 11-711 Advanced NLP - Distillation, Quantization, and Pruning

(Day 75) Retrieval-based LMs training and applications

March 16, 2024

Hello :) Today is Day 75!

A quick summary of today:

covered Section 4: retrieval-based LMs training of ACL 2023
covered Section 5: Applications
Found out about NVIDIA GTC 2024 which is next week and registered for some of the events

(Day 74) Retrieval-based LMs

March 15, 2024

Hello :) Today is Day 74!

A quick summary of today:

Started taking notes and studying the ACL 2023 talk on retrieval-based LMs

(Day 73) MBR and FUDGE - decoding mechanisms; pre vs post layer normalization

March 14, 2024

Hello :) Today is Day 73!

A quick summary of today:

Covered lecture 6: Generation algorithms and 9: Experimental design and human annotation from the CMU 11-711 Advanced NLP course, from which I found out about:
- Minimum Bayes Risk (MBR)
- FUDGE decoding
- Why pre layer normalization is better than post layer normalization in transformers

(Day 72) Carnegie Mellon University - Advanced NLP Spring 2024 - assignment 1

March 13, 2024

Hello :) Today is Day 72!

A quick summary of today:

Today I found 11-711 Advanced NLP by CMU

(Day 71) Backprop, GELU, Tricking ChatGPT, and Stealing part of an LLM

March 12, 2024

Hello :) Today is Day 71!

A quick summary of today:

Finished up the manual backprop code to make it more clear
Read some research papers
Gaussian Error Linear Units (GELUs)
Using Hallucinations to Bypass RLHF Filters
Stealing Part of a Production Language Model

« Prev 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 Next »