(Day 167) Learning about model monitoring

Ivan Ivanov · June 16, 2024

Hello :) Today is Day 167!

A quick summary of today:

  • started Module 5: Monitoring of the MLOps zoomcamp
  • read a bit of Multivariate statistical methods: a primer (which I bought yesterday)

Firstly, about ML model monitoring

image

ML monitoring involves looking at things like service health, model performance, data quality and integrity, data and concept drift, performance by segment, model bias/fairness, outliers, explainability.

image

Here are the materials over which I studied today. Using evidently’s python library we can great nice looking reports and dashboards like the below.

In this, 2 created reports can be seen.

image

Below is one of the default suggested dataset summary tables

image

Along with variable info

image image image

And this is an example of a simple dashboard we can use to monitor specific items from our reports

image

Secondly, some things from the book

  • examples of multivariate data
  • matrix algebra (vectors, matrices, eigenvalues/vectors)
  • visualising multivariate data (draftsman’s plot - scatter plot matrix, lines that show profile of variables)
  • tests of significance with multivariate data (Hotelling’s T-squared test, Box’s M-test, Levene’s test, Van Valen’s test, Wilk’s lambda statistic, there were many more that appeared throughout the 20th century)

That is all for today!

See you tomorrow :)