Log Production Predictions with the Model's Version and Input Data

March, 2023 • Alex Serban, Koen van der Blom, Joost Visser

33 / 46 • Deployment •

This practice was ranked as medium.
Click to read more. •

This practice helps to increase
the traceability of ML components.
Click to read more.

Intent

Enhance debugging, enable traceability, reproducibility, compliance and incident management.

Motivation

Tracing decisions back to the input data and the model's version can be difficult. It is therefore recommended to log production predictions together with the model's version and input data.

Applicability

Prediction logging should be implemented in any production-level ML application.

Description

Debugging production models is difficult if one does not have access to the input data. Moreover, tracing decisions and mitigating incidents without access to the input data is almost inconceivable.

In order to mitigate these issues, but also enhance traceability, it is recommended to log production prediction together with the model’s version and input data.

If model and data versioning is done properly, the model’s version will lead to the training data repository and enable complete reproducibility.

Adoption

Use Versioning for Data, Model, Configurations and Training Scripts