Debugging production models is difficult if one does not have access to the input data. Moreover, tracing decisions and mitigating incidents without access to the input data is almost inconceivable.
In order to mitigate these issues, but also enhance traceability, it is recommended to log production prediction together with the model’s version and input data.
If model and data versioning is done properly, the model’s version will lead to the training data repository and enable complete reproducibility.
- ModelOps: Cloud-based lifecycle management for reliable and trusted AI
- Model Governance Reducing the Anarchy of Production
- Managing Machine Learning Projects