Enable Shadow Deployment

29 / 45 Deployment This practice was ranked as medium. Click to read more.


Intent

Test a model's behaviour on production data, without any impact on the service it provides.

Motivation

Before pushing a model into production, it is wise to test its quality and performance on data from production. In order to facilitate this task, one can deploy multiple models to 'shadow' each other.

Applicability

Shadow deployment should be implemented in any production-level ML application.

Description

Instead of deploying a model straight into production, one can assess its quality and performance using the data from production without allowing the model to make final decisions. This involves deploying the a model to “shadow” or “compete” with the model in production, and redirect the data to both models. The model that is already deployed will still handle all decisions, until the shadow model is assessed and promoted to production.

Using shadow models allows teams to avoid unintended behaviours in production – coming from skews between training and test data. However, it introduces more complexity in the deployment infrastructure. Luckily, tool support for shadow or canary deployment has already matured.

Adoption

Related

Read more



29 / 45 Deployment This practice was ranked as medium. Click to read more.