Capture the Training Objective in a Metric that is Easy to Measure and Understand

10 / 46 Training This practice was ranked as basic.
Click to read more.


Intent

Ensure the machine learning objective is easy to measure and it is a good proxy for the true objective.

Motivation

Many times the true objective is hard to capture in a metric, and may lead to entangled measurements. Choosing a simple, observable metric as a proxy simplifies things, leads to better interpretability, and enhances communication within the team.

Applicability

All training objectives should be captured in an easy to comprehend metric.

Description

Choosing an objective to optimize is not trivial because (1) the objective may be hard to capture in a metric, and (2) the objective evolves over time.

In both cases, over-engineering a metric may lead to entangled measurements, which are hard to comprehend or assess.

Simple metrics, that are easy to measure and comprehend are considered better proxies for the true objective of a machine learning application. Working together with business or data analysts to ensure the metrics reflect business values helps to align the measurements with the true objective.

A great example can be found in the 13th rule for machine learning by Martin Zinchevich.

Adoption

Related

Read more



10 / 46 Training This practice was ranked as basic.
Click to read more.