Decide Trade-Offs through Defined Team Process

March, 2023 • Alex Serban, Koen van der Blom, Joost Visser

38 / 46 • Team •

This practice was ranked as medium.
Click to read more. •

This practice addresses requirements
from the EU guidelines for trustworthy ML.
Click to read more.

Intent

Define methodology for generation, evaluation, prioritisation and selection of solution alternatives.

Motivation

Software development is characterised by multiple objectives and constraints, uncertainty and incomplete information. Moreover, the problem parameters change very often.

Applicability

Trade-off processes should be applied to any machine learning application.

Description

Software development can be seen as a multi-objective optimisation problem, where trade-offs between quality attributes of a system have to be made and recorded. In most cases, decisions have to be made under uncertainty and incomplete information. In order to enhance the ability of a team to make the right decisions at the right time, it is imperative to define and enforce a standard process across team members.

Machine learning adds more uncertainty to a system, and increases the importance of this practice. Besides software related trade-offs – such as selecting the right architectural style or prioritising the maintenance cycle for certain components – new trade-offs regarding machine learning quality attributes must be considered.

For example, a team process can include the prioritisation of interpretable models over black-box models, or a procedure to roll-back production systems when the accuracy drops significantly. A team process can empower team members to take action even though other members responsible for decisions are not present. Large companies, with vast experience in software development, adopt company wide policies that are followed by all teams (e.g. Amazon’s 14 leadership principles). Defining a process that can be adopted by multiple teams in the organisation enhances collaboration and communication between teams.

Adoption

Employ Interpretable Models When Possible