Effects of the practices

Using the survey responses, we analysed the relationship between sets of practices and four desired effects. We found out that the effect can be predicted with high accuracy from the following sets of practices (read more about the method in our publication).

Agility

• Enable Parallel Training Experiments
• Use Continuous Integration
• Automate Model Deployment
• Enable Shadow Deployment
• Continuously Monitor the Behaviour of Deployed Models
• Enable Automatic Roll Backs for Production Models
• Communicate, Align, and Collaborate With Others

Software quality

• Assign an Owner to Each Feature and Document its Rationale
• Actively Remove or Archive Features That are Not Used
• Peer Review Training Scripts
• Run Automated Regression Tests
• Use Continuous Integration
• Use Static Analysis to Check Code Quality

Team Effectiveness

• Share a Clearly Defined Training Objective within the Team
• Use A Collaborative Development Platform
• Work Against a Shared Backlog
• Communicate, Align, and Collaborate With Others

Traceability

• Write Reusable Scripts for Data Cleaning and Merging
• Make Data Sets Available on Shared Infrastructure (private or public)
• Use Versioning for Data, Model, Configurations and Training Scripts
• Continuously Monitor the Behaviour of Deployed Models
• Log Production Predictions with the Model's Version and Input Data
• Work Against a Shared Backlog