Thanks for this summary, Hennie, it's very useful.

I was wondering if you should have a mention of Pareto Optimisation as another heuristic to consider.

I appreciate that it requires additional cognitive efforts for a development environment. But considering that packages like `optuna` have multi-objective capabilities baked in, readers of this article may benefit from knowing about this option.

I delivered in PyData a talk on the topic

http://bit.ly/improved-decisions-pareto

and am working on a Medium draft on this matter.

Since it's simpler to implement, I realise that most applications will use single objective optimisation, which your article gives great detail, but for the sake of those who need to optimise for multiple trade-off objectives Pareto Optimisation is a powerful tool.