The importance of data quantity in machine learning: how small is too small?

ABSTRACT: The past decade has seen rock engineering become more data-driven, resulting in increased use of machine learning (ML). ML is a type of artificial intelligence that involves the development of mathematical models, resulting in a computer system capable of making predictions with minimal human involvement. Such a powerful tool can help rock engineers efficiently uncover complex relationships between data and has been used to predict rock mass properties, mining and tunnelling hazards, and slope stability. However, the success and reliability of ML models are directly linked to the quality and quantity of data available. ML models for rock engineering applications are generally trained using either poor-quality or limited data. This inherently leads to poor and unreliable results with potential real-life adverse impacts. Both data quality and quantity pose significant challenges in rock engineering due to the subjective nature of many commonly used rock engineering parameters (resulting in poor-quality data) and the limited data available in the early stages of the design process. While there has been an increased awareness of the importance of data quality for ML among rock engineers, there has yet to be a comparably increased awareness of the importance of data quantity. Many rock engineering research articles focused on ML are training their ML models on only a few hundred data points, with some as few as 80 data points. However, these results can be misleading due to the stochastic nature of ML models and how the data is shuffled before data splitting, resulting in unreliable models. Using synthetic data and surrogate models, this paper aims to demonstrate the importance of data quantity in ML, and it recommends using caution when using small datasets for ML.