Another challenge with data is the fact that the data might need to be a better fit for your domain. We could describe them as data gaps, as they reflect the imperfection of real-world data.

The gaps can be observed as follows:

• Domain Gaps: training data does not match the real world where the model is used.
• Data Bias: the collected data might have a bias impacting individuals and society.
• Data Noise: data with a low signal-to-noise ratio. A frequent problem is mislabeled data.

Data Augmentation can help mitigate those imperfections with a data-centric approach in mind. It is a technique where the training data is artificially augmented by generating a modified version of existing data.

The topic of Data Augmentation deserves a more in-depth analysis, and we will leave that for a future blog. However, the central concept is to add additional training data to overcome the data gaps.

One approach is self-supervision, to use existing data and make minor changes. When dealing with images, one could, for example, apply cropping, resizing, or changing the contrast to generate additional
training data.

The second approach is to generate synthetic data. Synthetic data is generated artificially without using the original dataset. Creating synthetic data is a new concept but a significant advancement. The idea has already been introduced in AI fields such as Healthcare, self-driving cars, Natural Language Processing (NLP), and Automatic Speech Recognition.

The techniques behind synthetic data generation often use DNNs (Deep Neural Networks) and GANs (Generative Adversarial Networks).

When discussing benchmarking an AI solution, one should consider measuring the model’s accuracy, speed, and efficiency. The benchmark gives us a mechanism for consistent measurements and a process to validate improvements to our model. Such a benchmark is MLPerf^TM from ML Commons.

Data is considered the new bottleneck and limiting future advancements. To acknowledge this problem, it is essential to put data at the enter and create meaningful metrics. It is impossible to improve datasets without metrics and to measure against the metrics. The metrics help measure the vital qualities of data and indicate where improvements are needed.

The people at ML Commons have started the DataPerf initiative to create appropriate training and testing data to deliver new machine learning (ML) solutions. In other words, benchmarking datasets to make better datasets.

The AI industry is moving towards a Data-centric AI approach, and tools to build, maintain and evaluate datasets easier, cheaper, and more repeatable will be front and center.

DataPerf is backed by a strong community and always looking for people to help. Please reach out to them if you are considering contributing.

It is essential to understand not just the algorithms but what data, from where, is doing to give you the most accurate results. Assess the data regularly as you test your algorithms.  And know that understanding where the data lives and its value is vital to a cost-effective AI strategy. More data is better than less data. But better data beats more data!