Most image recognition systems are trained on huge databases that contain millions of photos of everyday objects. They are tagged by people, be it shots of snakes, milkshakes, cars or shoes. In this way, an AI system gradually learns to differentiate between objects.
Japanese researchers have now been able to show that such systems can be trained just as well with computer-generated fractals to recognize everyday objects. The idea may sound strange at first, but it could bear great fruit. The automated production of training data is an interesting trend within machine learning. Working with an almost inexhaustible amount of synthetic images and no longer with photos that come from the Internet could eliminate many problems with existing, hand-tagged datasets.
Hanging-Tagged Datasets: Elaborate and Biased
In the pre-training phase, an AI system learns basic skills before it is trained with specific data. Existing models are often used so that this phase does not have to be repeated completely. Instead, details are adjusted, for example to recognize images in a medical diagnostic environment.
The pre-training is done with a database that contains everyday objects – such as ImageNet with more than 14 million photos. A smaller database with medical images is then used for the fine-tuning – the training then continues until the model – often even invisible to humans – recognizes inconspicuous signs of an illness.
The problem is that manually assembling a dataset like ImageNet takes a tremendous amount of time and effort. These images are usually assigned by poorly paid crowdworkers. The data sets could also contain sexist or racist tags, so that the model is barely noticeably influenced by prejudices. And it is quite possible that pictures are used by people who have not given their consent. It has been proven that biases often arise during pre-training.
“FractalDB”: What can fractals do?
Fractals are natural shapes. They occur everywhere: in trees and flowers, clouds and waves. This piqued the team at Japan’s National Institute of Advanced Industrial Science and Technology (AIST), Tokyo Institute of Technology, and Tokyo Denki University: Could these patterns be used to teach automated systems the principles of image recognition? In this way you could do without photos and real objects. This is how FractalDB was born, a database with a gigantic amount of computer-generated fractals.
Some resemble leaves, others resemble snowflakes or the shell of a snail. Each group with similar patterns was automatically assigned a name. Researchers used FractalDB as a base to pre-train a Convolutional Neural Network, a deep learning model commonly used in image recognition systems, before completing the learning unit with a set of actual images.
As a result, the model trained in this way performed just as well as the others that were trained with recognized data sets such as ImageNet or Places and their 2.5 million outdoor photos.
Abstract patterns can be a problem
Anh Nguyen from Auburn University in Alabama, who was not involved in the study, is not yet convinced that FractalDB processes like ImageNet can hold a candle. In his research, he is concerned with how abstract patterns can mislead image recognition systems. “There is a connection between this work and examples showing how machines can be fooled,” he says, and would like to begin by examining in detail how this new approach works.
The Japanese researchers, on the other hand, believe that computer-generated data sets such as FractalDB could, after some optimization, replace existing databases without confusing systems.
In the study, the AI system was also trained with other abstract images, including those made with Perlin Noise that created special fantasy patterns, and Bézier curves, parametrically modeled curves as used in computer graphics. But fractals produced the best results. “Fractal geometry is something that exists in the background of the world,” says lead author Hirokatsu Kataoka at AIST.