Today’s advanced artificial-intelligence development is still influenced by the computer science philosophies of the 1950s, including the idea that an AI model is only as good as the data it’s trained on (“garbage in, garbage out”). Data scientists must train AI models with large and diverse datasets to perform tasks from suggesting a new movie to advanced cancer screenings — and everything in between. Yet authentic data, which is often protected for privacy reasons, can be difficult and expensive to source and, to boot, may lack the desired diversity.

Fortunately, synthetic datasets can aid in training AI models. These computer-generated simulations create completely anonymous training data through various methods, such as general adversarial networks or simulators using more non-AI procedures, and are made to closely resemble authentic data. By using synthetic datasets, AI developers benefit from higher-performing and more robust models.

A dupe for data

As developers reach the limits of readily available data, they will soon need to look elsewhere to improve their models.

Synthetic data is information that computer simulations or algorithms generate as an alternative to real-world data to fill the gap between model needs and data availability.

Data scientists have many ways to generate synthetic data. Simulations and 3D renderings are excellent starting points. For example, a self-driving car is often trained by having it drive thousands of miles of virtual roads before it ever rolls on a real one. General adversarial networks, generative models that create new data, can also be used for data production. Thanks to these, synthetic data collecting has become more accessible and efficient than ever.

Analyst firm Gartner recently reported that synthetic data is on a trajectory to go from a sideshow to becoming the main force behind the future of AI.1 In a study, Gartner notes, “Synthetic data democratizes the playing field by allowing smaller organizations to create AI models without a lot of data, effectively solving their cold-start problem.”

Artificial data addresses AI’s critical need

AI is already ubiquitous, as it has been integrated into our lives across health care, retail, entertainment, AVs, smart spaces, and more with smart devices and technologies that are accelerating us into the future.

Using AI as a digital mirror is the next step in its evolution. Yet variations in a particular environment can be innumerable. A shirt’s color may have many shades and hues. A room’s lighting changes with the sun’s movement or the turning on of lamps and lights.

Capturing the complexity of conditions makes diverse synthetic datasets essential for AI model-making. Synthetic data can be collected to power digital twins with far less time and expense than are required to gather data from primary sources. This maximizes access to large amounts of diverse data and adds the benefit of being free from privacy concerns.

Noting the importance of this AI asset, Gartner also notes, “Synthetic data is often seen as a lower-quality substitute, useful only when real data is inconvenient to get, expensive, or constrained by regulation. This misses the true potential of synthetic data. The fact is, you won’t be able to build high-quality, high-value AI models without synthetic data.”

Reality is random

Diverse training datasets are key for building AI models, but real-world data can fall short. The built-in feature for domain randomization enables Nvidia Isaac Sim, a robotics simulation application and synthetic data generation tool, to randomly vary the texture, colors, lighting, and placement in simulations.

The same is true for Nvidia Drive Sim, a simulation platform for testing AVs. It can change the size or language of a street sign or the position of the sun.

These capabilities are emphasized in the O’Reilly Media report “Accelerating AI with Synthetic Data”, which emphasizes that safety and efficiency are priorities in simulations. According to the report, “Some problems that can be tackled by using synthetic data would be too costly or dangerous (e.g., in the case of training models controlling autonomous vehicles) to solve using more traditional methods, or simply cannot be done otherwise.”

Randomizing conditions, like lighting, colors, and object placement, is essential for creating diverse synthetic training data for more accurate AI models. The variations in these digital worlds mirror the variations that appear in real life, where the unexpected and unpredictable occur regularly.

In factories, for example, an object handled by one worker may end up in a different position when a different worker handles the same object. The variations in environmental conditions, such as positioning, are significant when training robots how to work in a real factory using synthetic data and simulations. These abilities have enabled the production of robust smart factories and cities.

The critical link between graphics and AI

Beyond virtual cities and factories, synthetic data has paved the way for a renaissance within computer graphics, as simulating worlds in 3D is now a key component for training AI models. In a 3D world, objects should fall, body parts should bend, and skin should be textured to closely resemble all the moving parts of humans.

The different ways in which an individual can appear in a virtual world, with natural bodily variations, facial features, and behaviors, illustrates the true power of synthetic data. Diverse synthetic data can bridge the gap between virtual and real worlds, with precision in features varying from gravitational laws to bodily actions to skin texture.

Humans differ from each other with varying skin colors, reactions, and expressions that can be displayed in media productions and digital replicas. Digital humans are only one part of the puzzle, as environmental conditions like lighting and object positioning are just as important in computer graphics and simulations.

For example, a self-driving car needs to be able to respond when the sun is low in the sky, potentially hindering visibility. Synthetic data can help to improve simulated worlds by creating more realistic virtual environments that are true digital twins of reality. Generating physically accurate, physically based environments and humans is extremely challenging and requires advanced simulation, performant computing resources, and large amounts of data.

AI advancing its own future

The ability for AI to improve itself using synthetic data makes it a uniquely powerful technology. Synthesizing data is the key to enhanced quality and quantity of robust training data for advanced models and simulations.

Each wave of AI innovation builds upon the last. The opportunity for synthetic data will extend beyond its use in current AI applications to industries across agriculture, AVs, health care, robotics, and more.

When developing data sources for AI, don’t let the words “artificial” and “synthetic” deter you. The data may be artificially created, but the results are essential to real success. Soon, an incredibly accurate digital mirror of reality will exist, built efficiently and accurately using synthetic data.