Nvidia’s Canvas AI Painting Tool Turns Color Blobs into Realistic Imaginaries

Nvidia’s New Canvas tool, which is now available as a beta-free version, is a real-time painting tool GauGAN that can be used by anyone with an NVIDIA RTX GPU.

It allows creators to create a rough landscape by painting blobs and then it fills it with convincingly photorealistic content.

Each distinct color on the tool represents a different type of feature: water, mountains, grass, ruins, and others.

The crude sketch is passed to a generative adversarial network when colors are lobbed onto the canvas. GANs essentially pass content to AI creator who tries to make a realistic image. After that, the detector AI evaluates how realistic that image is. Together, they make a fairly realistic depiction of what was suggested.

It is a prototype of GauGAN which was shown at CVPR in 2019. It is much more user-friendly, smoother around the edges, can run on any Windows computer, and produces better imagery with a decent Nvidia graphics card.

However, unlike Google’s Chimera Creator, this app is run locally and requires Nvidia video.

The tool aims to create realistic faces, animals, and landscapes. However, there is usually some kind of “tell” that can be spotted in the picture. But, it’s important to note that the Canvas app isn’t trying to make something indistinguishable from reality.

Jama Jurabaev, a concept artist, explains that it’s more about being able to experiment freely with imagery that is more detailed than a doodle.

For instance, while moldering ruins in a field, a quick pencil sketch can only tell you so much. During this time, you have no idea how the final piece might look. You might have one way in your head, but then after two hours of the painting, you realize that the sun is setting and it’s making shadows awkward in the foreground.

If you scribble these features into Canvas, this will become clear to you right away, and then you can move on to the next idea. With Canvas, you get the option that lets you quickly change the time of day, palette, and other high-level parameters. This makes it fairly simple to quickly evaluate options.

“I’m not afraid of blank canvas anymore,” said Jurabaev. “I’m not afraid to make very big changes, because I know there’s always AI helping me out with details, I can put all my effort into the creative side of things, and I’ll let Canvas handle the rest.”