Neural Nets Give Low-End Phone Pics DSLR Look

Researchers have found a way to use neural networks to create DSLR-quality photos from snapshots taken with low-end smartphones.

A team of scientists at the ETH Zurich Computer Vision Lab recently published a paper describing a deep learning approach that uses neural networks to translate photos taken by cameras with limited capabilities into DSLR-quality photos automatically.

"We tackle this problem by introducing a weakly supervised photo enhancer -- a novel image-to-image GAN-based architecture," wrote Andrey Ignatov, Nikolay Kobyshev, Kenneth Vanhoey, Radu Timofte and Luc Van Gool.

GAN, or generational adversarial networks, is a new type of algorithm that can produce artificial images and even video, explained Aditya Kaul, a research director at Tractica.

"The technology is moving fast because the source code is open and people can experiment with the algorithm and improve it," he told TechNewsWorld.

Network Foolery

GAN technology uses a bit of trickery to get its results.

"It uses two neural networks that try to fool each other," Kaul said. "One network generates images and tries to fool the other network that they're real. At the end, you get a set of images where you can't tell the difference between the real image and the artificially generated image."

In their experiments, the researchers used two data sets -- one made up of images from a phone camera, the other composed of high-quality photos. Then they used several neural networks to bring the quality of the phone photos up to that of the high-quality images captured with a DSLR.

The researchers posted a number of examples at the CVL website. The side-by-side shot comparisons show how the scientists' method improves the phone photos.

What the researchers are doing is in the same vein as what photographers have been doing for years, said Rob Enderle, principal analyst at the EnderleGroup.

"It's not taking a better picture," he told TechNewsWorld. "It's rendering the picture so it's better than it would otherwise be without using artificial intelligence. That's what we've been doing in darkrooms and with Photoshop for some time."

Not for the Fussy

As impressive as the researchers' results are, pro and semi-pro photographers won't be shelving their DSLRs any time soon.

"This will please people who aren't overly fussy but want to get better pictures," suggested David D. Busch, creative director of the David Busch photography guides.

"It can simulate a lot of things so the pictures do look better," he told TechNewsWorld. "

However, many serious photographers would disapprove of the changes made to the photos on the researchers' samples page, Busch noted, "even though they did look better than the originals."

The changes added a lot of contrast and lost detail, he said. "If someone is a poor photographer, this has some applications -- but I don't see it threatening digital SLRcameras."

Improving the Baseline

"This is a good example of how technology can be used to improve the baseline of what's acceptable," said Jeff Orr, a senior practice director for mobile devices at ABI Research.

The researchers tried to determine what makes an image good and apply those things to an unknown image to make it better, he told TechNewsWorld. "It doesn't always work, but it's a good example of how software can make suggestions to enhance a photo, and it can only get better."

This research is trying to bridge the gap between what a smartphone can capture and the aesthetics of a DSLR, said Ross Rubin, principal analyst at ReticleResearch.

However, that doesn't mean that an algorithm can replace the quality of a DSLR.

"The algorithm can't do anything with data that isn't there," Rubin told TechNewsWorld, "so it's helpful to have both the highest quality input with the algorithm."

Future Threat?

As the processing power in phones increases, they could pose a threat to DSLRs.

"You can already compensate with software for some things that a lens can do," Rubin said.

DSLR camera makers should be concerned about developments like those from Zurich, in his view.

"They've seen the point-and-shoot market evaporate as smartphone image quality improved," Rubin pointed out. "Now, as the photo quality is augmented by the incredible processing power in these phones, which far exceed what we see in DSLRs, phones pose even more of a threat to camera makers."

The CVL researchers' technology has applications beyond making smartphone pictures prettier, Tractia's Kaul noted. It can be used to identify cancer cells in photos, enhance starlight and augment automated driving applications, for example.

There's also a dark side to the technology, Kaul said. "There is a fear that this technology can be misused for fake news and things like that."