Photography Bay

Digital Camera News, Reviews and Tips

  • Reviews
  • Recommended Cameras & Gear
  • Learn Photography
  • Contact
  • About
  • Subscribe

Google Research’s Handheld Multi-frame Super-resolution Explained

May 27, 2019 By Eric Reagan

If you’ve been wondering what makes the images in the Google Pixel 3 look so great, the below video and the accompanying research paper help explain some of the incredible computational photography research that Google is working on.

The paper abstract is below. You can read the full research paper here.

Compared to DSLR cameras, smartphone cameras have smaller sensors, which limits their spatial resolution; smaller apertures, which limits their light gathering ability; and smaller pixels, which reduces their signal-to-noise ratio. The use of color filter arrays (CFAs) requires demosaicing, which further degrades resolution. In this paper, we supplant the use of traditional demosaicing in single-frame and burst photography pipelines with a multi-frame super-resolution algorithm that creates a complete RGB image directly from a burst of CFA raw images. We harness natural hand tremor, typical in handheld photography, to acquire a burst of raw frames with small offsets. These frames are then aligned and merged to form a single image with red, green, and blue values at every pixel site. This approach, which includes no explicit demosaicing step, serves to both increase image resolution and boost signal to noise ratio. Our algorithm is robust to challenging scene conditions: local motion, occlusion, or scene changes. It runs at 100 milliseconds per 12-megapixel RAW input burst frame on mass-produced mobile phones. Specifically, the algorithm is the basis of the Super-Res Zoom feature, as well as the default merge method in Night Sight mode (whether zooming or not) on Google’s flagship phone.

Filed Under: Technology Tagged With: computational photography, google, Google Pixel 3

 

Support Photography Bay


Recent Posts

  • Reader Photos Roundup: March 19, 2023
  • Computational Photography & AI: Where Do We Go From Here?
  • Reader Photos Roundup: January 15, 2023
  • Panasonic Lumix S5 II and S5 IIX Cameras Announced with 6K Video and Improved AF
  • Reader Photos Roundup: January 8, 2023

Pages

  • 2022 Black Friday Camera and Photo Deals
  • About
  • Camera and Photo Deals Newsletter
  • Canon 1D X
  • Canon 1Ds Mark IV
  • Canon 5D Mark III
  • Canon Rebel T4i
  • Contact
  • Disclosures
  • DSLRs
  • How to Subscribe
  • Learn Photography
  • NAB 2016 Live Blog
  • Nikon D4
  • Nikon D800
  • Photography Laws
  • Recommended Cameras & Accessories
  • Reviews
  • Sony Alpha A77

Copyright © 2023 | Photography Bay