Blur-Aware Image Downsizing

One pervasive trend in imaging hardware is the ever increasing pixel count of image sensors. Today even inexpensive cameras far outperform common display technologies in terms of image resolution. For example, very few cameras remaining on the market, including mobile devices, capture an image at low enough resolution to show on a 1080p HDTV display without resizing. In an extreme case, the preview screen on one Nikon professional digital SLR can only display 1.5% the pixels captured by the sensor. The cellphone camera owner and the 4K cinematographer face the same problem of getting an accurate depiction of the image when they can't see all the pixels.

While high resolution images are needed for a number of applications such as on-camera previewing, print output or cropping, the image is often previewed on a display of lower resolution. As a result, image downsampling has become a regular operation when viewing images. Conventional image downsampling methods do not accurately represent the appearance of the original image, and lowering the resolution of an image alters the perceived appearance. In particular, downsampling can cause blurred regions to look sharp and the resulting image often appears higher quality than its full-size counterpart. While the higher quality images can be desirable for purposes such as web publishing, the change is problematic in cases where the downsampled version is to be used to make decisions about the quality of the full-scale image, for example in digital view finders.

In this paper, we aim to develop an image downsampling operator that preserves the appearance of blurriness in the lower resolution image. This is a potentially complex task - the human visual system's ability to differentiate blurs is dependent on spatial frequency,and edges blurred by different amounts may be perceived as different at one scale but equal at another. Additionally, there is potential for content-dependent blur perception where the same amount of blur is perceived differently, depending on the type(s) of object(s) shown.

We approach this problem by conducting a perceptual study to understand the relationship between the amount of blur present in an image, and the perception of blur at different image sizes. Our study determines how much blur must be present in a downsampled image to have the same appearance as the original. We find a complex and mostly image-independent relationship between matching blur levels in images at different resolutions. The relationship can be explained by a linear model when the blur magnitude is analyzed in terms of spatial frequency.

Using the results of this study, we develop a new image resizing operator that amplifies the blur present in the image while downsampling to ensure it is perceived the same as the original. While our algorithm is compatible with any combination of methods for producing a spatially-variant estimate of image blur and spatially-variant image filtering, we base our implementation around a modified version of the algorithm by Samadani et al. The result is a fully-automatic method for downsampling images while preserving their appearance, the performance of which we verify with another user study.