Reconstructing dynamic refractive shapes from a high-speed video. A light field probe located behind a transparent, refractive object codes the position and incident angle of refracted light rays on the background in observable changes in color and intensity (upper left). This information is used to reconstruct the surface gradients of thin refractive shapes, such as fluid surfaces, from a single photograph (lower left). The surface gradients or normals can be integrated to estimate refractive shapes (upper right), which can subsequently be used for computer graphics applications (lower right).


Abstract

Acquiring transparent, refractive objects is challenging as these kinds of objects can only be observed by analyzing the distortion of reference background patterns. We present a new, single image approach to reconstructing thin transparent surfaces, such as thin solids or surfaces of fluids. Our method is based on observing the distortion of light field background illumination. Light field probes have the potential to encode up to four dimensions in varying colors and intensities: spatial and angular variation on the probe surface; commonly employed reference patterns are only two-dimensional by coding either position or angle on the probe. We show that the additional information can be used to reconstruct refractive surface normals and a sparse set of control points from a single photograph.



Results

Schematic showing how both position and incident angle of a refracted ray are color coded by a light field probe (top). Our probe prototypes consist of a light box, transparencies, and a lenslet array (bottom); these are positioned behind a refractive object when photographed.


Synthetic results for a refractive sinusoidal object. Normals and positions are shown for the original object (left column), and for reconstructions (other columns) from simulated camera images with an increasing amount of noise (top row).


Three thin refractive objects under room illumination (left column) and in front of a light field probe (center left column). The distorted colors of the probe allow us to estimate refractive surface normals from a single image (center row), which can be integrated to reconstruct the geometry of transparent, refractive objects (right). For full resolution image "Rick Click -> View Image".
Camera images and reconstructed surfaces of dynamic water surfaces. The upper rows shows a drop falling into the water, whereas the lower rows depict water being poured into the tank. For full resolution image "Rick Click -> View Image".


Evaluation of reconstruction with respect to noise and refractive index mismatch. A 1D parabolic surface (left, dotted red) is simulated to be captured with a light field probe and reconstructed with different amounts of camera noise and mismatches in the refractive index of the medium (left). While noise results in high frequency artifacts, a mismatch in the refractive index causes low frequency distortions. We show the mean squared error of surfaces (top right) and gradients (bottom right) for an increasing amount of sensor noise. For full resolution image "Rick Click -> View Image".


Files

Paper [pdf] [bib]

Related Projects

Hand-Held Schlieren Photography with Light Field Probes
Time-resolved 3D Capture of Non-stationary Gas Flows
Schlieren and Shadowgraph Techniques (Textbook)


© IEEE, 2011. This is the author's version of the work. It is posted here by permission of IEEE for your personal use. Not for redistribution.