Illumination Problems in Computer Augmented Reality

The ability to merge a real video image (RVI) with a computer-generated image (CGI) enhances the usefulness of both. To go beyond "cut and paste" and chroma-keying, and merge the two images successfully, one must solve the problems of common viewing parameters. common visibility and common illumination. The result can be dubbed Computer Augmented Reality (CAR). The solution needs contributions from both computer graphics and computer vision. The problems of common illumination are especially challenging, because they test our understanding and practice of shadow and global illumination computation. In this paper we will describe and illustrate work in our laboratory where the emphasis is on extracting illumination information from real images and computing the common illumination between the real and the computer generated scene.