Mihoko Shimano, Hiroki Okawa, Yuta Asano, Ryoma Bise, Ko Nishino, and Imari Sato
National Institute of Informatics, Tokyo Institute of Technology, and Drexel University
Visual recognition of wet surfaces and their degrees of wetness is important for many computer vision applications. It can inform slippery spots on a road to autonomous vehicles, muddy areas of a trail to humanoid robots, and the freshness of groceries to us. In the past, monochromatic appearance change, the fact that surfaces darken when wet, has been modeled to recognize wet surfaces. In this paper, we show that color change, particularly in its spectral behavior, carries rich information about a wet surface. We derive an analytical spectral appearance model of wet surfaces that expresses the characteristic spectral sharpening due to multiple scattering and absorption in the surface. We derive a novel method for estimating key parameters of this spectral appearance model, which enables the recovery of the original surface color and the degree of wetness from a single observation. Applied to a multispectral image, the method estimates the spatial map of wetness together with the dry spectral distribution of the surface. To our knowledge, this work is the first to model and leverage the spectral characteristics of wet surfaces to revert its appearance. We conduct comprehensive experimental validation with a number of wet real surfaces. The results demonstrate the accuracy of our model and the effectiveness of our method for surface wetness and color estimation
Wetness and Color from A Single Multispectral Image
M. Shimano, H. Okawa, Y. Asano, R. Bise, K. Nishino, and I. Sato,
in Proc. of Conference on Computer Vision and Pattern Recognition CVPR’17, Jul., 2017. (Oral)
[ paper ][ project ][ talk ]
Overview
There are two fundamental components to this appearance change. The first is the decreased intensity across the spectrum, whose magnitude is proportional to the degree of wetness. This monochromatic characteristic of wet surface appearance has been modeled and used for recognition and image synthesis in the past. Perhaps less known is the second characteristic appearance change of wet surfaces: the effect of water (liquid) absorbed by the surface on its spectral behavior. The spectral distribution of a wet surface region becomes sharper than that of a dry region with the same surface color. As we show in this paper, this is due to the complex interplay of light absorption and multiple scattering inside the surface. Although this important spectral characteristic of wet surfaces has been empirically studied for image synthesis, to our knowledge, it has not been rigorously modeled in the past. In this paper, we derive a comprehensive spectral appearance model of wet surfaces. We specifically focus on the general appearance change induced by liquid absorbed in the surface itself. Our model explains both the monochromatic darkening and spectral sharpening of wet surfaces. It also rigorously relates the degree of surface wetness to its appearance change. Collectively, the model enables the recovery of the degree of wetness and the original (dry) surface color from a single observation.
Results
Our model lends itself to a simple yet powerful method for recovering the surface wetness and original color from a single observation. We show that we can recover a spatial map of surface wetness and the original (dry) spectral distribution from a single multispectral image. The recovered wetness is absolute if there are completely dry and completely wet points of the same material in the image. It becomes relative to the driest or wettest point if neither are present. Experimental results of surface wetness and color recovery for various real-world surfaces: felt-green, felt-blue, cotton-red, cotton-green, and gauze-blue. (a) Input image. (b) A 3D γ map showing the spatial distribution of the degree of wetness. (c) Graphs for intensity (red), γw (blue) along the black line of (a), and the inverse intensity (magenta). (d) Visualization of recovered dry spectral distribution matrix A (upper) and multiple scattering liquid (water) absorption matrix B (bottom), where the plots on the left show the spectral distributions for single scattering (i.e., first column of the matrices). (e) Comparison of the input multispectral distribution and the recovered distribution using estimated model parameters for dry and saturated surfaces, where the horizontal axis is the wavelength and vertical axis is the magnitude.
We also demonstrate that we can recover the underlying texture and spatial wetness of a wet textured surface. The left three columns have red wine split on textured cloth. The recovered spatial wetness and dry texture (albedo) agree well with what we would expect from the input image.
