[Editors] MIT shows how brain tells glossy from grainy surfaces

[Elizabeth Thomson]

MIT News Office
Massachusetts Institute of Technology
Room 11-400
77 Massachusetts Avenue
Cambridge, MA  02139-4307
Phone: 617-253-2700
http://web.mit.edu/newsoffice/www

======================================
MIT shows how brain tells glossy from grainy surfaces

--Work could lead to better robotic vision

======================================

For Immediate Release
THURSDAY, APR. 19, 2007
Contact: Elizabeth A. Thomson, MIT News Office
Phone: 617-258-5402
Email: thomson at mit.edu

PHOTO, GRAPHIC AVAILABLE

CAMBRIDGE, Mass.--Imagine looking at a pool of spilled milk. Your 
brain knows that it's milk and not another white substance like 
sugar, or cottage cheese, but how does it know?

MIT researchers and colleagues investigating how the brain interprets 
the appearance of surfaces think they have an answer. They have found 
that the perception of reflectance and gloss are correlated with 
certain statistical properties of the image. These properties could 
be coded by neurons that respond differentially to light and dark 
spots.

The research team, a collaboration between MIT and the NTT 
Communication Science Labs in Japan, reported its findings in the 
April 18 online issue of Nature.

Studying how the brain analyzes surface appearance is not only 
important to understanding the workings of the human brain, but could 
also help scientists develop better visual systems for robots.

"We know a lot about the perception of objects, but much less about 
the perception of the materials that the objects are made of," said 
Edward Adelson, an author of the paper and professor of visual 
science in the Department of Brain and Cognitive Sciences at MIT. 
"Our studies show that statistical skewness, which quantifies an 
asymmetry between light and dark patterns, has a strong influence on 
the way a material is perceived."

Analyzing visual attributes such as color, texture, and gloss, is 
critical in everyday tasks such as deciding whether a patch of 
pavement is icy, whether a pancake is cooked, or whether skin is 
healthy, according to the researchers.

In their experiments, the research team asked subjects to rate the 
lightness and glossiness of natural materials such as stucco or 
fabric.  The surfaces have a mix of dark shadows and bright 
highlights, giving rise to distinctive visual patterns.

"Natural surfaces are complicated," said Adelson. "They have bumps 
and dips, and the light reflects in complex ways, producing 
characteristic statistical patterns." These patterns serve as 
signatures both for the shape and the material composing the surface.

The researchers quantified the images in terms of "luminance 
histograms," which plot the distribution of pixel values. They also 
estimated the histograms of responses of neurons that respond to 
light and dark spots in an image. In both cases, they found that the 
"skewness" of the histogram, which measures its asymmetry, was 
correlated with the subject's perceptions of surface qualities. 
Positive skewness led to darker and glossier surface appearance.

The researchers also found that they could manipulate subjects' 
perceptions of glossiness by digitally manipulating the skewness of 
the images, said study author Lavanya Sharan, an MIT graduate student 
at the Computer Science and Artificial Intelligence Lab.

Technology based on this research could also be useful for autonomous 
vehicles, said Sharan.

"You want to know what kind of surface you're on-is the road dry or 
wet or icy? Are you on a dirt road?  A machine vision system needs to 
make these judgments based on the surface appearance," she said.

The exact neural mechanism for detecting image skewness is unknown, 
but the researchers have offered a tentative model.

In the retina and brain, there are cells that are preferentially 
sensitive to either bright patterns or dark ones. After pooling 
responses within each cell population, the brain could compare the 
balance between very bright and very dark patterns. This balance 
determines the skewness.

"We don't know what the underlying physiology is, but what we do know 
is that the computation of skewness is something that could be done 
very easily with the hardware that does exist in the early parts of 
the brain's visual system," said Adelson.

The researcher's skewness model is supported by experiments with 
visual aftereffects, which showed that the human visual system can 
adapt to skewness. Similar to what happens when you stare at an image 
of a certain color and then see an after-image, staring at patterns 
with a high degree of positive skewness will cause the next pattern 
you look at to appear negatively skewed, which causes the material to 
appear lighter and less glossy.

The lead author on the Nature paper is Isamu Motoyoshi of the Human 
and Information Science Lab at NTT Communication Science Labs in 
Japan. Shin'ya Nishida of NTT is also an author on the paper.

The research was funded by NTT (Nippon Telegraph and Telephone 
Corporation) and the National Science Foundation.

--END--

Written by Anne Trafton, MIT News Office