Researchers from MIT and NTT Communication Science Labs in Japan are investigating the mechanisms of human perception of surface texture:
“We know a lot about the perception of objects, but much less about the perception of the materials that the objects are made of,” said Edward Adelson, an author of the paper and professor of visual science in the Department of Brain and Cognitive Sciences at MIT. “Our studies show that statistical skewness, which quantifies an asymmetry between light and dark patterns, has a strong influence on the way a material is perceived.”
Analyzing visual attributes such as color, texture, and gloss, is critical in everyday tasks such as deciding whether a patch of pavement is icy, whether a pancake is cooked, or whether skin is healthy, according to the researchers.
In their experiments, the research team asked subjects to rate the lightness and glossiness of natural materials such as stucco or fabric. The surfaces have a mix of dark shadows and bright highlights, giving rise to distinctive visual patterns.
“Natural surfaces are complicated,” said Adelson. “They have bumps and dips, and the light reflects in complex ways, producing characteristic statistical patterns.” These patterns serve as signatures both for the shape and the material composing the surface.
The researchers quantified the images in terms of “luminance histograms,” which plot the distribution of pixel values. They also estimated the histograms of responses of neurons that respond to light and dark spots in an image. In both cases, they found that the “skewness” of the histogram, which measures its asymmetry, was correlated with the subject’s perceptions of surface qualities. Positive skewness led to darker and glossier surface appearance.
The researchers also found that they could manipulate subjects’ perceptions of glossiness by digitally manipulating the skewness of the images, said study author Lavanya Sharan, an MIT graduate student at the Computer Science and Artificial Intelligence Lab.
Technology based on this research could also be useful for autonomous vehicles, said Sharan.
“You want to know what kind of surface you’re on–is the road dry or wet or icy? Are you on a dirt road? A machine vision system needs to make these judgments based on the surface appearance,” she said.
The exact neural mechanism for detecting image skewness is unknown, but the researchers have offered a tentative model.
In the retina and brain, there are cells that are preferentially sensitive to either bright patterns or dark ones. After pooling responses within each cell population, the brain could compare the balance between very bright and very dark patterns. This balance determines the skewness.
Press release: MIT shows how brain tells glossy from grainy surfaces …