Carnegie Mellon scientists have successfully performed the first rudimentary mind reading using functional magnetic resonance imaging, essentially identifying one of a number of words (“airplane” as opposed to “celery” in the image scan to the right) that subjects inside the machine were thinking about.
From the Carnegie Mellon press release:
In the study, nine subjects underwent fMRI scans while concentrating on 60 stimulus nouns – five words in each of 12 semantic categories including animals, body parts, buildings, clothing, insects, vehicles and vegetables.
To construct their computational model, the researchers used machine learning techniques to analyze the nouns in a trillion-word text corpus that reflects typical English word usage. For each noun, they calculated how frequently it co-occurs in the text with each of 25 verbs associated with sensory-motor functions, including see, hear, listen, taste, smell, eat, push, drive and lift. Computational linguists routinely do this statistical analysis as a means of characterizing the use of words.
These 25 verbs appear to be basic building blocks the brain uses for representing meaning, Mitchell said.
By using this statistical information to analyze the fMRI activation patterns gathered for each of the 60 stimulus nouns, they were able to determine how each co-occurrence with one of the 25 verbs affected the activation of each voxel, or 3-D volume element, within the fMRI brain scans.
To predict the fMRI activation pattern for any concrete noun within the text corpus, the computational model determines the noun’s co-occurrences within the text with the 25 verbs and builds an activation map based on how those co-occurrences affect each voxel.
In tests, a separate computational model was trained for each of the nine research subjects using 58 of the 60 stimulus nouns and their associated activation patterns. The model was then used to predict the activation patterns for the remaining two nouns. For the nine participants, the model had a mean accuracy of 77 percent in matching the predicted activation patterns to the ones observed in the participants’ brains.
The model proved capable of predicting activation patterns even in semantic areas for which it was untrained. In tests, the model was retrained with words from all but two of the 12 semantic categories from which the 60 words were drawn, and then tested with stimulus nouns from the omitted categories. If the categories of vehicles and vegetables were omitted, for instance, the model would be tested with words such as airplane and celery. In these cases, the mean accuracy of the model’s prediction dropped to 70 percent, but was still well above chance (50 percent).
Press release: Carnegie Mellon Computer Model Reveals How Brain Represents Meaning
(hat tip: Drudge Report)
