BRAIN-scanning technology has been used for the first time to recreate simple images by decoding the brain activity of people looking at them. Previously, the best that had been achieved was to pick which of a predetermined set of images the subject was observing.
Using functional magnetic resonance imaging, at ATR Computational Neuroscience Laboratories in Kyoto, Japan, initially scans his subjects鈥� brains while they look at various images made up of a 10-by-10 array of black and white pixels. Software developed by Kamitani鈥檚 team analyses the scans to find patterns of activity that are associated with certain pixels being blacked out. It then uses this information to discover signature patterns of brain activity for each pixel.
Kamitani then shows the subjects fresh patterns while their brains are being scanned, and feeds the scans to his software. By comparing the patterns in the scan with its list of signature patterns, the software can reconstruct the 10-by-10-pixel images the subjects are watching ().
Advertisement
In June, and colleagues at the University of California, Berkeley, showed that they could tell from a brain scan which of a set of images someone was looking at by comparing patterns of brain activity with those produced by looking at a separate set of 鈥渢raining鈥� photos (Nature, ). Kamitani鈥檚 work goes further by recreating the images from scratch. 鈥淚t really is a very significant step forward,鈥� says of the Max Planck Institute for Human Cognitive and Brain Sciences in Leipzig, Germany.
The images that Kamitani recreated are quite primitive, but by combining several scans, the researchers were able to read off the word 鈥渘euron鈥� (see graphic) and several numbers and shapes that people were shown. It is an important proof of principle, Haynes says.
As brain-scanning technology improves, Kamitani says it should be possible to recreate higher-quality images with many more pixels. He also wants to find out if it is possible to create images from shapes that people are imagining rather than actually seeing. Haynes suggests this might make it possible to 鈥渕ake a videotape of a dream鈥�.