Sheffield
FOR most of us, the ability to see and interpret our surroundings is such an
automatic part of everyday experience that we seldom pause to consider what an
astonishing feat of analysis it is. We perceive colour, form and motion without
even thinking about it. Yet computer scientists have struggled to develop robot
vision that can deal with anything like the level of detail available to us in
any visual field—the window on the world in front of our eyes. We
instantly recognise familiar objects and quickly categorise unfamiliar ones. We
recognise a chair as a chair whatever angle we see it from, even if it is upside
down or partly hidden by a table. The subtle differences between human faces are
enough for us to identify them instantly as those of particular
individuals—even if we cannot always put a name to them. More astonishing
still, we are able to appreciate the many facets of an intricate visual image
simultaneously.
How do we manage to see the world around us in all its complexity? This
question has kept philosophers busy for literally thousands of years. Yet only
in the past century or so have the techniques of investigative neurobiology
begun to reveal how the brain deals with all the information entering it from
the eye. Our understanding has advanced to a stage that illuminates the workings
of visual pathways, and even the very mechanisms of consciousness itself.
Advertisement
At the beginning of the present century the Spanish anatomist Santiago Ramon
y Cajal and the British physiologist Charles Sherrington between them laid the
foundations for the modern science of neurobiology. They showed that the secret
of the brain’s amazing abilities lies in its connectivity, the millions of
interconnections between different groups of interacting nerve cells. So the
best way to understand vision is to follow the pathways that visual information
takes from the eye to the brain.
All visual information reaches us in the form of light at wavelengths from
the visible part of the spectrum (about 300 to 700 nanometres), which is
reflected from objects in the world around us. It enters the eye through the
transparent window of the cornea and is focused by the lens, forming an
image on the retina. This image is upside down, like the image in a pinhole
camera: the top half of the retina receives light from the bottom half of the
visual field, and vice versa. Similarly, the left side of each retina receives
light from the right visual field, while the right side receives light from the
left field.
This has an interesting consequence for the route the visual pathways take to
the main visual centres in the brain. The cerebral cortex—the outer layer
of brain tissue where most of the nerve cells are found and most of the
information processing takes place—has two halves or hemispheres, each
dealing with information from the opposite side of the body. For example,
sensory information and motor instructions relating to the right side of your
body are dealt with by the cortex on the left side of your brain. This is also
true with vision.
The left visual cortex, situated at the back of the brain, processes
information from the right visual field. In the left eye, this information falls
on the left side of the retina. Long fibres called axons, which come from nerve
cells in this part of the retina, enter the optic nerve and pass on to
waystations on the same side of the brain. But in the right eye, the axons of
nerve cells on the left side of the retina must cross over in a structure called
the optic chiasm, so that they also reach the left side of the brain. In this
way information from both eyes relating to the same part of the visual field
reaches the same part of the brain.
On the way to the cortex, the new grouping of axons transmitting information
from the visual field of the opposite side passes in the optic tract to the
lateral geniculate nucleus. This highly organised structure in the midbrain has
six layers. The axons from each eye terminate in separate layers, three for each
eye. The separate inputs from the two eyes are not combined until they reach the
cortex. Axons from the nerve cells of the lateral geniculate nucleus leave in a
bundle called the optic radiation and terminate in connections with the visual
cortex at the back of the brain.
Image analysis
Feature by feature
The analysis of the massive amount of information in a visual image begins in
the eye itself. The retina of each eye contains 126 million photoreceptor cells,
but only one million axons leave the retina in the optic nerve. These axons
carry the output of the retinal ganglion cells, each of which integrates the
responses of photoreceptors on a small patch of the retina known as its
receptive field. The way these receptive fields are organised is critical to the
retina’s preliminary analysis of form, colour and movement.
Each receptive field is a circular patch of retina, and the ganglion cell
responds differently depending on whether light falls on the centre of the
circle or on the surrounding area. “On-centre, off-surround” cells, as their
name suggests, increase their activity if light falls on the centre of their
receptive field, but decrease it if the light is in the outer ring. Other
ganglion cells have the opposite response. The combined efforts of the ganglion
cells send a map of the visual field to the brain that highlights areas where
there are changes in the levels of illumination, such as the edges of objects.
The map also includes information about colour: a proportion of the ganglion
cells integrate inputs from the three types of cones in the retina,
photoreceptors that are sensitive to blue, green or red light.
Once the retina has recorded where everything is in the visual world, the
rest of the visual system uses that map as it conducts an even more detailed
analysis. Cells dealing with adjacent parts of the visual field also tend to be
physically close to one another in the brain. The lateral geniculate nucleus,
for example, more or less faithfully repeats the map created by the ganglion
cells.
The cortical area where the optic radiation terminates is called the primary
visual cortex, or V1. It has a distinctive line running through it when viewed
under a microscope, so it is also known as the striate cortex. The surrounding
cortical areas are called the prestriate cortex or “association cortex”. Until
very recently, scientists assumed that the primary visual cortex carried out
most of the analysis of visual information, and then passed the result on
to the association areas. Here visual images would be “associated” with previous
visual memories, as well as with input from other senses, eventually giving rise
to conscious perceptions.
But this rather simple scheme turns out to be seriously flawed. Often in
science the most dangerous and misleading assumptions are those which are not
even recognised as such. In the case of vision, we all have a very strong
subjective feeling of what it is like to see. When we look at a scene we
instantly see all its visual attributes—colour, form, texture, motion and
so on. It seems perfectly natural to suppose that all these facets are analysed
together in one area of the brain. And the obvious candidate for this area is
the striate cortex, the first cortical area to receive all the information
coming from the eyes. The very precise point-to-point mapping of the visual
fields in this area seems to lend weight to this idea.
Yet it turns out that our unitary visual experience is not an appropriate
model of how our brains actually work. In recent years neurobiologists have
provided impressive evidence that the different attributes of a visual image are
in fact analysed in different areas of the brain. Much of what was vaguely
thought to be association cortex plays a much more fundamental role in the
analysis of form, motion and colour. In a few short years, some carefully
gathered experimental data have rendered obsolete centuries of philosophising.
And it has neatly explained some of the strange effects experienced by patients
who have suffered strokes affecting the visual cortex (see Box 1).
Early attempts to define the areas of the cortex that were specialised for
different functions were focused largely on the arrangement of the layers of
cells visible under the microscope. Because much of the visual cortex had a
fairly uniform structure, neuroscientists assumed that its function was also
uniform. But modern methods that enable neuroscientists to record the responses
of living cells and trace their connections have revealed subdivisions within
the visual cortex. Some of the most exciting research in this area is carried
out on humans using positron emission tomography (PET), a scanning technique
that reveals changes in local blood flow in the living brain. An advantage of
these experiments is that humans can report their subjective experiences at the
same time as researchers collect the experimental data.
Visual cortex
Making pictures
Of all the visual areas, V1 contains the most detailed point-to-point map of
the retina. Its cells are organised into a stunningly complex system of distinct
modules. Alternating columns of cells show a preference for responding to
stimuli coming from one eye or the other. These ocular dominance columns are
further subdivided in a regular manner into columns of orientation-selective
cells, which respond to an edge or bar in their receptive fields only when it is
held at a particular orientation. All the cells in one column respond to one
orientation, cells in the adjacent column respond to an orientation a few
degrees off from the first, and so on until all possibilities are covered. There
are other groupings of cells within the ocular dominance columns which are not
orientation selective, but instead show a tendency to respond to light at
particular wavelengths. In this way V1 preserves the segregation of form and
colour that begins in the retina.
But V1 carries out a more elaborate analysis on these data. It contains
further groupings of cells that respond only to a stimulus that is not just of
the correct orientation, but also moving in a particular direction. And its
orientation-selective cells are sensitive not only to real boundaries, but to
illusory ones (Box 2), created when the cortex begins to reconstruct a mental
world of objects from the patterns of light and dark transmitted from the
retina. The intricate detail of V1 helps to explain why it is the largest visual
area, since it scans the field of view for all the features of the visual scene,
which are represented within it in a multiple series of overlapping maps.
Research on laboratory monkeys over the past couple of decades has revealed
several further distinct visual areas, labelled V2, V3, V4 and so on. Techniques
such as PET scanning in humans indicate that we have separate specialised visual
areas connected to V1, which are similar, but not always identical, to those in
monkey brain.
Most of V1’s output goes to an area immediately surrounding it called V2,
where there is a similar series of overlapping maps representing all the visual
features. As well as cells sensitive to colour, motion and orientation, V2
contains cells that are sensitive to disparity—the slightly different view
from each of the two eyes that is the basis of stereoscopic vision. V1 and V2
have intricate connections with each other and with other more specialised
visual areas.
V4 specialises in the perception of colour. This is not nearly as
straightforward a task as it might seem. The cones in the retina respond to
light at different wavelengths—but there is no straightforward
relationship between wavelength and the colour we perceive. The wavelengths of
light reflected from an object vary enormously according to lighting conditions.
Yet the leaves on trees still appear green whether at dawn or dusk, in the
midday sun or in the darkening sky of an approaching storm. Achieving this
colour constancy is one of the main jobs of V4, and it does it by comparing the
wavelengths reflected by groups of adjacent objects with their overall
brightness.
V5‘s function is to analyse motion, while V3 is concerned with the analysis
of form and depth—how far away an object is. Even more complex, the
recently-discovered area V6 appears to be responsible for analysing the absolute
position of an object in space. This is what makes you aware that a magazine in
front of you stays in the same place even when you turn to look at someone
coming into the room.
As information passes from one visual area to the next, cells become less
concerned with where an object is than with what it is. V1 cells will respond
only to objects in a small section of the visual field. But cells in the more
specialist areas tend to have much larger receptive fields. Some respond to
certain categories of object regardless of where their images appear on the
retina. The old idea that visual information passed up a rigid hierarchy of
cells until it reached a single cell that would respond only to a specific
image, such as that of your grandmother, has long been discredited. But what
does seem possible is that there are visual areas that encode information about
complex objects, including faces, in relatively small networks of perhaps 100 or
so cells.
Our perception results from selection and synthesis of available
information—we do not record things simply like a video camera does. What
we see depends largely on our past experience of the way the visual world is
organised, a fact which forms the basis of many visual illusions.
Seeing and knowing
Conscious experience
This new understanding of the way in which the brain handles visual
information has profound implications for our understanding of consciousness,
that most mysterious and elusive property of our minds. Older ideas suggested
that the primary analysis of visual information happened in the striate cortex,
which then fed this information forward to be associated with information from
other senses. The implication appeared to be that the associated information
would then be fed forward somewhere else until eventually a place was reached
where perception and consciousness would be generated.
Instead we now know that different parts of our awareness—from colour
to the expression on a person’s face—are generated simultaneously in
different specialised cortical areas. And if one of the specialised areas is
damaged we lose the relevant perception, causing strange alterations of
consciousness such as the awareness of colour without form, or the ability to
see form but not motion (Box 1). This indicates that all parts of our cortex
contribute directly to consciousness, which is the result of ongoing activity in
many intimately connected, specialised cortical areas. Indeed, the
interconnections are so complex that their description and analysis will provide
plenty of work for several more generations of neuroscientists.
* * *
1: Missing parts of the picture
OF all the organs in the body, the brain is the most critically dependent on
its blood supply. Interruption of the flow of blood for even a few minutes
causes irreversible damage to the region of brain affected—a stroke. Since
different regions of the brain are specialised for performing specific
functions, strokes vary in their effects depending on which part of the brain is
involved. For example, damage to the motor area of the cortex causes paralysis,
while damage to the primary visual cortex causes blindness.
Strokes often affect quite large areas of the brain, with widespread and
tragic consequences for the patient. But the problems caused by smaller areas of
damage have provided neurologists with some remarkable insights into the way the
human brain handles information and controls behaviour. Particularly strange
things can happen when strokes affect the visual areas. Damage to the primary
visual cortex produces a complete blind spot in the opposite visual field. But
localised damage to the more specialised areas can disturb some aspects of
vision while leaving others intact.
Louis Verrey, a Swiss ophthalmologist, described in 1888 the case of a
60-year-old woman who suffered a stroke affecting the visual cortex of the
left cerebral hemisphere. As a result, she could no longer see the world in
colour in the right half of her field of view. Instead, everything she saw in
that half appeared in shades of grey. Although strokes with this effect are
quite rare, they have been described many times, providing strong evidence that
colour is analysed separately from the other elements of a visual scene. This
cortical colour blindness or achromatopsia is quite different from the common
type of colour blindness which affects the whole visual field, and is due to an
abnormality of the wavelength-sensitive light receptors in the retina.
Even stranger cases include a 43-year-old woman who suffered a stroke, and
found that she could no longer see objects which were in motion, though
stationary objects presented no problem. This caused considerable difficulty.
For example, she found it hard to pour a cup of tea because the moving liquid
appeared frozen like a glacier, and she could not stop pouring at the right time
because she could not see the cup filling up. She also had problems crossing
roads: a car would seem to be far away, then suddenly would be very near as she
went to cross. This cortical motion blindness is strong evidence that motion is
also analysed in its own special area.
The type of visual disturbance made famous by the neurologist Oliver Sacks in
his book The Man Who Mistook His Wife for a Hat goes by the splendid, if
rather tongue-twisting name of prosopagnosia. These patients suffer an inability
to recognise familiar faces, including their own. They understand what a face is
and can see various features, such as the eyes, nose and mouth, but they just
cannot recognise it as a particular face. Even more bizarre is the fact that
some patients with this condition, unable to identify anyone from their face,
nevertheless retain the ability to recognise the expression on a face,
indicating that there is another cortical area that specialises in the
analysis of facial expressions.
More extraordinary still is the phenomenon of blindsight. Some people with
damage to the primary visual cortex, who deny being able to see anything in the
part of the visual field affected, can still make correct judgments about the
position, wavelength, or direction of movement of objects in the blind spot if
forced to do so . One patient, for example, could follow a moving striped object
with his eyes, even though he said he could not see it. The most likely
explanation is that the information is coming from surviving visual pathways
beneath the cortex, such as the lateral geniculate nucleus but, because it does
not reach the cortex, it is not available to conscious awareness.
* * *
2: There’s more to vision than meets the eye
FROM early in this century psychologists have been fascinated by the
phenomenon of visual illusions, which give a powerful sense of a reality that
simply is not there. Some of these, such as the illusion of movement you
experience when you look through the window of a train that has stopped at a
station, are simply the result of adaptation to prolonged stimulation in part of
the system. But others occur because the brain is trying, on the basis of its
past experience of the visual world, to come up with a “best guess” about what
is really there.
Most of the time this does not cause any problems, because the most likely
interpretation is probably the right one. Ambiguous figures (
Figure a) are an
exception. The brain can decide that the figure is a vase, or it can decide that
it is two faces; but it cannot see both interpretations at once, and tends to
alternate between one and the other.FIG-20739004.gif
Other illusions arise because certain features in the visual environment
provide such strong clues about the position of objects in a scene that it
becomes impossible to ignore them. Painters make use of the fact that light
falling on objects causes shadows to create the illusion of depth in their
pictures. Converging lines also suggest distance, and many simple optical
illusions use this device to fool you into making false judments about the size
or shape of objects (
Figure b).FIG-20739005.gif
A third example is that of illusory contours, where the brain extends partial
outlines to create shapes that do not exist (Figure c). The striking feature
here is the powerful sense that the illusory triangle glows with more brightness
than the surrounding white space. Making a reasonable hypothesis about the
elements of the scene, the brain decides that there is a bright triangle
slightly in front of the other objects.
- A Vision of the Brain by Semir Zeki (Blackwell Scientific,
1993), an excellent account of the neurological basis of vision. - An Introduction to the Visual System by Martin Tovée
(Cambridge University Press, 1996), an up-to-date textbook.