Perception is Inference

A new scientific understanding of perception has emerged in the past few decades, and it has overturned classical, centuries-long beliefs about how our brains work—though it has apparently not penetrated the medical world yet. The old understanding of perception is what neuroscientists call “the naïve view,” and it is the view that most people, in or out of medicine, still have. We’re inclined to think that people normally perceive things in the world directly. We believe that the hardness of a rock, the coldness of an ice cube, the itchiness of a sweater are picked up by our nerve endings, transmitted through the spinal cord like a message through a wire, and decoded by the brain.

In a 1710 “Treatise Concerning the Principles of Human Knowledge,” the Irish philosopher George Berkeley objected to this view. We do not know the world of objects, he argued; we know only our mental ideas of objects. “Light and colours, heat and cold, extension and figures—in a word, the things we see and feel—what are they but so many sensations, notions, ideas?” Indeed, he concluded, the objects of the world are likely just inventions of the mind, put in there by God. To which Samuel Johnson famously responded by kicking a large stone and declaring, “I refute it thus!”

Still, Berkeley had recognized some serious flaws in the direct-perception theory—in the notion that when we see, hear, or feel we are just taking in the sights, sounds, and textures of the world. For one thing, it cannot explain how we experience things that seem physically real but aren’t: sensations of itching that arise from nothing more than itchy thoughts; dreams that can seem indistinguishable from reality; phantom sensations that amputees have in their missing limbs. And, the more we examine the actual nerve transmissions we receive from the world outside, the more inadequate they seem.

Our assumption had been that the sensory data we receive from our eyes, ears, nose, fingers, and so on contain all the information that we need for perception, and that perception must work something like a radio. It’s hard to conceive that a Boston Symphony Orchestra concert is in a radio wave. But it is. So you might think that it’s the same with the signals we receive—that if you hooked up someone’s nerves to a monitor you could watch what the person is experiencing as if it were a television show.

Yet, as scientists set about analyzing the signals, they found them to be radically impoverished. Suppose someone is viewing a tree in a clearing. Given simply the transmissions along the optic nerve from the light entering the eye, one would not be able to reconstruct the three-dimensionality, or the distance, or the detail of the bark—attributes that we perceive instantly.

Or consider what neuroscientists call “the binding problem.” Tracking a dog as it runs behind a picket fence, all that your eyes receive is separated vertical images of the dog, with large slices missing. Yet somehow you perceive the mutt to be whole, an intact entity travelling through space. Put two dogs together behind the fence and you don’t think they’ve morphed into one. Your mind now configures the slices as two independent creatures.

The images in our mind are extraordinarily rich. We can tell if something is liquid or solid, heavy or light, dead or alive. But the information we work from is poor—a distorted, two-dimensional transmission with entire spots missing. So the mind fills in most of the picture. You can get a sense of this from brain-anatomy studies. If visual sensations were primarily received rather than constructed by the brain, you’d expect that most of the fibres going to the brain’s primary visual cortex would come from the retina. Instead, scientists have found that only twenty per cent do; eighty per cent come downward from regions of the brain governing functions like memory. Richard Gregory, a prominent British neuropsychologist, estimates that visual perception is more than ninety per cent memory and less than ten per cent sensory nerve signals. When Oaklander theorized that M.’s itch was endogenous, rather than generated by peripheral nerve signals, she was onto something important.

The fallacy of reducing perception to reception is especially clear when it comes to phantom limbs. Doctors have often explained such sensations as a matter of inflamed or frayed nerve endings in the stump sending aberrant signals to the brain. But this explanation should long ago have been suspect. Efforts by surgeons to cut back on the nerve typically produce the same results that M. had when they cut the sensory nerve to her forehead: a brief period of relief followed by a return of the sensation.

Moreover, the feelings people experience in their phantom limbs are far too varied and rich to be explained by the random firings of a bruised nerve. People report not just pain but also sensations of sweatiness, heat, texture, and movement in a missing limb. There is no experience people have with real limbs that they do not experience with phantom limbs. They feel their phantom leg swinging, water trickling down a phantom arm, a phantom ring becoming too tight for a phantom digit. Children have used phantom fingers to count and solve arithmetic problems. V. S. Ramachandran, an eminent neuroscientist at the University of California, San Diego, has written up the case of a woman who was born with only stumps at her shoulders, and yet, as far back as she could remember, felt herself to have arms and hands; she even feels herself gesticulating when she speaks. And phantoms do not occur just in limbs. Around half of women who have undergone a mastectomy experience a phantom breast, with the nipple being the most vivid part. You’ve likely had an experience of phantom sensation yourself. When the dentist gives you a local anesthetic, and your lip goes numb, the nerves go dead. Yet you don’t feel your lip disappear. Quite the opposite: it feels larger and plumper than normal, even though you can see in a mirror that the size hasn’t changed.

The account of perception that’s starting to emerge is what we might call the “brain’s best guess” theory of perception: perception is the brain’s best guess about what is happening in the outside world. The mind integrates scattered, weak, rudimentary signals from a variety of sensory channels, information from past experiences, and hard-wired processes, and produces a sensory experience full of brain-provided color, sound, texture, and meaning. We see a friendly yellow Labrador bounding behind a picket fence not because that is the transmission we receive but because this is the perception our weaver-brain assembles as its best hypothesis of what is out there from the slivers of information we get. Perception is inference.

[link]