Home Experience Services Contact Us Seminars Attorney's Guide to Perception Resources
"The Grand Illusion": Assigning Blame In Failure To See

Marc Green

This article discusses how we see. The topic my not be on everyone's lips, but that does not make it unimportant. In fact, I shall explain that a knowledge of seeing is essential for understanding accidents and for assigning blame in court. It is going to be a rough ride for some people because the discussion introduces ideas that run contrary to intuition and to subjective experience. This is unavoidable because most peoples' intuitions about seeing are incorrect and because subjective experience masks the true nature of seeing by making it automatic and effortless. Moreover, a discussion of seeing delves into issues such as the nature of consciousness and of reality. Much of this may sound like some egghead philosophical naval-gazing. On the contrary, it is practical and absolutely essential for the fair and just assignment of blame in court, as well as in life.

Seeing is an important legal issue because courts frequently assess blame based on whether a viewer should have seen some object or piece of information. Unfortunately, most people, including judges, jurors, attorneys and even most "experts," don't understand how seeing works, so they do not appreciate how various factors affect the ability to see. As a consequence, there are many tragic mis-assignments of blame on viewers who were simply acting normally, adaptively and even intelligently. I've seen so many mis-assignments of blame lately that I feel compelled to explain the basics of seeing. The starting point for correcting this situation is to recognize that:

Basic Rule 1: You cannot determine whether a viewer is at fault for failing to see a pedestrian, or anything else, unless you understand how seeing works.

Basic Rule 1 has great generality, I .e., you cannot fairly judge why a person fell down stairs unless you understand normal stair descent; you cannot fairly judge why a person failed to avoid a collision unless you understand how people normally avoid collisions; you cannot fairly judge why a person failed to comply with a warning unless you understand how people are normally process information into existing schemata, etc. Unfortunately, the courts seem overflowing with "experts" who give opinions despite an utter lack of background or knowledge about the way humans perform basic tasks like seeing, walking, thinking, remembering, etc.


Most people have never thought about how they see and have probably never thought that seeing even requires explanation. It just happens. At most, they have a vague intuitive notion that vision scientists call the "Homunculus theory." The eyes are like cameras that project an image on to an inner screen where a little man, the Homunculus, views it. There are many reasons why this scheme is wrong, most obviously that it begs the question of how the Homunculus sees the image. (Is there a second Homunculus inside the first, and third inside second, on and on into infinite regress?) Moreover, most people suffer from "na´ve realism", the false notion that seeing is a passive process where the eyes transmit a complete and objective reality directly to our consciousness. Na´ve realism is na´ve because seeing is active, selective and highly subjective.

Na´ve realism has powerful consequences for blame assignment, which is why it is so important to understand its na´vitÚ. For example, suppose that a driver strikes a pedestrian on a dark road. During a re-creation, an observer standing at the driver's viewpoint can see the pedestrian. The observer then goes to court and claims that the driver was at fault because he could have seen the pedestrian. The observer might even have taken photographs to bring into court as a means for demonstrating his conclusion's validity.

However, this conclusion is based on na´ve realism. The observer assumes that seeing is an objective, complete, and passive process that just happens when a viewer looks at a scene. Said another way, the sole determinant of seeing is the light entering the viewer's eyes. This assumption is wrong and reflects a fundamental misunderstanding about the nature of seeing.

It is easy to understand why people make this assumption and don't understand seeing - it occurs automatically and without conscious awareness. We just look, and we see. There is little or no conscious thought or effort, so we must not be doing anything. I also run into the same problem when I testify about falls. It is highly unintuitive that the ability to remain upright and to walk is a complex task that requires an intricate sensorimotor integration process. Unfortunately, most fail to appreciate that nature has thankfully made most of our everyday survival skills (seeing, walking, etc.) occur effortlessly so that our limited conscious resources can be directed toward novel problems for which we have no hard-wired or automatic behaviors.

I won't go into detail about the unnoticed complexity of seeing here, as I have explained the issues elsewhere
(Green, et al. 2017). However, even a quick look at the physical events involved in seeing reveals some key points:
  1. In most cases, light from a source (sun, headlamp, etc) reflects off a surface (pedestrian, sign, etc.) and enters the viewer's eyes. In a few cases, light goes directly from the source to the eye;
  2. The light entering the eye forms an image on a light-sensitive layer, the retina, which is very roughly analogous to the film in a camera;
  3. The retina contains nerve cells called "photoreceptors" that convert the light energy into electrical impulses through a chemical process;
  4. The electrical impulses propagate through a network of other nerve cells in the retina before traveling up the primary visual pathway to the brain. At every stage in the pathway neurons perform some processing on the signal;
  5. Seeing occurs. That is, you consciously perceive something, but not everything.
Now stop and think about this process. First, notice that we never see the world directly. Instead, we only directly see the image that it casts on the retina. This is why all visual analysis should start with the retinal image. Second, notice that we don't even see the retinal image. There are no pictures in the head and no inner viewing screen - just nerve impulses. The Homunculus, even if he existed, would be useless.

So what do we actually see? The answer is highly unintuitive but inescapable: we see something that the brain creates out of a subset of the electrical signals from the retinal image and from stored knowledge. This something has a rough correlation to some parts of the physical world, but it is not the physical world. The best example is color, which is not a property of light but which instead is purely a construction of the brain. Admittedly, nobody has the faintest clue as to how the brain actually turns these electrical impulses into conscious perceptions. But make no mistake that when we see, we are seeing something created by a pattern of electrical impulses and not directly the world or even the retinal image. Perception experts often refer to this world that we conscious perceive as "The Grand Illusion."

If you doubt this, consider a few facts. First, you don't need the "reality," retinal images or even light to see. You can push on the eye or hit yourself in the head to have visual sensations. You can close your eyes and mentally "see" pictures. You can even "see" objects that are entirely creations of you mind, for example, a purple and magenta elephant with 9 tusks or even invent completely new animals, if you like. Second, the world appears three-dimensional, yet the retinal image is only two-dimensional. Where does the extra dimension come from? The answer is that our brains must create it because it certainly isn't in the 2D retinal image1 (unless you are a strict Gibsonian). Third, you walk into a movie theater during the day. It is very black and you can see little. After a while, your eyes adapt, the scene looks brighter and you can see much that was previously invisible. Which is the "real" world - the dark theater or the bright theater? The answer is obviously neither. Your brain created both.

Basic Rule 2: Conscious perception is the result of two factors, the light that enters our eyes and the stored memories and knowledge that we carry around in our heads as mental models, schemas and scripts.

Helmoltz's Rule elaborates Basic Rule 2 by saying that viewers will consciously see the interpretation that they unconsciously deem the most likely, given the constraints created by the light. At most, the light constrains likely perceptions but does not determine them.

Basic Rule 2 also summarizes the common understanding of the many scientists who have been studying seeing over the last few hundred years. Various authors may use different terminology, "sensory vs. cognitive," "sensory vs. nonsensory," "knowledge in the head vs. knowledge in the world" or "bottom-up" vs. "top-down", but all agree on the basic notion that seeing employs two sources of information one from inside and one from outside, plus some unconscious reasoning process to combine them. For example:

"We see our world of organized scenes and moving objects by combining information from the light which reaches our eyes with knowledge stored in memory about the structure and identity of objects and scenes. We understand our surroundings partly by seeing them but also by inferring what is thereůVision is therefore a genuine cognitive function, not merely a sensory one." - Hampson and Morris (1996);

"Vision is not merely a matter of passive perception, it is an intelligent process of active construction. What you see is, invariably, what your visual intelligence constructs. Just as scientists intelligently construct useful theories based on experimental evidence, so your visual system intelligently constructs useful visual worlds based on images at the eyes. The main difference is that the constructions of scientists are done consciously, but those of your visual intelligence are done, for the most part, unconsciously." -Hoffman (1998); and

"We do not see what we sense. We see what we think we sense. Our consciousness is presented with an interpretation, not the raw data. Long after presentation, an unconscious information processing has discarded information, so that we see a simulation, a hypothesis, an interpretation; and we are not free to choose." -Norretranders (1998).

I am especially fond of Norretranders's summary since it captures all of the essentials: We do not see the real world. We do not see the object or even the retinal image. We see an interpretation. And we have no choice. I realize that this is hard to accept, but it is the reality of seeing, at least as best we can ever hope to understand reality. (The more philosophically inclined reader may wish to consult Immanuel Kant for his views on the impossibility of knowing reality.)

Now, I know what you are thinking. If we do not perceive an objective reality and if we construct our own perceptions, then why do people generally agree about what they are seeing? There are several answers to this question. First, the sensory information by itself is usually enough to constrain possible perceptions to a narrow range, but it does not specify them. Since we are all the members of the same species with essentially the same sensory neural wiring that evolution has found useful in helping us survive in the real world (whatever it is), a high level of agreement would be expected. Further, we all likely share similar cognitive neural wiring, although this varies more across individuals. Second, we all live in the same environments, have similar experiences and perform similar tasks with similar goals, so we develop very similar mental models and perceptual and motor schema. However, our experiences are not identical. Third, we seldom directly compare our perceptions with others. When we do, much of the comparison relies more on social convention than on actual perception. Perceptions are private, internal experiences that cannot be directly compared among people. Instead, we make the comparisons using language, which relies heavily on social conventions. Even red-green colorblind people will describe an apple as being red. Fourth, the disagreement among viewer perceptions is far greater than people generally suppose. In most cases, we don't notice the discrepancies because they have no important consequences. When they do, as in a court testimony, then startling differences among viewers are commonly revealed.


Now let's go back to the example where the observer goes to the accident re-creation, sees the pedestrian and concludes that a normal, "attentive" driver should have seen the pedestrian. This statement is based on a faulty understanding of seeing - it is assuming that perception rests 100% on the light entering the eye and that's all there is to seeing. As I have explained, this is never true. You cannot separate the perception and perceiver:

"The world is not a fixed solid array of objects out there for it cannot be fully separated from our perception of it. If shifts under our gaze and must be interpreted by us. Whatever fundamental units the world is put together from, they are more delicate, more fragile and more fugitive and startling than we can ever catch in the butterfly net of our senses." Jacob Bronowski (1981).

Unlike the driver, the observer standing on the roadway knows exactly where to look, what he will find and what is going to happen. He can look directly at the pedestrian for as long a he likes. Perhaps most importantly, he has no expectations, no goals, no purposes, and no competing tasks. The jurors who look at the photographs will similarly have all the same advantages as the observer. They are different viewers from the driver, and they cannot be made to see what the driver would, and yes could, see. Remember Norretranders: "and we are not free to choose." As for the photographs, the argument that they contain "what was there to be seen" misses the point.

With this proper understanding of seeing, here is a very rough outline of a real scientific and expert analysis.

1. Analyze the retinal image and the bottom-up information. For a given scene, this bottom-up sensory information should be roughly the same for all normally-sighted viewers. The peripheral parts of the visual system, retina, photoreceptors and the early visual pathway, etc. are primitive and hard-wired into our species. There is no reason to believe that viewers will differ, except for factors such as aging, visual abnormality and the like. However, there are many relevant variables, such as duration, retinal locus, luminance, adaptation level, etc. Even at this stage, however, top-down cognitive information is important, since it is the prime determinant of where the driver will point his eyes. The applicable science at this stage is "visual psychophysics," a discipline which mixes physics, psychology and physiology in order to examine how the senses operate. It defines the "boundary conditions" for seeing. If information is below sensory threshold, then the viewer can never see it. Usually, the relevant sensory threshold is for sensing contrast. In cases involving collision, however, the critical sensory threshold is often for motion.

2. Examine how top-down cognitive factors would influence seeing and response. Drivers have 1) a mental model of how the roadway works, 2) a set of schemas, common situations that they have frequently encountered, 3) a collection of scripts, frequently occurring sequences of events and 4) a set of goals that he is trying to achieve. These largely determine what the viewer consciously perceives, how he interprets it and which response is appropriate. Moreover, a fundamental property of human nature is its ability to adapt and to improve efficiency by making behavior more and more automatic (requiring less attention) through experience. At every opportunity, people switch behavioral control from the conscious and resource-draining processing of attending the light entering the eyes to an automatic and unconscious mental model. In a sense, we develop packaged subroutines that we can trigger and forget because they run off with minimal supervision. In other words, we act primarily on expectations. The central issues are often: How did the situation compare to a normal driver's mental model and schemas? Were any part of the model, I e., major expectations, violated? The applicable science here is cognitive psychology. Lastly, humans have some hard-wired top-down predispositions in perception. These include the tendency to see bright and warm colors as foreground and dark and blue colors as background, Gestalt Pragnanz principles of organization, etc.

3. Determine how the top-down cognitive factors would affect processing of the bottom-up sensory information in the retinal image. Examine the relative influence of the retinal image information vs the information stored in the head as knowledge and expectations about where to look for task-related information, what is likely there, what is likely to occur, how are other road users likely to act, etc.

Several factors can affect the balance between the bottom-up and the top-down. When viewers are in a familiar situation and/or engaged in a familiar behavior, they rely more on stored knowledge in the mental model. The familiarity may be achieved either directly through experience or vicariously by learning from other people. This same notion is expressed as a contrast between behavioral control by the past and the present:

Basic Rule 3: Behavior that is automatic and overlearned is more about the past than the present.

When viewers are in more novel situations for which they do not have stored ready-to-go subroutines of behaviors, then they rely more on the light. The balance can also be influenced by strength of the light entering the eye. A bright flash might engage attention better than a dim one. However, the ability of flashes, bright colors, etc. to control behavior is much less powerful than most people believe (Green, et al. 2008).

The observer who goes to the scene after the fact does none of this. The photographs that he shows the jurors contain none of this. His statements about what could or could not be seen and his photographs are at best useless and at worst misleading.


Seeing is not a passive process of responding to some objective world or even to the retinal image created by that world. It is not a mechanical process where the light entering the eyes determines what the viewer will see. Rather, it is an active process of interpretation that somehow results from a pattern of brain electrical activity created both by the light that enters the eyes and by knowledge stored in the head. An unconscious reasoning process combines the two information processes to produce conscious perception. And we are not free to chose.

The judgment of whether a normal viewer should have seen an object such as a pedestrian must take both bottom-up and top-down factors into account. The perceptions of an observer who views the scene after the fact says very little about what the driver would or could have seen at the time of the accident. Similarly, juror perceptions of scene photographs are likely to be misleading.


Bronowski, J. (1981) The Ascent of Man. London: Futura.

Green, M. Allen, M., Abrams, B. & Weintraub, L. (2008) Forensic Vision: With Application To Highway Safety. Tucson: Lawyers & Judges Publishing.

Hampson, P. and P. Morris. (1996). Understanding Cognition. Oxford: Blackwell.

Hoffman, D. (1998). Visual Intelligence: How We Create What We See. New York: W. W. Norton & Company.

Norretranders, T. (1998). The User Illusion Cutting Consciousness Down to Size. New York: Viking.


1If you think that physics or any other science is more "objective" and "real" than perceptual psychology, then think again. All scientific research results must be perceived and interpreted by humans. Since the science of physics is based on such results, it is as thoroughly subjective as any other perception. This observation led Bertrand Russell to conclude that physics is simply a subdiscipline of psychology.

Other Topics
Personal Injury: Road Accidents
  • Is The Moth-Effect Real?
  • Human Error in Road Accidents
  • Reaction Time
  • Let's Get Real About Perception-Reaction Time
  • Why PRT Is Not Like Gravity
  • Vision in Older Drivers
  • Weather and Accidents: Rain & Fog
  • Accidents At Rail-Highway Crossings
  • Seeing Pedestrians At Night
  • Underride Accidents
  • Rear End Collision: Looming
  • Night Vision
  • Distracted Pedestrians
  • Failure To See
  • Perception-Reaction Time (PRT) Programs
  • Twilight (3.3 lux) As A visibility Criterion
  • Human Error And Fault Tolerance
  • Why Pedestrians Die
  • Bicyclists! Read This To Save Your Life
  • Personal Injury: Warnings & Product Defects
  • Warnings and Warning Labels
  • Warning Effectiveness Checklist
  • The Psychology of Warnings
  • Drugs, Adverse Effects & Warnings
  • Are Warnings Effective?
  • Human Error Vs. Design Error
  • Product Misuse And "Affordances"
  • Safety Hierarchy: Design Vs. Warning
  • Thinking Like A Human Factors Expert
  • Personal Injury: Other
  • Diving Accidents in Pools
  • Falls Down Steps
  • Medical Error
  • Computer & Medical Error
  • Criminal & Police
  • Errors in Eyewitness Identifications
  • Perceptual Error in Police Shootings
  • Eyewitness Memory Is Unreliable
  • Human Factors In Forensic Evidence
  • Intellectual Property
  • "Any Fool Can See The Trademarks Are Different"
  • Measuring Confusion For Intellectual Property
  • Color in Trademark and Tradedress Disputes
  • Visual Human Factors
  • 33 Reasons For Not Seeing
  • Seeing Color
  • Determining Visibility
  • "Inattentional Blindness" & Conspicuity
  • Computer animation has perceptual limitations
  • Photographs vs. Reality
  • The Six Laws Of Attention
  • What is "inattention?"

  • | Home | Experience | Services | Contact Us  | Seminars/CLE | Attorney's Guide  | Resources |

    Send this link to someone

    Copyright © 2013 Marc Green, Phd
    Home Page:
    Contact Us