.
Human Factors In Forensic Evidence
Marc Green
An FBI publication says, "Of all methods of identification, fingerprinting alone has proved to be both infallible and feasible." This might come as a surprise to Brandon Mayfield and Shirley McKie, among others. The FBI incorrectly matched Mr. Mayfield's fingerprint to one of a terrorist bomber of the Madrid subway. Scottish police incorrectly matched Ms. McKie's fingerprint to one found at a crime scene. This article explains reasons why such errors occur. They usually originate from innate human limitations and predispositions in perceptual judgment, rather than from intentional deception or falsification. The mental processes causing the errors are normal and usually rational, but they are not infallible. Examining the causes of these "normal" errors has two benefits. First, it can suggest methods for their prevention. Second, it underlines the reality that changing human nature is difficult, so the ultimate solution lies more in changing the system, not the examiner.
Expert forensic opinion decides many court cases. The
opinion often concerns individualization, where the expert examiner has
visually compared objects taken from a crime scene (fingerprint, handwriting,
bite marks, hair sample, etc.) with an exemplar taken from a suspect or a
database. The expert first judges whether the evidence is usable, and if so,
then classifies the comparison as a match, an exclusion or as inconclusive.
Individualization judgment is so powerful that juries may convict defendants
solely on the basis of the expert's opinion.
Courts have generally accepted examiner judgment with little
question. Although there has long been some skepticism about the infallibility
of forensic experts1 both
research studies and high-profile forensic individualization errors have
recently moved this skepticism to the forefront.
The research studies have demonstrated two important facts
about forensic examiners. First, they make individualization errors2.
This is not surprising in domains such as handwriting, where there is obviously
great uncertainty. However, expert examiners also make errors in domains such
as fingerprints, which authorities have long held to be virtually 100% objective
and infallible. Second,
examiners can be biased by "contextual factors," sources of information that
are independent of the evidence itself4
The highest profile case is the Mayfield fingerprint error.
The FBI ran a partial print left by a Madrid subway bomber through AFIS (Automated
Fingerprint Identification System, a database of 40 million sets
of fingerprints. One of the candidate matches was Brandon Mayfield, an Oregon attorney who had converted to
Islam and defended Muslim suspects. A first FBI expert declared that Mayfield's
fingerprints matched the partial. A second and then third FBI expert verified
the match. Next, an independent, defense fingerprint expert also verified the
FBI's match. However, the match proved to be an error when Spanish police
subsequently identified the real bomber.
Neither such examiner errors nor the biasing of perception
and judgment by context is surprising to those of us who scientifically study
vision and cognition. Visual matching tasks like those performed by
individuating examiners have been the subject of intense scientific research
for many years. Perceptual scientists have performed many thousands of research
studies on this topic and understand the sources of errors and biases
reasonably well. Until recently, however, the forensic world's assumption of
infallibility and objectivity has largely prevented application of this
understanding to the evaluation and improvement of expert judgment.
Below, I briefly outline some of the scientific knowledge
needed to understand how examiners make perceptual judgments and where the
sources of bias and error lie. The discussion frequently refers to the Mayfield
case as an example of these factors in operation. Lastly, I also discuss the
need for blind testing to reduce the amount of bias.
The discussion also includes comments on eyewitness
identification, since the same scientific principles apply. The eyewitness and the forensic examiner
perform very similar tasks: the examiner matches one physically present image
to another while the eyewitness matches one physically present image to a
remembered image. In fact, the same sensory and nonsensory factors that bias
forensic examiner judgment apply across a wide range of law enforcement
domains, such as police judgment in the use of force. For example, a police
officer patrolling at night sees a suspect holding an indistinct object in his
hand. Is it a gun? Should he fire?5
Biasing Forensic Judgments
The fundamental scientific principle of perception is highly
unintuitive: all conscious perception is an interpretation. We are not passive
recipients of light from the world who experience an objective reality. Rather,
we actively construct our perceptions and interpretations using two distinct
classes of information, sensory and nonsensory6.
Moreover, we reason with this information using heuristics,
rules-of-thumb, that help simplify reasoning in complex situations7.
Viewers derive sensory information directly from the
senses. It is the physical properties of images as measured objectively by a
light meter. For example fingerprint images contain friction ridge patterns,
which are light and dark shapes of differing contrast and brightness, perhaps
on a cluttered background. Theoretically, sensory information is objective and
unbiased in the sense of being independent of the particular viewer's cognitive
processes, knowledge, beliefs, goals and expectations, etc. It is presumably an
objective reality.
The other set of information is called nonsensory
because it does not arise from the image or the senses. The most common
nonsensory information concerns probabilities, and payoffs. Knowledge about
probabilities affects viewer expectation while knowledge of payoffs for various
judgments biases response in one direction over another.
The moment the examiner views the sensory information, his
cognitive processes spring into action using nonsensory information to
organize, to interpret and to judge the image's meaning. Sometimes these nonsensory
cognitive processes bias judgment. Scientifically, the term "bias" is the
tendency to favor one response category over another. It does not imply any
errant behavior or misdeed. In fact, I will explain that biased judgment is
normal and rational and is often intelligent behavior for improving
performance.
The probability information suggests the a priori
likelihood that one response category is the correct choice. All perceptual
scientists know that seeing is believing but that it is just as true that believing
is seeing. A viewer is more likely to give a response that is consistent
with pre-existing expectations and with other factors affecting probability
that the suspect is guilty. For example, one study8
found that examiners were more likely to find a fingerprint match after
learning that the suspect had confessed. Conversely, the examiners viewing the
same fingerprints were less likely to find a match when told that the suspect
was in police custody at the time of the crime.
Another common example occurs when the forensic laboratory
requires that a second examiner verify positive matches from the first. The
second examiner knows that someone else has already made a positive match, so
there is a very strong a priori likelihood that it really is an
individualization. The second examiner will then be biased toward judging the
comparison a match. The effect will be stronger if the original examiner is
senior and has high credibility or if multiple examiners have already confirmed
the match. The Mayfield case is an extraordinary clear example because the
independent defense expert confirmed the errant match made by three
prosecution experts. An FBI review9 of the error concluded, "Once the mind-set occurred with the initial
examiner, the subsequent examinations were tainted"
The expectancy effect in the Mayfield case was a classic
case of the "representative heuristic,"
which says that two objects sharing one attribute likely share another. The
reasoning likely was 1) Muslims have the attribute terrorist, 2) Mayfield had
the attribute Muslim, and 3) so Mayfield most likely shares the attribute
terrorist. (Try substituting "black" for "Muslim" and "drug dealer" for "terrorist.")
Use of nonsensory
information is often perfectly rational. Fingerprints from a subject who
confessed are more likely to match. Objects sharing one attribute often share
another. The problem is that such reasoning is not infallible. Humans
are inherently "satisficers", meaning
we seek "good enough" solutions within reasonable bounds of information and
effort. Humans have evolved reasoning based on biases and heuristics to aid
satisficing, but they do not ensure 100% correctness. However, courts assume
and expect forensic experts to be correct 100% of the time. Obviously, there is
a mismatch between the court's demands and normal human judgment. This is the
crux of the problem, and I'll suggest some solutions later.
Once judgment occurs, however, viewers tend toward
"confirmation bias,"10 the strong tendency to perceive and to make decisions consistent with prior
held beliefs and to ignore evidence which contradicts the prior beliefs. It is
perhaps the most power heuristic in human thinking and has been frequently
cited as a problem in the examination of forensic evidence11 and as well as a source of error and accident12.
Confirmation bias is why FBI agents, having found a good candidate would then tend to seek out
evidence to confirm their conclusion and to dismiss contradictory evidence. It is also why law
enforcement personnel and courts in general frequently ignore subsequent, contradictory
evidence once having decided on a suspect's guilt.
Lastly, payoffs are the other class of nonsensory information. Viewers automatically consider the
consequences of their decisions on their goals. The legal system has a general built-in payoff bias
because it is presumably "better to let 100 guilty men go free than to convict one innocent man."
While this presumably biases the examiner away from matches, there are also personal goals and
payoffs to consider.
An examiner might want to win the approval of the local police by helping them solve a high
profile or particularly heinous crime13. Some examiners have large egos and want to prove their
skill by finding matches in particularly uncertain evidence and where other examiners have
"failed." Finding a match in a high-profile case can mean earn recognition and praise. A junior
examiner is unlikely to disconfirm a match by a more senior examiner, especially one who is his
boss. (Apparently at the FBI, "To disagree was not an expected response."14 Continued failure
to find matches may result in loss of status or even employment. If the suspect is a known "bad
guy," the public would be better served by putting him away. It isn't hard to spot the biasing
payoffs for the FBI agents in the Mayfield case.
Unbiasing Forensic Judgments
Before considering methods to reduce error and eliminate bias, I summarize the key take-home
points to remember about bias in perception and judgment:
1. All human perception and judgment is based on a combination of sensory and nonsensory
factors and is inherently subjective;
2. When an image is clear and unambiguous, the viewer is likely to judge primarily on the
sensory information and approach the court's ideal. The biasing effects of nonsensory
information increase as the image becomes more ambiguous and uncertain. Bias further
increases under time pressure and stress causing viewers to prefer simpler solutions based on
less information;
3. Reasoning on nonsensory information is largely unconscious. Certainly, there may be
deliberate attempts by a few examiners to falsify forensic evidence, but normally the
nonsensory factors operate without awareness or volition;
4. The use of nonsensory information is normal. It is the typical human solution to making
decisions with uncertain information;
5. The use of nonsensory information is generally rational. Matches made by a senior examiner
are highly likely to be correct. In many situations, completely ignoring the nonsensory
information would depart from normal reasoning and would be considered irrational.
In sum, bias occurs unconsciously and is perfectly normal and rational. It is human nature to be
unconsciously biased by nonsensory information because it is usually helpful. Merely giving
viewers instructions to do so is futile. Once learned, nonsensory information cannot be
completely ignored or forgotten, even if later proved to be incorrect. An examiner who learns
that the suspect confessed, for example, will still be affected by that information even if he later
learns that the suspect had recanted or even that the confession never existed.
The courts assume that the examiners should use only sensory information because it is
objective. This is naïve thinking because all perception and judgment is subjective. The only
issue is the degree to which this subjectivity is biased one way or the other. Obviously, there is a
conflict between the court's idealized view and real human nature15. As we all know, however,
you can't change human nature.
So how can bias be reduced from individualization without changing human nature? The answer
is to change the circumstances under which the examiner works. Two obvious solutions suggest
themselves.
First, improve the sensory information. Bias effects are smaller when the judgment is less
ambiguous and less uncertain. Other domains such as radiology, where viewers must also make
categorical judgments from uncertain images, have done extensive research on finding image
processing techniques which clarify the critical features. There is little similar research in the
field of forensics.
It may also be possible to choose examiners who have greater innate skill at viewing the sensory
information and who would then be less influenced by nonsensory factors. Many domains, such
as aviation, have developed tests of candidate aptitude. Simple acuity or other visual tests might
help weed out unsuitable candidates. It might also be possible to teach examiners how to better
extract sensory information from the image. Current training is highly variable from place to
place and not based on any scientific understand of how examiners perform their job.
Second, use blind procedures which isolate the examiner from most nonsensory information. The
examiner should know nothing about the circumstances of the crime, the opinion of the police
about the suspect's identity, the origin of the fingerprints, the nature of the crime, the opinion of
other examiners, etc.
It is especially critical to isolate examiners from police and others who may have knowledge of
the case. Humans are extraordinarily keen at detecting social cues such as subtle voice
inflections, changes in posture, facial expressions, etc. In eyewitness identification procedures,
for example, those conducting eyewitness identifications should not know the suspect's identity.
They can provide subtle cues which both direct the eyewitness and reinforce another human bias,
"overconfidence."16
Blind judgment is also critical in verification. Given what is known about confirmation bias, it is
no surprise that studies17 have found blind verification far better at catching errors than non-blind
verification.
Moreover, some,18 have gone so far as to claim that the lack of the blind procedures, among other
reasons, renders the current ACE-V fingerprint individualization procedures as scientifically
unproven. One experienced Ontario Provincial Police fingerprint examiner19 concluded that ACE-V is inherently
subjective and noted that "Reaching these [fingerprint classification] decisions, however,
requires something not found in science. In essence they require us to use our training, our
judgment and our knowledge, to arrive at an opinion," That is, the examiner must use his
cognitive processes, which are susceptible to biasing by nonsensory factors.
Conclusion
The first step in understanding and correcting examiner error and bias is a realistic assessment of
perception and judgment. Courts frequently have an idealized of human abilities which assumes,
expects and desires infallibility or at least perfect "objectivity." The reason that courts would
take this view is easy to understand. It greatly simplifies their task in weighting evidence.
Moreover, humans typically prefer black-and-white solutions that minimize complexity,
especially in a system under severe time and resource constraints.
Unfortunately, simple is not always better. The vast scientific and applied research literature on
human perception and judgment shows that the idealized view is untenable. This knowledge on
the sources of error and bias is ready and waiting to help evaluate and to correct the defects in
the current system. It cannot be applied, however, until courts adopt a more realistic view of
human nature and do not expect the impossible. The first step to a solution is to admit that there
is a problem.
Endnotes
1 Miller, L. (1987). Procedural bias in forensic science examination of human hair. Law And
Human Behavior, 11, 157-163.
2 E. g., Haber, L. & Haber, R. (2003).Error rates for human latent fingerprint examiners. In
Ratha, N. & Bolle, R. (Eds.), Advances in Automatic Fingerprint Recognition. New York:
Springer-Verlag.
3 E. g., Federal Bureau of Investigation, (1985). The Science of Fingerprints: Classification and
Uses. "Of all methods of identification, fingerprinting alone has proved to be both infallible and
feasible."
4 Dror, I.E., Peron, A., Hind, S., & Charlton, D. (2005). When emotions get the better of us: The
effect of contextual top-down processing on matching fingerprints. Applied Cognitive
Psychology, 19(6), 799-809.
5 Green, M. (2005). "Is It A Gun? Or Is It A Wallet?" Perceptual Factors In Police Shootings of
Unarmed Suspects, Police Marksman, July/August, 52-54.
http://www.visualexpert.com/Resources/policeshooting.html
6 This distinction also goes by other names: "psychophysical" vs. "cognitive" and "bottom-up"
vs. "top-down." The "sensory" vs. "nonsensory" terminology come from Signal Detection
Theory, a perceptual approach to separating bias from accuracy. Lastly, some refer to nonsensory
effects as "context effects," or "observer effects."
7 Green, et al. (2008). Forensic Vision: With Applications To Highway Safety. Tucson: Lawyers
& Judges Publishing.
8 Dror, I., Charlton, D., & Peron A. (2006). Contextual information renders experts vulnerable to
making erroneous identifications Forensic Science International, 156 (1), 74-78.
9 Stacey, R. (2004). A report on the erroneous fingerprint individualization in the Madrid train
bombing case. Journal of Forensic Identification, 54, 706-718.
10 In one classic example of confirmation bias, police arrested a suspect who had been identified
in a photo array as a rapist. The man protested that at the time of the rape he was sitting next to
an assistant police commissioner on a live TV show in front of millions of viewers. The police
dismissed his alibi saying, "Yes, I suppose you've got Jesus Christ, and the Queen of England,
too."
11 E. g., Risinger, D. Saks, M., Thompson, W., & Rosenthal, R. (2002). The Daubert/Kumho
Implications of Observer Effects in Forensic Science: Hidden Problems of Expectation and
Suggestion, California Law Review, 90, 1-55.
12E. g., Green, M. (2004). Nursing error and human nature. Journal of Nursing Law, 9, 37-44.
13 Dror, I., Peron, A., Hind, S., & Charlton, D. (2005). When emotions get the better of us: The
effect of contextual top-down processing on matching fingerprints. Applied Cognitive
Psychology, 19(6), 799-809.
14See endnote 9.
15 This is very common in law. For example, statutes often say that a viewer should see whatever
is available to be seen. This is unrealistic and ignores all the normal cognitive process of
attention, expectation and other cognitive processes discussed here plus much more. See footnote
and 5 and Marshall, J. (1966). Law And Psychology In Conflict. New York. Bobs-Merrill
16 Green, M. (2003). Proper Eyewitness Identification Procedures," Law and Order, Oct., 195-
198.
17 E .g., Arvizu, J. (2002) Testimony on Mr. Plaza's motion to exclude the government's latent
fingerprint identification evidence, hearing before Judge Louis Pollak, USA v. Plaza, et al., U.S.
District Court for the Eastern District of Pennsylvania, February 24,2002.
18 Haber, L. & Haber, R, (2008).Scientific Validation of Fingerprint Evidence Under Daubert,
Law, Probability & Risk, 7, Issue 2, pp. 87-109, 2008.
19Ashton, W. Fingerprint Identifications (Science or Not). Retrieved from
http://www.4intel.com/images/fingerprints.pdf 2 Feb 09.
Footnotes
*A version of this article was published under the title, "Human Fallibility Woven Into Forensic Science" in For The Defense, 2009.
A division of 2057949 Ontario, Inc.
Copyright © 2024 Marc Green, Ph. D.
Home Page
Contact Us
The highest profile case is the Mayfield fingerprint error. The FBI ran a partial print left by a Madrid subway bomber through AFIS (Automated Fingerprint Identification System, a database of 40 million sets of fingerprints. One of the candidate matches was Brandon Mayfield, an Oregon attorney who had converted to Islam and defended Muslim suspects. A first FBI expert declared that Mayfield's fingerprints matched the partial. A second and then third FBI expert verified the match. Next, an independent, defense fingerprint expert also verified the FBI's match. However, the match proved to be an error when Spanish police subsequently identified the real bomber.
Neither such examiner errors nor the biasing of perception and judgment by context is surprising to those of us who scientifically study vision and cognition. Visual matching tasks like those performed by individuating examiners have been the subject of intense scientific research for many years. Perceptual scientists have performed many thousands of research studies on this topic and understand the sources of errors and biases reasonably well. Until recently, however, the forensic world's assumption of infallibility and objectivity has largely prevented application of this understanding to the evaluation and improvement of expert judgment.
Below, I briefly outline some of the scientific knowledge needed to understand how examiners make perceptual judgments and where the sources of bias and error lie. The discussion frequently refers to the Mayfield case as an example of these factors in operation. Lastly, I also discuss the need for blind testing to reduce the amount of bias.
The discussion also includes comments on eyewitness identification, since the same scientific principles apply. The eyewitness and the forensic examiner perform very similar tasks: the examiner matches one physically present image to another while the eyewitness matches one physically present image to a remembered image. In fact, the same sensory and nonsensory factors that bias forensic examiner judgment apply across a wide range of law enforcement domains, such as police judgment in the use of force. For example, a police officer patrolling at night sees a suspect holding an indistinct object in his hand. Is it a gun? Should he fire?5
Biasing Forensic Judgments
The fundamental scientific principle of perception is highly unintuitive: all conscious perception is an interpretation. We are not passive recipients of light from the world who experience an objective reality. Rather, we actively construct our perceptions and interpretations using two distinct classes of information, sensory and nonsensory6. Moreover, we reason with this information using heuristics, rules-of-thumb, that help simplify reasoning in complex situations7.
Viewers derive sensory information directly from the senses. It is the physical properties of images as measured objectively by a light meter. For example fingerprint images contain friction ridge patterns, which are light and dark shapes of differing contrast and brightness, perhaps on a cluttered background. Theoretically, sensory information is objective and unbiased in the sense of being independent of the particular viewer's cognitive processes, knowledge, beliefs, goals and expectations, etc. It is presumably an objective reality.
The other set of information is called nonsensory because it does not arise from the image or the senses. The most common nonsensory information concerns probabilities, and payoffs. Knowledge about probabilities affects viewer expectation while knowledge of payoffs for various judgments biases response in one direction over another.
The moment the examiner views the sensory information, his cognitive processes spring into action using nonsensory information to organize, to interpret and to judge the image's meaning. Sometimes these nonsensory cognitive processes bias judgment. Scientifically, the term "bias" is the tendency to favor one response category over another. It does not imply any errant behavior or misdeed. In fact, I will explain that biased judgment is normal and rational and is often intelligent behavior for improving performance.
The probability information suggests the a priori likelihood that one response category is the correct choice. All perceptual scientists know that seeing is believing but that it is just as true that believing is seeing. A viewer is more likely to give a response that is consistent with pre-existing expectations and with other factors affecting probability that the suspect is guilty. For example, one study8 found that examiners were more likely to find a fingerprint match after learning that the suspect had confessed. Conversely, the examiners viewing the same fingerprints were less likely to find a match when told that the suspect was in police custody at the time of the crime.
Another common example occurs when the forensic laboratory requires that a second examiner verify positive matches from the first. The second examiner knows that someone else has already made a positive match, so there is a very strong a priori likelihood that it really is an individualization. The second examiner will then be biased toward judging the comparison a match. The effect will be stronger if the original examiner is senior and has high credibility or if multiple examiners have already confirmed the match. The Mayfield case is an extraordinary clear example because the independent defense expert confirmed the errant match made by three prosecution experts. An FBI review9 of the error concluded, "Once the mind-set occurred with the initial examiner, the subsequent examinations were tainted"
The expectancy effect in the Mayfield case was a classic case of the "representative heuristic," which says that two objects sharing one attribute likely share another. The reasoning likely was 1) Muslims have the attribute terrorist, 2) Mayfield had the attribute Muslim, and 3) so Mayfield most likely shares the attribute terrorist. (Try substituting "black" for "Muslim" and "drug dealer" for "terrorist.")
Use of nonsensory information is often perfectly rational. Fingerprints from a subject who confessed are more likely to match. Objects sharing one attribute often share another. The problem is that such reasoning is not infallible. Humans are inherently "satisficers", meaning we seek "good enough" solutions within reasonable bounds of information and effort. Humans have evolved reasoning based on biases and heuristics to aid satisficing, but they do not ensure 100% correctness. However, courts assume and expect forensic experts to be correct 100% of the time. Obviously, there is a mismatch between the court's demands and normal human judgment. This is the crux of the problem, and I'll suggest some solutions later.
Once judgment occurs, however, viewers tend toward "confirmation bias,"10 the strong tendency to perceive and to make decisions consistent with prior held beliefs and to ignore evidence which contradicts the prior beliefs. It is perhaps the most power heuristic in human thinking and has been frequently cited as a problem in the examination of forensic evidence11 and as well as a source of error and accident12. Confirmation bias is why FBI agents, having found a good candidate would then tend to seek out evidence to confirm their conclusion and to dismiss contradictory evidence. It is also why law enforcement personnel and courts in general frequently ignore subsequent, contradictory evidence once having decided on a suspect's guilt.
Lastly, payoffs are the other class of nonsensory information. Viewers automatically consider the consequences of their decisions on their goals. The legal system has a general built-in payoff bias because it is presumably "better to let 100 guilty men go free than to convict one innocent man." While this presumably biases the examiner away from matches, there are also personal goals and payoffs to consider.
An examiner might want to win the approval of the local police by helping them solve a high profile or particularly heinous crime13. Some examiners have large egos and want to prove their skill by finding matches in particularly uncertain evidence and where other examiners have "failed." Finding a match in a high-profile case can mean earn recognition and praise. A junior examiner is unlikely to disconfirm a match by a more senior examiner, especially one who is his boss. (Apparently at the FBI, "To disagree was not an expected response."14 Continued failure to find matches may result in loss of status or even employment. If the suspect is a known "bad guy," the public would be better served by putting him away. It isn't hard to spot the biasing payoffs for the FBI agents in the Mayfield case.
Unbiasing Forensic Judgments
Before considering methods to reduce error and eliminate bias, I summarize the key take-home points to remember about bias in perception and judgment:
1. All human perception and judgment is based on a combination of sensory and nonsensory factors and is inherently subjective;
2. When an image is clear and unambiguous, the viewer is likely to judge primarily on the sensory information and approach the court's ideal. The biasing effects of nonsensory information increase as the image becomes more ambiguous and uncertain. Bias further increases under time pressure and stress causing viewers to prefer simpler solutions based on less information;
3. Reasoning on nonsensory information is largely unconscious. Certainly, there may be deliberate attempts by a few examiners to falsify forensic evidence, but normally the nonsensory factors operate without awareness or volition;
4. The use of nonsensory information is normal. It is the typical human solution to making decisions with uncertain information;
5. The use of nonsensory information is generally rational. Matches made by a senior examiner are highly likely to be correct. In many situations, completely ignoring the nonsensory information would depart from normal reasoning and would be considered irrational.
In sum, bias occurs unconsciously and is perfectly normal and rational. It is human nature to be unconsciously biased by nonsensory information because it is usually helpful. Merely giving viewers instructions to do so is futile. Once learned, nonsensory information cannot be completely ignored or forgotten, even if later proved to be incorrect. An examiner who learns that the suspect confessed, for example, will still be affected by that information even if he later learns that the suspect had recanted or even that the confession never existed.
The courts assume that the examiners should use only sensory information because it is objective. This is naïve thinking because all perception and judgment is subjective. The only issue is the degree to which this subjectivity is biased one way or the other. Obviously, there is a conflict between the court's idealized view and real human nature15. As we all know, however, you can't change human nature.
So how can bias be reduced from individualization without changing human nature? The answer is to change the circumstances under which the examiner works. Two obvious solutions suggest themselves.
First, improve the sensory information. Bias effects are smaller when the judgment is less ambiguous and less uncertain. Other domains such as radiology, where viewers must also make categorical judgments from uncertain images, have done extensive research on finding image processing techniques which clarify the critical features. There is little similar research in the field of forensics.
It may also be possible to choose examiners who have greater innate skill at viewing the sensory information and who would then be less influenced by nonsensory factors. Many domains, such as aviation, have developed tests of candidate aptitude. Simple acuity or other visual tests might help weed out unsuitable candidates. It might also be possible to teach examiners how to better extract sensory information from the image. Current training is highly variable from place to place and not based on any scientific understand of how examiners perform their job.
Second, use blind procedures which isolate the examiner from most nonsensory information. The examiner should know nothing about the circumstances of the crime, the opinion of the police about the suspect's identity, the origin of the fingerprints, the nature of the crime, the opinion of other examiners, etc.
It is especially critical to isolate examiners from police and others who may have knowledge of the case. Humans are extraordinarily keen at detecting social cues such as subtle voice inflections, changes in posture, facial expressions, etc. In eyewitness identification procedures, for example, those conducting eyewitness identifications should not know the suspect's identity. They can provide subtle cues which both direct the eyewitness and reinforce another human bias, "overconfidence."16
Blind judgment is also critical in verification. Given what is known about confirmation bias, it is no surprise that studies17 have found blind verification far better at catching errors than non-blind verification.
Moreover, some,18 have gone so far as to claim that the lack of the blind procedures, among other reasons, renders the current ACE-V fingerprint individualization procedures as scientifically unproven. One experienced Ontario Provincial Police fingerprint examiner19 concluded that ACE-V is inherently subjective and noted that "Reaching these [fingerprint classification] decisions, however, requires something not found in science. In essence they require us to use our training, our judgment and our knowledge, to arrive at an opinion," That is, the examiner must use his cognitive processes, which are susceptible to biasing by nonsensory factors.
Conclusion
The first step in understanding and correcting examiner error and bias is a realistic assessment of perception and judgment. Courts frequently have an idealized of human abilities which assumes, expects and desires infallibility or at least perfect "objectivity." The reason that courts would take this view is easy to understand. It greatly simplifies their task in weighting evidence. Moreover, humans typically prefer black-and-white solutions that minimize complexity, especially in a system under severe time and resource constraints.
Unfortunately, simple is not always better. The vast scientific and applied research literature on human perception and judgment shows that the idealized view is untenable. This knowledge on the sources of error and bias is ready and waiting to help evaluate and to correct the defects in the current system. It cannot be applied, however, until courts adopt a more realistic view of human nature and do not expect the impossible. The first step to a solution is to admit that there is a problem.
Endnotes
1 Miller, L. (1987). Procedural bias in forensic science examination of human hair. Law And Human Behavior, 11, 157-163.
2 E. g., Haber, L. & Haber, R. (2003).Error rates for human latent fingerprint examiners. In Ratha, N. & Bolle, R. (Eds.), Advances in Automatic Fingerprint Recognition. New York: Springer-Verlag.
3 E. g., Federal Bureau of Investigation, (1985). The Science of Fingerprints: Classification and Uses. "Of all methods of identification, fingerprinting alone has proved to be both infallible and feasible."
4 Dror, I.E., Peron, A., Hind, S., & Charlton, D. (2005). When emotions get the better of us: The effect of contextual top-down processing on matching fingerprints. Applied Cognitive Psychology, 19(6), 799-809.
5 Green, M. (2005). "Is It A Gun? Or Is It A Wallet?" Perceptual Factors In Police Shootings of Unarmed Suspects, Police Marksman, July/August, 52-54. http://www.visualexpert.com/Resources/policeshooting.html
6 This distinction also goes by other names: "psychophysical" vs. "cognitive" and "bottom-up" vs. "top-down." The "sensory" vs. "nonsensory" terminology come from Signal Detection Theory, a perceptual approach to separating bias from accuracy. Lastly, some refer to nonsensory effects as "context effects," or "observer effects."
7 Green, et al. (2008). Forensic Vision: With Applications To Highway Safety. Tucson: Lawyers & Judges Publishing.
8 Dror, I., Charlton, D., & Peron A. (2006). Contextual information renders experts vulnerable to making erroneous identifications Forensic Science International, 156 (1), 74-78.
9 Stacey, R. (2004). A report on the erroneous fingerprint individualization in the Madrid train bombing case. Journal of Forensic Identification, 54, 706-718.
10 In one classic example of confirmation bias, police arrested a suspect who had been identified in a photo array as a rapist. The man protested that at the time of the rape he was sitting next to an assistant police commissioner on a live TV show in front of millions of viewers. The police dismissed his alibi saying, "Yes, I suppose you've got Jesus Christ, and the Queen of England, too."
11 E. g., Risinger, D. Saks, M., Thompson, W., & Rosenthal, R. (2002). The Daubert/Kumho Implications of Observer Effects in Forensic Science: Hidden Problems of Expectation and Suggestion, California Law Review, 90, 1-55.
12E. g., Green, M. (2004). Nursing error and human nature. Journal of Nursing Law, 9, 37-44.
13 Dror, I., Peron, A., Hind, S., & Charlton, D. (2005). When emotions get the better of us: The effect of contextual top-down processing on matching fingerprints. Applied Cognitive Psychology, 19(6), 799-809.
14See endnote 9.
15 This is very common in law. For example, statutes often say that a viewer should see whatever is available to be seen. This is unrealistic and ignores all the normal cognitive process of attention, expectation and other cognitive processes discussed here plus much more. See footnote and 5 and Marshall, J. (1966). Law And Psychology In Conflict. New York. Bobs-Merrill
16 Green, M. (2003). Proper Eyewitness Identification Procedures," Law and Order, Oct., 195- 198.
17 E .g., Arvizu, J. (2002) Testimony on Mr. Plaza's motion to exclude the government's latent fingerprint identification evidence, hearing before Judge Louis Pollak, USA v. Plaza, et al., U.S. District Court for the Eastern District of Pennsylvania, February 24,2002.
18 Haber, L. & Haber, R, (2008).Scientific Validation of Fingerprint Evidence Under Daubert, Law, Probability & Risk, 7, Issue 2, pp. 87-109, 2008.
19Ashton, W. Fingerprint Identifications (Science or Not). Retrieved from http://www.4intel.com/images/fingerprints.pdf 2 Feb 09.
Footnotes
*A version of this article was published under the title, "Human Fallibility Woven Into Forensic Science" in For The Defense, 2009.
A division of 2057949 Ontario, Inc.
Copyright © 2024 Marc Green, Ph. D.
Home Page
Contact Us