NAS Recommendation 3: The Research That No One Has Done Yet

The third recommendation in the report from the National Academy of Sciences (NAS) titled “Strengthening Forensic Science in the United States: A Path Forward” states the need to establish a solid foundation of scientific research, validation, techniques, and collaboration quite clearly, so I will simply repeat it here for anyone who has not seen the actual recommendation.

Research is needed to address issues of accuracy, reliability, and validity in the forensic science disciplines. The National Institute of Forensic Science (NIFS) should competitively fund peer-reviewed research in the following areas:

So what does this recommendation mean exactly? And why would any scientist object being held to these standards?

Few organizations, such as the International Association for Identification (IAI) have welcomed the call for more research to further support their fields. Most have insisted that they have already fulfilled this requirement. I know that some forensic scientists honestly believe that they have fulfilled this obligation. There is a communication gap between the world of practicing forensic scientists and the world of research science.

Part A: Establishing the Scientific Basis

I will use my own field, firearms examination, as an example for the first part of this recommendation, establishing a scientific basis demonstrating validity. The professional organization to which I belong, the Association of Firearm and Tool Mark Examiners (AFTE), has its own journal and many people who have done research of varying degrees of scientific validity. Firearm and tool mark identification is an area that is under constant attack from attorneys as being subjective and unsupported and is singled out in the NAS report in a few places as needing more research to support and validate it as truly scientific.

I’ll break it down into three stages of research that need to be carried out for firearm and tool mark identification, some of which have been done quite thoroughly:

1. The Basics:
1. Metals and Metallurgy
1. Crystallization and fracture patterns of metals and their alloys.
2. Metalworking and machining processes used to shape metals and their alloys.
3. How different patterns are made when one metal surface moves across another and what affects those patterns (the machines, the metals, temperature, angle, process type, etc.).
4. Etc.
2. Do the same for other compounds that tools or gun parts may be made from.
2. Firearms and Tools
1. Consecutively manufactured barrels, breach faces, tools, etc.
2. Studies on the effects of use of a firearm or tool.
3. The effects of the markings left by parts of a firearm or tool on different types of surfaces (painted, lacquered, plastic, metal, wood, etc.).
3. External Ballistics (firearms)
1. Where did the bullet go once it left the gun?

There has been a lot of research in the second and third areas, but not in the first. That’s where firearm and tool mark examination is weak and where we need to catch up. Once that is done, Daubert and Frye heartburn will be a thing of the past. The same is true to varying degrees for other fields of forensic science.

Parts B and C: Proving We Know What We’re Doing and Defining “Peer Review”

Establishing quantifiable measures of the reliability and accuracy in realistic case scenarios is absolutely necessary. Pure and simple. Just because the loudest voices in the field weren’t the ones to think of it first does not mean it cannot be done. Before someone has a fit, let me give a qualifier: just because something can be explained scientifically and quantified on one scale does not mean it can be quantified on every scale. For example, if the question is “What color is it?” and the answer is “It’s blue,” there may not always be an answer to the question, “Well, how blue is it?” But you can figure out how many times the same thing comes up blue. And who can tell whether or not it’s blue.

In forensic science “peer reviewed” is very often more like “buddy reviewed.” In the world of research science, “peer reviewed” does not mean “somebody I like and approve of and who thinks like me looked at it and thinks it’s fine.” In research science “peer reviewed” means that anyone who is trained as a scientist (almost any kind of scientist, within reason) can look at your research, understand what you did, and find your results believable and your conclusions and discussion logical. They also expect to be able to reproduce your results without having to be trained by you.

Also, when a journal article is published in, for example, the journal Neurotoxicology, it is unlikely that the author is personal acquaintances with anyone who the journal’s editors choose to review the article before publishing it or that the author would be consulted on who gets to review their article.

Another item that is absolutely crucial is statistics. You cannot just start applying any statistics you want to your research because it will invalidate everything you’ve done and you won’t be able to publish your results in a “respected scientific journal”. Don’t believe me? Go ahead and try. You also have to decide on which of the many types of statistical methods you are going to use before you start your project. It is simply a part of basic research design. It isn’t too difficult, but it is something that you need to look in to before you start your project.

Part D: Automation

While automation can and has help streamline some processes in forensic science there are other areas where its application has been very limited due to the overwhelming number of variables that limit those types of analyses strictly to the interpretation of human examiners. Therefore, we have to work toward making these human observations as objective as possible through training, oversight, and transparency.

Finally, I continue to urge the forensic science community to try to understand the positive intent of the NAS Committee and to take this opportunity to begin a collaborative relationship with the research science community to build our science into what it could and should be.

See all NAS posts.