January 3, 2023 | Mark Paradies

Human Performance Tools: The Good, The Bad, and the Ugly!

Which Human Performance Tools Work and Which Ones Don’t?

You might have heard about something called HU or HuP (Human Performance Tools) or HPT (Human Performance Technology). These are names for a set of tools developed in the nuclear industry to try to achieve zero human errors (or at least improved human performance). I’ve been interested in these tools and the concepts behind them because of my background in human factors (I am a Certified Professional Ergonomist – Human Factors Professional – certificate #85) and my work to find the root causes of human errors.

This article lists the tools and provides a critique of one of the tools to help people understand that despite the hubbub about HU, not all the tools work. In fact, some may actually be counterproductive.

What Are the Human Performance Tools

The HU tools have many variations. Some were developed in the military, especially the Nuclear Navy. The Institute of Nuclear Power (INPO) further developed the tools and promoted their use in the commercial nuclear industry back in the 2000s and 2010s. Since then, they have taken on a life of their own with many variations taught by various consultants. Thus the list below is just a compilation of the different techniques referred to by many consultants. You may have heard of slightly different techniques or learned to use them in a slightly different way; that’s to be expected.

Here is our summary list of HU tools:

Procedure Use/Adherence
Place Keeping
Independent Verification
Three-Way Communications
Pre-Job Brief/Personal Safety Assessment
Observation and Coaching
Post-Job Brief
Concurrent Verification
Questioning Attitude
STAR
Time Out (Stop When Unsure)
Attention to Detail
Management of Change
Error Traps and Precursors
Validate Assumptions
Do Not Disturb Sign
Conservative Decision-Making

That’s a long list. If you have any additions, please post the technique and describe it in a comment below.

Do All These HU Tools Work?

Some of these tools have been promoted for almost 20 years, but I have never seen any comprehensive research to show that they are effective. Therefore, I started assessing them by applying my knowledge of human factors and did reviews of the literature and discovered that some tools do have research that suggests they are effective, while others violate certain human factors principles.

Let’s look at one example that is often cited that violates a human factors principle. The HU Tool is called having a “Questioning Attitude.”

What is a Questioning Attitude? The US Nuclear Regulator Commission Safety Culture Policy Statement defines a Questioning Attitude as:

Individuals avoid complacency and continually challenge existing conditions and activities in order to identify discrepancies that might result in an error or inappropriate action.

US Nuclear Regulatory Commission Safety Culture Policy Statement

This may sound more like a philosophy, but after an incident, investigators often point to a lack of a questioning attitude as a potential root cause. Thus, operators, mechanics, and supervisors are encouraged to have a questioning attitude to prevent accidents.

Questioning Attitude Example

Here is an accident described in US NRC’s publication Safety Culture Trait Talk:

A hospital was conducting a cancer treatment with a high-dose rate brachytherapy remote after loading system using an iridium-192 source. Just prior to the cancer treatment, the hospital had replaced the source and upgraded the software.
When entering the data into the treatment system, the medical physicist was unable to electronically transfer the patient’s treatment plan from the planning system to the treatment system due to an error message. After several failed attempts by staff, the medical physicist entered the treatment plan manually into the treatment systems control console, rather than question why he was seeing the error message.
Due to a bug in the software upgrade, the treatment system software created an unexpected source step size change in the treatment parameters. When the medical physicist entered the data manually for the source dwell times, the software automatically changed the entered data to the default parameters for the source step size. The medical physicist faced an unexpected condition with the software error and failed to recognize the change in the source step size.
The patient was then treated with a mispositioned source.
The medical physicist failed to verify that the treatment computer system was correct after data entry and prior to treatment. In addition, the hospital failed to follow its procedure of performing an independent review of the treatment plan prior to patient treatment.
As a result, the patient received a radiation dose to tissue outside the treatment area and an underdose to the treatment site.
This scenario illustrates equipment (software) errors as the initial precipitating event. Had the medical physicist used a questioning attitude, he could have identified the equipment failure and the hospital could have corrected this failure before treating the patient”

The article clearly states that if the medical physicist had used the Questioning Attitude tool, he could have corrected the software failure and prevented the error in the treatment of the patient.

What do you think? Is the Questioning Attitude tool an effective Safeguard?

Read on, and we will discuss the technique and its advantages and limitations.

When to Use the Questioning Attitude Tool

Of course, it would be nice to exhibit a questioning attitude continually. But to use it as a tool, one must be able to recognize when to apply the tool. Therefore consultants teaching the tool come up with lists of when an operator, supervisor, or mechanic should apply the Questioning Attitude tool. Here is a list I made that combines several different sources (including the Human Performance Improvement Handbook, Volume 2):

When you feel uncertain
When your gut tells you there is something wrong
When something unanticipated happens
When you are using the human performance tool STAR
When making important decisions
When you are confused or in doubt
When you find something missing
When someone says, “We’ve always done it this way.”
When you encounter inconsistencies
Before doing something important (like a critical procedure step)
When plans and procedures don’t agree
When plant conditions are different than you expected
When you encounter something new/different
When you hear: “I assume” or “probably” or “I think …”
When normal work processes fail/don’t produce the expected results

That’s a nice list.

Peter Parker (Spider-Man) would add, “When my spidey senses are tingling!“

When do we recommend having a Questioning Attitude? Always!

The only time we are sure you will have had a Questioning Attitude is AFTER an accident because:

Hindsight is 20/20.

And that is the secret to this human performance tool.

What’s Wrong With the Questioning Attitude Tool

After an accident, it is easy to point out where you should have noticed an error, asked a question, or doubted your senses. But, in the heat of battle, while the clock is ticking, when you have lots to do and only eight hours to do it, spotting all the errors and correcting them is NEVER a sure thing. Thus, you are doomed by fate to be caught without your Questioning Attitude when you need it the most. That’s why it is easy for the Inspector from the NRC will say that the medical physicist should have had a questioning attitude but difficult for the medical physicist to detect that he should have had a Questioning Attitude at that moment.

And, of course, people sometimes do catch and prevent accidents just in the nick of time.

Once, when I was in the Nuclear Navy on a Nuclear Powered Cruiser, I woke up in the middle of the night and had an uneasy feeling that made me get up, get dressed, and go down to the Auxilary Machinery Room. Sure enough, I found a major issue and stopped work before anyone was hurt. That was a Questioning Attitude while I was asleep!

But that amount of Questioning Attitude isn’t normal. And I don’t think it can be trained.

Let’s look at why it is so difficult to realize that you need a Questioning Attitude just in time to stop an accident.

Monitoring and Detecting an Error

FIRST, to make a Questioning Attitude work, people need to monitor their work environments continuously and proactively detect errors and stop accidents from happening. In some cases, this happens. Workers catch mistakes and prevent accidents. But how reliable are people at continuously questioning, challenging assumptions, and detecting anomalies? That depends.

Research shows that the brain consistently makes corrections to fill in blank spots and make sense of what we see. This happens literally when the brain fills in the blank spot in your vision (see Being Wrong, Adventures in the Margin of Error, Kathryn Schulz, Harper-Collins Publishers, NY, 2010).

I’ve tried to find reliable human factors research that shows that humans can be trained to deny their human nature and become robotic, error-proof error checkers that constantly check and recheck the accuracy of their actions and the actions of those around them, but I can’t find any such research.

What can I find? The brain can detect errors. It senses when something is wrong. But there is a problem. Humans aren’t consistent. For example, they catch one typo and miss the next.

In research on detecting spreadsheet errors (yes, people are paid to detect errors in spreadsheets), several researchers found that the odds of individual inspectors finding a seeded error were 50% to 60%. (See “Does Spell-Checking Software Need a Warning Label?” Communications of the ACM 48(7), pp. 82-86, July 2005, by D.F. Galletta, A. Durcikova, A. Everard, and B. Jones and Human Error Website by Ray Panko, 2007.)

A sample of the data he presented about catching errors includes:

Work – Error Detection Rate in %

Postal workers typing ZIP codes – 50%
Programmers finding coding errors – 43%
Nuclear workers detecting execution errors in a simulator exercise – ~ 66%
Nuclear workers detecting misdiagnosis errors in a simulator exercise – 0%

Perhaps those nuclear operators in a simulator needed a better questioning attitude!

In the paper, Thinking is Bad: Implications of Human Error Research for Spreadsheet Research and Practice, Ray Panko writes:

The second fundamental thing to understand from human error research is that making large spreadsheets error free is theoretically and practically impossible and that even reducing errors by 80% to 90% is extremely difficult and will require spending about 30% of project time on testing and inspection. Trusting “eyeballing” and “checking for reasonableness” to reduce errors to a reasonable level flies in the face of a century of human error research and is a testament to the profound human tendency to overestimate their ability to control their environment even when they cannot.

Yes, you can catch some errors. Yes, it is good to try to catch all the errors. You may even be good at it (perhaps 66%). But you won’t be perfect. And when you are imperfect, someone will say:

“You should have had a questioning attitude.”

So, have a Questioning Attitude but don’t think of it as a magic error reduction tool.

Use your Questioning Attitude on anyone who tells you that Questioning Attitude is a reliable error reduction technique. Question their belief that a Questioning Attitude is a foolproof tool to stop human errors.

Assessing a Questioning Attitude

SECOND, one more problem with implementing the Questioning Attitude tool. Management and supervisors can’t assess the use of a Questioning Attitude because they don’t know when the worker should be actively questioning something. Penalizing someone for not having a Questioning Attitude after an incident looks like and is … BLAME.

So, we recommend that people have a Questioning Attitude, but we don’t see this as a foolproof tool to stop human errors.

It’s a Philosophy

FINALLY, some people will say that a Questioning Attitude isn’t a tool but rather is more of a philosophy or an attitude that people should always have. To that, I respond that it doesn’t make any difference what you call it (tool, attitude, or philosophy); it is still just as ineffective as a Safeguard to stop errors or incidents.

Stop Human Errors

If you find the discussion above interesting and you would like to learn more about improving human performance, I would suggest attending this course:

Stopping Human Error

To see the next scheduled public Stopping Human Error Course, CLICK HERE.

Why should you attend the course? Ask these questions:

Do you want to achieve excellent human performance?
Would you like to understand the methods you can apply to effectively stop major accidents and incidents by “stopping” human error?
Do you need to understand what are the most effective human performance improvement techniques/tools and which ones are counterproductive (yes, some techniques don’t work and may cause people to hide errors)?

If any of the above questions received a “Yes!” answer, you should attend the Stopping Human Error Course.

The instructors will help you understand:

The causes of human error
Human factors design best practices
Methods to find error-likely situations
CHAP (Critical Human Action Profile)
Human Performance improvement Technology (Hu – Human Performance Tools)
Designing your human performance improvement program.

You will leave this course with a clear understanding of methods to improve human performance and a plan to apply those methods at your company to achieve great gains in safety, quality, or operational and maintenance performance (all of which depend on human performance).

Participants will also receive the book, Stopping Human Error, a $99.95 value, as part of the course materials. Participants will also receive a certificate of completion and a 90-day subscription to TapRooT® Software, our dynamic cloud-based software that computerizes the TapRooT® Root Cause Analysis Techniques.

See the schedule for upcoming public Stopping Human Error courses HERE. Or contact us to schedule a course at your site.

One more idea to help you improve human performance. Attend the 2023 Global TapRooT® Summit. You can attend the Stopping Human Error Course on April 24-25 and then attend the 2023 Global TapRooT® Summit on April 26-28 (both being held near Houston, TX, at the Margaritaville Lake Resort) and SAVE $300 off the course price.