May 5, 2021 | Mark Paradies

Measuring Safety/Quality Improvement

Why TRIR is a Poor Measure of Safety and Other Considerations When Measuring Safety and Quality

People often use the Total Recordable Injury Rate (TRIR) as a way to measure and compare safety performance. The problem with the TRIR is that it is not a very reliable measure and the better your safety program is, the less there is to measure. That problem was highlighted in an article published by the CSRA (Construction Safety Research Alliance) and in Process Safety (a journal of the American Society of Safety Professionals) titled: “The Statistical Invalidity of TRIR as a Measure of Safety Performance.”

My goal here isn’t to repeat the discussion in the article. You can read that at the link above. However, key points of the article are (just to name a couple):

TRIR is not associated with fatalities. Why? Because many (or most?) TRIR injuries could never become a fatality and therefore aren’t indicative of future fatalities or serious injuries.
TRIRs are almost totally random. They don’t occur at repeatable intervals.

However, we have seen companies significantly reduce both their fatalities/serious injuries rates and their TRIR and have seen other companies that have a bad record of fatalities/serious injuries and TRIR.

Thus, there must be measures and ways to trend infrequently occurring safety statistics (and other statistics, for example, some types of quality statistics).

First, Develop Your Metrics

At almost every company, trending fatalities is not a good measure. Hopefully, they hardly ever occur. Plus, having a fatality is a poor way to be alerted that safety has declined.

For any negative performance indicator (safety or quality) you may measure a negative outcomes indicator (fatalities, customer complaints, accidental environmental releases, …), but a better indicator would be leading (not lagging) and predictive (not results of past performance – thus, reactive).

Managing a business by using lagging, reactive measures is like driving by looking in the rear-view mirror. Yes, you should occasionally look in the rear-view mirror, but, as they say in the financial industry:

Past performance is not indicative of future results.

So, you must know the system you are trying to improve (safety performance, quality results, environmental performance, …) and then find measurable indicators of future performance.

In the book, Performance Measures and Trending for Safety, Quality, & Business Performance, the following proactive performance measures (leading and proactive) are suggested as possible measures for safety and quality performance:

SAFETY

Pre-job safety meetings conducted divided by the number of jobs completed in a month
Percent of safe work behaviors observed per total behavior observations in a month
Safety budget
Number of hours spent performing safety audits/observations per month
Number of safety improvements implemented per month
Results (score) of an employee safety attitude survey
Total hours spent in safety meetings (including toolbox safety briefs) and pre-job safety planning per 200,000 hours worked
Number of precursor incidents investigated

QUALITY

Number of quality corrective actions implemented per month
Number of quality improvement projects started per month
Number of quality improvement projects completed per month
Quality improvement budget
Number of management hours spent on quality improvement per month
Number of quality audits performed per month

Note that these indicators may or may not work at your facility. It depends on the state of your program and the processes that you have in place.

Once again, the book quoted above says:

In some industries, there are no adequate measures. For example, in the late 1990s, the Institute of Medicine issued a report: To Err is Human: Building a Safer Health System (November 1999). In the report, they estimated that as many as 98,000 people per year may be killed by medical errors. Why did they have to make an estimate of the number of people killed by medical errors? Because there is no standard for or reporting of deaths by medical error.

You might ask … if there isn’t a reliable outcome measure, how do patient safety advocates know if patient safety is getting better or worse? That’s an excellent question.

An idea for developing hospital measures is proposed on page 19 of the book, Performance Measures and Trending for Safety, Quality, & Business Performance.

For more ideas about developing measures, watch this video: