Tuesday, May 26, 2009

Success?

How does an agency like OSHA measure success? Maybe the better question is: How should an agency like OSHA measure success?

This whole discussion is predicated on the idea that we must provide some measure of our efforts. Some people will make the argument that we should just do our jobs and not waste time measuring what we're doing, after all, the resources used to make those measurements could be used to do more inspections. While I am certainly sympathetic to that argument, I'm also enough of a pragmatist to recognize that Congress won't let us get away with that, nor should they I suppose. As someone who pays taxes I expect departments and agencies like EPA, or CDC, or DOE to justify their existence, and OSHA should be no different.

As long as we must provide some measure of our success, the question remains: How?

Historically we've used several measures, although everyone seems to fixate on the number of inspection we do every year. Personally, I've aways found inspection numbers amusing. Not necessarily a great measure of success, yet at the same time, not something to get too worked up about. We're all familiar with the end of the fiscal year ritual, towards the end of August the AD looks at the total inspection numbers for the office, sees that we're short, and he goes running around the office screaming "the sky is falling, the sky is falling." We, of course, then scatter like cockroaches when the light is turned on and start looking for construction sites. Some people refer to this as the "great lie," but here's the thing, we're inspecting a seasonal industry (at least in the northern states) that has some of the highest fatality and injury rates out there, at a time when they are at their greatest production. How is this a bad thing?

Local police departments will often use what are known as "saturation patrols" in areas where crime rates may have had a sudden spike. These police patrols don't necessarily produce a lot of arrests, but the increased presence has been shown to reduce crime in that area (at least temporarily). To me, the end of the year construction surge is, at the very least, a lot like saturation patrolling. Most construction companies aren't run by stupid people, and many, especially the bigger ones or those that have been around for awhile, have figured out our end of summer surge and are more conscientious about safety at that time of year. I ask again, how is this bad thing? Certainly it's unfortunate that they don't keep that level of awareness up all year, but even short term improvement is still improvement.

So should we get away from total inspection numbers? I wouldn't mind it and I'm sure there are ADs who would like too as well, but inspection numbers are easy for Congress to understand and the unions point to them all the time, so I don't see them going away.

What about using BLS data? It seems to me that this is a much better measure of success, after all our goal is to reduce fatalities/injuries/illnesses. But, as many people ave pointed out, the data is not necessarily accurate. If the data was accurate? The unions wouldn't want us using BLS numbers because that would suggest that business is making progress (regardless of whether or not you think OSHA had a hand in it or not), and then what would they rally against? Please don't construe this as anti-union, it's not (I'm in the union), but I do recognize that labor and management will use anything they can find against each other (sad as that may be to those of us caught in the middle).

The people at the OSHA Underground don't want us using inspection numbers, and the Pump Handle thinks the BLS data is worthless, so now what?

Keeping in mind that we MUST provide some sort of measurables, I ask you: How do we measure our success?

6 comments:

  1. The use of the data collected for the Site Specific Targeting Program might be a starting point since the same sic codes are used most of the time A comparision of older establishment lists with newer lists could help establish repeats and provide an indicator of improvement.
    Just a thought....

    ReplyDelete
  2. OSHA success may be so small that it is hard to measure! No one high in OSHA wants to measure this because it might cause the OSHA Administrators to have a very red face. Granted, workplace safety has improved over the decades but it might be a tough sell if the results of OSHA efforts are singled out from other workplace safety forces such as the high cost of workplace accidents from worker's comp rates, lawsuits, etc. So, I don't think OSHA management really wants the answer to this question cause they might not like it and then they would be put in the position of having to defend the OSHA program. For now, OSHA management wants to let sleeping dogs lie and to not disturb them.

    ReplyDelete
  3. That's fine (original post) but if you don't measure then you CANNOT make any claims that they Agency is doing well NOR doing poorly.

    ReplyDelete
  4. The problem with using inspection numbers as a metric of productivity is that from the CSHO's perspective, it diminishes the incentive to do complex work. A drive by of a constructions site counts as much as a refinery fatality.

    ReplyDelete
  5. That's absolutely true in an office where CSHOs are held accountable for so many numbers, but theoretically that doesn't happen (yeah, right). Certainly the reality is that some ADs do still count CSHO inspections, although my doesn't.

    As long as inspection numbers are limited to AO totals, I think we can still do complex work and get credit for it, which I personally think are professionally more interesting and prefer to a standard fall protection or trench drive-by.

    Having said that, I'll ask again, if we don't use numbers, what do we do? As the previous commenter pointed out, if we don't measure what we do we can't claim we're making a difference.

    ReplyDelete
  6. Wayne Gray and John Scholz have published research indicating that OSHA inspections do make a difference. You can check out their paper "Do OSHA Inspections Reduce Injuries? A Panel Analysis." Also "Inside the Black Box," co-authored by Gray and John Mendeloff, which was published in the March 2005 issue of Law and Policy.

    ReplyDelete