Management Practices Demonstrating the
Value of the Profession

Introduction

A study was designed to identify components of management systems that have demonstrated the value that Industrial Hygiene and Occupational Health (IH/OH) program practices can have on an organizations business results. A survey instrument was distributed by ORC to its participating companies late in 2007. Twenty-four companies responded. To identify these components, a survey instrument was developed and based on the Malcolm Baldrige Award Criteria for Performance Excellence (Criteria).

The Criteria are a set of empirically-derived, model business approaches pulled together into a system that represents the best practices of many of the best performing companies in the world. They form a comprehensive, complete business architecture and are looked upon internationally as a strategy for sustained performance excellence. They have also been refined and improved over their 20-year existence as more effective business approaches are demonstrated.

The National Institute for Standards and Technology (NIST) manages the Malcolm Baldrige National Quality Award program for the Department of Commerce. This award program recognizes those few excellent businesses that can demonstrate to a group of national award examiners that the implementation of the first six categories of Criteria produce the Business Results required in Category seven. The award is about excellent results using recognized best systems. A company cannot win the award without demonstrating sustained excellent business results through sustained implementation of the Criteria.

For this study, the Criteria served as an architecture upon which to base survey questions. The Criteria comprehensively covers all aspects of a business and would therefore catch most opportunities to identify the demonstrated value that Industrial Hygiene and Occupational Health (IH/OH) program practices can have on an organizations business results.

The survey questions dealt specifically with an organizations IH/OH focus through the lens of these Criteria. The aim was to examine the relationships between Business Results and the six Baldrige Criteria areas:

  • Leadership
  • Strategic Planning
  • Customer Focus
  • Management, Analysis and Knowledge Management
  • Workforce Focus
  • Process Management

Executive Summary Results

In general, the analysis of the survey responses showed a positive relationship between selected yes answers to Leadership, Strategic Planning, Customer Focus, and Workforce Focus questions and yes answers to Business Results questions.

Specifically, the following relationships were statistically significant, i.e., high numbers of Yes answers to these questions corresponded to high numbers of Yes answers to Business Results questions. In other words, there was a statistically significant relationship between the following conditions and improved business results.

Leadership: When senior leaders regarded a productive IH/OH program as a competitive advantage.

Strategic Planning: When the IH/OH program made performance improvement projections as part of the strategic or action planning process.

Customer Focus: When external customers required evidence of a specific level of IH/OH program performance or proficiency.

Workforce Focus: When (1) IH/OH considerations were included as part of employee (including leadership) development, and (2) the company used IH/OH programs as a tool to recruit, hire and retain employees.

Although a direct cause and effect relationship cannot be statistically determined with limited data available, it appears that companies that pay particular attention to IH/OH programs and practices through the aspects listed above in leadership, strategic planning processes, customer issues, and workforce development are more likely to see improved business results than those companies that dont.

Methods

Data Collection and Analyses

Seven business aspects related to Industrial Hygiene and Occupational Health programs were measured. Each was measured using 8 questions: 4 of the questions required a Yes/No response, and another 4 questions were aimed at a quantitative measure of each Yes/No response. Unfortunately, the number of missing answers to the quantitative responses was too high to examine relationships using the quantitative measures in the analysis.

Three distinct statistical analyses were used to examine relationships between the Business Results responses and the responses associated with Leadership; Strategic Planning; Customer Focus; Management, Analysis and Knowledge Management; Workforce Focus; and Process Management. Three analyses were used in order to obtain three different perspectives on the significance of the results. The 24 respondents offer a reasonable picture of the population of companies and insights into IH/OH practices. However, the sample size of 24 companies presented challenges when attempting to use all 48 predictive components of Business Results. For that reason, the predictive variables were combined for two of the analyses conducted.

Analysis 1: Test of Proportions

The first analysis was a hypothesis test to determine if the proportion of Yes Answers to each of the questions posed was the same as the proportion of yes answers to each Business Results question. Assume that i is a placeholder for each of the 24 questions asked in the survey, and j a placeholder for each of the 4 Business Results questions. Ninety-six hypothesis tests were run:

The test was run using Fishers exact test, and any test with a p-value < .05 was highlighted. [Note that with 96 tests, the average number of tests wrongly identified as significant would be about 5].

The results of this test are shown in Table 2. It shows the p-value of each of the 96 hypotheses tested. When the test showed a statistically significant difference, it practically means that the proportion of Yes answers of Business Results changed depending on the proportion of Yes answers of the dimension tested. The results from these tests showed that when answers to the questions were No, Business Results answers were typically No. However, when the answers to the questions were yes, the answers to the Business Results questions were just as often Yes as No, and sometimes more often Yes. These relationships are shown in Figures 1 and 2.

Analysis 2: Regression Analyses

The purpose of the regression analysis was to determine if knowing information about each of the Baldrige categories (e.g., Leadership, Strategic Planning) could help to predict improved Business Results. This set of analyses required a compilation of the questions from each dimension surveyed. Rather than examining each question separately, each dimension compiled all 4 of the Yes/No questions by taking the average number of yes responses, and comparing it to the average number of yes responses for Business Results. It was necessary to reduce the number of independent variables because of the limited sample size of the survey. Note that this analysis offers insights into potential relationships. However, the analysis violates some underlying assumptions that should be in place to ensure the statistical rigor of regression, and results should be considered in that light. Based on scatter plots of the data, a quadratic strategy was fit for the following relationships.

Two hypothesis tests were conducted for each strategy:

and p-values are shown for each strategy. When these tests were significant (p < .05), the strategy was constructed and the relationship described with a graph and the strategy listed. In addition to the strategy, prediction intervals were constructed for the strategies.

There are two key take-aways from this analysis. The first is that there were several significant relationships, i.e., knowing the proportion of Yes responses to the dimensions allows prediction of the proportion of Business Result Yes responses. The second takeaway is that even though there are statistically significant strategies, examination of the prediction intervals shows that the preciseness of prediction is limited. This is a function of sample size as well as underlying variability in the prediction.

Analysis 3: Analysis of Variance

The final analysis tested whether the proportion of Yes answers to BR questions changed depending on the proportion of Yes answers seen on the survey for each dimension measured. The following hypothesis test was conducted for each of the six dimensions measured in the survey:

H0:

%BR Responses when 0% yes responses for dimension in study =
%BR Responses when 25% yes responses for dimension in study =
%BR Responses when 50% yes responses for dimension in study =
%BR Responses when 75% yes responses for dimension in study =
%BR Responses when 100% yes responses for dimension in study

H1: at least 1 inequality

When this test was significant, the means that were different were identified at the end of the analysis.

Survey Questions

Below is a list of the survey questions used. Also below is Table 1 that links the survey question number (e.g., 1, 7, 12, 18) to the Baldrige category and question number within the category.

Leadership Questions

  1. Have senior leaders implemented a company-level policy/other guidance to support the IH/OH program?
  2. Is there a senior leadership sponsor for the IH/OH program?
  3. Do senior leaders regard a productive IH/OH program as a competitive advantage?
  4. Does senior leadership include IH/OH objectives (e.g. safe products and processes), as part of the communicated organizational vision and ethics expectation?

 

Strategic Planning Questions

  1. Is the development of IH/OH strategic objectives part of your company's strategic planning process?
  2. Do your IH/OH strategic objectives focus on meeting one or more strategic challenges?
  3. Are specific action plans deployed to the organization to ensure the achievement of IH/OH strategic objectives?
  4. Does your IH/OH program make performance improvement projections as part of the strategic or action planning process?

 

Customer Focus Questions

  1. Do any company external customers require evidence of a specific level of IH/OH program performance or proficiency?
  2. Do the IH/OH program functions use systematic approaches for listening to and learning from their internal customers in order to meet changing customer requirements?
  3. Does the IH/OH program perform annual or other assessments of the health and safety hazards of the company?
  4. Are any IH/OH program function metrics systematically monitored to ensure the function is consistently meeting customer requirements?

 

Measurement, Analysis and Knowledge Management Questions

  1. Do IH/OH program functions align their performance measures with higher-level organizational performance measures that indicate achievement of IH/OH strategic objectives?
  2. Do IH/OH program functions have a systematic approach for analysis of their performance against action plans?
  3. Do IH/OH program functions utilize both leading (process) and lagging (outcome) metrics?
  4. Do IH/OH program functions actively utilize the best practices identified by IH/OH and other functions in the company?

 

Workforce Focus Questions

  1. Does your company use its IH/OH programs as a tool for increasing worker engagement?
  2. Are IH/OH considerations included as part of employee (including leadership) development?
  3. Does your company use IH/OH programs as a tool to recruit, hire and retain employees?
  4. Does your company use IH/OH programs as a tool to maintain a safe, secure and supportive work climate?

 

Process Management Questions

  1. Does the IH/OH program use a systematic, written procedure for assessing a potential internal hazard?
  2. Do different IH/OH program functions systematically share lessons learned?
  3. Are IH/OH program functions systematically audited for consistent use of approved procedures?
  4. Does the IH/OH program have a corrective action system?

 

Business Results Questions

  1. Is IH/OH program performance objectively linked to product or service cost reductions?
  2. Is IH/OH program performance objectively linked to increases in employee productivity?
  3. Is IH/OH program performance objectively linked to customer satisfaction with safer products or services?
  4. Does your company use IH/OH achievements (e.g. dust, toxin, mold elimination) as evidence of being corporately responsible?

 

These questions have been labeled to facilitate the analysis described in the objective session. The labeling scheme is shown below in Table VI-1.

 

Significant p-values are highlighted in yellow. The results from this analysis suggest that questions L3, SP8, CF9, W18 and W19 are linked to Business Results with statistical significance. The charts in Figure VI-1 and Figure VI-2 show that when these predictors are answered No, the vast majority of companies answered No to the Business Results questions as well. However, when the answer to these predictors was Yes, some of the Business Results questions were answered predominately Yes, and some of the Business Results questions were answered Yes a higher proportion of the time. A possible conclusion is that Yes answers to the predictor questions do not necessarily guarantee a Yes answer to the Business Results questions, but you cannot achieve Yes answers to the Business Results questions unless you answer Yes to the predictor questions. For example, question SP8 is,


Regression Analysis

In order to understand the potential contribution of each of the categories from the IH/OH-based Baldrige assessment on Business Results, a metric was calculated to create a single numeric response for each of the six categories. This metric measured the average number of yes responses for each dimension by each organization.

Each category Leadership; Strategic Planning; Customer Focus; Measurement Analysis and Knowledge Management; Workforce Focus; and Process Management - was measured in the survey by four yes or no questions. The number of yes answers was counted for each organization that responded for the dimension measured, and the proportion of yes answers was calculated

As an example:

  • L1=yes, L2=yes, L3=no, L4=yes: L =.75
  • L1=no, L2=no, L3=no, L4=no: L = 0
  • L1=yes, L2=yes, L3=yes, L4=yes: L = 1

Seven new metrics were created:

  • L = proportion of yes answers for each of the 4 Leadership questions
  • SP = proportion of yes answers for each of the 4 Strategic Planning questions
  • CF = proportion of yes answers for each of the 4 Customer Focus questions
  • M = proportion of yes answers for each of the 4 Measurement, Analysis and Knowledge Transfer questions
  • W = proportion of yes answers for each of the 4 Workforce Focus questions
  • PM = proportion of yes answers for each of the 4 Process Management questions
  • BR = proportion of yes answers for each of the 4 Business Results questions

The relationship between each of the Baldrige categories and Business Results is shown in Figure 4. Figure 3 is presented to illustrate the relationships observed in the survey.

The data point highlighted in the blue circle is organization 7, which answered yes to 1 of the 4 Business Results questions, and yes to 2 of the 4 Leadership questions.

The data points highlighted in the red circle are organization 3, 18 and 22, which each answered yes to 4 of the 4 Business Results questions, and yes to 4 of the 4 Leadership questions.

It appears that organizations that answered No to the dimensions measured also answered No to the Business Results responses, and organizations that answered Yes to the dimensions, had varied Business Results responses.

In order to evaluate the influence that each of the categories had on Business Results, a regression analysis was performed. Each dimension was used to predict Business Results using a quadratic fit. In addition to testing whether the regression was statistically significant, prediction limitswhich represent a range that a single new observation is likely to fall given specified settings of the predictor (the specific dimension in the analysis)were constructed for the strategy. An examination of the prediction limits for each of the analyses show that even though there is a statistical relationship, knowing the level of support expressed does not definitively yield the Business Results response. Results follow.

Analysis of Variance

An Analysis of Variance was conducted that tested whether the proportion of Yes answers to BR questions was the same for each level of the proportions of Yes answers for each of the six categories measured in the survey. Results from those analyses follow:

Business Results as a function of Leadership (L)

 

This analysis shows that when companies answered yes to all 4 of the Leadership questions, there was a higher proportion of yes answers to Business Results questions than when 3 questions were answered yes, and when 1 was answered yes.

Business Results as a function of Strategic Planning (SP)

 

This analysis shows that when companies answered yes to all 4 of the Strategic Planning questions, there was a higher proportion of yes answers to Business Results questions than when 3 questions were answered yes.

Business Results as a function of Customer Focus (CF)

 

This analysis shows that when companies answered yes to all 4 of the Customer Focus questions, there was a higher proportion of yes answers to Business Results questions than when 2 questions were answered yes.

Business Results as a function of Measurement, Analysis and Knowledge Management (M)

This analysis showed no difference in Business Results answers when Measurement, Analysis, and Knowledge Management questions varied.

Business Results as a function of Workforce Focus (W)

 

This analysis shows that when companies answered Yes to all 4 of the Workforce Focus questions, there was a higher proportion of Yes answers to Business Results questions than when 3, 2, 1 or 0 questions were answered Yes. It also shows that when 3 Workforce Focus questions were answered Yes, the proportion of Yes answers to Business Results questions were higher than when 2 or 1 Workforce Focus questions were answered Yes.

Business Results as a function of Process Management (PM)

This analysis showed no difference in Business Results answers when Process Management Questions varied.

Conclusions

It is difficult to demonstrate with statistical rigor, with the limited data available, that a single Baldrige category of Leadership; Strategic Planning; Customer Focus; Measurement, Analysis, and Knowledge Management; Workforce Focus; or Process Management influences Business Results. However, there were specific questions in some categories (the predictors listed below) that did appear to influence Business Results responses. When these predictors were answered No, Business Results responses were predominately No. Only when these predictor questions were answered Yes was there a reasonable chance of Yes answers to the Business Results questions. There were no statistically significant correlations between Process Management responses and improved Business Results, or Measurement Analysis and Knowledge Management responses and improved Business Results. Although this is true when looked at in the light of IH/OH programs, it may be different if other programs, such as machine maintenance, budget planning, or business development were analyzed.

Predictors

Leadership

  • Do senior leaders regard a productive IH/OH program as a competitive advantage?

Strategic Planning

  • Does your IH/OH program make performance improvement projections as part of the strategic or action planning process?

Customer Focus

  • Do any company external customers require evidence of a specific level of IH/OH program performance or proficiency?

Workforce Focus

  • Are IH/OH considerations included as part of employee (including leadership) development?
  • Does your company use IH/OH programs as a tool to recruit, hire and retain employees?

Response

Business Results

  • Is IH/OH program performance objectively linked to product or service cost reductions?
  • Is IH/OH program performance objectively linked to increases in employee productivity?
  • Is IH/OH program performance objectively linked to customer satisfaction with safer products or services?
  • Does your company use IH/OH achievements (e.g. dust, toxin, mold elimination) as evidence of being corporately responsible?

Yes answers to the Business Results questions mean the respondents indicated that, for their companies, IH/OH program performance is objectively linked to cost reductions in products or services, increases in employee productivity, and customers being satisfied with safer products or services. As well, IH/OH program achievements are used as evidence of being corporately responsible.

The executive summary for this section states that it appears that companies that pay particular attention to IH programs and practices through leadership, strategic planning processes, and customer and workforce focus are more likely to see improved business results than those companies that do not. More specifically, only when the five predictors were answered Yes was there a reasonable chance of improved Business Results.

So what is the studys suggestion for the IH/OH program managers if they want to demonstrate their programs value to the company and the profession?

Based on this study, a Malcolm Baldrige Award Examiner could suggest the following with confidence.

Ensure that Senior Leaders regard a productive IH/OH program as a competitive advantage. Sustaining success is most generally supported at the top. This means that we could expect to see IH/OH program metrics in proposals submitted for contract work or in publications to stockholders and other stakeholders. Senior leaders in this context include the head of an organization and their direct reports.

During the strategic planning process, the IH/OH programs should develop performance improvement projections as part of their action planning activities. This may tend to solidify commitment to their achievement.

Listen to your customers. One might expect that when a company regards its productive IH/OH programs as a competitive advantage, that its external customers (or at least some of them) also value IH/OH program performance or proficiency. Expect improved business results when customers expect higher levels of IH/OH program performance.

Use productive IH/OH programs as a tool to recruit, hire and retain employees, as well as a tool for employee development. Among all the Baldrige Categories, Workforce Focus was most highly correlated with improved Business Results. Hiring employees that value productive IH/OH programs can strengthen a workforce already exhibiting a culture with a focus on productive industrial hygiene and occupational health.

Recommendations

As with many studies, additional research with larger sample sizes (even if the same survey instrument is used) would yield stronger correlations and possibly cause and effect relationships. With only 24 respondents this study was able to show results with statistical significance, yet half of the survey questions had to be set aside due to the absence of responses for these questions.

It may also be of value to correlate the six criteria categories to other types of business results such as stock value, stock market stability, and company growth. Examining in detail the demographics of certain industries and the responses to the questions on the instrument used for this study may show stronger correlations for specific industries such as chemical, service, or manufacturing.