An update on SOII undercount research activities

The Survey of Occupational Injuries and Illnesses (SOII) is an annual employer survey conducted by the U.S. Bureau of Labor Statistics (BLS) and is the only comprehensive national source of data on the number and rate of occupational injuries and illnesses in the United States. This article updates stakeholders on the ongoing BLS-sponsored research into the completeness of the occupational injury and illness counts from the SOII. Previous updates on SOII undercount research activities include “Examining evidence on whether BLS undercounts workplace injuries and illnesses” by John W. Ruser in 2008¹ and “Examining the completeness of occupational injury and illness data: an update on current research” by William J. Wiatrowski in 2014.² Ruser discusses work by BLS research economists and external researchers who matched SOII microdata and workers’ compensation records from several states so they could evaluate the completeness of the SOII injury and illnesses counts. A short time after publication of that article, BLS received additional funding from Congress to establish an ongoing research program to systematically investigate the completeness of the SOII injury and illness counts and address concerns about an undercount of occupational injuries and illnesses.

BLS began by partnering with three states and one contractor to fund three research projects from 2009 to 2012:

1. Multisource enumeration—Using multiple data sources (including SOII microdata, workers’ compensation records, and hospital and emergency room data) to compile a comprehensive count of all work-related amputation and carpal tunnel syndrome cases across 2 or 3 years
2. SOII–workers’ compensation matching—Matching SOII data and workers’ compensation records to build on previous case matching research
3. Employer interviews—Conducting a small number of interviews with employers regarding their injury and illness recordkeeping practices

Researchers from both BLS and Washington State conducted the employer interviews, which resulted in published reports.³ While the interviews were qualitative in nature, they generated considerable interest from BLS, our research partners, and other SOII stakeholders for quantitative data on injury and illness recordkeeping practices drawn from a larger sample of employers. A detailed discussion of all three projects is beyond the scope of this article but can be found in a Monthly Labor Review article published in 2014.⁴

To fulfill the demand for quantitative data on employer injury and illness recordkeeping practices, BLS partnered with four states to conduct a much larger number of employer interviews from 2012 through 2014. Unlike the qualitative data from interviews conducted from 2009 to 2012, the data collected from the 2012–14 interviews are generalizable to all employers in each of the four states and provide BLS with important information on employer injury and illness recordkeeping practices and barriers to the reporting of injuries and illnesses to the SOII.

In addition to conducting employer interviews as a partner in the four-state study, BLS funded two state-specific research projects in Washington State from 2012 through 2014. The first was a multiyear match of SOII microdata to workers’ compensation records to identify and evaluate any undercount trends over time. The second project involved a series of hypothesis-generating interviews with employers in Washington. In these interviews, researchers documented various reasons employers gave for not reporting to the SOII specific injury and illness cases with days away from work despite the existence of corresponding workers’ compensation claims for these injuries and illnesses.  

The rest of this article focuses on the employer interviews from the four states and the two Washington research projects. It concludes with a brief discussion of current and future undercount research projects.

Employer interviews across four states

In the fall of 2012, BLS entered into cooperative agreements with the following four State Workforce Agencies for the expanded employee interview project:

·       The Minnesota Department of Labor and Industry

·       The New York State Department of Health

·       The Oregon Department of Consumer and Business Services

·       The Washington State (hereafter, “Washington”) Department of Labor & Industries

To facilitate communication and cooperation throughout the duration of the project, BLS and the states arranged for regular conference calls. Regular communication between all participants in the project was vital for development of the survey instrument and for collaborative problem solving and sharing of ideas as the project advanced from the initial survey development into data collection, and then estimation and tabulation of the collected data.

Development of the survey instrument

From fall 2012 to spring 2013, BLS and the states worked together to develop a survey instrument to be used by all four states when conducting the employer interviews. A draft survey instrument, similar to the one developed for the qualitative employer interviews from round 1, served as the starting point. The survey instrument was revised over the course of several months in late 2012 into early 2013. Following pretesting by all states in spring 2013, the survey instrument was finalized that May.

The final questionnaire consisted of 47 core questions asked of employers in each state. Most were yes/no or multiple-choice questions. Minnesota, New York, and Washington also had a small number of state-specific questions of interest that were included on their respective survey instruments. The final survey instrument was divided by topic into six sections, with questions on

·       company/establishment characteristics;

·       employee roles within the company;

·       injury and illness recordkeeping, in general;

·       Occupational Safety and Health Administration (OSHA) recordkeeping;

·       SOII recordkeeping; and

·       workplace practices and recordkeeping scenarios.

Data and sample selection

BLS provided state-specific microdata files with SOII respondent information to each state. From this file, each state drew a representative sample of employers to contact according to guidance provided by BLS. To ensure comparability to the SOII, each state’s sample was stratified by ownership, industry, and establishment size according to the definitions in use by BLS on the SOII. Two states—Minnesota and Oregon—elected to sample unique respondents from the 2010 and 2011 SOII in order to draw from a larger pool of employers. Washington and New York sampled from 2011 SOII respondents only.

Conducting the interviews

States began contacting respondents in the late spring or early summer of 2013. Employers were notified of their selection in this survey, and were encouraged to participate, via a prenotification letter sent by postal mail (or by email, if available) from the state to the SOII point of contact for each employer. Employers are required by federal law to participate in the SOII if contacted by BLS, but participation in this followback survey was voluntary.

After employers received the notification letter or email, states began contacting the employers to recruit their participation in the study and to complete an interview. States made a minimum of three contact attempts to employers, varying the day of the week and time of day of the contact. If the individual listed as the contact was no longer with the employer, states attempted to locate and interview his or her replacement. A typical interview took 25–30 minutes to complete. All interviews were conducted over the telephone. All states completed interviews with participating employers by early 2014.

Employer responses

In total, the states contacted or attempted to contact over 6,000 establishments that were selected to participate in this survey.⁵ Response rates reported by Minnesota, New York, and Washington were around 50 percent, while Oregon reported a response rate over 60 percent. There were some interstate differences in how response rates were calculated, so direct comparisons between states are not possible.

Respondent characteristics across states followed similar patterns by establishment size and ownership. Larger establishments, state and local government establishments, and establishments that were part of a larger company or organization with multiple locations were all generally more likely to respond to state interview requests than smaller establishments, private sector establishments, and single-location establishments.

Initial results

The results presented below, unless otherwise noted, come from the final reports each state provided to BLS at the conclusion of the project.⁶ A more detailed analysis of the microdata files each state provided to BLS as part of this project will be forthcoming. (See the concluding section for more details.)

The primary objective of this project was to obtain quantitative data on the recordkeeping decisions employers make when choosing whether to record occupational injuries and illnesses. Employers selected to participate in the SOII must record and report to BLS all injuries and illnesses throughout the calendar year that qualify as a recordable case according to OSHA criteria.⁷ In addition, employers are required to provide detailed worker demographic and case circumstance information for any injury or illness where the worker missed at least 1 calendar day away from work following the day of injury or illness.

Preliminary findings from across the four states point toward confusion among employers regarding several aspects of OSHA recordkeeping criteria. A few of these scenarios are discussed below.

Employer confusion about case types. According to OSHA recordkeeping rules, a situation in which a worker experiences at least 1 day of restricted work duty following an injury or illness but does not miss any days away from work is a “Days of Job Transfer or Restriction” (DJTR) case. If a worker misses at least 1 calendar day of work following an injury or illness, employers should record this as a “Days Away From Work” (DAFW) case. The total number of days of job transfer or job restriction or days away from work are counted beginning with the next calendar day following the injury or illness for DJTR and DAFW cases. Injuries and illnesses that are recordable but don’t result in DJTR or DAFW are collectively referred to as “other” recordable cases.

The state reports on the results of the employer interviews indicate confusion on the part of employers regarding the appropriate type of case to record for injuries and illnesses and the correct number of days to assign for DAFW cases. For DAFW (and DJTR) cases, the number of calendar days, not scheduled shifts or work days, should be recorded as the number of days away from work (or restricted duty in the case of DJTR cases). Many employers reported counting scheduled work days or shifts instead of calendar days for DAFW cases. States reported that when employers were presented with a hypothetical recordkeeping scenario,⁸ most employers correctly identified the injury as OSHA recordable, but many did not correctly record the injury as a DAFW case.

These results are of concern to BLS because detailed case and demographic data are currently collected and published only for DAFW cases. In addition to providing a potentially distorted picture of the severity of injuries and illness that befall workers, the misclassification of DAFW cases as either “other” or DJTR cases may also contribute to the “missing” SOII cases noted by various researchers who have matched SOII cases with workers’ compensation records. More work will be needed to confirm whether this is a national trend, and BLS is conducting additional work (discussed more below) that will better help us understand this issue nationwide.

Reliance on workers’ compensation definitions. Employers are instructed to use OSHA recordkeeping criteria when recording injuries or illnesses and when responding to the SOII. This ensures a consistent recordkeeping standard is used across the nation. The interviews across the four states revealed that many employers use their state’s workers’ compensation definitions to record injury and illness cases onto their OSHA log.

Unlike OSHA recordkeeping guidelines, workers’ compensation guidelines vary by state. Using workers’ compensation rules as the basis for recording an injury or illness can result in cases being either erroneously included or excluded by employers who use their state’s definition of a compensable case when they respond to the SOII. When employers record injury and illness cases by relying on workers’ compensation definitions, they introduce unintended variability into the SOII estimates.

Temporary workers. There appears to be considerable confusion among employers regarding how to record the injuries and illnesses of temporary workers hired through an outside agency. If a temporary worker is supervised on a day-to-day basis by the employer, OSHA regulations require that any occupational injuries and illnesses of that worker be included on the sampled employer’s log, rather than on the log of the staffing agency.⁹ Initial results reported by the states indicate that many establishments using temporary workers who are supervised at the establishment do not record their injuries and illnesses on their OSHA log. Although temporary workers are used in a relatively small number of establishments, the exclusion of their injuries and illnesses could potentially bias estimates in industries where they are frequently used by employers.

Washington projects

In addition to their work on the employer interviews as part of the four-state study, the Washington State Department of Labor & Industries also completed work on two other projects funded by BLS.

SOII–workers’ compensation case matching, 2000–11

Previous work during the first round of undercount research matched only 2 years of SOII microdata and workers’ compensation records in California and Massachusetts (2007–08) and 3 years in Washington (2006–08). Given such a short timeframe, BLS could not identify matching rates and trends over time when analyzing these results. For this project, Washington expanded on its initial research by refining and enhancing its methods while extending the match to SOII microdata and workers’ compensation records from 2002 to 2011.¹⁰

The goals of the Washington study were to

·       compare the annual reporting of injuries and illnesses between the SOII and the workers’ compensation system,

·       evaluate any trends over time, and

·       identify establishment or case characteristics associated with differential reporting of injuries and illnesses between the two systems.

BLS supplied the relevant SOII microdata from Washington employers. Washington already had access to workers’ compensation records because it oversees the fund that state law requires most employers pay into for workers’ compensation insurance. Washington also had an interagency agreement to access state unemployment insurance information, which was used to bridge the workers’ compensation records and SOII data using a unique employer identifier that is found in all Washington administrative databases. This unique arrangement within Washington helped facilitate matching between the SOII microdata and workers’ compensation records.

To ensure no potential matches were inadvertently excluded, all workers’ compensation records were included in the match with SOII case data. After the match was complete, Washington applied exclusions to workers’ compensation records for cases that did not have any missed days of work, for records associated with other company establishment locations not sampled by the SOII, and for records with days of missed work that occurred after the SOII survey year. This was done to include only those workers’ compensation records within the scope of the SOII. The state applied these exclusions to both matched and unmatched records in order to ensure the estimates generated for SOII underreporting included only workers’ compensation records that met SOII eligibility criteria.

View Chart Data

View Chart Data

In a recently published analysis of the matching results, Washington found that only about 70 percent of SOII-eligible workers’ compensation claims were reported in the SOII. In looking at underreporting by establishment size, Washington found the largest establishments (those with more than 1,000 employees) had the highest percentage of workers’ compensation claims reported in the SOII. (See figure 1.) When Washington researchers examined underreporting by ownership and industry, they found state and local government establishments had the highest percentage of eligible claims reported to the SOII, followed by private sector manufacturing establishments. (See figure 2.) In a multivariable analysis of establishment size, ownership, and industry, Washington found the highest rates of underreporting occurred in large private construction establishments (those with more than 250 employees) and small private educational services establishments (those with fewer than 50 employees). The lowest rates of underreporting were found in state and local government establishments.¹¹

Washington also found differential match rates by the physical characteristics, or nature,¹² of the worker’s injury or illness. Acute injuries (such as fractures, bruises, and instances of a worker experiencing multiple traumatic injuries) were more likely to be matched to a corresponding worker’s compensation claim than sprains, strains, tears, and other nontraumatic disorders and illnesses. Washington observed a slight increase in the rate of underreporting across the reporting period (2002–11). These matching results suggest that the SOII is doing a better job of capturing acute or highly visible traumatic injuries (such as fractures) than less visible traumatic injuries (such as sprains, strains, and tears) or nontraumatic illnesses and disorders.¹³

One important limitation of this project, and other similar SOII–workers’ compensation matching work, is that SOII underreporting was assessed only in relation to workers’ compensation claims. Since SOII-eligible injuries and illnesses may be missing from the workers’ compensation systems, the overall rate of eligible injuries and illnesses not captured by the SOII may be higher than Washington estimated in its analysis. More details on this project are available in the state’s final report submitted to BLS and in a recently published peer-reviewed paper by the Washington research team.¹⁴

“Real time” interviews with SOII respondents

To estimate the percentage of missing injury and illness cases from the SOII, undercount research from 2009 to 2012 involved matching individual SOII cases to workers’ compensation claims. In this earlier work, employers were not contacted to discuss specific reasons why potentially eligible cases had not been reported to the SOII. The goal of this Washington study was to identify reasons why specific workers’ compensation claims from employers were not included in the SOII case data reported to BLS. The objective was not to obtain quantitative data generalizable to a larger population of employers, but instead to generate a comprehensive list of different reasons employers gave for submitting a workers’ compensation claim but not reporting that injury or illness to the SOII.

In order to identify workers’ compensation cases not reported to the SOII, Washington matched 2012 SOII respondent data provided by BLS to SOII-eligible workers’ compensation claims using state unemployment insurance data to help bridge these two data sets. Washington then conducted a standard structured interview with respondents using the phone instrument developed in the four-state study, followed by a more unstructured interview about the potential missing SOII cases. In total, Washington interviewed 103 establishments regarding 171 different workers’ compensation claims. Employers’ responses were categorized by the reason they gave for a case’s exclusion from the SOII. Table 1 lists these categories and shows employer-provided examples within each category.

Almost half of unlinked workers’ compensation claims were due to inaccurate OSHA’s Form 300 logs¹⁵— which employers use to record employees’ work-related injuries and illnesses—and another 15 percent were due to mistakes made by employers when responding to the SOII. About one-third of the unlinked claims were due to problems reconciling cases between SOII and workers’ compensation records. These harmonizing issues—instances in which a case was eligible for workers’ compensation wage replacement but was not considered an OSHA-recordable DAFW case by the employer—illustrate the challenges involved in accurately matching cases between an injury and illness surveillance system, such as the SOII, and an administrative records system, such as workers’ compensation data, that were designed for very different purposes.

The full report from Washington, with additional details on this project, is available online.¹⁶

Current and future research

Analysis of the four-state study data will continue in 2016, and analysis of a recently completed national survey of employers will begin. BLS is also continuing newly initiated research on the feasibility of conducting a worker survey of occupational injuries and illnesses.

Continued analysis of four-state study data

The four states that partnered with BLS to conduct the employer interviews provided BLS with a rich set of respondent data suitable for detailed analysis. One of the state research partners that participated in the four-state study was awarded a BLS/American Statistical Association/National Science Foundation research fellowship and is working with BLS to analyze this data set. The analysis will focus on (1) investigating the role of state-level differences in criteria used to record injuries and illnesses on the SOII, (2) looking at employer reliance on workers’ compensation data when reporting SOII case information, and (3) reviewing employer’s injury and illness recordkeeping practices. 

National recontact survey

Building on the success of the four-state study, BLS partnered with a contractor to conduct a national recontact survey of 2013 SOII respondents. One of the primary goals of this project is to obtain nationwide data on the effect of “late cases” on the counts of workplace injuries and illnesses estimated by the SOII. Late cases include injuries or illnesses that

1.       occurred late in the calendar year,

2.       were discovered or reported by employers after they had responded to the SOII, or

3.       were initially not recordable but became recordable at a later date.

Preliminary results from the four-state study indicate that these late cases may not be consistently recorded on employer OSHA logs, and then not reported to BLS when the employer is responding to the SOII. Thus, these late cases may be undercounted. The recontact study should assist BLS is assessing the magnitude of this issue nationwide.

In addition to being asked about late cases, employers were asked a series of OSHA recordkeeping questions similar to those in the four-state study. This series of questions was designed to obtain national data on the scope of employer misunderstanding of OSHA recordkeeping criteria. The data will help inform BLS decisionmaking aimed at improving SOII data collection procedures.

OSHA’s Form 300 log was also collected from employers for reference year 2013 as a part of this study. Many employers use the form to record employees’ work-related injuries and illnesses that occur throughout the year and then use the completed form to report these injuries and illnesses to BLS when responding to the SOII. Analysis of these logs will help BLS evaluate the results of the recontact survey.

Initial analysis of the results of this study will begin in late 2016.

Research on a worker survey of occupational injuries and illnesses

The SOII is an establishment survey and is designed, pursuant to the Occupational Safety and Health Act of 1970, to collect injury and illness data directly from employers. As discussed in “Occupational injury and illness surveillance: conceptual filters explain underreporting,”¹⁷ there are many “filters” that can lead to OSHA-recordable injuries and illnesses going unreported by workers to their employer, which then go unreported by the employer to BLS. For example, a worker may refrain from reporting an occupational injury for fear of employer retribution, or may choose to use private insurance to get medical care instead of reporting a work-related illness and going through the workers’ compensation system. One possible way to circumvent filters to occupational injury and illness reporting is to contact workers directly. BLS is currently sponsoring exploratory research on how occupational injury and illness data may be collected directly from workers. The goal of this data collection would be to complement, not replace, the valuable data we get from employers as a part of the SOII.

As in the past, BLS will report on the results of all SOII undercount research through various publications, such as the Monthly Labor Review and other economic and public health journals; sessions at relevant conferences; and on the SOII undercount web page.¹⁸