Data from the occupational safety and health statistics program of the Bureau of Labor Statistics are watched closely by various stakeholders and have been the subject of debate from time to time. Many program changes have come about as a result of this careful inspection and discussion. In the past few years, several researchers have identified new concerns about BLS workplace injury and illness data; these concerns are based on comparisons of individual cases from the BLS Survey of Occupational Injuries and Illnesses in selected states with case data from state workers’ compensation records. These studies suggest that BLS undercounts worker injuries and illnesses, although estimates of the extent of that undercount vary widely depending upon the research methodology and state studied. Other research and analysis concludes that the size of the undercount is small. On the heels of both this research and media reports of unsafe work places, Congress provided funding to BLS to follow up and expand on the previous research so as to understand the nature and magnitude of any undercount and attempt to identify solutions. This article updates an August 2008 Monthly Labor Review article that provided an overview of the BLS Survey of Occupational Injuries and Illnesses and introduced the research that led to concerns about an undercount. Details on three separate research projects conducted between 2009 and 2012 are discussed along with recommendations for the future.
Counting occupational injuries and illnesses is complex; potential issues include employee fear about reporting, lack of employer awareness of reporting requirements, complex definitions of workplace injuries, difficulty determining whether an injury actually occurred at work or is otherwise work related, disputes over responsibility, measurement issues, and more.1 The Bureau of Labor Statistics provides the nation’s official statistics on workplace injuries and illnesses, including estimates of the number and rate of nonfatal injuries and illnesses by industry and state and, for selected cases, details about the worker involved and the circumstances of the incident.2These data are based on Occupational Safety and Health Administration (OSHA) recordkeeping requirements, which are a set of definitions and explanations used by employers nationwide to maintain workplace safety records for their establishment.3 The Occupational Safety and Health Act of 1970 (the OSH Act) requires employers selected by BLS to maintain these records, which are used by employers to complete the annual Survey of Occupational Injuries and Illnesses (SOII). From time to time, the completeness of the SOII estimates has been examined, most recently through a series of BLS-sponsored research projects. The goals and results of that research are the subject of this article.
More than once since its inception in the early 1970s, the BLS occupational safety and health statistics program has been the subject of scrutiny, often leading to program changes. For example, concern in the late 1980s that a sample survey could not provide a complete count of workplace fatalities led to the development of the BLS Census of Fatal Occupational Injuries (CFOI), which has provided a comprehensive count of fatal work injuries annually since 1992. Also in 1992, BLS expanded the SOII to include “case and demographic” details. These data elaborate on the worker involved (including occupation, age, and gender) and the circumstances surrounding the incident (including nature of injury and part of body affected). Since 1992, case and demographic data have only been available for cases that result in at least 1 day away from work, although BLS recently began publishing the results of pilot tests of these details for cases that result in no days away from work but lead to a job transfer or restricted work.4
New concerns about BLS workplace injury and illness data were identified in the mid-2000s by researchers who attempted to compare data on individual cases from the SOII in selected states with case data from state workers’ compensation records. The results varied, with estimates of an undercount ranging from 20 percent to 70 percent of cases depending upon the research methodology and state studied.5 Other research and analysis concluded that the size of the undercount is small.6 On the heels of this research, and media reports of unsafe work places, Congress held hearings on the subject and later identified funding for BLS to follow up and expand on the previous research so as to understand the nature and magnitude of any undercount and attempt to identify solutions. At the same time, Congress provided funding to OSHA and to the National Institute for Occupational Safety and Health to conduct complementary research into issues surrounding the completeness of work injury data.7
BLS used the new funding to sponsor research in three areas:
BLS staff, several state agencies, and one university researcher conducted this initial round of research over 3 years, with final results completed in summer 2012. At that time, BLS convened all those involved in the research as well as non-BLS stakeholders to hear presentations on research results and to develop consensus recommendations for moving forward. Each research topic is discussed below, with a focus on the Bureau’s interpretation of the results and impact on the program. These summaries are followed by a look at future activities.
This article updates an August 2008 Monthly Labor Review article, “Examining evidence on whether BLS undercounts workplace injuries and illnesses,” by John Ruser.8 The earlier work provides considerable detail about the BLS survey and the early matching studies that led to concerns that the SOII undercounts workplace injuries and illnesses. Many of those details are not repeated here, except where needed to provide context for new findings.
The first two research projects, which involve the matching of data from different sources, are intended to identify a broader universe of injuries and then determine which of these injuries should be reported to SOII. In the case of the SOII-workers’ compensation match, comparable cases are those which meet OSHA recordability definitions and extend beyond any workers’ compensation waiting period.
The multisource matching project goes beyond OSHA-recordable (and SOII-eligible) cases to identify a complete count of all nonfatal occupational injuries and illnesses, including cases that are not OSHA recordable. This broader measure is of interest to public health professionals seeking to identify interventions and safety protocols related to the total burden imposed by workplace injuries and illnesses, including injuries and illnesses to workers who are not currently included in the SOII—such as federal government employees9—and those to self-employed individuals (the self-employed are not identified in SOII because it is a survey of employers). Further, this broader interest includes cases that may not meet OSHA definitions and may not exist in the OSHA records used in the SOII, such as long-latency illnesses.
The SOII sets out to estimate OSHA-recordable injuries and illnesses occurring within a given year, as identified by employers. While there may be an undercount of such cases, this more narrow scope is still vitally important to public health officials and others. It may be possible for SOII data to be used as one input as the public health community attempts to identify the various components of the total burden of workplace injuries and illnesses, but complete data on such burden will have to come from a variety of employer-based and nonemployer-based sources.
Concerns raised in the mid-2000s about the completeness of SOII data stem from projects that matched data on individual cases from the SOII with case data from individual state workers’ compensation systems. When the OSH Act authorized the Department of Labor to develop workplace injury and illness recordkeeping definitions and procedures, the result was the first consistent set of data across states and industries. Workers’ compensation, on the other hand, is a state program with state-to-state differences in definitions and procedures.10 Differences between the SOII and workers’ compensation systems had long been suspected, based on previous comparisons of aggregate counts and studies of selected industries or injuries.11 The mid-2000s studies were the first to match case data from the two systems: one researcher matched data for Michigan, and a second researcher matched data for six different states. The results suggested that the SOII and workers’ compensation systems each missed cases and that there were cases missed by both systems.
A number of challenges were identified in the matching of cases from these two systems. Of particular note is the share of cases that must be excluded from the matching process. Because the SOII data available in the mid-2000s had details about only those cases resulting in days away from work, this subset of cases serves as the starting point for matching. Further, many workers’ compensation systems begin paying time-off-from-work benefits after a waiting period, such as 3 or 5 days. Although the researchers first attempted to match establishments and cases without regard to the number of days off from work, ultimately the matches were limited to cases with more days than the waiting period. In some instances, these limitations meant that no more than about 1 in 4 SOII cases were included in these matching studies (see figure 1).
The quality of matches between SOII and workers’ compensation often depended on the ability of the researcher to parse workers’ compensation data. In some instances, days-away-from-work injuries may not be identified as such in workers’ compensation, perhaps because the employer chose to leave the worker “on salary” during the period of disability or because the number of days recorded for workers’ compensation purposes was adjudicated to a number other than what appears on the OSHA log. Further, cases of permanent partial disability may result in workers’ compensation payments but may not be recorded as having days away from work. Because the detailed SOII data were only available for days-away-from-work cases, it is possible that some workers’ compensation cases were included in the SOII (as “job transfer” or “other” cases) but without the case details necessary to identify a match.
Another issue in matching SOII and workers’ compensation data is the scope of the two data sources. SOII is a sample of establishments, which are typically single locations (even for employers who have more than one location in a region or a state). Workers’ compensation records, conversely, are a universe of all cases within the state and are maintained at the firm level (meaning all locations of an employer within a state are combined). Deciding which workers’ compensation case should have been included in the SOII is therefore difficult, as the SOII only has cases for a subset of locations, and these cases are more narrowly defined than in workers’ compensation records. Matching employers who have only one location in a state proved easier than matching employers who have multiple locations, from which SOII generally samples one or a few locations. To further complicate the match, for some employers the SOII captures only a sample of cases.
SOII and workers’ compensation also serve different purposes, which can affect the data match. SOII is designed to estimate employers’ OSHA recording of workplace injuries and illnesses over a fixed period of time—1 year. In contrast, workers’ compensation is an administrative dataset that is used for adjudication of payments; information is updated over time.
The results of research on matching by BLS economists using data from Wisconsin have been previously published in the Monthly Labor Review12; the non-BLS researchers who recently completed matching activities using data from California and Washington are currently preparing their work for publication. Some of their results include the following:
While the matching projects attempt to limit the scope of records being matched to those which appear likely to be in both systems, there are reasons why specific injury or illness cases might appear in one system and not the other. For example, injuries that occur while preparing food for personal consumption (such as during a lunch break) are not included in OSHA recordkeeping but in some states might be compensable under workers’ compensation.13 Conversely, injuries that result from horseplay at work might not result in workers’ compensation but are OSHA recordable. In addition, differing rules apply for injuries that occur while commuting, in an employer parking lot, or off of the employer’s premises.14 These examples point to complexities in the two systems that may result in inconsistent reporting.
Differences between injuries reported on the OSHA log and those found in a state workers’ compensation system may also be the result of administrative differences between the two systems. For example, workers’ compensation records in California do not include information on number of days away from work. An estimate of the number of days away from work was made if information for time-away-from-work benefits was available. A few examples from other states attest to the potential difficulties in matching workers’ compensation and OSHA data. In New York, for a case to be accepted by the workers’ compensation board, there must be a medical report. The burden to submit the medical report to the workers’ compensation board is on the injured worker. In Minnesota, days away from work are calculated beginning with the first day of any lost time from work, including a fraction of a day. This differs from the OSHA definition in that days away from work are counted beginning with the first day after the day of incident. Finally, differences and OSHA-allowable discretion in whether aggravated injuries are treated as a new case or as a continuation of an existing case can result in inconsistent recording in OSHA and workers’ compensation.
A further complication in matching occurs because of the focus in OSHA recordkeeping on the workplace event and the corresponding nature of the injury, while the focus in workers’ compensation is on the ultimate medical outcome for which compensation is being provided. This dichotomy can also be seen in the multisource enumeration projects described next, where work-related and medical-related sources often provide different types of information. OSHA recordkeeping guidelines require employers to record updates to an injury, including sequelae (further medical issues that follow the original injury), on their OSHA log (the form that counts the number of injuries and illnesses), but there is no requirement that employers update the detailed case information after it is initially recorded. That individual case detail is the SOII information used in the matching projects, such as the nature of the injury or the number of days away from work. To further complicate matters, OSHA’s ability to issue citations to employers who fail to record employee injuries or illnesses on their log was recently limited by a court ruling.15 These issues can result in differences between SOII and workers’ compensation records.
The various matching projects identified a number of additional research questions. For example, to what extent is the undercount that is identified through these matching techniques consistent over time? What variation occurs across industries or types of injuries? Further, how can BLS address the concern that late-year injuries and illnesses might be missed by the SOII? Can BLS learn about changes made to OSHA logs after SOII data have been collected? BLS has begun work on some of these issues, as discussed at the end of this article.
The multisource enumeration project was designed to test the availability of, and feasibility of using, additional records—beyond the employer-based SOII and workers’ compensation data—to identify workplace injuries and illnesses. This project stemmed from recommendations of earlier researchers that the SOII use techniques to identify all nonfatal workplace injuries and illnesses similar to those used by the BLS Census of Fatal Occupational Injuries (CFOI). The CFOI uses multiple sources to identify and confirm the work relationship of all fatal work injuries in the country. Each case is supported by an average of 4 source documents, which may include death certificates, OSHA fatality reports, police statements, coroner reports, and media reports. Through this process, the CFOI has over the past 20 years become highly regarded as a complete count of fatal work injuries.
The theory behind a multisource enumeration of nonfatal injuries and illnesses is similar to the CFOI process: each case would be supported by one or more sources, and sources would be matched to ensure cases were counted only once. In practice, however, using multiple sources to identify an unduplicated count of nonfatal cases poses numerous challenges, including the following:
The multisource enumeration study looked at two types of injuries, amputations and carpal tunnel syndrome. These injuries were chosen because it was thought that amputations would be easily identified in both SOII and workers’ compensation, while the long onset and lack of clarity as to the work relationship of carpal tunnel syndrome might mean this injury is less easily identified in the two systems. As it turned out, neither condition proved easy to identify.
The work surrounding the study of amputations identified some unanticipated issues, including the following:
Other challenges existed in the carpal tunnel syndrome study. The results suggested that information was more consistent across data sources when surgery was involved. Further, OSHA recordkeeping guidelines indicate that employers may consider recurrences of existing injuries that are not brought on by a new event or exposure in the workplace to be the same case rather than being recorded as a new case. This guideline, along with the potential long lag between onset and identification or treatment, may further complicate the ability to match cases across data sources.
Beyond the SOII and workers’ compensation records, most of the sources looked at for this study were medical in nature, such as trauma registries, hospital discharge records, and emergency room records. These records have little information on work relationship or worker characteristics, such as occupation. Often the only evidence of work relationship is an indication that the medical-care payor is the workers’ compensation system. Further, the medical sources often lacked information on whether the injury resulted in at least one day away from work. Without this information, the data from the medical sources could not be matched to the SOII case data.
The results of these studies indicated that multiple sources of information can be used to identify more amputation and carpal tunnel syndrome cases than are currently included in the SOII. When attempting to identify SOII-eligible cases, and using a variety of methods, data from one state indicated that SOII captures up to half of all carpal tunnel syndrome cases and up to three-quarters of all amputations, although the results vary widely. Another concern raised by the multisource studies was the potential for any bias in the SOII data, such as more complete reporting among establishments in certain industries or size categories. Such differential reporting may be related to familiarity with OSHA recordkeeping. This same issue was also identified in the interviews with employers, as described in the next section.
While the two matching projects above describe how OSHA recordable cases and other systems aren’t always in agreement, the employer interviews explored how employers interact with both systems. Interviews were first done with a small selection of employers in the Washington, DC, area. Over the past few years, the locations were expanded to include groups of employers in Kentucky and Washington State. These states were selected to correspond with the SOII–workers’ compensation matching work. However, while some employers being interviewed were identified as representing establishments that had consistent SOII and workers’ compensation data (and some were identified as representing establishments that did not have consistent SOII and workers’ compensation data), the limited number of interviews could not be used to identify any relationships between employer practices and the consistency of employer data.
The interviews attempted to identify the person or persons who were responsible for completing the SOII; these individuals were often responsible for maintaining OSHA logs but were not necessarily responsible for maintaining workers’ compensation records. These respondents were asked about the processes they go through when completing these reports. For example, questions were designed to determine how well respondents understood which cases are to be reported on an OSHA log. Some respondents indicated that they followed OSHA criteria, while others reported recording all injuries—that is, injuries included on workers’ compensation reports, those which involved doctor visits, or those which required medical attention. Additional questions were asked about any training received on OSHA recordability, including the source and quality of that training.
Another area of investigation involved recording workplace injuries for contract workers. According to OSHA guidelines, injuries for such workers are to be recorded by the establishment that provides supervision to the injured worker. In many cases, this is the employer who is contracting for the services, not the contracting agency. In contrast, workers’ compensation for such contract workers would be the responsibility of the contracting agency. To further complicate this situation, even if the establishment where the injury occurred is responsible for recording the injury on its OSHA log and to the SOII, the contracting agency is likely to replace the injured worker with another worker, making it difficult for the individual completing the OSHA log to know the number of days away from work associated with the injury. The employer interviews identified numerous respondents as being uncertain of their responsibility for recording workplace injuries for contractors.
An interesting finding of the employer interview project is the potential that SOII and workers’ compensation are too highly correlated. As noted, differing definitions mean that certain cases belong in SOII and not in workers’ compensation, and vice versa. Further, data items can differ between the two systems. For example, SOII is designed to capture the number of days away from work while workers’ compensation captures the number of days for which compensation is paid. If employers are using workers’ compensation rules to complete their OSHA logs (and ultimately their SOII reports), this could introduce state-to-state variation in SOII data (on the basis of state differences in workers’ compensation) that is not intended. BLS has revised its instructions to employers and provided training to its state partners to help guard against this kind of incorrect reporting.
The information gleaned from these employer interviews is anecdotal in nature, as it was not derived from a statistical sample of employers. Nonetheless, this work opened a window into some of the issues faced by employers in attempting to meet their OSHA recordkeeping requirements. These results generated considerable interest in having quantitative data on employer recordkeeping practices that can be generalizable to a larger population of employers. As noted below, BLS has now funded additional research to address this interest.
In July 2012, BLS brought together researchers from both within and outside the Bureau who had been working on these various projects over the previous 3 years. They presented their findings to each other and to a group of occupational safety and health stakeholders. As a group, the researchers were asked to identify their key findings and to make recommendations for future activities.
These are key researcher findings:
The researcher recommendations fell into five broad categories: improvements to the SOII, expansion of occupational injury and illness data beyond the SOII, further research, recommendations for OSHA, and communications and outreach. While many of the recommendations can be addressed, some are not possible to bring about (perhaps because of lack of resources or lack of authority) and some are not desirable (often because they involve inspection and enforcement issues, which are not part of the BLS mandate to capture statistical data). These recommendations were used to guide the selection of further research in 2013 and beyond to improve the completeness and accuracy of estimates from the SOII. Current research activities include the following:
While subsequent research activities have not been finalized, BLS is particularly interested in learning more about whether employers update their OSHA recordkeeping logs after SOII data are collected and, if so, whether it is feasible for BLS to capture those changes.
BLS is also implementing a number of improvements to the SOII. In 2013, BLS began publishing case details and worker characteristics for a subset of nonfatal injury and illness cases that result in job transfer or restriction in addition to comparable data previously available for cases that result in days away from work. For now the job-transfer-or-restriction data are limited to cases in a few select industries, but these data may shed light on whether cases thought to be missing from the SOII are in fact captured as something other than days-away-from-work cases. In addition, BLS has captured and is currently assessing the quality of data on whether injured or ill workers are hospitalized, with hopes of publishing this information in the future.
BLS AND ITS RESEARCH PARTNERS will report the results of all SOII research through various publications such as the Monthly Labor Review and peer-reviewed journals, as well as through contributed sessions at relevant conferences including the Council of State and Territorial Epidemiologists and the American Public Health Association.
William J. Wiatrowski, "Examining the completeness of occupational injury and illness data: an update on current research," Monthly Labor Review, U.S. Bureau of Labor Statistics, June 2014, https://doi.org/10.21916/mlr.2014.24.
1 For a good overview of the “filters” that can influence the identification and reporting of occupational injuries and illnesses, see Lenore S. Azaroff, Charles Levenstein, and David H. Wegman, “Occupational injury and illness surveillance: conceptual filters explain underreporting,” American Journal of Public Health, September 2002, http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1447253/pdf/0921421.pdf.
4 The most recent information on worker characteristics and case circumstances for occupational injuries and illnesses resulting in days of job transfer or restricted work is available at https://www.bls.gov/iif/oshwc/osh/case/djtr2012.pdf.
5 Kenneth D. Rosenman, Alice Kalush, Mary Jo Reilly, Joseph C. Gardiner, Mathew Reeves, and Zhewui Luo, “How much work-related injury and illness is missed by the current national surveillance system?” Journal of Occupational and Environmental Medicine, April 2006, pp. 357–365, and Leslie I. Boden and Al Ozonoff, “Capture–recapture estimates of nonfatal workplace injuries and illnesses,” Annals of Epidemiology, June 2008, pp. 500–506.
6 For a thorough description of prior studies on this issue, see John W. Ruser, “Examining evidence on whether BLS undercounts workplace injuries and illnesses,” Monthly Labor Review, August 2008, https://www.bls.gov/opub/mlr/2008/08/art2full.pdf.
7 Information about OSHA activities is available at https://www.osha.gov/OshDoc/Directive_pdf/CPL_02_10-07.pdf. NIOSH occupational safety and health surveillance activities are described at https://www.cdc.gov/niosh/programs/.
8 See Ruser, “Examining evidence.”
9 BLS added state and local government employees to the SOII in 2008. BLS is currently working with OSHA on plans to capture occupational injury and illness data for federal government employees. Although BLS plans to publish data for federal workers, a time frame for the inclusion of these additional data has not yet been determined.
10 A good source of information on workers’ compensation programs in the United States is the Workers’ Compensation: Benefits, Coverage, and Cost annual report produced by the National Academy of Social Insurance, available at http://www.nasi.org/research/workers-compensation.
11 For a good discussion of prior studies, see Ruser, “Examining evidence.”
12 See Nicole Nestoriak and Brooks Pierce, “Comparing Workers’ Compensation claims with establishments’ responses to the SOII,” Monthly Labor Review, May 2009, pp. 57–64, https://www.bls.gov/opub/mlr/2009/05/art4full.pdf.
13 For example, under workers’ compensation provisions in Washington State, injuries occurring during lunchtime on the employer premises are covered injuries.
14 An injury occurring during commuting is one area where OSHA recordkeeping rules may differ from individual state workers’ compensation laws. Under OSHA rules, injuries while commuting are not recordable. Further, injuries arising from motor vehicle accidents that occur on a company parking lot or access road while commuting to and from work are also not recordable. In contrast, workers in Oregon who are paid for commuting time are covered by workers’ compensation during that paid time.
15 The U.S. Court of Appeals for the District of Columbia Circuit ruled that OSHA may no longer issue citations alleging that an employer failed, more than 6 months before, to record employee injury on the employer’s recordkeeping log. Previously, citations had been issued for up to 5 years after the alleged improper recordkeeping occurred. Details of the case are available at http://ehstoday.com/osha/volks-decision-what-does-it-mean-employers.