Technical Note
Methodology
Data for the 2021 BRS were collected from July 27 through September 30, 2021. The BRS relied on the
existing data collection instrument of the BLS QCEW program’s Annual Refiling Survey (ARS). BRS survey
responses were solicited via email and printed letters. Responses were collected online using the
platform that is consistently relied on by the ARS. This allows for a large, nationally representative
sample to be surveyed with minimal financial costs to BLS.
Definitions
Establishments. An individual establishment is generally defined as a single physical location at
which one, or predominantly one, type of economic activity is conducted. Most employers covered under
the state UI laws operate only one place of business.
North American Industry Classification System (NAICS) codes. NAICS codes are the standard used by
federal statistical agencies in classifying business establishments for the purpose of collecting,
analyzing, and publishing statistical data. The BRS is based on 2017 NAICS.
Large/small. For these data, establishments with 2020 annual average employment greater than 499 are
considered large.
Sample Design and Selection Procedures
For the 2021 BRS, BLS selected a stratified sample of 322,560 establishments from a universe of just
over 8.6 million establishments. The universe source was the set of establishments from the 2020 fourth
quarter BLS Business Register that were identified as in-scope for this survey.
The BLS Business Register is a comprehensive quarterly business name and address file of employers
subject to state Unemployment Insurance (UI) laws. It is sourced from data gathered by the QCEW program.
Each quarter, QCEW employment and wage information is collected and summarized at various levels of
geography and industry. Geographic breakouts include county, Metropolitan Statistical Area (MSA), state,
and national. Industry breakouts are based on the six-digit NAICS.
The QCEW covers all fifty states, the District of Columbia, Puerto Rico, and the U.S. Virgin Islands.
The primary sources of data for these 53 entities are the Quarterly Contributions Reports (QCRs) submitted
to State Workforce Agencies (SWAs) by employers subject to state UI laws. The QCEW program also gathers
separately sourced data for Federal Government employees covered by the Unemployment Compensation for
Federal Employees (UCFE) program.
There were a little over 10.5 million establishments on the 2020 fourth quarter BLS Business Register that
served as the source of the BRS’s sampling universe. However, about 1.9 million of these establishments
were determined to be out-of-scope for the survey. Establishments that were excluded from the universe:
•Public Administration & Government (NAICS 92)
•Private Households (NAICS 814110)
•U.S. Postal Service (NAICS 491110)
•Services for the Elderly and Disabled Persons (NAICS 624120) with Establishment Size = 1
•Unclassified Accounts (NAICS 999999)
•U.S. Virgin Islands (State FIPS 78)
The 2021 BRS leveraged the technical and collection infrastructure of the ARS. While the synchronization of
the two surveys was efficient, it created a need to adapt the BRS sample in accordance with some of the
constraints imposed on the ARS sample. Regarding ARS sampling constraints, establishments with one to three
employees are never administered the ARS and, of the establishments that are eligible for the ARS, roughly
one-third are administered the ARS in any given year. The determination as to which ARS eligible establishments
are active for any year’s ARS is based on a random mechanism. During BRS sample selection, active ARS eligible
establishments and ARS ineligible establishments were “selectable,” whereas inactive ARS eligible establishments
were disallowed from selection, in part as a means of managing respondent burden over time.
To integrate the BRS sample into the ARS framework, each establishment in the BRS sampling universe was
categorized into one of the following groups:
•ARS Eligible Establishments – Active for this Year’s ARS (BRS Selectable)
•ARS Eligible Establishments – Inactive for this Year’s ARS (BRS Not Selectable)
•ARS Ineligible Establishments (BRS Selectable)
Each BRS sampling stratum consisted of establishments from one or more of the groups above. Within strata
containing only active ARS eligible establishments or only ARS ineligible establishments, sample selection
proceeded with no restrictions using simple random sampling. Strata containing only inactive ARS eligible
establishments ended up being imputed because there were no selectable establishments and, therefore, no survey
results. For any stratum containing a mix of ARS eligible and ARS ineligible establishments, stratum sample
sizes were allocated proportionately to each sub-population. Within the stratum’s ARS ineligible sub-population,
sample selection then proceeded with no restrictions using simple random sampling. Within the stratum’s ARS eligible
sub-population, sample selection proceeded by taking a simple random sample from amongst only the active/selectable
establishments.
Note that for any stratum containing both active ARS eligible and inactive ARS eligible establishments, the
sample was selected from amongst only the active portion of the stratum. This selection was still considered
to be representative of all ARS eligible establishments in the stratum, regardless of active/inactive status,
since the determination of ARS active/inactive status was random. Because of this, and because stratum sample
sizes were proportionately allocated to eligible/ineligible sup-populations, sample units were equally weighted
within (but not across) strata and survey question combinations.
When designing the survey and determining sample sizes, BLS researchers, analysts, and methodologists collaborated
to identify the key research goals. As part of this process, a balance had to be struck between producing precise
estimates for various establishment aggregations and the costs associated with fielding a sample that could deliver
on those goals. Based on the types of administrative data available for establishments on the BLS Business Register
and based on the team’s experience analyzing similar establishment-based surveys, research goals centered on creating
survey estimates for different combinations of establishment geography, industry type, and/or establishment size.
This motivated the decision to choose a design that stratified on all three factors. A decision was then made to
define granular strata to keep the strata homogeneous and to facilitate the construction of a wide array of broader
composite estimates as functions of the more narrowly defined strata estimates. In the end, for the 2021 BRS, strata
were defined jointly on the following factors:
•State
{All states plus the District of Columbia and Puerto Rico}
•Industry Type, Based Primarily on Two-Digit NAICS
{11-21, 22, 23, 31-33, 42, 44-45, 48-49Mod, 4811, 484, 51, 52-53, 54-56, 61, 62Mod, 71, 72, 81}
•Establishment Size, Based on Employment
{1-4, 5-9, 10-19, 20-49, 50-99, 100-249, 250-499, 500-999, 1,000+}
In the industry type list above, industry grouping 48-49Mod excludes industries with NAICS classifications of 484
and 4811. Industry 62Mod excludes industries with a NAICS classification of 624120 that also have an establishment
size of one.
In the establishment size list above, all nine “narrow” size groupings are given. Some BRS analyses were conducted
using two other broader establishment size groupings – a “medium-width” grouping and a “broad,” or large/small,
grouping. The medium-width size classes were 1-19, 20-99, 100-499, and 500+. The large/small groupings were 0-499
and 500+.
At the time the survey was designed, it was clear to researchers and analysts that different industries and
establishment size classes would have different pandemic-related programs and policies targeted towards them. Because
of this, specific state – industry, state – size class, and industry – size class establishment aggregations were
identified as the key levels at which to produce estimates to a certain degree of precision while still being realistic
about survey costs and burden. These aggregations were used to drive sample size determination.
Specifically, they were:
•State by Goods-Producing/Services-Producing Industry Type Categorization {52*2 = 104 estimation cells}
•State by Medium-Width Establishment Size {52*4 = 208 estimation cells}
•Modified NAICS Sector by Medium-Width Establishment Size {15*4 = 60 estimation cells}
•Narrow Establishment Size {9 estimation cells}
Researcher interest was not, and is not, limited to these aggregations. However, because these were the aggregates
initially identified as the most important ones, the sample was designed to achieve a desired precision when estimating
specifically for these groupings. Alternatively, the sample was not designed to achieve a desired precision when estimating
for other groupings, although in some cases the desired precision was achieved anyway. Note that researchers were certainly
interested in estimating with precision at broader levels such as national, state, modified NAICS sector, and narrow size
class. But it was easy to see that a sample that allowed for the generation of precise estimates for the four aggregates
listed above would certainly allow for the generation of precise estimates for these broader level aggregates.
For each estimation cell within each of the four key aggregates listed above, sample sufficiency counts were determined
based on estimating proportions to an agreed upon degree of precision. The formula for the sample sufficiency of an
estimation cell was based on the deconstruction of the formula for the variance of a proportion (using simple random
sampling within the cell). Estimation cell sample sufficiency counts were then allocated proportionately to all strata
within each cell. The result was a set of four “allocated sufficiency counts” per stratum. For each stratum, the maximum
of the four sufficiency counts was chosen. Each stratum’s chosen sufficiency count was then divided by an estimated
survey response rate to derive a stratum sample size. If the chosen value exceeded the number of selectable establishments
in a stratum, the stratum’s final sample size was set equal to its number of selectable establishments. In that case, the
truncated sample size was reallocated to other strata mapping to the same estimation cell. Once sample sizes were finalized,
samples were selected within each stratum as described earlier when discussing the composition of strata in terms of active
ARS eligible, inactive ARS eligible, and ARS ineligible establishments.
Response Rate
The 2021 BRS consisted of 25 questions to which establishments could respond. A survey was considered usable if the
respondent answered at least 5 of the 25 questions. Estimates were generated from usable surveys only.
Of the 322,560 sampled establishments, about 5,300 were deemed uncollectible prior to fielding the sample. These
uncollectible establishments were treated as non-responders. Typically, these were establishments that changed status
between the time when the universe was drawn and a point in time closer to fielding the sample, such that the establishment’s
new status indicated it could not be contacted and/or could not respond to the survey. Thus, the 2021 BRS was administered
to about 317,000 establishments.
Of the establishments that were given the opportunity to take the survey, 85,254 participated to some degree, and 82,487
were usable (answered 5 or more questions). Thus:
•Survey Participation Rate (relative to the full sample) = 26.4%
•Survey Participation Rate (relative to the collectible sample) = 26.9%
•Usable Response Rate (relative to the full sample) = 25.6%
•Usable Response Rate (relative to the collectible sample) = 26.0%
•Usability Rate Amongst Survey Participants = 96.8%
For full technical documentation visit:
https://www.bls.gov/brs/methods/2021-technical-notes.htm
United States Department of Labor
