Debunking common misconceptions about the Job Openings and Labor Turnover Survey: BLS research disproves overcounting and other myths in JOLTS data

The U.S. Bureau of Labor Statistics (BLS) Job Openings and Labor Turnover Survey (JOLTS) program produces monthly and annual estimates of job openings, hires, and separations for the nation.¹ JOLTS data are available by ownership (private versus public), region, supersector, and select industry sectors. At the total private level, estimates are produced by establishment size class. The JOLTS program also produces estimates for all 50 states and the District of Columbia. State estimates are produced monthly and published annually at the total nonfarm level for job openings, hires, and separations including quits, layoffs and discharges, and total separations.²

Many JOLTS measures exhibited sharp changes at the start of the COVID-19 pandemic and for several years after. Job openings fell steeply at the onset of the pandemic in March and April 2020, increased sharply to a series high in March 2022, and then began to trend down. (See chart 1.) In this article, we use national JOLTS data and research to show that the steep upward trend in JOLTS job openings from April 2020 through March 2022 was attributable more to economic factors rather than to noneconomic reasons. The research provided in this article shows that the sharp increase in JOLTS job openings was not caused by data collection errors or by respondents overreporting job openings but primarily by real economic changes over that timeframe. In addition, common misconceptions about JOLTS methodology are addressed.

One explanation for the steep increase in job openings was the changing economic landscape during the pandemic, which was characterized by an increase in demand for labor, fewer jobseekers, and an increase in firms seeking to hire more remote workers.³ However, there was skepticism about the explanation.⁴ For example, it was suggested that an increase in remote work may cause an issue with businesses advertising the same job opening across multiple locations, thereby causing overreporting or overcounting in JOLTS and, as a result, overestimation.⁵ There was speculation that the steep increase in job openings was tied to noneconomic processing issues, like changes in technology making it easier for firms to post and recruit for a job.⁶

Common misconceptions about JOLTS methodology: overcounting online job listings

A common misconception is that JOLTS sources information from online job listings and, therefore, JOLTS would be susceptible to overcounting. However, JOLTS methodology does not rely on online job listings. JOLTS job openings data are sometimes compared with analogous measures that utilize information from online job boards.⁷ Employers can post positions across numerous online platforms, countries, regions, states, and cities. Sometimes, vacancy postings may be left online even after the position has been filled.⁸ The complexities involved with measuring job openings based on online job platforms can cause some to hypothesize that it may lead to the overcounting of job openings.⁹

JOLTS is an establishment-based survey that collects data directly from the sampled establishments.¹⁰

JOLTS defines job openings (job vacancies) as all positions that are open (that is, not filled) on the last business day of the month. A job is open when it meets all three of the following conditions:

• A specific position exists and there is work available for that position. The position can be full time or part time, and it can be permanent, short term, or seasonal.

• The job could start within 30 days, regardless of whether the establishment finds a suitable candidate during that time.

• The employer is actively recruiting workers from outside the establishment location that has the opening.¹¹

JOLTS has rigorous procedures and training to ensure accurate reporting

JOLTS has robust data collection procedures to ensure accurate reporting. All JOLTS data collection staff are trained on JOLTS concepts, and staff also train and help survey respondents provide data. For the first several months that establishments are in the JOLTS sample, they are contacted by telephone so that interviewers can instruct and readily clarify any points of confusion. As survey respondents become more familiar with providing JOLTS data, they are offered the opportunity to self-report by web collection or email methods. Throughout the sampled establishment’s time in the survey, trained staff will continue to contact respondents as needed to verify they are still adhering to proper data collection practices, definitions, and criteria.

JOLTS employs proactive quality assurance methods to ensure accurate data collection

Beyond training, JOLTS data collection staff engage in continual data-quality assessments and practices. Data collectors are monitored by supervisors to check that data are being collected correctly. Data collectors follow up with respondents to verify data correctness when extreme month-to-month changes are reported.

Diagnostic programs that flag anomalous data are run and assessed by JOLTS data analysts. These analysts are experienced in evaluating anomalous and/or fluctuating JOLTS data to determine if they are within acceptable ranges or if they merit follow-ups with respondents to double-check that the flagged data were reported and collected properly.

Thus, given JOLTS definitions and collection practices, although some businesses may continue to post job openings even though they do not have vacancies, the JOLTS job openings measure is unlikely to be substantially impacted. Furthermore, JOLTS data collection and review have historically not suggested that the overcounting of job openings is common or widespread among sampled establishments.

Two empirical approaches show no evidence of JOLTS job openings overestimation

The empirical research described in this article finds no evidence that the dramatic rise in JOLTS job openings is caused by issues with overcounting. We used two separate approaches to demonstrate this assertion. First, a micro-level approach examines establishment-level JOLTS data as reported to the survey by the respondents. Second, a macro-level approach compares JOLTS job openings trends for businesses with a single worksite to businesses with more than one location.

Defining the terms and concepts of the research

Before describing these approaches in more detail, some terms and concepts are important to define.

JOLTS defines an establishment as a single physical location where business is conducted or where services or industrial operations are performed. Some establishments are single-unit businesses, which are referred to as “singles.” Other establishments are associated with broader business entities (for example, firms and corporations) that have oversight over multiple establishments. Those multi-unit businesses are referred to as “multis.”

The JOLTS sample includes both singles and multis. Within a particular multi-unit business, JOLTS may randomly sample one, more than one, or no establishments. When singles are sampled in JOLTS, the contact person responding to the survey for the establishment is generally someone that works at the specific location sampled. However, for multis, the person best suited to respond to the survey may not work at the actual physical location of the sampled establishment or establishments. Furthermore, the same contact person may supply responses for more than one of the business’s sampled establishments. Individuals who respond for more than one establishment are referred to as multireporters.

Some have speculated that overcounting may be happening at the data collection stage. One theory is that multi-unit businesses are reporting business-wide job openings instead of establishment-level job openings. For example, suppose a restaurant chain with two different locations has a total of four job openings for the firm, with one at the first location and three at the second location. Further, suppose that both locations were sampled in JOLTS, and a multireporter provides a response of four job openings for each location. In this case, job openings for the first location would be overcounted by three, and job openings for the second location would be overcounted by one. Remote job openings may be particularly likely to be overreported because they may not be tied to a particular establishment. In these cases, data collection staff are instructed to inform the survey respondent that each remote job opening should be reported only for the establishment to which a new hire would be expected to report.

Micro-level research: examining JOLTS microdata shows low prevalence of overcounting

One misconception about JOLTS data is that multireporters may be reporting firm-level job openings instead of establishment-level job openings or that the same remote job openings are reported in multiple establishments. We analyzed JOLTS microdata from June 2021 through May 2022 and found that there is little evidence of this occurrence. This period includes the job openings peak in March 2022, as shown in chart 1. Each observation represents a sampled establishment’s survey answers for a particular month and year. This is known as a JOLTS report. Exhibit 1 shows relevant variables and definitions for each JOLTS report.

For the purposes of this research, we searched for observations that matched on month and year, employee identification number (EIN), reporting contact, and job openings values, because matching on these variables would be indicative of the possibility of overcounting. We identified 563 sets of observations that matched all four variables. However, 451 of these matches involved job openings values of zero. These cases were excluded from further analysis because reports with zero job openings are not atypical and not suggestive of incorrect reporting. Furthermore, even if the matched zero reports were somehow erroneous, they would serve to deflate job openings values rather than inflate them. After removing the duplicate zero cases, there were 112 nonzero matches. We examined these 112 cases further to assess which ones should be flagged as likely overcount cases. If in doubt after reviewing the evidence, they were flagged. There were two primary means of investigation:

• Comparison of other JOLTS measures: hires, quits, layoffs and discharges, other separations, and total separations

• JOLTS case notes review and reporting history analysis

When a set of reports matched on month and year, EIN, reporting contact, and number of job openings, but mismatched on at least one other element—that is, hires, quits, layoffs and discharges, and other separations—the default position was to initially assume that the matched job openings values were not evidence of overcounting. This was because the reporting contact provided at least some differing elements across reports, supporting the idea that the matched set's job openings values matched coincidentally and not due to overcounting. On the other hand, when a set of JOLTS reports matched on all other elements, the default position was to assume that the matched job openings values were potentially indicative of overreporting. However, these default positions were sometimes changed after further research.

The results of these comparisons are supplemented with information deduced from JOLTS case notes and reporting histories. Case notes document communication between survey respondents and data collectors during the survey collection and review process. Consider a hypothetical multi with two JOLTS-sampled establishments with matching job openings values. Case notes may clarify that the matching values were coincidental and legitimate or, perhaps, they may suggest the opposite. Respondent histories reveal the past tendencies of establishments and/or reporting contacts. For example, a respondent history exhibiting a spotty survey cooperation record, with a tendency to supply matching variable values repeatedly over time, would cast doubt on the ability to confidently declare the matched job opening values as valid.

JOLTS subject-matter experts evaluated the 112 cases identified for further review on a case-by-case basis to determine each report's accuracy. Ultimately, the investigation yielded 22 matches that were flagged as likely overcount cases. These 22 matches, comprising 69 JOLTS reports, are summarized in table 1.

To more rigorously assess the impact of these potential and/or likely job openings overcount cases, consider the following two conditions that must be true for overcounting to be responsible for a sharp rise in job opening levels and rates:

• Condition 1: There must be enough cases of overcounting to influence the overall job openings level and rate.
• Condition 2: The job openings rate for the overcount cases must be considerably higher than the overall rate.

Table 1 touches on the first condition, showing that only 0.08 percent of JOLTS reports were flagged as likely job openings overcounts. To address the second condition, we took a conservative analytical approach, meaning we assumed the worst-case scenario that all 112 reviewed matches were actual overcount cases. The rationale was that if the analysis shows that these 112 cases have little impact on JOLTS job openings rates, it is a stronger refutation of the hypothesis that overcounting led to the spike in JOLTS job openings than if the analysis was done on only the 22 matches flagged as likely overcounts. We computed employment-weighted job openings rates over the 112 reviewed matches, which spanned 277 JOLTS reports. We also computed employment-weighted job openings rates over all JOLTS reports that were not part of the 112 matches. Finally, we computed employment-weighted job openings rates over the combined set of JOLTS reports. These job openings rates were computed according to the following formulas and definitions:

${\mathrm{w}\mathrm{g}\mathrm{t}\mathrm{j}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{e}}_{S,t}=\frac{{\mathrm{w}\mathrm{g}\mathrm{t}\mathrm{j}\mathrm{o}}_{S,t}}{{\mathrm{w}\mathrm{g}\mathrm{t}\mathrm{e}\mathrm{m}\mathrm{p}}_{S,t}+{\mathrm{w}\mathrm{g}\mathrm{t}\mathrm{j}\mathrm{o}}_{S,t}}\times 100$${\mathrm{w}\mathrm{g}\mathrm{t}\mathrm{j}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{e}}_{S,t}=\frac{{\mathrm{w}\mathrm{g}\mathrm{t}\mathrm{j}\mathrm{o}}_{S,t}}{{\mathrm{w}\mathrm{g}\mathrm{t}\mathrm{e}\mathrm{m}\mathrm{p}}_{S,t}+{\mathrm{w}\mathrm{g}\mathrm{t}\mathrm{j}\mathrm{o}}_{S,t}}$

where

$\mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e}$

$\mathrm{w}\mathrm{g}\mathrm{t}\mathrm{j}{\mathrm{o}}_{S,t}$${\mathrm{w}\mathrm{g}\mathrm{t}}_{\mathrm{j}{\mathrm{o}}_{S,t}}$${\mathrm{w}\mathrm{g}\mathrm{t}}_{\mathrm{j}{\mathrm{o}}_{S,t}}$${\mathrm{w}\mathrm{g}\mathrm{t}\_\mathrm{j}\mathrm{o}}_{S,t}$= employment-weighted job openings for sample subset S and month and year t, $$

$\mathrm{w}\mathrm{g}\mathrm{t}\mathrm{e}\mathrm{m}{\mathrm{p}}_{S,t}$= weighted employment for sample subset S and month and year t, and${\mathrm{w}\mathrm{g}\mathrm{t}}_{\mathrm{e}\mathrm{m}{\mathrm{p}}_{S,t}}$${\mathrm{w}\mathrm{g}\mathrm{t}}_{\mathrm{e}\mathrm{m}{\mathrm{p}}_{S,t}}$${\mathrm{w}\mathrm{g}\mathrm{t}}_{\mathrm{e}\mathrm{m}{\mathrm{p}}_{S,t}}$${\mathrm{w}\mathrm{g}\mathrm{t}}_{\mathrm{e}\mathrm{m}{\mathrm{p}}_{S,t}}$${\mathrm{w}\mathrm{g}\mathrm{t}}_{\mathrm{e}\mathrm{m}{\mathrm{p}}_{S,t}}$${\mathrm{w}\mathrm{g}\mathrm{t}}_{\mathrm{e}\mathrm{m}{\mathrm{p}}_{S,t}}$${\mathrm{w}\mathrm{g}\mathrm{t}}_{\mathrm{e}\mathrm{m}{\mathrm{p}}_{S,t}}$$\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{w}\mathrm{h}\mathrm{e}\mathrm{r}\mathrm{e}$

where

$S=\left\{\begin{array}{c}\mathrm{P}\mathrm{O}\mathrm{C},i.e.,samplesubsetofpotentialovercounts\\ \mathrm{N}\mathrm{O}\mathrm{C},i.e.,samplesubsetofnonpotentialovercounts\\ \mathrm{C}\mathrm{o}\mathrm{m}\mathrm{b},i.e.,combinedsample\end{array}\right\}$

$S=\left\{\begin{array}{c}\mathrm{P}\mathrm{O}\mathrm{C},i.e.,samplesubsetofpotentialovercounts\\ \mathrm{N}\mathrm{O}\mathrm{C},i.e.,samplesubsetofnonpotentialovercounts\\ \mathrm{C}\mathrm{o}\mathrm{m}\mathrm{b},i.e.,combinedsample\end{array}\right\}$

Table 2 shows the results of these monthly calculations from June 2021 to May 2022. It is worth noting that the weighted job openings rates in table 2 were calculated without incorporating the birth–death model and alignment procedure.¹²

The prevalence of JOLTS reports identified in this analysis as potential overcounts is quite low. Specifically, only 0.27 percent of JOLTS reports were identified as such and these reports accounted for only 0.31 percent of weighted employment. The impact of the JOLTS reports identified in this analysis as potential overcounts is also very low. The overall weighted job openings rate taken over all JOLTS reports was 6.82 percent, which is the same as the rate taken over only those JOLTS reports that were not categorized as potential overcounts, at least when taken to 2 decimal places. In other words, whether the 277 potentially overcounted JOLTS reports were included in the weighted job openings rate calculations or not, the result was virtually the same. It is also worth noting that the weighted job openings rate over JOLTS reports categorized in this analysis as potential overcounts was lower, at 6.06 percent, than the overall rate of 6.82 percent.

The results in tables 1 and 2 run contrary to the hypothesis that overcounting may explain the sharp increase in JOLTS job openings. They show that neither of the two conditions listed earlier hold. That is, based on this research, there are not enough cases of overcounting to influence the overall job openings levels and rates. Furthermore, over the timeframe of this analysis, the job openings rate for the potentially overcounted JOLTS reports is lower than the overall rate, but it would need to be substantially higher for this type of overcounting to be the driver of the sharp JOLTS job openings increase.

The micro-level research did not support the theory that JOLTS multireporters overcounting job openings values across establishments caused the JOLTS job openings to spike during the pandemic. We investigated only those multi-unit businesses for which more than one of their underlying establishments were sampled in JOLTS. For multis that had only one underlying establishment sampled, there was no way to consistently assess if overcounting was occurring. To the extent that overcounting occurred in such cases, this micro-level analysis could not reflect it.

Macro-level research: comparing JOLTS job openings trends for single-unit businesses and multi-unit businesses shows little evidence of overcounting

As detailed in the previous section, the micro-level research found a very low prevalence of multireporters overcounting job openings across sample establishments within the same business. The impact these cases had on JOLTS estimation was minimal and did not explain the sharp increase in JOLTS job openings that occurred during the COVID-19 pandemic. However, the micro-level research was unable to detect potential overcounting in multis for which only a single establishment was sampled. Because of this limitation, we conducted further macro-level research.

In this section, we examine the suggestion that some reporters may be attributing the same job opening to multiple establishments within the firm. This type of overcounting can occur only in multis. It cannot occur in single-unit businesses. If this kind of overcounting caused the spike in JOLTS job openings, job openings trends for singles and multis would likely diverge during the timeframe of the spike, with the multis increasing more sharply than the singles. To investigate, we used a macro-level approach to compare job openings rates for singles to those for multis.

When conducting this research, there were three possible analytical outcomes:

• Outcome 1: Trend divergence (e.g., job openings trend unchanged for singles, but changed upward for multis)

• Outcome 2: Trend consistency (job openings for singles and multis exhibit similar trends)

• Outcome 3: Ambiguous (something between outcomes 1 and 2)

In the case of outcome 1, the overcounting hypothesis could indeed hold. If so, and given the results of the micro-level research, it may suggest that overcounting was rampant in multis for which only a single establishment was sampled. On the other hand, outcome 2 would very strongly suggest that the overcounting hypothesis has little merit, because in this outcome, there would be no trend divergence in job openings rates between singles and multis, and trend divergence would be an almost necessary condition for the overcounting hypothesis to be true. In the case of outcome 3, conclusions would be less clear. Interpretation of the results would depend upon how closely the results leaned towards outcome 1 or outcome 2.

Table 3 compares JOLTS job openings rates across 2-year timeframes from 2019 to 2020 and from 2021 to 2022, broken out by single-unit and multi-unit establishments. The 2019 to 2020 timeframe was used because the sharp increase in job openings began in May 2020 and lasted about 2 years. However, by the end of 2020, job openings had recovered to roughly prepandemic levels. The 2021 to 2022 timeframe was used because the more unusual part of the spike in job openings occurred during that time.

Overall, from 2019 to 2020, the job openings rate was 4.2 percent. From 2021 to 2022, the job openings rate rose to 6.4 percent. While establishments in both singles and multis saw a substantial rise in job openings rates across the two periods, the job openings rate for singles rose more (+2.3 percentage points) than the rate for multis (+2.0 percentage points).

To provide broader context, chart 2 depicts historical job openings rates from 2013 to 2024, calculated over biennial intervals, and broken out by single establishments and multis. The chart clearly shows that job openings rates for the singles and multis categories have tended to move together consistently over time. The trend from 2019 to 2020 and from 2021 to 2022 was typical in that regard.

The findings illustrated in table 3 and chart 2 contradict the overcounting hypothesis, in which the expectation would be for the increase in job openings rates for the multis category to sharply outpace the singles during the timeframe of the spike. Because the job openings rate trends over these timeframes were consistent for both singles and multis, we can categorize this as trend consistency (outcome 2 described earlier). Consequently, this result, combined with the results of the micro-level research, provides solid evidence that the sharp increase in JOLTS job openings was not caused by overcounting.

Conclusion

JOLTS relies on job openings data collected directly from businesses, not from external sources such as online job boards. Therefore, JOLTS is not affected by some of the challenges that may be associated with measuring job openings based on indirect sources. The research described in this article provides evidence that the sharp rise in JOLTS job openings during the pandemic is reflective of real economic change.

It is worth noting that JOLTS response rates have decreased substantially in recent years, coinciding with a period in which job openings are at historically high levels. If nonresponse is not random, it could potentially contribute to the increase in job openings. Further research is needed to assess whether there is any link between falling response rates and increased job openings rates.