Department of Labor Logo United States Department of Labor
Dot gov

The .gov means it's official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Handbook of Methods Occupational Requirements Survey Calculation

Occupational Requirements Survey: Calculation

The Occupational Requirements Survey (ORS) provides estimates of job requirements and categories within each requirement have estimates conveyed as the percentage of workers, the mean (in hours, days, percentage of a workday, or pounds), percentile, or mode for each occupation or occupational group. See Exhibit 7 at the end of this section for a full list of occupational requirements with published estimates as well as a list of the various types of estimates.  

Most physical demands and environmental conditions last for a specified duration of time. These correspond with the amount of time that a worker performs a physical demand or the length of exposure to an environmental condition. The table below provides the duration levels with the corresponding percent of the workday that workers perform physical demands or are exposed to environmental conditions.  Also see Exhibit 8 at the end of this section for a list of job requirements with duration levels.

Table 2. Duration levels and the percent of the workday associated with each level
Duration level Presence of the requirement in the workday

Not present

Requirement is not present and there is no duration

Seldom

Up to 2 percent of the workday

Occasionally

2 percent and up to 1/3 of the workday

Frequently

1/3 up to 2/3 of the workday

Constantly

2/3 or more of the workday 

Source: U.S. Bureau of Labor Statistics.

BLS calculates a percentage-of-workers estimate for each duration level. In addition, estimates of some physical demands use means and percentiles to convey duration, such as sitting and standing/walking. For example, BLS measures sitting in hours, so mean and percentile estimates (10th, 25th, 50th, 75th, and 90th percentiles) are calculated for both hours and the percentage of the workday spent sitting for a specific occupation or occupational group. BLS also calculates mean and percentile estimates for education, training, and experience requirements.

For physical demands and environmental conditions, BLS identifies the mode (the duration level with the largest weighted number of workers).

The general formulas used to calculate these estimates are shown below. The type of estimator used depends on the job requirement and whether it is categorical or continuous. For categorical job requirement estimates, BLS calculates a percentage of workers and mode estimates for these percentages. For continuous job requirement estimates (such as duration in hours or days and maximum weight lifted or carried elements), BLS calculates mean and percentile estimates.

For other job requirements that do not have duration levels associated with them, BLS still determines the mode; however, it is calculated across all categories for the specific job requirement. For example, the minimum education that is the most common for security guards is a high school diploma.

Percentage of workers. The formula for the percentage of workers with a given job requirement out of all workers in the domain (such as an occupation) is

    NRAF cell = i v cell, viable w i emp i i u cell, usable w i emp i

where:

I  is the total number of establishments,

Gi  is the total number of sampled jobs in establishment i,

 is the establishment,

g is the occupation within establishment i,

 OccFWig is the final sampled job weight for occupation g in establishment i.

Xig is 1 if sampled job ig meets the condition set in the domain (denominator) condition and 0 otherwise.

Zig is 1 if sampled job ig meets the condition set in the requirement condition and 0 otherwise.

 

Average (mean). The formula for the average (mean) estimate of a job requirement is

  w i

where:

I is the total number of establishments,

Gi is the total number of sampled jobs in establishment i,

i is the establishment,

g is the occupation within establishment i,

OccFWig is the final sampled job weight for occupation g in establishment i.

Xig is 1 if worker ig meets the condition set in the domain (denominator) condition and 0 otherwise.

Zig is 1 if worker ig meets the condition set in the requirement condition and 0 otherwise.

Qig is the value of a quantity for a specific requirement for occupation g in establishment i.

Percentiles. The following percentiles p are calculated: 10⁠th, 25⁠th, 50⁠th (median), 75⁠th, and 90⁠th. The pth percentile is the value Qigwhere the value of a quantity is for a specific category for occupation g in establishment i, such that  

  • the sum of final sampled job  weights (OccFWig) across sampled jobs with a value less than Qig is less than p percent of all final sampled job weights and
  • the sum of final sampled job weights (OccFWig) across sampled jobs with a value more than Qig  is less than (100 – p) percent of all final sampled job weights.

It is possible that there is no specific sampled job ig for which both of these properties hold. This occurs when there exists a sampled job for which the OccFWig of records whose value is less than Qig equals p percent of the total weighted sampled job employment. In that situation, the pth percentile is the average of Qig and the value of the sampled job with the next-lowest value.

Mode. The mode is the highest percentage estimate within a job requirement category.

Education, training, and experience

Although BLS bases most of the estimates for job requirements on establishment responses about the selected jobs’ various tasks, some require an additional level of calculation. One of these is the Specific Vocational Preparation (SVP) level which is the amount of preparation time required for the worker to develop the skills needed to perform the job. The job requirements that contribute to this preparation time are the minimum education level with respect to formal degree types, pre-employment training, previous work experience, and required on-the-job training.  These requirements’ associated time are then aggregated and used to determine the SVP level needed for the job. The table below shows these levels.

Exhibit 4. Preparation time necessary for each specific vocational level
Specific Vocational Preparation level Preparation time   

1

Short demonstration only (4 hours or less)

2

Anything beyond short demonstration up to and including 1 month

3

Over 1 month up to and including 3 months

4

Over 3 months up to and including 6 months

5

Over 6 months up to and including 1 year

6

Over 1 year up to and including 2 years

7

Over 2 years up to and including 4 years

8

Over 4 years up to and including 10 years

9

Over 10 years

Source: U.S. Bureau of Labor Statistics.

Strength

BLS derives strength estimates from several job requirements’ estimates; and measures it with five levels: sedentary, light work, medium work, heavy work, and very heavy work. The levels are determined by how much weight a worker is required to lift or carry, how often, and whether standing or walking is required as part of the workday, in some special cases. BLS determines the strength level when at least one of the lifting or carrying conditions shown in the table below are satisfied, or as defined by the “Strength Level - Special Cases” table. The highest strength level satisfied is the level that represents that sampled job. For example, if a job requires a worker to lift or carry 11–20 pounds occasionally, then it is classified as light work. However, if that same job were to require lifting or carrying that same weight frequently, then it is medium work.

Exhibit 5. Determining strength level based on duration of lifting or carrying
Strength level Duration of lifting or carrying
Seldom Occasionally Frequently Constantly

Light work

11-20 pounds 11-20 pounds 1-10 pounds Negligible weight

Medium work

21-50 pounds 21-50 pounds 11-25 pounds 1-10 pounds

Heavy work

51-100 pounds 51-100 pounds 26-50 pounds 11-20 pounds

Very heavy work

>100 pounds >100 pounds >50 pounds >20 pounds

Source: U.S. Bureau of Labor Statistics.

As noted, there are special cases for strength. The following table outlines the special cases. In instances where field economists are unable to determine certain job requirements from the respondent, they record these data as “unknown” and strength level handle derivation through imputation. See the section “Weighting, imputation, and benchmarking” for more information.

Exhibit 6. Special cases for calculating strength level
Strength level Description

Unknown

If it is unknown whether lifting or carrying occurs occasionally, frequently, or constantly or none of the conditions in the strength level chart are met and standing or walking or sitting are unknown.

Sedentary

If none of the conditions in the strength level chart are met and standing or walking occurs less than or equal to 1/3 of the work schedule.

Light work

If none of the conditions in the strength chart are met and does not meet the special conditions for unknown or sedentary.

Source: U.S. Bureau of Labor Statistics

Weighting, imputation, and benchmarking

ORS faces obstacles with nonresponse and sampling frame coverage. Nonresponse occurs because participation in the survey is voluntary as a company official may refuse to participate and the associated establishment would then be “nonresponding.” An establishment is “responding” if it provided information for at least one usable sampled job. Some establishments selected from the sampling frame may be out of the scope of the survey or may have gone out of business. The ORS program addresses both of these sampling frame coverage problems with adjustments.  

In order to address nonresponse, specifically unit and item nonresponse, the ORS program adjusts the weights of the responding establishments and imputes missing data values during the estimation process. To mitigate the effects of sampling frame coverage issues, the ORS program using benchmarking. Both of these are done to ensure that occupational requirement estimates are representative of requirements for civilian workers. This section will describe the different nonresponse and sampling frame problems that arise and the weighting, imputation, and benchmarking methods that BLS implements to account for these problems. 

Weighting

BLS adjusts for unit (establishment) nonresponse. A “nonresponding” establishment is one that is unable to provide at least one usable sampled job. BLS treats establishment nonresponse with adjustments that redistribute the weights of nonrespondents to similar respondents. BLS groups similar respondents into cells that are defined by characteristics such as the industry, size class, and geographic area of the establishment. For example, if the nonresponding establishment was in the manufacturing industry and had an employment of 350 workers, ORS would adjust the weights of responding manufacturing establishments with 250–499 workers during estimation. Applied at the establishment level, this adjustment is a nonresponse adjustment factor (NRAF), and it is calculated using the following formula: 

BMF id,cr = CES id,cr i e id,cr w i × e i  

 where:

   cell weight x NRAF x BMF x reported data element weighted employment of all usable establishments in the nonresponse cell

  D = Hires_sa - Seps_sa - ∆Cesemp  weighted employment of all viable but not usable establishments in the nonresponse cell

If there are no responding establishments to reweight within the industry/employment group, then additional responding units from similar geographic areas are considered. Establishments no longer in operation or out of the scope of the survey, and establishments with no workers within the scope of the survey, are considered unviable and excluded from survey estimates.

Situations also arise in which BLS adjusts weights for sampled job nonresponse, which is a situation in which an establishment does not provide any occupational requirements data for a given sampled job. BLS addresses sampled job nonresponse during the interview with an adjustment that redistributes the weights of nonresponding sampled jobs to responding sampled jobs in the same occupational group, ownership, industry, and size class.  

BLS applies additional adjustment factors to special situations that may have occurred during data collection. For example, when a sample unit is one of two establishments owned by a given company and the respondent provides data for both locations combined instead of data for the sampled unit, BLS adjusts the weight of the sampled unit to reflect the employment data for the sampled unit.

Imputation

Item nonresponse is a situation in which an establishment responds to the survey but is unable or unwilling to provide some of the occupational requirements data for a given sampled job. Item nonresponse is addressed through item imputation in certain situations. Item imputation replaces missing values for an item or items with values derived from sampled jobs within similar establishments with similar worker characteristics that have a value for the item. For ORS estimates, items with missing values are imputed within groups of ORS job requirements that are related. For example, one ORS group refers to categorical variables only and includes such requirements as hearing, vision, and driving. Within the group, BLS imputes values by a process that matches sampled jobs using occupational information from similar occupations in similar establishments. Imputation of one group of ORS requirements does not affect the imputation for any other group.

BLS uses additional imputation procedures to align previously collected data with current survey definitions. Where possible, 3 years of ORS samples are included in the estimates, the scope changes implemented before the third year of collection resulted in conceptual inconsistencies for most physical-demand and environmental-conditions requirements across the 3 years. Because of scope change, the first 2 years of the sample were imputed based on response distributions from occupational requirements data collected during the third year. To achieve this, BLS used a multiple imputation approach employing an iterative logistic regression imputation procedure. BLS created several sets of imputations, added the third year of collected data to each set, calculated estimates for each set, and averaged the estimates across the sets to create final ORS estimates. 

Benchmarking 

BLS uses benchmarking to adjust the weight of each establishment in the survey and match the most current distribution of employment by industry. The ORS establishment sample is drawn from the Quarterly Census of Employment and Wages (QCEW) Longitudinal Database and a file of units reporting to the Railroad Retirement Board, and the Current Employment Statistics (CES) survey. The QCEW and the railroad information provide historical employment data, but since these sources do not have current employment data, BLS uses CES to make an adjustment to employment. The benchmark process updates the initial establishment weights, assigned during sampling, by current employment. Benchmarking ensures that survey estimates reflect the most current industry composition–that is the employment counts in proportions consistent with private industry, state government, and local government sectors (hereafter, ownership).

As an example of the benchmarking process, 40 private industry, 10 local government, and 5 state government units in the service sector were selected from the ORS sampling frame. These units consist of establishments employing 200,000 private workers, 30,000 local government workers, and 10,000 state government workers. If, by the time of survey processing, the private service sector experienced an employment increase of 10,000 workers (5 percent) and there is no increase in employment in the service sectors of state and local government, then the sample would underrepresent current employment in the private industry service sector in the absence of benchmarking. In this example, ORS would adjust the sample weights of the 40 service sector firms in private industry to ensure that the number of workers in establishments in the sampling frame rises to 210,000. The ownership employment counts for the private industry service sector would then reflect the current proportions of 84 percent for private industry, 12 percent for local government, and 4 percent for state government employment.

For more information, see the Estimation and Validation within the Research section of the ORS website. 

Reliability of ORS estimates

To assist users in confirming the reliability of ORS estimates, BLS publishes standard errors. Standard errors provide users with a measure of the precision of an estimate to ensure that it is within an acceptable range for their intended purpose. The standard errors are calculated from collected and imputed data. BLS is researching methods for estimating the variance excluding imputed values. For additional information, see www.bls.gov/ors/se.htm.

BLS derives ORS estimates from sampled jobs within responding establishments. Two types of errors are possible in an estimate based on a sample survey: sampling and nonsampling errors. Sampling errors occur because the sample makes up only a part of the population it represents. The sample used for the survey is one of a number of possible samples that could have been selected under the sample design, each producing its own estimate. A measure of the variation among sample estimates is the standard error. Nonsampling errors are data errors that stem from any source other than sampling error, such as data collection errors and data-processing errors.

Standard errors can be used to measure the precision with which an estimate from a particular sample approximates the expected result of all possible samples. The chances are about 68 out of 100 that an estimate from the survey differs from a complete population figure by less than the standard error. The chances are about 90 out of 100 that this difference is less than 1.6 times the standard error. Statements of comparison appearing in ORS publications are significant at a level of 1.6 standard errors or better. This means that, for differences cited, the estimated difference is more than 1.6 times the standard error of the difference.

The ORS program uses balanced repeated replication (BRR) to estimate the standard error. The procedure for BRR entails first partitioning the sample into variance strata composed of a single sampling stratum or clusters of sampling strata, and then splitting the sample units in each variance stratum evenly into two variance primary sampling units (PSUs). Next, ORS chooses half-samples so that each contains exactly one variance PSU from each variance stratum. Choices are not random but are designed to yield a “balanced” collection of half-samples. By using half-samples, we can compute a “replicate” estimate with the same formula for the regular, or “full-sample,” estimate, except that the final weights are adjusted. If a unit is in the half-sample, its weight is multiplied by (2 – k); if not, its weight is multiplied by k. For all ORS publications, k = 0.5, so the multipliers are 1.5 and 0.5.

The BRR estimate of the standard error with R half samples is 

D = 40 - 60 - - 25 = 5 ,

where:

the summation is over all replicates of half-samples r = 1,...,R,

  PAHires_sa = Hires_sa Hires_sa + Seps_sa xD  is the rth replicate estimate, and

  PAHires_sa = 40 40 + 60 x5 = 2  is the full-sample estimate. 

Quality assurance programs mitigate collection and processing errors using data collection reinterviews, observed interviews, computer edits of the data, and systematic professional review of the data. These programs also serve as a training device to provide feedback to field economists, or data collectors, on errors and the sources of errors that can be remedied by improved collection instructions or computer-processing edits. Field economists receive extensive training to maintain high standards in data collection.

Once estimates of occupational requirements are produced, the estimates are validated. The focus of the validation is to compare the estimates with expectations for them. Expectations are based on values of the ORS estimates from prior years as well as similar estimates from other sources of data, such as the Occupational Information Network (O*NET). In addition, ORS estimates between similar occupations are compared. 

BLS investigates estimates that deviate from their expectations to ensure that their underlying data are consistent with ORS collection procedures, and that their calculation is consistent with ORS statistical procedures. They designate estimates that are consistent with these procedures as “fit-for-use” for publication.

Before publishing any estimate, BLS reviews it to make sure that it meets specified statistical reliability and confidentiality requirements. The review prevents the publication of an estimate that has a large standard error or that could reveal information about a specific establishment. See data review and estimate validation for additional information.

Exhibit 7. List of calculated occupational requirements by category and estimate type
Occupational requirement Categorical Continuous

Physical demands

Percentage Mode Mean Percentile ⁠(1)

Sitting or standing and walking

Standing and walking

Sitting

Sitting vs standing at will

Auditory and vision

Hearing

One on one

Group

Telephone

Other sounds

Pass a hearing test

Vision

Near visual acuity

Far visual acuity

Peripheral vision

Verbal communication

Driving

Climbing

Ramps/stairs: structural only

Ramps/stairs: work-related

Ladders/ropes/scaffolds

Lifting and carrying

Strength

Weight (range) lifted/carried- seldom

Weight (range) lifted/carried - occasionally

Weight (range) lifted/carried - frequently

Weight (range) lifted/carried - constantly

Most weight ever lifted/carried (pounds)

Reaching and manipulation

Reaching overhead

One or both

Reaching at or below the shoulder

One or both

Foot/leg controls

One or both

Gross manipulation

One or both

Fine manipulation

One or both

Keyboarding: traditional

Postural

Crawling

Crouching

Stooping

Kneeling

Pushing and pulling

With hand/arm

One or both

With foot/leg

One or both

With feet only

One or both

Environmental conditions

Extreme Cold (non-weather related)

Extreme Heat (non-weather related)

Wetness (non-weather related)

Humidity

Heavy vibration

High, exposed places

Proximity to moving mechanical parts

Outdoors

Hazardous contaminants

Noise Intensity Level

Education, training, and experience

Specific vocational preparation (SVP)

Minimum formal education or literacy required

Degree by type

Associates degree time (days)

Vocational associates degree time (days)

High school vocational time (days)

Literacy (if no high school required)

Other training and experience

Pre-employment training (license, certification, other)

Prior work experience

Post-employment training

Pre-employment training (certification)

Pre-employment training (license)

Pre-employment training (educational certification)

Pre-employment training (other)

Footnotes:

(1) Percentile estimates are calculated at the 10th, 25th, 50th (median), 75th, and 90th.

Note: √ = Potential estimate for occupational requirement and … = No estimate for this occupational requirement.

Source: U.S. Bureau of Labor Statistics.

Exhibit 8. List of calculated physical and environmental occupational requirements with duration
Occupational requirement Duration levels ⁠(1) calculated

Physical demands

Sitting or standing and walking

Standing and walking

⁠(2)

Sitting

⁠(2)

Auditory and vision

Verbal communication

Climbing

Ramps/stairs: work-related

Ladders/ropes/scaffolds

Reaching and manipulation

Reaching overhead

Reaching at or below the shoulder

Foot/leg controls

Gross manipulation

Fine manipulation

Keyboarding: traditional

Postural

Crawling

Crouching

Stooping

Kneeling

Pushing and pulling

With hand/arm

With foot/leg

With feet only

Environmental conditions

Extreme cold (non-weather related)

Extreme Heat (non-weather related)

Wetness (non-weather related)

Humidity

Heavy vibration

High, exposed places

Proximity to moving mechanical parts

Outdoors

Hazardous contaminants

Footnotes:

⁠(1) Duration levels include seldom, occasionally, frequently, and constantly, as described earlier in this section.

⁠(2) Available as a continuous estimate.

Note: v = Potential estimate for occupational requirement.

Source: U.S. Bureau of Labor Statistics.

Last Modified Date: April 29, 2019