United States Department of Labor

The .gov means it's official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

# Handbook of Methods Occupational Requirements Survey Calculation

## Occupational Requirements Survey: Calculation

The Occupational Requirements Survey (ORS) calculates categorical and continuous estimates of job requirements by occupation and occupational group. Categorical estimates measure the percentage of workers by job requirement. For many of the categorical estimates, the ORS also identifies the mode within a categorical grouping. Continuous estimates measure the average (or mean in hours, days, percentage of workday or pounds) or percentile for the job requirement. See exhibit 8 at the end of this section for a full list of published occupational requirements as well as a list corresponding estimate types. This section includes the formulas used to calculate the ORS estimates.

### Estimation formulas

Percentage of workers. The formula for the percentage of workers with a given job requirement in the domain (occupation or occupational group) is:

$∑ i = 1 I ∑ g = 1 Gi OCCFW ig × X ig × Z ig ∑ i = 1 I ∑ g = 1 Gi OCCFW ig × X ig × 10 0$

where,

I is the total number of establishments,

Gi is the total number of sampled jobs in establishment i, i is the establishment,

g is the occupation within establishment i,

$ΟccFW ig$$ΟccFW ig$$ΟccFW ig$$ΟccFW ig$$ΟccFW ig$ is the final sampled job weight for occupation g in establishment i,

Xig is 1 if sampled job ig meets the condition set in the domain (denominator) condition and 0 otherwise, and

Zig is 1 if sampled job ig meets the condition set in the requirement condition and 0 otherwise.

Average (mean). The formula for the average (mean) estimate of a job requirement is:

$∑ i = 1 I ∑ g = 1 Gi OCCFW ig × X ig × Z ig × Q ig ∑ i = 1 I ∑ g = 1 Gi OCCFW ig × X ig × Z ig × 10 0$

where,

I is the total number of establishments,

Gi is the total number of sampled jobs in establishment i, i is the establishment, g is the occupation within establishment i,

is the final sampled job weight for occupation g in establishment i,

Xig is 1 if worker ig meets the condition set in the domain (denominator) condition and 0 otherwise,

Zig is 1 if worker ig meets the condition set in the requirement condition and 0 otherwise, and

Qig is the value of a quantity for a specific requirement for occupation g in establishment i.

Percentiles. The 10th, 25th, 50th (median), 75th, and 90th, percentiles are calculated. The pth percentile is the value Qig, where the value of a quantity is for a specific category for occupation g in establishment i, such that:

• the sum of final sampled job weights () across sampled jobs with a value less than Qig is less than p percent of all final sampled job weights, and
• the sum of final sampled job weights () across sampled jobs with a value more than Qig is less than (100 - p) percent of all final sampled job weights.

It is possible that there is no specific sampled job ig for which both of these properties hold. This occurs when there exists a sampled job for which the  of records whose value is less than Qig equals p percent of the total weighted sampled job employment. In that situation, the pth percentile is the average of Qig and the value of the sampled job with the next lowest value.

### Duration

Duration corresponds with the amount of time that workers perform physical demands or the length of exposure to environmental conditions. Exhibit 4 provides the duration levels with the corresponding percent or fraction of the workday that workers perform physical demands or are exposed to environmental conditions. See exhibit 9 at the end of this section for a list of job requirements with associated duration levels.

Exhibit 4. Duration levels and amount of the workday associated with each level
Duration level Presence of the requirement in the workday

Not present

Requirement is not present and there is no associated duration

Seldom

Up to 2 percent of the workday

Occasionally

2 percent and up to 1/3 of the workday

Frequently

1/3 up to 2/3 of the workday

Constantly

2/3 or more of the workday

Source: U.S. Bureau of Labor Statistics, Occupational Requirements Survey.

The ORS calculates a percentage-of-workers estimate for each duration level. In addition, estimates of some physical demands use averages (means) and percentiles to convey duration, such as sitting and standing. For example, the ORS measures sitting in hours, as well as the average (mean) and percentile estimates (10th, 25th, 50th, 75th, and 90th percentiles) are calculated for both hours and the percentage of the workday spent sitting for a specific occupation or occupational group. The ORS also calculates average (mean) and percentile estimates for education, training, and experience requirements.

For some estimates the ORS identifies the mode, which is the estimate (such as duration level) with the largest weighted number of workers.

### Specific vocational preparation (SVP)

Although the ORS calculates most job requirement estimates from establishment responses about the selected jobs' various tasks, some require an additional level of calculation. One of these is the specific vocational preparation (SVP) level, which is the amount of preparation time required for the worker to develop the skills needed to perform the job. The job requirements that contribute to the SVP are the minimum formal education, credentials, prior work experience, and on-the-job training. These requirements' associated time are then aggregated and used to determine the SVP level needed for the job. (See exhibit 5.)

Concurrent time due to credentials necessary for jobs that also require minimum formal education level, experience, or on-the-job training are not included separately in SVP. Concurrent time is reflected in the education, training, and experience requirements where the time overlaps with time necessary to obtain licenses, certifications, or other non-degree credentials.

Exhibit 5. Preparation time necessary for each specific vocational level
Specific vocational preparation (SVP) level Preparation time

1

Short demonstration only (4 hours or less)

2

Anything beyond short demonstration up to and including 1 month

3

Over 1 month up to and including 3 months

4

Over 3 months up to and including 6 months

5

Over 6 months up to and including 1 year

6

Over 1 year up to and including 2 years

7

Over 2 years up to and including 4 years

8

Over 4 years up to and including 10 years

9

Over 10 years

Source: U.S. Bureau of Labor Statistics, Occupational Requirements Survey.

### Strength

Strength levels are derived from several physical requirements. The estimates reflect the amount of weight workers are required to lift or carry, how often, and whether standing or walking is required to perform critical tasks in the workday. The strength levels indicate whether jobs are considered sedentary, light work, medium work, heavy work, and very heavy work. Exhibits 6 and 7 provide the conditions necessary for estimates to reflect the strength levels.

The highest strength level satisfied is the level that represents that sampled job. For example, if a job requires workers to lift or carry 11–25 pounds occasionally, then it is classified as light work. However, if that same job were to require lifting or carrying that same weight frequently, then it is classified as medium work.

Exhibit 6. Determining strength level based on duration of lifting or carrying
Strength level Duration of lifting or carrying
Seldom Occasionally Frequently Constantly

Light work

11–25 pounds 11–25 pounds 1–10 pounds Negligible weight⁠1

Medium work

26–50 pounds 26–50 pounds 11–25 pounds 1–10 pounds

Heavy work

51–100 pounds 51–100 pounds 26–50 pounds 11–25 pounds

Very heavy work

>100 pounds >100 pounds >50 pounds >25 pounds

⁠1 Negligible weight includes anything lifted or carried weighing less than 1 pound.

Source: U.S. Bureau of Labor Statistics, Occupational Requirements Survey.

As noted, there are special cases for strength. The following table outlines the special cases. In instances where field economists are unable to determine certain job requirements from the respondent, they record these data as "unknown" and strength level is derived through imputation. See the section "Benchmarking, weighting, and imputation" for more information.

Exhibit 7. Determining strength level based on sitting or standing
Strength level Description

Unknown

If it is unknown how often lifting or carrying occur or none of the conditions in exhibit 5 are met and standing or sitting are unknown. The estimate is treated as item of nonresponse.

Sedentary

If none of the conditions in table 3 are met and standing occurs less than or equal to 1/3 of the workday.

Light work

If none of the conditions in table 3 are met and does not meet the special conditions for unknown or sedentary.

Source: U.S. Bureau of Labor Statistics, Occupational Requirements Survey.

### Low postures

The low postures estimates indicate whether workers are required to perform critical tasks while crawling, crouching, kneeling, or stooping. The amount of the workday workers are required to perform critical tasks in low postures is provided as the percentage of workers by duration level. Individual estimates of low postures are also published as required or worker choice. Job tasks may require workers to perform low postures but workers may choose whether to crawl, crouch, kneel, or stoop to complete the work activities.

### Benchmarking, weighting, and imputation

The ORS program addresses establishment refusals, item nonresponse, and out of business and out of scope units. Benchmarking compares the ORS sample to the Quarterly Census of Employment and Wages (QCEW) data, using it as a universe. After benchmarking, the ORS program adjusts the weights of the responding establishments and imputes missing data values during the estimation process in order to address nonresponse (specifically unit nonresponse). Imputation is used to address item nonresponse, which is when an establishment responds to the survey but is unable or unwilling to provide all of the occupational requirements data needed to include the data in estimation.

#### Benchmarking

The ORS uses benchmarking to adjust the weight of each establishment in the survey and match the most current distribution of employment by several establishment and occupation characteristics. The ORS benchmark process uses the Quarterly Census of Employment and Wages (QCEW) Longitudinal Database and a file of units reporting to the Railroad Retirement Board, the Occupational Employment Statistics (OES) survey, and the Current Employment Statistics (CES) survey. The QCEW and the railroad information and OES survey provide historical employment data, but since these sources do not have current employment data, the ORS uses CES to make an adjustment to employment. The benchmark process updates the initial establishment weights, assigned during sampling, by current employment. Benchmarking ensures that survey estimates reflect the most current employment distribution by industry, employment size, geographic area, and occupational group.

As an example of the benchmarking process, 40 private industry, 10 local government, and 5 state government units in the service sector were selected from the ORS sampling frame. These units consist of establishments employing 200,000 private workers, 30,000 local government workers, and 10,000 state government workers. If, by the time of survey processing, the private service sector experienced an employment increase of 10,000 workers (5 percent) and there is no increase in employment in the service sectors of state and local government, then the sample would underrepresent current employment in the private industry service sector in the absence of benchmarking. In this example, the ORS program would adjust the sample weights of the 40 service sector firms in private industry to ensure that the number of workers in establishments in the sampling frame rises to 210,000. The ownership employment counts for the private industry service sector would then reflect the current proportions of 84 percent for private industry, 12 percent for local government, and 4 percent for state government employment.

#### Weighting

An establishment is considered responding if it provided information for at least one sampled job. Similarly, a nonresponding establishment is one that is unable or unwilling to provide information for at least one sampled job. If the contact person (respondent) for an establishment refuses to participate, then the associated establishment is considered nonresponding. The ORS program adjusts weights for unit (establishment) nonresponse by redistributing the weights of nonresponding establishments to similar establishments. The ORS program groups similar respondents into cells that are defined by characteristics such as the industry, size class, and geographic area of the establishment. For example, if the nonresponding establishment was in the manufacturing industry and had an employment of 350 workers, the ORS program would adjust the weights of responding manufacturing establishments with 250–499 workers during estimation. Applied at the establishment level, this adjustment is a nonresponse adjustment factor (NRAF), and it is calculated using the following formula:

$NRAF= ∑ A + ∑ B ∑ A$

where,

A = weighted employment of all usable establishments in the nonresponse cell, and

B = weighted employment of all viable but not usable establishments in the nonresponse cell.

If there are no responding establishments to reweight within the industry and/or employment group, then additional responding units from similar geographic areas are considered. Establishments no longer in operation or out of the scope of the survey, and establishments with no workers within the scope of the survey, are considered unviable and excluded from survey estimates.

Situations also arise in which the ORS program adjusts weights for sampled job nonresponse, which is a situation in which an establishment does not provide any occupational requirements data for a given sampled job. The ORS program addresses sampled job nonresponse during the interview with an adjustment that redistributes the weights of nonresponding sampled jobs to responding sampled jobs in the same occupational group, ownership, industry, and size class.

The ORS program applies additional adjustment factors to special situations that may have occurred during data collection. For example, when a sample unit is one of two establishments owned by a given company and the respondent provides data for both locations combined instead of data for the sampled unit, the ORS program adjusts the weight of the sampled unit to reflect the employment data for the sampled unit.

#### Imputation

Item nonresponse is a situation in which an establishment responds to the survey but is unable or unwilling to provide some of the occupational requirements for a given sampled job. Item nonresponse is addressed through item imputation in certain situations. Item imputation replaces missing values for an item or items with values derived from sampled jobs within similar establishments with similar worker characteristics that have a value for the item. For ORS estimates, items with missing values are imputed within groups of ORS job requirements that are related. For example, one ORS group refers to categorical variables only and includes such requirements as vision, and driving. Within the group, the ORS imputes values by a process that matches sampled jobs using occupational information from similar occupations in similar establishments.

For more information, see estimation within the research section of the ORS website.

### Reliability of ORS estimates

To assist users in confirming the reliability of ORS estimates, the ORS publishes standard errors. Standard errors provide users with a measure of the precision of an estimate to ensure that it is within an acceptable range for their intended purpose. The standard errors are calculated from collected and imputed data. The ORS program is researching methods for estimating the variance excluding imputed values. Examples on how to build confidence intervals using standard errors are included in the standard error section of the ORS website.

The ORS derives estimates from sampled jobs within responding establishments. Two types of errors are possible in an estimate based on a sample survey: sampling and nonsampling errors. Sampling errors occur because the sample makes up only a part of the population it represents. The sample used for the survey is one of a number of possible samples that could have been selected under the sample design, each producing its own estimate. A measure of the variation among sample estimates is the standard error. Nonsampling errors are data errors that stem from any source other than sampling error, such as data collection errors and data-processing errors.

Standard errors can be used to measure the precision with which an estimate from a particular sample approximates the expected result of all possible samples. The chances are about 68 out of 100 that an estimate from the survey differs from a complete population figure by less than the standard error. The chances are about 90 out of 100 that this difference is less than 1.6 times the standard error. Statements of comparison appearing in ORS publications are significant at a level of 1.6 standard errors or better. This means that, for differences cited, the estimated difference is more than 1.6 times the standard error of the difference.

The ORS uses balanced repeated replication (BRR) to estimate the standard error. The procedure for BRR entails first partitioning the sample into variance strata composed of a single sampling stratum or clusters of sampling strata, and then splitting the sample units in each variance stratum evenly into two variance primary sampling units (PSUs). Next, the ORS program chooses half-samples so that each contains exactly one variance PSU from each variance stratum. Choices are not random but are designed to yield a "balanced" collection of half-samples. By using half-samples, the ORS program can compute a "replicate" estimate with the same formula for the regular, or "full-sample," estimate, except that the final weights are adjusted. If a unit is in the half-sample, its weight is multiplied by (2 - k); if not, its weight is multiplied by k. For all ORS publications, k = 0.5, so the multipliers are 1.5 and 0.5.

The BRR estimate of the standard error with R half samples is:

$SE Y ^ = 1 (R(1 - k) 2 ∑ r = 1 R ( Y ^ r - Y ^ ) 2$

where,

the summation is over all replicates of half-samples r = 1,...,R,

$Y ^ r$ is the rth replicate estimate, and

$Y ^$  is the full-sample estimate.

### Quality assurance

The ORS program uses a variety of quality assurance programs to mitigate collection and processing errors by using data collection reinterviews, observed interviews, computer edits of the data, and systematic professional review of the data. These programs also serve as a training device to provide feedback to field economists, or data collectors, on errors and the sources of errors that can be remedied by improved collection instructions or computer-processing edits. Field economists receive extensive training to maintain high standards in data collection.

Once estimates of occupational requirements are produced, the estimates are validated. The focus of the validation is to compare the estimates with expectations for them. Expectations are based prior year estimates as well as similar estimates from other sources of data, such as the Occupational Information Network (O*NET.) In addition, ORS estimates between similar occupations are compared.

The ORS program investigates estimates that deviate from their expectations to ensure that their underlying data are consistent with ORS collection procedures, and that their calculation is consistent with ORS statistical procedures.

Before publishing any estimate, the ORS program reviews it to make sure that it meets specified statistical reliability and confidentiality requirements. See data review and estimation tabs on the research section of the ORS website. Estimates that are consistent with these procedures are designated as "fit-for-use" and released in BLS publications.

Exhibit 8. List of occupational requirements by availability of estimate type
Occupational requirement   Categorical Continuous
Percentage Mode Mean Percentile ⁠1

Cognitive and mental requirements

Ability to pause work

Machinery, equipment, or software

Numerical performance targets

People

Interpersonal skills

Telework available

Problem solving

Verbal interactions

Work around crowds

Work pace

Varying

Consistent, generally slow

Consistent, generally fast

Work review

Frequency of work being checked

Presence of supervisor

Supervising duties

Education, training, and experience requirements

Credentials

Certification

Educational certification

Minimum formal education

Type of formal education

Literacy, if no formal education

On-the-job training

Prior work experience

Specific vocational preparation, SVP

Environmental conditions

Extreme cold (non-weather related)

Extreme heat (non-weather related)

Hazardous contaminants

Heavy vibration

Heights

Humidity

Proximity to moving mechanical parts

Outdoors

Noise intensity level

Quiet

Moderate

Loud

Use of personal protective equipment

Wetness (non-weather related)

Physical demands

Climbing

Structural ramps or stairs

Work-related ramps or stairs

Ladders, ropes, or scaffolds

Driving

Fine manipulation

One or both

Foot and leg controls

One or both

Gross manipulation

One or both

Hearing

In person speech

Other remote speech

Other sounds

Telephone

Keyboarding

Lifting and carrying

Most weight lifted or carried

Low posture

Crawling⁠4

Crouching⁠4

Stooping⁠4

Kneeling⁠4

Pushing and pulling

With foot or leg

One or both

With hand or arm

One or both

Reaching

Reaching at or below the shoulder

One or both

One or both

Sitting⁠2 or standing⁠3

Sitting

Standing

Choice of sitting or standing

Strength level

Sedentary

Light

Medium

Heavy

Very heavy

Vision

a

Far

Near

Peripheral

⁠1 Percentile estimates are calculated at the 10th, 25th, 50th (median), 75th, and 90th.

⁠2 Sitting estimates includes requirements for sitting or laying down for critical tasks.

⁠3 Standing estimates include requirements for standing or walking for critical tasks.

⁠4 Estimates include required, not required, and choice to perform critical tasks in a particular low posture.

Note: ✓ = Potential estimate for occupational requirement and dash (–) = Indicates no estimate for this occupational requirement.

Source: U.S. Bureau of Labor Statistics, Occupational Requirements Survey.

Exhibit 9. List of physical and environmental occupational requirements with associated duration
Occupational requirement Duration levels ⁠1 Percentiles

Education, training, and experience

Credentials

⁠2

Certification

⁠2

⁠2

Educational certificate

⁠2

Prior work experience

⁠3

On-the-job training

⁠2

Environmental conditions

Extreme cold ⁠4

Extreme heat ⁠4

Hazardous contaminants

Heavy vibrations

Outdoors

Proximity to moving mechanical parts

Wetness ⁠4

Physical requirements

Climbing

Ladders, ropes, or scaffolds

Work related ramps or stairs

Fine manipulation

Foot or leg controls

Gross manipulation

Keyboarding

Lifting or carrying

⁠5

Low postures

Reaching

Reaching at or below the shoulder

Pushing or pulling

With hands or arms

With feet or legs

Sitting ⁠6

⁠7

Standing ⁠8

⁠7

Speaking

⁠1 Duration levels correspond to seldom, occasionally, frequently, constantly

⁠2 Estimates provided as number of associated days.

⁠3 Estimates provided as number of associated hours.

⁠4 Nonweather related exposure.

⁠5 Estimates provided as number of associated pounds.

⁠6 Sitting estimates include time spent sitting or laying down.

⁠7 Estimates provides as percentage of the workday and number of the hours in the workday.

⁠8 Standing estimates include time spent standing or walking.

Note: ✓ = Potential estimate for occupational requirement and dash (–) = Indicates no estimate for this occupational requirement.

Source: U.S. Bureau of Labor Statistics, Occupational Requirements Survey.