## Occupational Requirements Survey: Calculation

The ORS estimates provide information about the physical demands; environmental conditions; education, training, and experience; and mental requirements of jobs in the U.S. economy. Job requirement categories in ORS have estimates conveyed as the percentage of workers, mean (in hours, days, pounds, or percentage of a day), percentiles, and mode estimates for each occupational definition. Most physical demands and environmental conditions have duration associated with their requirements, which are grouped within a duration level, and a percentage-of-workers estimate is calculated for each of these levels (see the “Duration Level” table below).

Some categories have estimates that use the mean to convey duration, such as sitting or standing/walking and various education, training, and experience components. For example sitting is captured using hours, so mean and percentile estimates (10⁠th, 25⁠th, 50⁠th, 75⁠th, and 90⁠th percentiles) are calculated for both hours and the percent of day spent sitting for a specific occupation or occupational group.

Duration level
Not present

Requirement is not present and there is no duration

Seldom

Up to 2 percent of the workday

Occasionally

2 percent and up to 1/3 of the workday

Frequently

1/3 up to 2/3 of the workday

Constantly

2/3 or more of the workday

For physical demands and environmental conditions, the mode of the category is identified, that is, which duration level has the largest weighted number of workers. For other categories that do not have duration levels associated with them, the mode is also determined. For example, the minimum education that is the most common for security guards is a high school diploma.

Field economists collect ORS job requirements for over 70 categories; however, as explained above, many estimates can be calculated from one collected job requirement. This results in many more calculated ORS estimates per occupation (or occupational group). For a full list of calculated elements, please see Exhibit A at the end of this section.

The formulas used to calculate these estimates are shown below. The type of estimator used depends on the job requirement category. For categorical job requirement estimates, a percentage of workers is calculated, and a mode identified for these percentages. For continuous job requirement estimates (such as duration in hours or days and maximum weight lifted/carried elements), mean and percentile estimates are calculated. Exhibit A provides the breakdown of estimate type by job requirement category.

Percentage. The formula for the percentage of workers with a given job requirement out of all workers in the domain (such as an occupation) is

$∑ i = 1 I ∑ g = 1 G i OccFW ig × X ig × Z ig ∑ i = 1 I ∑ g = 1 G i OccFW ig × X ig × 100$

where:

i is the establishment,

g is the occupation within establishment i,

I is the total number of establishments,

Gi is the total number of quotes (or sampled jobs) in establishment i,

Xig is 1 if worker ig meets the condition set in the domain (denominator) condition and 0 otherwise.

Zig is 1 if worker ig meets the condition set in the requirement condition and 0 otherwise.

OccFWig is the final quote weight for occupation g in establishment i.

Average (mean). The formula for the average value of a quantity for a requirement is

where:

i is the establishment,

g is the occupation within establishment i,

I is the total number of establishments,

Gi is the total number of quotes (or sampled jobs) in establishment i,

Xig is 1 if worker ig meets the condition set in the domain (denominator) condition and 0 otherwise.

Zig is 1 if worker ig meets the condition set in the requirement condition and 0 otherwise.

OccFWig is the final quote weight for occupation g in establishment i.

Qig is the value of a quantity for a specific requirement for occupation g in establishment i.

Percentiles. The grouping of various categories are used to describe the distribution of a numeric value. The following percentiles p are calculated: 10⁠th, 25⁠th, 50⁠th (median), 75⁠th, and 90⁠th. The pth percentile is the value Qig, where the value of a quantity is for a specific category, such that

· the sum of final quote weights (OccFWig) across quotes with a value less than Qig is less than p percent of all final quote weights and

· the sum of final quote weights (OccFWig) across quotes with a value more than Qig is less than (100 – p) percent of all final quote weights.

It is possible that there is no specific quote ig for which both of these properties hold. This occurs when there exists a quote for which the OccFWig of records whose value is less than Qig equals p percent of the total weighted quote employment. In that situation, the pth percentile is the average of Qig and the value of the quote with the next-lowest value.

Mode. The mode is the highest percentage estimate within a job requirement category. Refer to Appendix A at the end of this section for a list of elements that have mode estimates.

### Education, training, and experience

Although most of the estimates for these requirements are simply based on establishment responses about the selected jobs’ various tasks, there are some that require an additional level of calculation. One of these is the Specific Vocational Preparation (SVP) level which is determined by the amount of preparation time required by the worker in order to develop the skills needed to perform the job. The job requirement categories that make up this preparation are the minimum education level with respect to formal degree types, pre-employment training, previous work experience, and on-the-job training required by a job. These categories’ associated time are then aggregated and used to determine the SVP level needed for the job shown in the table below:

Specific vocational preparation levelPreparation time
1

Short demonstration only (4 hours or less)

2

Anything beyond short demonstration up to and including 1 month

3

Over 1 month up to and including 3 months

4

Over 3 months up to and including 6 months

5

Over 6 months up to and including 1 year

6

Over 1 year up to and including 2 years

7

Over 2 years up to and including 4 years

8

Over 4 years up to and including 10 years

9

Over 10 years

### Strength

Another job requirement that is based off several categories’ data is strength. It is measured in five levels: sedentary, light work, medium work, heavy work, and very heavy work. The levels are determined by how much weight a worker is required to lift and/or carry, how often, and standing/walking in some special cases. The strength level is determined by satisfying at least one of the lifting/carrying conditions shown in the table below, or as defined by the “strength special cases” table. The highest strength level satisfied will be the level that represents that sampled job. For example, if a job requires a worker to lift or carry 11–20 pounds occasionally, then it would be classified as light work. However, if that same job were to require lifting or carrying that same weight frequently, then it would be classified as medium work.

Strength level

Lifting/carrying

Light workMedium workHeavy workVery heavy work

Seldom

11–20 pounds21–50 pounds51–100 pounds>100 pounds

Occasionally

11–20 pounds21–50 pounds51–100 pounds>100 pounds

Frequently

≤10 pounds11–25 pounds26–50 pounds>50 pounds

Constantly

Negligible weight≤10 pounds11–20 pounds>20 pounds

The following table outlines the special cases for strength. In instances where field economists are unable to determine certain job requirements from the respondent , they record these data as unknown. See the section “Weighting, Nonresponse Adjustment, Imputation, and Benchmarking” for more information.

Strength level—special cases
Unknown

If lifting occasionally, frequently, or constantly is unknown. Or if none of the conditions in the strength chart is met and stand/walking or sitting is unknown.

Sedentary

If none of the conditions in the strength chart are met and standing/walking is less than or equal to 1/3 of the work schedule

Light work

If none of the conditions in the strength chart are met and does not meet the special conditions for unknown or sedentary.

### Reliability of ORS estimates

To assist users in ascertaining the reliability of ORS estimates, standard errors are published along with each estimate. Standard errors provide users with a measure of the precision of an estimate to ensure that it is within an acceptable range for their intended purpose. The standard errors are calculated from collected and imputed data. BLS is researching methods for estimating the variance excluding imputed values. For additional information, see www.bls.gov/ncs/ors/or/se.htm.

ORS estimates are derived from sampled jobs within responding establishments. Two types of errors are possible in an estimate based on a sample survey: sampling and nonsampling errors. Sampling errors occur because the sample makes up only a part of the population it represents. The sample used for the survey is one of a number of possible samples that could have been selected under the sample design, each producing its own estimate. A measure of the variation among sample estimates is the standard error. Nonsampling errors are data errors that stem from any source other than sampling error, such as data collection errors and data-processing errors.

Standard errors can be used to measure the precision with which an estimate from a particular sample approximates the expected result of all possible samples. The chances are about 68 out of 100 that an estimate from the survey differs from a complete population figure by less than the standard error. The chances are about 90 out of 100 that this difference is less than 1.6 times the standard error. Statements of comparison appearing in ORS publications are significant at a level of 1.6 standard errors or better. This means that, for differences cited, the estimated difference is more than 1.6 times the standard error of the difference.

The ORS uses balanced repeated replication (BRR) to estimate the standard error. The procedure for BRR entails first partitioning the sample into variance strata composed of a single sampling stratum or clusters of sampling strata, and then splitting the sample units in each variance stratum evenly into two variance primary sampling units (PSUs). Next, half-samples are chosen, so that each contains exactly one variance PSU from each variance stratum. Choices are not random but are designed to yield a “balanced” collection of half-samples. By using half-samples, we can compute a “replicate” estimate with the same formula for the regular, or “full-sample,” estimate, except that the final weights are adjusted. If a unit is in the half-sample, its weight is multiplied by (2 – k); if not, its weight is multiplied by k. For all ORS publications, k = 0.5, so the multipliers are 1.5 and 0.5.

The BRR estimate of standard error with R half samples is

$SE Y ^ = 1 R 1 - k 2 ∑ r = 1 R Y r ^ - Y ^ 2$ ,

where:

the summation is over all replicates of half-samples r = 1,...,R,

$Y ^ r$ is the rth replicate estimate, and

$Y ^$ is the full-sample estimate.

Data collection and processing errors are mitigated primarily through quality assurance programs that include the use of data collection reinterviews, observed interviews, computer edits of the data and systematic professional review of the data. The programs also serve as a training device to provide feedback to field economists, or data collectors, on errors and the sources of errors that can be remedied by improved collection instructions or computer-processing edits. Field economists receive extensive training to maintain high standards in data collection.

Once estimates of occupational requirements are produced, the estimates are verified, or validated. The focus of the validation is to compare the estimates with expectations for them. The expectations are based on values of the ORS estimates from prior years as well as similar estimates from other sources of data, such as the Occupational Information Network (O*NET). In addition, ORS estimates between similar occupations are compared.

Estimates that deviate from their expectations are further investigated to ensure that their underlying data are consistent with ORS collection procedures, and their calculation is consistent with the ORS statistical procedures. Estimates that are consistent with these procedures are designated as “fit-for-use” for publication.

Before any estimate is published, it is also reviewed to make sure that it meets specified statistical reliability and confidentiality requirements. The review prevents the publication of an estimate that could reveal information about a specific establishment or that has a large sampling error.

For additional information on data review and estimate validation, see the Data Review and Validation portion of the research section on the ORS website.

### Weighting, nonresponse adjustment, imputation, and benchmarking

Participation in the survey is voluntary; therefore, a company official may refuse to participate in the survey. In addition, some establishments selected from the sampling frame may be out of the scope of the survey or may have gone out of business. To address the problems of nonresponse and missing data, the ORS program adjusts the weights of the remaining establishments and imputes missing values, to ensure that occupational requirement estimates are representative of requirements for civilian workers during the estimation process. This section describes the current weight adjustments, imputation, and benchmarking methods.

Weight adjustments and imputation are made in accordance with the following steps:

1. Unit nonresponse adjustment: An establishment is considered responding if it provided information for at least one usable quote (or sampled job). A quote is classified as usable if the following data are present: occupational attributes (full-time or part-time schedule, union or nonunion status, and time or incentive type of pay), work schedule, and occupational requirements data for any of the job requirement categories.

An establishment is considered nonresponding if it is unable to provide at least one usable quote. Establishment nonresponse is treated with adjustments that redistribute the weights of nonrespondents to similar respondents by characteristics such as the industry, size class, and geographic area of the establishment. For example, if the nonresponding establishment was in the manufacturing industry and had an employment of 350 workers, the ORS program would adjust the weights of responding manufacturing establishments with 250–499 workers by a nonresponse factor during estimation. This nonresponse adjustment factor (NRAF) at the establishment level is calculated using the following formula:

$NRAF= ∑ A + ∑ B ∑ A$ ,

where:

$∑ A$= weighted employment of all usable establishments in the nonresponse cell

$∑ B$ = weighted employment of all viable but not usable establishments in the nonresponse cell

If there are no responding establishments to reweight within the industry/employment group, then additional responding units from similar geographic areas are considered.

Establishments no longer in operation or out of the scope of the survey, and establishments with no workers within the scope of the survey, are excluded from the survey estimates.

2.  Quote nonresponse adjustment: Quote nonresponse is a situation in which an establishment refuses to provide any occupational requirements data for a given sampled job (quote). Quote nonresponse during the interview is addressed by an adjustment that redistributes the weights of nonresponding quotes to responding sample quotes in the same occupational group, ownership, industry, and size class.

3.  Item nonresponse is a situation in which an establishment responds to the survey but is unable or unwilling to provide some of the occupational requirements data or worker attributes for a given sampled job. Item nonresponse is addressed through item imputation in certain situations. Item imputation replaces missing values for an item or items with values derived from establishments with similar establishment and worker characteristics that have a value for the item. For ORS estimates, items with missing values are imputed within groups of ORS job requirements that are related. For example, one ORS group refers to categorical variables only and includes such requirements as hearing, vision, and driving. Within the group, values are imputed using occupational information from similar occupations in similar establishments. Imputation of one group of ORS requirements does not affect the imputation for any other group.

Additional adjustment factors are applied to special situations that may have occurred during data collection. For example, when a sample unit is one of two establishments owned by a given company and the respondent provides data for both locations combined instead of data for the sampled unit, the weight of the sampled unit is adjusted to reflect the employment data for the sampled unit.

4.  Benchmarking (poststratification). Benchmarking is the process of adjusting the weight of each establishment in the survey to match the most current distribution of employment by industry.

The ORS establishment sample is drawn from the Quarterly Census of Employment and Wages (QCEW) Longitudinal Database and a file of units reporting to the Railroad Retirement Board, and the Current Employment Statistics (CES) survey. The QCEW and the railroad information provide employment data, but because these sources do not  have current employment data, the CES is used to adjust employment. The benchmark process updates the initial establishment weights, assigned during sampling, by current employment. Establishment weights reflect employment at the time of sampling, not collection. Benchmarking ensures that survey estimates reflect the most current industry composition–employment counts in proportions consistent with the private industry, state government, and local government sectors (hereafter, ownership).

For example, 40 private industry, 10 local government, and 5 state government units in the service sector were selected from the ORS sampling frame. These units consist of establishments employing 200,000 private workers, 30,000 local government workers, and 10,000 state government workers. If, by the time of survey processing, the private service sector experienced an employment increase of 10,000 workers (or 5 percent) and there is no increase in employment in the service sectors of state and local government, then the sample would underrepresent current employment in the private industry service sector in the absence of benchmarking. In this example, the ORS would adjust the sample weights of the 40 service sector firms in private industry to ensure that the number of workers in establishments in the sampling frame rises to 210,000. The ownership employment counts for the private industry service sector would then reflect the current proportions of 84 percent for private industry, 12 percent for local government, and 4 percent for state government employment.

For more information, please see the Estimation and Validation portion of the research section on the public ORS website.

Exhibit A: List of Calculated Occupational Requirements By category and estimate type
Occupational requirementCategorical (⁠1),(⁠2)Continuous (⁠1),(⁠2)

Physical Demands

PercentageModeMeanPercentile (⁠3)

Sitting or standing & walking

Standing and walking

Sitting

Sitting vs standing at will

Auditory & Vision

Hearing one on one

Hearing in a group

Hearing over the telephone

Hearing other sounds

Pass a hearing test

Near visual acuity

Far visual acuity

Peripheral vision

Verbal communication

Driving, type of vehicle

Climbing

Ramps/stairs: structural only

Ramps/stairs: work-related

Lifting & Carrying

Strength

Weight (range) lifted/carried- seldom

Weight (range) lifted/carried - occasionally

Weight (range) lifted/carried - frequently

Weight (range) lifted/carried - constantly

Most weight ever lifted/carried (pounds)

Reaching & Manipulation

One or both hands/arms

Reaching at or below the shoulder

One or both hands/arms

Foot/leg controls

One or both feet/legs

Gross manipulation

One or both hands

Fine manipulation

One or both hands

Keyboarding: touch screen

Keyboarding: 10-key

Keyboarding: other

Keyboarding: any keyboarding

Postural

Crawling

Crouching

Stooping

Kneeling

Pushing/Pulling

With hand/arm

One or both

With foot/leg

One or both

With feet only

One or both

Environmental conditions

Extreme Cold (non-weather related)

Extreme Heat (non-weather related)

Wetness (non-weather related)

Humidity

Heavy vibration

High, exposed places

Proximity to moving mechanical parts

Outdoors

Hazardous contaminants

Noise Intensity Level

Education, training, and experience

Specific vocational preparation (SVP)

Minimum formal education or literacy required

Degree by type

Associates degree time (days)

Vocational associates degree time (days)

High school vocational time (days)

Literacy (if no high school required)

Other training & experience

Pre-employment training

Prior work experience

Post-employment training

Pre-employment training (certification)

Pre-employment training (licesnse)

Pre-employment training (educational certification)

Cognitive and mental demands

Regular contacts: type of interaction

Regular contacts: frequency of contact

Other contacts: type of interaction

Other contacts: frequency of contact

Decision-making

Work review

Pace of work

Control of work pace

Pace of work with faster/slower work periods

(⁠1) √ = Potential estimate for occupational requirement

(⁠2) … = No estimate for this occupational requirement

(⁠3) Percentile estimates are calculated at the 10th, 25th, 50th (median), 75th, and 90th

Exhibit B. List of calculated physical and environmental occupational requirements with duration
Occupational requirementDuration levels(⁠1,⁠2) calculated

Physical demands

Sitting or standing & walking

Standing and walking

(⁠3)

Sitting

(⁠3)

Auditory & Vision

Verbal communication

Climbing

Ramps/stairs: work-related

Reaching & Manipulation

Reaching at or below the shoulder

Foot/leg controls

Gross manipulation

Fine manipulation

Keyboarding: touch screen

Keyboarding: 10-key

Keyboarding: other

Postural

Crawling

Crouching

Stooping

Kneeling

Pushing & Pulling

With hand/arm

With foot/leg

With feet only

Environmental conditions

Extreme Cold (non-weather related)

Extreme Heat (non-weather related)

Wetness (non-weather related)

Humidity

Heavy vibration

High, exposed places

Proximity to moving mechanical parts

Outdoors

Hazardous contaminants

(⁠1) Duration levels include seldom, occasionally, frequently, and constantly, as described earlier in this section.

(⁠2) √ = Potential estimate for occupational requirement

(⁠3) Available as a continuous estimate.