The Occupational Requirements Survey: estimates from preproduction testing

The Occupational Requirements Survey: estimates from preproduction testing.

This article introduces preliminary estimates and is a followup to previous articles that examined ORS data elements in the context of collection, review, and estimation.

 

Pre-production Testing Information

 

Fiscal Year 2015 Plans

In Fiscal Year 2015, BLS plans to:

  • Conduct a large-scale nationwide pre-production test to complete the evaluation of whether ORS can produce estimates of sufficient quantity and quality to move the survey into a production environment.
  • Continue working on areas from the Fiscal Year 2014 testing, including further refinement and testing of the new mental cognitive data elements, survey design processes, protocols, aids, and collection procedures; developing best practices to collect data from large employers and conducting additional research regarding the methods of collection and approaches for testing the validity and reliability of occupational requirements data.

 

Fiscal Year 2015 pre-production documentation:

  1. Occupational Requirements Reliability and Validity Literature Research - the report provides advice on achieving high levels of reliability, validity, and accuracy given the ORS' parameters. The views expressed in this document are those of the author and not necessarily those of BLS or SSA.

    • Executive Summary (PDF)

    • Full Report (PDF)

  2. ORS Pre-Production Final Report, September 2015 (HTML) (PDF)
    • ORS Pre-production Estimation and Validation Report, September 2015 (HTML) (PDF)

    • ORS Pre-production Collection Activities Report, June 2015 (HTML) (PDF)
    • ORS Pre-production Job Observation Report, November 2015 (HTML) (PDF)

  3. American Statistical Association, Joint Statistical Meeting (JSM) articles
    • Occupational Requirements Survey Sample Design (HTML) (PDF)
    • Using Calibration Training to Assess the Quality of Interviewer Performance (HTML) (PDF)
    • Building Quality Assurance for the Occupational Requirements Survey (HTML) (PDF)
    • Occupational Requirements Survey (ORS) Data Review Process (HTML) (PDF)
    • Estimation Processes Used in the Occupational Requirements Survey (HTML) (PDF)
    • Validation of Estimates in the Occupational Requirements Survey (HTML) (PDF)

 

ORS Pre-Production Materials

  1. Occupational Requirements Survey Visual Aid NEW
  2. ORS Information Sheet - October 2014 (PDF) (en Español) (PDF)
  3. Watching Work: The 2015 Job Observation Test (HTML) (PDF) NEW

 

Fiscal Year 2014 Occupational Requirements Testing Results

 

In Fiscal Year 2014, the Bureau of Labor Statistics successfully completed a series of tests to assess the viability of using the National Compensation Survey platform to accurately and reliably capture data relevant to the Social Security Administration's (SSA) disability program. These tests can be grouped into two categories: (1) tests of the feasibility of collecting mental-cognitive requirements of work that were not part of prior phases of collection test efforts but are essential to the SSA disability program and (2) tests to aid the BLS in refining survey operations for the collection of the Occupational Requirements Survey (ORS).

The outcomes confirmed that mental-cognitive requirements of work are collectable and fit within the collection of NCS information that is typically obtained when determining the duties and responsibilities and work level for the occupation. Although additional refinements to survey operation to support ORS collections will continue, the BLS believes the NCS survey platform is viable for the collection of occupational requirements survey data for use by the SSA.

For additional information, see the Fiscal Year 2014 Summary Report.

  • Fiscal Year 2014 Test Plans (PDF)
  • Phase 3 Summary Report (PDF)
  • Phase 2 Summary Report (PDF)
  • Phase 1 Summary Report (PDF)
  • Article - Testing the Collection Requirements Data (PDF)
  • Article - Sample Design Evaluation (PDF)
  • Article - Estimation Considerations (PDF)
  • Article - Analysis of Approaches to Validating Data (PDF)

 

Last Modified Date: December 5, 2016