Comparing the DOT, O*NET and eDOT

#5: What is the Sample Size?

For occupational classification systems, sample size must be considered in terms of number of:

• jobs surveyed

• analyses made per job

O*NET sample size

Jobs surveyed. When it comes to the jobs surveyed, there are 2 drawbacks to O*NET methodology.

• O*NET targets the high-tech industry
• O*NET ignores self-employed individuals and small employers

O*NET data collection plans target "high growth industries as well as high- and low-technology occupations." It focuses on collecting data for new jobs in the U.S. economy. However, this most likely means that lower-level work is underrepresented in O*NET.

In addition, O*NET's use of SOC and OEWS data neglects domestic workers and certain types of farm laborers (a political consequence of the funding of the OES Prevailing Wage Survey in 1995).

Certain lower-level jobs simply do not exist in O*NET.

Furthermore, O*NET employer samples are taken from companies of 5 or more employees. That means it eliminates all self-employed individuals, including temporary employees, consultants, etc. This creates a small, but growing group of individuals who are not represented.

It also raises issues of bias, since some jobs are performed quite differently for small employers than for larger employers.

Example: Self-employed individuals may be required to have skills and abilities (such as the ability to self-manage, perform minor accounting duties, deal with customers, etc.) that individuals working for an employer might not be required to exhibit.

This data is especially important because small employers and self-employment often provide the best opportunities for disabled workers.

Number of analyses. To ERI's knowledge, no research exists as to the number of job analysis submissions that are required for validity when:

• employees do the scoring
• the main part of the questionnaire is divided into 3 parts, with 3 separate individuals describing "the elephant from their respective sides"

O*NET states its target sample size per occupation is at least 15 respondents.

But is this really enough? Let's take a look...

Consider the fact that O*NET's target for analyses is 15 analyses per occupation. That means 45 employees must be surveyed (each completing one of 3 questionnaires) for 1 OU. But each OU can incorporate up to 500 DOT/eDOT positions (since O*NET collapses all jobs into less than 1,000 OUs). Thus, a single OU represents a large range of work.

Can 15 surveys really adequately assess 500 separate jobs?