Comparing the DOT, O*NET and eDOT

COURT CHALLENGES

Now that you are familiar with these 3 occupational classification systems, we need to look at a crucial question:

Does their data hold up in court?

This is an important question for anyone who uses occupational classification data to make hiring decisions, set pay or assess transferable skills in disability cases.

To feel confident that the data you use can hold up under court challenge, you must be able to answer the following questions:

• Who collected the data?
• How was the data collected?
• What measures are used?
• What does the collected data represent?
• What is the sample size?
• What is the rate of error?

The following sections will examine how O*NET and eDOT answer these questions. We will NOT examine the reliability of the DOT, since all parties agree that it is outdated.

#1: Who Collected the Data?

Finding out who collected the data is crucial to determining reliability.

Let's take a look...

O*NET uses job incumbents

O*NET is highly unusual in that it primarily collects data directly from job incumbents via questionnaires.

The questionnaire instructions do NOT specify whether a job incumbent should rate the position based on:

• his/her own level of competency
• the minimum needed to do (and keep) the job
• the average level the job generally requires

Here is an example of how questions are structured on an O*NET questionnaire:

11. Thinking Creatively	Developing, designing, or creating new applications, ideas, relationships, systems, or products, including artistic contributions.

Scales. The numeric scales used in these questions are less straightforward than the DOT's functioning levels of "Seldom, Occasionally and Frequent." Unsurprisingly, most incumbents tend to rate their job requirements toward the midpoint and there is high correlation between the Importance and Level scales indicating that workers may not make much distinction between the two.

Reading level. In addition, the level of reading comprehension required by O*NET questionnaires may be too high. Questions require a relatively sophisticated comprehension level, seemingly above the proposed 6th grade reading level. And they rely on terminology used in organizational psychology. This may hinder O*NET's ability to obtain accurate responses from some respondents.