Great article by Bob Sutton on hiring.
Most remarkable to me is how unpredictable on-the-job performance can be, using:
- unstructured interviews (most common in my experience),
- reference checks (!), and
- job experience (!!)
The upshot of this research is that work sample tests (e.g., seeing if people can actually do key elements of a job – if a secretary can type or a programmer can write code ), general mental ability (IQ and related tests), and structured interviews had the highest validity of all methods examined (Arun, thanks for the corrections). As Arun also suggests, Schmidt and Hunter point out that three combinations of methods that were the most powerful predictors of job performance were GMA plus a work sample test (in other words, hiring someone smart and seeing if they could do the work), GMA plus an integrity test, and GMA plus a structured interview (but note that unstructured interviews, the way they are usually done, are weaker).
Note that this information about combinations is probably more important than the pure rank ordering, as it shows what blend of methods works best, but here is also the rank order of the 19 predictors examined, rank ordered by the validity coefficient, an indicator of how strongly the individual method is linked to performance:
Work sample tests (.54)
GMA tests …“General mental ability” (.51)
Employment interviews – structured (.51)
Peer ratings (.49)
Job knowledge tests (.48) Test to assess how much employees know about specific aspects of the job.
T & E behavioral consistency method (.45) “Based on the principle that past behavior is the best predictor of future behavior. In practice, the method involves describing previous accomplishments gained through work, training, or other experience (e.g., school, community service, hobbies) and matching those accomplishments to the competencies required by the job. a method were past achievements that are thought to be important to behavior on the job are weighted and score
Job tryout procedure (.44) Where employees go through a trial period of doing the entire job.
Integrity tests (.41) Designed to assess honesty … I don’t like them but they do appear to work. 9. Employment interviews – unstructured (.38)
Assessment centers (.37)
Biographical data measures(.35)
Conscientiousness tests (.31) Essentially do people follow through on their promises, do what they say, and work doggedly and reliably to finish their work.
Reference checks (.26)
Job experience –years (.18)
T & E point method (.11)
Years of education (.10)
Interests (.10)
Graphology (.02) e.g., handwriting analysis.
Age (-01)
Certainly, this rank-ordering does not apply in every setting. It is also important to recall that there is a lot of controversy about IQ, with many researchers now arguing that it is more malleable than previously thought. But I find it interesting to see what doesn’t work very well – years of education and age in particular. And note that unstructured interviews, although of some value, are not an especially powerful method, despite their widespread use. Interviews are strange in that people have excessive confidence in them, especially in their own abilities to pick winners and losers – when in fact the real explanation is that most of us have poor and extremely self-serving memories.
Original Article: bobsutton.typepad.com