In this rather lengthy video which ultimately may be considered a preview of his new book, Malcolm Gladwell regurgitates a thread from Moneyball and one which is the core topic of concern to any selection specialist. The focus is on the problem of identifying effective predictors of job performance. As in Moneyball, MG discusses the use of field tests as predictors of future performance in sport. He also uses examples of cognitive test in the National Football League, physical abilities tests and subjective tests as predictors of performance in hockey, football, and basketball.
The focus is on what MG has termed 'mismatch'. Mismatch means identifying the wrong person for the job and fundamentally rests on choosing the wrong predictors (as was the story in Moneyball). To generalize the problem, MG goes on to provide examples of the difficulty of predicting effective performance in teaching and in the legal profession.
In teaching, the question is what predicts teacher quality? This is clearly important as it has an impact on student learning. Traditional predictors include various dimensions of education and experience. To teach in the US you must have a first degree, certification, state licensing, and academic work specific to the specialty. As reported by MG apparently there is no relationship apparently between these predictors and teacher quality.
What is missing from this discussion is a definition of what being a good teacher is. It is true, that in part a good teacher is one who contributes to improvements in test scores of students. However, many would argue that simply seeing standardized test scores improve cannot be judged the only indicator. What if you are lucky enough to work in a school where you have bright students. They will learn despite your efforts to the contrary! What if you are unlucky enough to work in an environment where students have other serious distractions such as family troubles, poverty, gang violence? Can we judge a teacher as poor even if her achievement is simply to keep kids in school? How will we measure this criterion (similar issues for Doctors and Lawyers by the way)?
Unfortunately, in his rant, MG has made a fundamental error. He never considers the complexity of measuring the criterion - what is an excellent athlete, what is an excellent teacher or lawyer? By obfuscating this half of the equation, it is unlikely that any satisfactory analysis will be achieved.
However, Gladwell does pose a challenge to selection specialists everywhere. In a world that is complex and dynamic, where the required behaviors are uncertain, should we continue to search for predictors of performance? Here in my opinion, are some of the flaws in the logic of Gladwells argument:
1. He ignores the fact that even a weak predictor is actually likely to improve your odds over no predictor at all (chance) or a bad predictor (stereotypical ideas of a good performer). If you combine several modestly predictive measures you can significantly impact job performance of new hires.
2. Performance is complex and must be defined clearly. Sometimes we can predict parts of a job performance rather than overall job performance.
3. Using standardized predictors increases actual and perceived fairness. It helps us to ensure that blacks, whites, women, men, old and young etc. all receive consistent treatment. Furthermore, it enables us to hold companies accountable for how the decision is made.
4. Despite the examples of cases where selection systems do not work, MG ignores the fact that when done well, careful selection improves the average performance of the employees hired and that has a positive impact on organizational performance. This was the other side of the story in Moneyball!
I am looking forward to Mr Gladwell's book. He chooses interesting topics and writes beautifully. What is your opinion, is selection more or less important as the world increases in complexity?
No comments:
Post a Comment