How to avoid adverse impact in testing
05/11/2008, Author: Alan Redman
November 5th 2008 and the big news this morning are the great results - the life affirming news that after a long journey a controversial issue related to race was settled in exactly the way we all hoped.
Plus Barack Obama was elected. Hooray....
The results closer I'm talking about are related to a long term study into adverse impact and testing we have conducted. We have been analysing data going back to 2005 for over 22,000 candidates who completed our Utopia graduate tests online.
The great news is that we found no significant differences in test scores for UK-White candidates and other groups including Asian, Bangladeshi, Black, Chinese, Non-UK white, UK Irish and Latino-Hispanic. Only one group in the study showed any significant difference and that was UK-Pakistani, who scored 1 or 2 points lower as a group. We will of course publish these results in a more formal way along with analyses of other assessment methods completed by the same group.
In the meantime I think this is a good opportunity to reiterate what we at Criterion believe are the steps to take to avoid adverse impact and enjoy great results like this one.
So here they are; the critical success factors to avoiding adverse impact in testing:
1. Avoid heavily speeded tests. Tests with short time limits and a lot of questions will work against candidates with a slower reading speed, such as those taking the test in their second language or with some form of dyslexia etc.
2. Avoid tests with overly complex language. Use tests that employ business English appropriate for the role.
3. Use tests that include good quality practice materials. Not just sample questions but extensive and practical advice on how to do well in testing.
4. Avoid treating different candidates differently. Do not adjust administration times/conditions for individual candidates because you believe they the test might adversely affect them. Instead interpret their results sensitively based on what you know about the candidate. This ties in with the next point...
5. Get informed consent upfront. Communicate the nature of any testing effectively to candidates and ensure any potential difficulties they may experience are communicated back to you.
6. When choosing a test ask the developer what steps they took in its design to minimise potential bias. If they can't tell you then they probably didn't do it. This links with the next point..
7. Don't let test publishers blind you with science. Validity studies and adverse impact studies vary in quality. Question what you're being presented with.
8. Make sure that your test is content valid - it reflects the demands and content of the job itself. If your testing is ever challenged then this is what you'll need to demonstrate. This means doing some job analysis and having a job/person specification in place.
9. Make sure the comparison groups you use for scoring reflect the composition of your candidate group. Do not use separate language norms in recruitment.
10. Remember that the test is likely to be the least of your worries. Interviews and other assessments can be even more problematic in terms of bias since they include far more of the human element.
Maybe you know some more?
You can maybe find some more on this on the British Psychology Society's website. But to be honest this is not always a great resource for test users.
This blog talks about it in terms of personality testing and research conducted with a Big-5 measure (i.e. nothing you'd actually want to use in a modern work context). Plus it references a meta-analysis, which makes me want to reach for my gun.
Still, good fairness in testing data doesn't come along everyday - not least as much as 22,000 candidates. So with that and the US election a good day for a Wednesday.
Contact us if you like to receive more about this study once it's published.
If you find this post offensive please contact us
Comments
No Comments
You must be logged in to leave comments. You can register here».
