The current study investigated characteristics and behaviors believed to influence performance on a Test Preparation Manual (TPM) style test used for the selection of firefighters in order to determine whether the TPM resulted in lower adverse impact when compared to a measure of cognitive ability, which was administered during the same session. TPM-style testing is a type of trainability testing process during which job applicants study a job-related manual over an extended period of time and then answer questions from memory about information contained in that manual. The characteristics and behaviors investigated included the amount of time participants reported studying a firefighter training manual on which they were later tested, race, cognitive ability, motivation to study, study self-efficacy, number of times a practice test was taken, studying with a tutor or study group, and whether they had set aside certain times to study.
The study concluded that the Black-White standardized mean-score difference on the TPM test was less than half that obtained on a measure of cognitive ability. Also, the amount of time the TPM manual was studied was the primary variable that predicted TPM test performance, with the lowest Black-White mean-score difference on the TPM test being for those who reported studying the TPM manual the most. However, cognitive ability was also a significant predictor of performance on the TPM and cognitive ability was correlated with race.
During this study Black test takers benefited more than White test takers from increased study time during the final two weeks before the TPM test was administered, but this benefit was eliminated when cognitive ability was controlled. Furthermore, those with higher levels of cognitive ability benefited significantly more from increased study times than those with lower levels of cognitive ability.
The TPM test appears to offer a fair and valid method for measuring the cognitively-loaded aptitudes of reading and learning written text. When appropriate it should be considered as an alternative employment practice as specified in Section 3 of the federal Uniform Guidelines on Employee Selection Procedures (1978), if future research finds that it is a valid alternative to cognitive ability testing.