Despite the increasing use of personality inventories as a basis for personnel decisions, many users of these inventories are seriously concerned about the susceptibility to response distortion (i.e., faking). The basis for the concern is not just that response distortion is possible, but that it will cause deleterious effects to the validity and utility of personality inventories. Because of these concerns, several techniques have been developed to detect individuals who are responding dishonestly on personality inventories. Increasingly, these techniques are based on item response theory (Zickar, 2000).
This paper extends and tests an IRT-based technique called differential person functioning (DPF; Johanson & Alsmadi, 2002) to the detection of response distortion on measures of the five-factor model of personality. DPF is a technique that can be used to identify if the response for a given individual are different for different groups of items.
Two experimental studies were conducted to examine the accuracy and consistency of DPF with other response distortion detection techniques. In the first study, participants completed a personality inventory twice under different types of response instructions (e.g., respond honestly or dishonestly) to determine which of the items on the International Personality Item Pool could be distorted. In the second study, participants completed a personality inventory under one of three different types of response instructions to determine the accuracy of the DPF techniques in identifying individuals who are distorting their responses. The accuracy of the DPF techniques was compared to the accuracy of two response distortion scales and an IRT based person-fit index.
The results of the studies indicate that those who are instructed to respond dishonestly do respond to some items on the personality inventory differentially and that these individuals were detected by the response distortion detection methods, albeit at low levels of accuracy. It was also found that the differential person functioning techniques detected individuals who were responding dishonestly and did so at levels of accuracy comparable to other response distortion detection techniques. The discussion of these studies focuses on the implications of these findings and the differential person functioning techniques for personality assessment and personnel decisions.