CIFR | FAQ
Frequently Asked Questions
Why should I care about careless and insufficient-effort responders?
A great deal of research has suggested that the inclusion of C/IE responders in one’s data can have a number of psychometrically troubling consequences, including artificially increasing/decreasing observed effect sizes and creating phantom factors in otherwise unidimensional data. Methods—such as the use of infrequency/frequency scales—can be useful for identifying such responders.
Where do the items in the CIFR portal come from?
The CIFR portal includes 531 bespoke items created by Kay and Saucier (2023); 5 items based on work from Forer (1949), Paulhus (1991), and Snyder (1974); and 124 items collated from Beach (1988), Benning and colleagues (2018), Curran and Hauser (2019), Dunn and colleagues (2018), Fervaha and Remington (2013), Hargittai (2009), Huang and colleagues (2015), Kay (2021), Kay and Saucier (2022), Lilienfeld and Widows (2005), Lynam and colleagues (2011), Maniaci and Rogge (2014), and Meade and Craig (2012). See the references page for the full citations for each of these sources.
How many infrequency/frequency items should I include in my survey?
At a minimum, we recommend including two infrequency items and two frequency items in your survey. For longer surveys, we recommend including one infrequency item and one frequency item for every forty non-infrequency/frequency items.
How can I use the infrequency/frequency items to calculate an index of careless and insufficient-effort responding?
To create an index of careless and insufficient-effort responding, you can reverse score the frequency items and average them together with the infrequency items.
What cut-off score should I use to flag careless and insufficient-effort responders?
The appropriate cut-off score depends on a number of factors, including the specifics of the population being examined and the design of the study. Our recommendation would be to produce a histogram of the careless and insufficient-effort responding scores. Ideally, there will be two modes: one to the left, representing careful and sufficient-effort responders, and one to the right, representing careless and insufficient-effort responders. In this case, the cut-off to use would be the value that best separates these two groups.
That being said, we do recognize that some researchers would prefer a general “rule-of-thumb” cut-off value. If we had to recommend such a value, we would suggest flagging participants with scores greater than or equal to zero. A score of zero is equivalent to incorrectly selecting “strongly agree” to all of the infrequency items and correctly selecting “strongly agree” to all of the frequency items (or correctly selecting “strongly disagree” to all of the infrequency items and incorrectly selecting “strongly disagree” to all of the frequency items). As such, a cut-off value of greater than or equal to zero will, at a minimum, flag participants who straightline an entire survey. This is likely to miss a number of careless and insufficient effort responders, but we side with other reseachers (e.g., Curran, 2016) who believe it is better to miss a careless and insufficient-effort responder than accidentally remove a valid responder.
Where do the means and standard deviations in CIFR come from?
The means and standard deviations in the CIFR database come from a study of 818 undergraduate students at the University of Oregon and 348 participants nominated by the undergraduate students at the University of Oregon (Kay & Saucier, 2023). Each undergraduate student completed 220 randomly-selected items from the CIFR and each nominee completed between 22 and 220 randomly selected items from the CIFR, depending on how long they said they were willing to respond to the survey. On average, each item was responded to by 305 participants. For more information about the survey, we direct you to Kay and Saucier (2023).
Although we recommended using the version of the portal that combines the two samples (i.e., CIFR Portal), a version of the portal that includes only data from the undergraduate students (i.e., CIFR Portal - Human Subjects Pool Sample) and a version of the portal that includes only data from the participants nominated by the undergraduate students (i.e., CIFR Portal - Nominee Sample) are also available.
Why are some of the infrequency items frequent and why are some of the frequency items infrequent?
A case could be made for retaining only the highest-performing items in CIFR, such that the infrequency items would only be those that are highly infrequent and the frequency items would only be those that are highly frequent. However, we think it is important for researchers to know not only which infrequency and frequency items work but also which items don’t work. We, therefore, retain all tested items in CIFR.
Why are some of the items in CIFR not associated with a mean and standard deviation?
Items without a mean and standard deviation were added after the first wave of data collection. We intend to collect additional data on these items in the future.
Can I recommend new items be added to CIFR?
Absolutely! Recommendations for all new items can be sent to Cameron S. Kay (cameronstuartkay@gmail.com). Ideally, these items would be associated with a citation (so you can receive formal credit for your contribution), but all items accepted into the CIFR will be accompanied by your name in the CIFR database (unless you wish to stay anonymous).
What if I notice an error with one of the items in CIFR?
Please send all errors and corrections to Cameron S. Kay (cameronstuartkay@gmail.com).
Should I cite CIFR?
Please do! CIFR can be cited as:
Kay, C. S. & Saucier, G. (2023). The Comprehensive Infrequency/Frequency Item Repository (CIFR): An online database of items for detecting careless/insufficient-effort responders in survey data. Personality and Individual Differences.
We also request that you cite the original source for any items you use from CIFR. The original source for each item in the CIFR database is listed in the “Source” column on the portal page. The full citation for each of these sources can be found on the references page.
Can I see a list of publications that have used CIFR?
Yes! A list of publications that have used items from CIFR is available on the Publications page.
If you would like to add your publication to the list, please email Cameron S. Kay (cameronstuartkay@gmail.com).
Who funds CIFR?
Currently, CIFR does not receive funding from any public or private institution. The CIFR website is hosted for free on GitHub, and the domain was purchased by Cameron S. Kay. We plan to seek funding to collect additional data to further validate new and existing items in CIFR.