In recruiting, there is currently much discussion about the sensible use of AI. For example, the German Ethics Council HR has published its ten guidelines for dealing with AI systems in recruiting, which advocate a transparent and tested use of AI systems. We, 100W are not satisfied with the vagueness, but prove what our analysis can do. After our successful initial validation with a data set from the renowned motivational researcher Prof. Dr. Schultheiss, we have now started a second time to test the validity of our motivational categories. With our partner company A 47 Consulting, which is headed by Dr. Stefan Dörr, we are now achieving results that even exceed the previous validation.
But first of all: Why is it important to carry out a validation at all? In addition to the two other characteristics of objectivity and reliability, validity is a central quality criterion and indicates whether a test really measures what it claims to measure. In other words: Do our motive categories power, performance and relationship really measure motives and how good are they at it? Although there are already procedures that measure motives in texts (e.g. TAT Thematic Apperception Test), the evaluation of these tests takes a lot of time because experts have to read through the text and then evaluate it manually.
This is not practical from an economic point of view. An automatic evaluation saves a lot of time and money and makes the test applicable to a broader circle. For an automatic solution to be permissible, however, it must prove itself on the quality criteria mentioned above. In the first proof (2018), we have already achieved impressive results with our software. But, as the saying "double checking is always better" implies, we wanted to prove it a second time and validate the new technological breakthroughs that have gone into our core.
To do this, however, we needed data from the previous standard test in order to be able to compare ourselves with it. It is very difficult to find such data in good quality and sufficient quantity. Through our previous study we came into contact with Dr. Stefan Dörr, one of the motivational experts in Germany. During his many years of work he was able to collect more than 1000 motive stories. The special thing about it is that, unlike is often the case, his sample does not consist of students, but rather of managers from different industries. Dr. Dörr was kind enough to provide us with this data.
We analysed the texts of the standard tests and compared the results of our software with them. At the end we obtained a correlation, which is generally regarded as a measure of the correlation between two characteristics. A high correlation indicates that our tool measures similarly to the standard test. Our software achieved values between r = .47 to r =. 67, which is considered a strong effect in psychology. For comparison, the correlation between different versions of the standard test also lies between r = .45 and r = .72 (Winter, 1991).
Consequently, our study showed that the software can capture motifs of 100 words with the same quality as previous standard tests with "human" coding. This makes it possible for the first time to automatically capture valid and scalable motifs.
Especially with regard to recruiting and employee motivation the capture of motives is important. Motives are the psychological characteristic that explains professional success even before intelligence (Elder, 1974). By describing our approach, the methods used and the results in minute detail, we meet the demand for transparency and traceability that is placed on good research. By the way, in this way we also take into account the recommendations of the AI Ethics Council.
Click here for the full paper: