Wouldn't it be great if we could cut testing time in half without sacrificing accuracy – while enhancing test security? Well, that's exactly what we did. We have implemented Computer Adaptive Testing (CAT) within our system and launched adaptive versions of some of our most popular tests.
What is CAT?
Computer-Adaptive Tests (CAT) are assessments in which each question in a large question bank is calibrated regarding its difficulty level, ability to discriminate between candidates of different ability levels, and the likelihood of guessing the correct answer. In a CAT test, the next question is selected using the results of all questions answered so far by the candidate. The question that offers the most additional 'information' regarding the candidate's ability level is selected. Additionally, the test is stopped when the estimate has stabilized within prescribed confidence levels.
The central idea behind CAT is to choose questions based on their usefulness in estimating a particular candidate's ability level based on what is already known about that candidate's ability level based on previous answers. For example, if a candidate answers several easy questions incorrectly, asking a difficult question will add less value to the estimation process than asking a less difficult question, since it's very likely the candidate will get the difficult question wrong. Conversely, if the candidate gets several easy questions correct, asking a difficult question would be more helpful than asking another easy question.
To ensure that the test doesn't end too quickly, and make the wrong impression on a candidate or employee, all HR Avatar CAT tests have a minimum number of questions. Additionally, a topic balancing algorithm is active to ensure that the questions cover an appropriate range of subtopics within the test.
What are the benefits of CAT?
For employment testing applications, CAT offers two important benefits:
- Reduced Testing Time - CAT tests can often achieve the same measurement precision with half the number of questions.
- Enhanced Security - Every test is different and questions are typically drawn from a large question bank.
These advantages translate to a better candidate or employee experience, lower dropout rates, and increased confidence in the assessment results.
Are there any drawbacks?
In cases where you want every candidate to receive the exact same questions, CAT is not a good choice. Additionally, if you are sampling a large number of unconnected topics within the same assessment, CAT is not appropriate. That is because the adaptive algorithm selects questions using statistics based on previous answers, so candidates will not receive the same questions. While CAT tests, including HR Avatar's CAT system, employ topic balancing to ensure proportional sampling of different topics included in a test, there is no guarantee that all topics will be addressed before the test stops.
For pre-employment assessments, these drawbacks are more than offset by the shorter testing time and improved test security.
Are the Results Different?
HR Avatar's adaptive tests produce results along the same scales as our traditional format scales. The results can be interpreted the same way. However, we also provide the raw CAT estimation values for those who have an interest in these values.
Creating CAT Tests
Developing a CAT test has traditionally been expensive and time-consuming. However, thoughtful and responsible application of artificial intelligence tools can reduce both cost and development time.
Not all assessments are suited to adaptive testing. The most important qualifying feature is that all questions in the test must be closely related and must measure essentially the same thing. This is often called uni-dimensionality. However, many knowledge and skill assessments, by their very nature, are generally uni-dimensional, as are some cognitive tests.
Which Tests are available in CAT?
HR Avatar has released adaptive versions of seven of its most popular pre-employment assessments:
Click on any of the above to right to the product page for that test. All tests are available in our public test catalog.
Should I be using CAT?
In most cases, the advantages of CAT (shorter testing time and better security) far outweigh any drawbacks and the answer is YES. However, if you want every candidate to receive the same questions, or you want to ensure that all topics within a test are addressed evenly, CAT may not be the right choice.
How do I administer a CAT Test?
You administer CAT tests the same way as any other HR Avatar test. Simply select the test from our catalog and click 'Administer this Test' to create a test key or multi-administration link. You can also include adaptive tests in test batteries - just like any other test. There is nothing special you need to do. All of our CAT tests work with any of our remote test proctoring settings.
