Test Fairness is a term that can mean different things. We use the term to describe the efforts we make to ensure our tests are job-related, fair to examinees, and are in compliance with the relevant Legal Guidelines. Test Fairness has been interpreted in several different ways, some of them technical, but the current definition is a social rather than a psychometric concept. Our focus at HR Avatar is to design assessments that predict job performance, are more engaging than traditional test modalities, and are fair.
The Equal Employment Opportunity Commission Uniform Guidelines (EEOC) and the Office of Federal Contract Compliance Programs Guide (OFCCP) are designed to ensure that employment tests don’t discriminate against protected groups. OFCCP generally covers all Federal contractors and subcontractors (Contracts and subcontracts of less than $10,000 generally are exempt) and EEOC covers everyone else. It is important to be aware of and to be in compliance with the legal requirements that apply when tests and other assessment instruments are used to select employees. Other resources that we use at HR Avatar to ensure compliance include The Department of Labor / O*NET guide for understanding job requirements, selecting and using employment tests.
The EEOC (and OFCCP) require employers to use selection procedures that are job-related and fair. HR Avatar can assist you with selection and assessment-related services such as Job Analysis, Adverse Impact Studies and Validation Studies upon request to help you implement and maintain a valid selection process. We describe the published guidance in further detail below.
The EEOC Guidelines provide the foundation for legal requirements of selection procedures like HR Avatar assessments. Protected groups include race, color, religion, sex, or national origin. If a test has an adverse impact on any protected group, the test procedure must have been demonstrated to be job-related, through a validation process. The process involves building evidence that the test is related to the job, and there are three main ways to do so:
- Criterion-related validity: Demonstrating that those who score higher on the test have better job performance. This requires empirical data of test scores and measures of important elements of job performance.
- Content validity: Demonstrating that the content of the assessment is representative of important aspects of performance on the job. A selection procedure can be supported by a content validity strategy to the extent that it is a representative sample of the content of the job.
- Construct validity: Demonstrating that the assessment does measure a skill or trait that is known to be required in the job.
Additionally, the Uniform Guidelines require that a job analysis be conducted to review information about the job and document the work behaviors. For content validity evidence, job analysis should include an analysis of the important work behaviors required for successful performance and their relative importance and, if the behavior results in work products, an analysis of the work products. For construct validity evidence, the job analysis should show the work behaviors required for successful job performance, and an identification of the constructs believed to underlie successful job performance.
The EEOC Guidelines also require that validity evidence be documented, and record-keeping be done that tracks scores by group, and adverse impact, and extent of adverse impact. HR Avatar creates and maintains our assessments in compliance with the EEOC Guidelines, so you can feel confident in the assessments you use to hire the best people possible.
The OFCCP Guide describes the content and implementing regulations of Executive Order 11246. It is enforceable with any organization that has a federal contract greater than $10,000 with the Federal Government. Protected groups include race, color, religion, sex, or national origin, as in the EEOC Guidelines. However OFFCP extends the definition of protected groups which includes specific veteran groups, sexual orientation and gender identity. Finally, OFCCP enforces other laws that protect qualified individuals with disabilities and qualified protected veterans.
The basic EEO requirements of a Federal Contractor are:
- Don’t Discriminate
- Post EEO Posters
- Include the EEO Tag Line in Employment Advertising
- Keep Records
- Permit OFCCP Access to Books and Records During a Complaint Investigation or Compliance Evaluation
- File an Annual EEO-1 Report
With respect to employment tests, the OFCCP Guide states, "Professionally developed tests can be used to make employment decisions, so long as the tests are fair and nondiscriminatory." This is indeed the case with HR Avatar assessments.
There are several entities that provide additional information on appropriate test validation and use.
The SIOP Principles are provided by The Society for Industrial and Organizational Psychology, which is a group that has a great deal of expertise in validation and use of selection procedures. They provide additional detail and more updated information, relative to the EEOC Guidelines. One of the key differences between the SIOP Principles and the older EEOC Guidelines is that the Principles conceptualize validity more as evidence that is collected to demonstrate the relationship between the tests, or predictors, and important outcomes, as opposed to thinking of it as three distinct types of validity studies (criterion-related, content & construct). The SIOP Principles also provide information about generalizing accumulated validity evidence to justify its use in a new situation without conducting a local validation research study.
The American Psychological Association (APA) partners with the American Educational Research Association (AERA) and the National Council on Measurement in Education (NCME) to produce the Standards for Educational and Psychological Testing. The document is very detailed and covers a broad range of testing, in addition to employment selection testing.
The ONET assessment guidelines were developed to help managers and human resource professionals use assessment practices to reach their organizations’ HR goals. The Guide suggests using a whole-person approach to assessment. Thus, a personnel assessment process should combine and evaluate multiple sources of information. At HR Avatar, we follow the guidance provided in the ONET Employer’s Guide.
The ONET Guide adds more context and detail for employers. For example, it goes into significant detail about test reliability. This is a key aspect of tests that tells the extent to which the test provides dependable and consistent information, and is clearly critical for a test to be effective. The reliability puts an upper limit on the amount of validity, or proof of job-relatedness, of an assessment. There are four main types of reliability:
Job Simulation Assessments
"Simulations can do a better job of predicting which applicants will perform well on the job than many other commonly used assessments, and they can provide a greater degree of fairness in the process."
U.S. Merit Systems Protection Board
- Test-retest reliability indicates the repeatability of test scores with the passage of time.
- Alternate of parallel form reliability indicates how consistent test scores are likely to be if a person takes two or more forms of a test.
- Inter-rater reliability indicates how consistent test scores are likely to be if the test is scored by two or more raters.
- Internal consistency reliability indicates the extent to which items on a test measure the same thing.
The U.S. Merit Systems Protection Board generated the Merit System Principles, which provides a list of nine key principles to follow in managing the executive branch workforce. One of the key aspects of the first Merit System Principle is "…selection and advancement should be determined solely on the basis of relative ability, knowledge, and skills," which is the objective of HR Avatar assessments.
- Our tests are designed to be fair, valid, and reliable measures.
- Our test design is based on the analysis of job requirements embodied in the O*Net job research data set.
- We use a "Whole-Person" approach, using multiple assessment types.
- We incorporate simulations of job-related tasks (work-samples)
- We capture EEO data and monitor our tests' fairness.
- Our tests meet the adverse impact guidelines.
- We document our approach and test-performance data in our test manual.
- We continue to monitor and improve our tests.
We follow the relevant legal guidelines both in terms of the letter of the law and the intent, meaning that we are very focused on designing the best possible assessments to help hire the best possible people into your organization.
HR Avatar measures are designed to measure important job-related competencies. The assessment provides a "whole person" approach to measurement. We provide a robust process that includes:
US Department of Labor Guideline
"To help ensure both efficiency and diversity in your workforce, apply the whole-person approach to assessment. Use a variety of assessment tools to obtain a comprehensive picture of the skills and capabilities of applicants and employees. This approach to assessment will help you make sure you don’t miss out on some very qualified individuals who could enhance your organization’s success."
Testing and Assessment: An Employer's Guide to Good Practices, O*Net/ U.S. Department of Labor
- Simulations: Animated simulations where the candidate plays the role of a new job incumbent, doing tasks that are similar to those of the job. The simulations are valid measures based on content validity and criterion-related validity.
- Attitudes, Interests and Motivations: Characteristics of personality that are related to job performance. The Attitudes, Interests and Motivations competencies are valid measures based on criterion-related validity.
- Past Experience: Experiences from candidates’ past that we know predict future success. The Past Experience measures are valid measures based on criterion-related validity.
- Skills: Tests based on specific skills the job requires, such as typing test or customer skills. The skills tests are valid measures based on content validity.
HR Avatar can assist you with selection and assessment-related services such as Job Analysis, Adverse Impact Studies and Validation Studies upon request to help you implement and maintain a valid selection process.