Go to Home Page
Home
Contact Us
GET STARTED Log On
The HR Avatar Testing Platform
The HR Avatar Testing Platform
Back

Purpose

This document is intended to provide a high level overview of the HR Avatar testing platform and what makes it different from other, legacy testing platforms.

Overview

The HR Avatar testing platform (TP) is the first testing technology platform designed from the ground up to support the requirements of mobile computing and simulation-based testing. At a high level, the system supports the following functions:

System Functions

  • Simulation-Based Test Design and Construction
  • Testing Administration
  • Test Results Computation, Management, and Distribution
  • Test and Item-Level Data Analysis and Maintenance

These functions demonstrate that the TP covers all aspects of testing, from test creation through test analysis and maintenance.

Additionally, the TP is designed to achieve superior performance against the following criteria:
 

Design Criteria

  • Support Simulation-Driven Requirements
  • Mobile Device Compatibility
  • 99.9% System Reliability
  • Instant Scalability
  • Appropriate System Security
  • Test Content (and Results) Auditability
  • Ease of Use
  • Seamless Integration including Direct Test Embedding

Achievement of these criteria starts by designing a solid computing infrastructure. After careful analysis, we selected a cloud-based solution provided by Amazon Web Services.

The Amazon AWS Cloud

All operational components of the TP utilize cloud computing services provided by Amazon Web Services (AWS), a division of Amazon.com. AWS is one of the largest cloud computing services providers in the industry.  In 2014, Gartner placed AWS in the highest possible quadrant -- the Service Magic Quadrant" -- in a comparison to other major cloud service providers in terms of ability to execute and completeness of vision ( http://www.gartner.com/technology/reprints.do?id=1-1UKQQA6&ct=140528&st=sb ).

The AWS services which HR Avatar uses includes:

  • Elastic Computing Cloud (EC2) - Cloud-based computing resources
  • Elastic Load Balancing - Traffic distribution across EC2 Instances
  • Simple Storage Service (S3) - Fully-redundant data storage
  • Relational Database Service (RDS) - A fully scalable, managed database management system
  • Glacier - Long-term data storage and archiving
  • Simple Email Service (SES) - for scalable, reliable emailing
  • Simple Notification Service (SNS) - for automated message distribution and response across applications

Some noteworthy features of AWS include the following:

On-Demand (elastic) Computing Resources

Scalability

Provides ability to obtain and configure additional computing, storage, and network capacity easily and quickly, to respond to spikes in workload or steady growth. Machine images are pre-positioned and ready at all times.

Computing instances backed by permanent block storage

Reliability

Allows each computing instance to use storage that is retained if the instance is terminated.

99.95% Availability

Reliability

All infrastructure is professionally monitored and managed 24x7 by full-time dedicated staff.

Virtual Private Cloud (VPC)

Security

Restricts access to computing resources exclusively to other instances within the same VPC and specifically listed external IP addresses.

Customized Security Groups

Security

Provides consistently managed, tight control of access to and from all instances through one group of settings.

Database and OS Software management

Security and Reliability

Patches and upgrades are managed by AWS staff resulting in up-to-date security and functional capability.

Automated database replication and backup

Scalability and Reliability

Offloads compute intensive database tasks to a replicated database, such as reporting and results searches, leaving the primary database resources available for test administration tasks. Additionally, all backups are conducted automatically and at regular intervals.

Multi-Availability Zone database deployments

Scalability and Reliability

Introduces redundancy into the database management system across geographic data center locations.

Reliable, secure storage using Amazon Elastic Block Storage (EBS)

Reliability and Security

Each Amazon EBS volume is automatically replicated within its Availability Zone to protect you from component failure. Amazon EBS encryption provides seamless support for data at rest security and data in motion security between computing instances and EBS volumes.

Integration with the ClicFlic Authoring Tool

For simulation-based test authoring, the TP incorporates the SimBuilder layer, which is built as an additional programming layer that works directly with the ClicFlic intelligent media object authoring system. This layer adds additional capabilities to allow the creation of market-ready simulation-based assessments. For example, each simulation can be developed as a series of ClicFlic modules that are combined to create finished assessments. This allows each module to be managed in much the same way that an item would be managed on a legacy test development system. This modular architecture allows for reuse of modules and greatly simplifies test maintenance and enhancement.

SimBuilder also allows for numerous other features, including competencies to be defined and managed separately from modules, creation of common media libraries, format templates at the simulation and individual item level, customized reports, and many others. All of these features were incorporated to enhance productivity during test development and to support automated scoring during test administration.  

A key feature of the SimBuilder layer, is the ability to tightly control module versions and maintain an audit trail between each completed test and the exact versions of each module that it used. This is a standard feature of legacy item-bank systems today, and the TP uses SimBuilder to implement the same capability for simulations.  

Simulation-Driven Formatting, Scoring, and State Persistence

Simulation-based tests add a new level of complexity to an assessment system, in both item format and during scoring. A virtually unlimited number of item formats is possible. For example, a test taker may be required to evaluate a series of expense reports, write a memo, proofread a letter, evaluate a series of emails, and respond to co-worker questions all in one series - just like in real life. Each specific test-taker action may evaluate a different competency and each competency may use a different scoring technique.  

The simulation authoring tool was designed to afford the ability to format test items in virtually any way desired, while associating all types of responses with specific competencies and attaching relevant scoring parameters - during author time. At the end of the authoring cycle, two products are produced: (1) the simulation itself, and (2) the scoring documentation, in a standard machine-readable (XML) format, which can read by the scoring engine to specify exactly how the test should be scored. This approach allows for a standard scoring engine to be constructed that reads this XML 'descriptor' document to score the test.

Using this design, all HR Avatar tests currently use the same scoring engine, even though the tests themselves use widely varying item types and scoring schemes. However, to provide more flexibility should it be needed in the future, the TP was designed to allow for different scoring engines, so that truly unique or incompatible scoring requirements can be modeled to the precise requirements of any given test.

Another challenge caused by the simulation format is the ability to restart a partially complete simulation at the place where the test was previously left off. Unlike sequential item-based tests, simulations require both item responses and other 'state' indicators and values in order to accurately pick up where the test left off. To accommodate this capability, the TP uses an XML-based state document that includes all item responses and state variables as the format for reporting responses back to the server. When a partially completed simulation is restarted, the test object simply reloads the most recently produced XML state document to restore state to precisely the right spot.

Shifting the Intelligence to the Client and enabling Direct Test Embedding

Like their video-game cousins, simulations rely on the web client (the test taker's web browser) to modify their output as they adapt to inputs from the test taker. This is in contrast to legacy testing systems, where most, if not all, formatting and branching decisions were made by the server and then sent to the browser as static XML.  

This shift to the web client results in some interesting design considerations, as well as a new and exciting integration option we call Direct Test Embedding.  Direct Test Embedding simply means you can embed a test inside any webpage on any website, and still have all the control and security you would get on a dedicated testing system. This means that partners and clients can control the test taking experience more seamlessly when integrate HR Avatar assessments with their hiring processes. The test taker never needs to leave the client's system!

For additional information about Direct Test Embedding, please review the HR Avatar Test Integration Guide.

  .

Mobile Compatibility - a Must Have for the Future

There are now more mobile devices in use today than desktop computers, and the trend is pushing this ratio ever higher. Many job candidates only have access to the Internet via their mobile device. In order to stay relevant, any testing system must support mobile devices.

Supporting mobile devices implies that the testing system must have the ability to:

  1. Support multiple browser-based technologies
  2. Support widely varying screen sizes
  3. Support different rules for playing media on mobile devices
  4. Support different video formats for different web browsers
  5. Support widely differing network connection speeds

Traditionally, most simulation-based assessments were based on Adobe Flash technology. This was a good solution for desktop computers. However, many mobile devices, including all Apple devices and most modern Android devices do not natively support Adobe Flash. Instead, these devices require use of a set of web technologies known as HTML5.

Unfortunately, several web browsers still in use do not support HTML5, such as Internet Explorer 6,7, and 8. Since some browsers support only Flash (IE 6,7,8) and others support only HTML5 (many mobile devices), the only viable simulation solution is to support both Flash and HTML5.

The TP automatically determines which technology is best for a given web browser and uses that technology.

Next, when testing on a mobile platform, one must contend with the fact that screen sizes vary wildly across devices.  In some cases, the item format must be changed to accommodate different screen sizes. For instance, on a large screen, you can ask a series of Likert scale items on one page. However, on a small screen, a single item per page is more appropriate. The TP software is configured to detect screen size and vary item format when needed. This feature is employed in many HR Avatar tests.  

Additionally, most mobile devices enforce rules that do not allow video to play on the device without a 'positive action by the user,' while others play video in video players outside the web browser (Safari iPhone, Opera). These restrictions require simulation software that recognizes each condition and adapts to it in a way that is user friendly and easy for test takers to navigate.

Unfortunately, there is still not one single video format that all web browsers can play. While it has gotten significantly better, multiple formats of every video must be maintained in order to be compatible with all major browsers and formats.

Even if there was one single video format for all web browsers, different sized screens and the need to make the most of network bandwidth demands different versions of each video for different resolutions.

As a result of the need to have different formats for different browsers, and the need to adapt to screen size differences and maximize the use of connection bandwidth, all HR Avatar assessments use videos that are encoded into four different versions - H.264 MP4, WEBM, H.264 MP4 baseline encoded (for tablets), H.264 baseline encoded small screen resolution (for phones). The software is smart enough to recognize which version to use for each situation.

Using the Platform for Non-HR Avatar Tests

Thanks to it's modern, modular infrastructure and object-oriented software design, the TP can easily accommodate 3rd party tests and test/item formats. Legacy as well as simulation-based formats can be implemented reasonably quickly and at a reasonable cost. In fact, the system already hosts, delivers, scores, and distributes reports for several other clients who wish to take advantage of its unique features. Please contact us at sales@hravatar.com  if you would like to discuss feasibility or options.


What Our Customers Say
"We've been using HR Avatar's technology for several years now for complex custom assessments. Our clients love them!"
Suzanne Tsacoumis, Vice President, Human Resources Research Organization (HumRRO)
"These tests help ensure we concentrate our attention on the best candidates."
Simon Murray, Managing Director, Client Stepping LTD
"Simulation-Based Assessments give the candidate a virtual snapshot of the work they will be asked to do when they are on the job."
John Austin, CEO, Talegent

Go to Home Page