Selecting software for online assessments is an important decision. As the use and integration of computer-based testing and assessment increases exponentially, this tool could play an integral role in using assessment for selection, certification, education, or training for the next decade or more. Not being successful in matching the selection to the actual organization assessment needs can not only make life difficult for the staff for the foreseeable future, it could also unfairly have a negative impact on those taking the assessment who are counting on fair and accurate evaluations.
A little upfront effort and due diligence to make this critical decision can reap benefits for years to come. Here are some important things to consider when it is time to select assessment software.
Section by Committee
There are a lot of stakeholders involved with using and supporting the online exam software. Obviously it includes the ones who make and take the assessments, but stakeholders can also include the IT staff who has to support it or the school board who has to pay for it. They all have different concerns, different responsibilities, and different knowledge. One of the worst mistakes is to not actively involve the people who will regularly use the software and the other stakeholders.
So the best way to choose assessment software is to involve all the stakeholders, including people who can represent the assessment-taker point of view. Try to find committee members who are positive, forward thinking, and open to change. Negative thinkers or those resistant to new tools like online assessment will only sabotage the process. Involve people who will contribute in a positive way and adhere to a defined schedule.
Get to Know the Field
Even for experienced online assessment users, if they are not keeping current with advances in this field they might be surprised at what is available. Like a lot of facets of Information Technology, computer based (or smart device based) assessment is evolving rapidly. So before even starting to define requirements and specify features, the first step a selection committee should take is to get up-to-date on what is going on in the industry. Explore what is available in the marketplace, current trends, and common assessment approaches and pitfalls. For example, have some members of the committee research and give presentations on assessment software products. Other committee members could explore academic and industry literature for helpful information.
Establish Criteria List
The goal of reviewing products in the first step isn’t to select a software package, but to get a basic feel for what online assessment tools are out there, what kind of features they provide, how they are delivered, how much it costs, and so on. With that knowledge, the committee can start establishing what it is looking for in online assessment software.
Now that they know what is available in the real world of online assessment, committee members can start deciding priorities and what are the “must haves” versus the “nice to have” versus the “don’t need” versus the “avoid this” - as well as establishing budget parameters.
At this point the committee may be able to make some decisions like definitely going with a Software as a Service (SaaS) approach (which usually involves on-going subscription fees to access software hosted on the paid provider’s servers) or using a purchased software package (paid for one time and downloaded locally in-house).
Once a clear list of criteria and requirements are established, the selection process can become a little easier as only providers or vendors who have products that meet the criteria are explored in detail. Plus, with clear requirements it becomes easier to compare apples to apples and oranges to oranges.
Don’t Forget to Look Behind the Curtain
Of course the selection committee should consider performance, functionally, usability, effectiveness, and reporting capabilities, but there are other factors to consider as well. Some examples are:
- Life cycle Cost: How much will the software cost to support over its lifetime?
(This includes licenses and support man-hours (including internal IT support or Outside programmers/ system consultants)
- Supportability: What kind of on-going support is provided with the software? What happens if something goes wrong?
- Scalability: What happens if the needs or use of on-line assessment grows rapidly over time?
Give Everyone a Chance
The committee should keep its options open as long as possible. For example, make a commitment to see at least five presentations or demonstrations before making a selection. A lot of salespeople are very skilled - they can’t plant negative thoughts about competitors without prospects even realizing it while making their product seem like the shining city on the hill. Suddenly it can seem like a waste of time to even consider something else.
Make sure to give everyone a chance. Perhaps after a few more demonstrations the shine starts to come off the assessment software that seemed so perfect before.
Assessment software is important. By following these steps you can make sure you decide on the on-line assessment tools that will provide the most benefit and value over the foreseeable future.