All assessments, whether online or in person, on computer or on paper, must have high standards when it comes to generating accurate information in order to measure knowledge, learning and learning methods. To achieve this goal, every on-line assessment should go through a standard creation process to ensure quality and consistency.
Here is a basic outline of what that process should look like to serve as a basic starting point. Of course, it should be tweaked and adjusted to meet any educator’s or certifier’s particular needs depending on curriculum, age and skill levels, teaching philosophies, and so on.
Step 1: Determine Criteria and Standards
Creating an effective assessment requires proper planning, and good planning is often the most difficult part of creating an assessment. For creating online assessments, planning activities make up several steps of the process.
The first planning step is to ask and answer key questions. What do students need to know? What methods are appropriate for the subject matter? How will assessment results be used? Do any standards or regulations apply to the assessment? Even if adherence to standards isn’t required, reviewing and understanding established standards can help ensure best practices are consistently employed.
Like planning for any activity, this step probably requires brainstorming sessions, research and note-taking, reviewing published/personal materials, and using graphical tools like mind maps. Once you have a good feel for what the assessment is about, you can start planning the assessment itself.
Step 2: Determine Times or Periods for Assessment
Following up on Step 1, once goals and criteria are established the next step is to decide the best times to use an assessment or what an appropriate assessment period would be. Once they are placed online it is easy to use an assessment, but there is a fine line to walk here. The more data and measurements that are available, the clearer the assessment picture can be. On the other hand, overly frequent assessments cause students to devalue them. They start to feel pointless and a waste of time. You may gain lots of data, but the accuracy or meaningfulness of the information may be questionable. Make the most of how easy online assessments are to give, but don’t overdo it to the point where they start to lose meaning.
Step 3: Determine Assessment Methods
With goals and assessment periods in place, design the assessment by creating questions or inquires that will assess knowledge and understanding. Try to use the simplest form appropriate to achieve the assessment goal. Sometimes a true-false questions fits perfectly to test knowledge, so don’t automatically avoid the simple approach because it seems too easy. On the other hand, don’t avoid using more complex methods just because a true-false or a multiple choice approach is easier to implement with most online exam software.
Ask the kind of question that best meets the goals established in the plan, whether that requires a true-false question or a short description.
Step 4: Create a Criteria Rubric
Rubrics define what is expected from a student in the assessment, as well as establishes performance levels (i.e. excellent, competent, needs improvement). The basic pattern of rubrics is to establish dimensions for task or criteria. These could include dimensions like knowledge or application or it could assess various dimensions of the task like research and communication. Then for every dimension create a performance scale and associated criteria. For example, the scale for each dimensions may be 1 through 5, with each level’s standard clearly defined, such as a 5 for the research dimension defined as, “Demonstrates excellence at finding and integrating appropriate materials.”
A clearly defined rubric makes evaluating results more consistent, and it is often a necessary component if evaluation has automated elements that are common in computer based assessments.
Step 5: Implement and Check
Now that goals, timing, questions, and rubrics are clearly defined, create the actual assessment using the available assessment software. Every program works differently in how it is used to create, implement, and evaluate the assessment, so every educator, instructor, or evaluator has to know how to use their particular assessment software to achieve assessment goals.
One critical part of implementing an online assessment, especially if it has significant stakes, is testing and verifying the assessment. This can involve things like test or pilot runs with staff or with third party subjects (like a different class). Verify the user experience works as expected. Don’t just assume everything works as it should from the user perspective.
Using an online assessment tools share much in common with more traditional methods, but to take full advantage and benefit from this important new tool also requires a fresh, innovative outlook as well. Following this basic process will put you on the path to creating accurate and productive on-line assessments.