Purpose in Relation to Security and Environment

While there are some special issues surrounding online testing, both face-to-face and online courses face the problem of weighing testing purpose in relation to security requirements and practical considerations introduced by the testing process and environment.

The specific elements that must be considered are the

1. Purpose of the test and related security issues.
2. Test administration conditions.

Purpose of the Test

The purpose of a test should be the first consideration in determining security conditions required. Tests (and other types of assessments) can be divided into either "high-stakes" or "low-stakes," depending upon the effects of the test results on the students' final grades. If a test is high-stakes, then a significant part of a student's grade for the course depends upon his or her performance on the test (usually more than 20% of the final grade). High-stakes tests include end of unit tests (there may be three or four during the semester); mid-terms; and finals.

If the test is low-stakes, students may receive a relatively small number of points toward the course grade, or the lowest score may be eliminated, but the overall effect of low-stakes testing does not significantly contribute toward the student's final grade. Generally, 20% or less of the final grade is related to low-stakes testing. Low-stakes tests can be given for a variety of purposes such as to

1. take attendance
2. motivate students to maintain a regular schedule of studying the material
3. check progress to help both instructors and students assess student achievement
4. reveal learning problems in time to correct them.

High-Stakes Test Requirements

If a test is high-stakes, then the test and its administrative conditions must be appropriate and secure enough to help instructors fairly and reliably judge student performances in relation to the learning objectives or expected outcomes. Both high-stakes and low-stakes tests should be developed in accordance with good test practices to ensure validity and reliability. But because the purpose of high-stakes is to make judgments that affect student grades and even course passage, good test development is even more critical.

This means that the test itself should be developed to reduce testing error as much as possible while ensuring validity and reliability of the test items. That is

• the test directions and items should be clear and precise
• the items should represent the range of information, problems, or skills essential for demonstrating proficiency
• the number of items should be high enough for demonstrating proficiency and yet low enough for completing within the testing time period
• pools of test items or versions of the test are secure (not available to students in advance)

Of course, good instructional practice advocates that students be allowed study guides and sample test items or even sample tests no longer in use.

High-Stakes Test Administration Conditions

High-stakes testing also requires that the conditions for test administration must be carefully considered. To make judgments on a student's performance, the test should be taken by the student (and not by someone else) and should fit into the requirements for resource use and administration conditions specified by the instructor. These resource use and administration conditions include what students can use while testing, time limits, and whether all students are testing at the same time and in the same place.

Resources Use Decisions

Resource use decisions depend upon the type of student performance the instructor wants to test. For example, if the purpose of the test is to test student memorization and recall of information, then outside resources would be prohibited. If the purpose of the test is to test student use of higher-order thinking skills, then outside resources might be appropriate.

The decision to use outside resources for testing higher-order thinking skills and what these resources should be depends on the instructor's expectation of whether students should be able to

1. recall all the information they need
2. be able to consult definite and limited resources, or
3. be able to consult any resource they wish.

These resources would be used by students in solving problems, developing descriptions and providing explanations, and analyzing and making recommendations or judgments. Outside resources could mean that students could be allowed to collaborate, or even required to do so in group project work.

The important element is that instructors need to determine if resource use is appropriate, and, if so, what resources are allowed. To do this, instructors should consider the learning objectives or outcomes for the material being tested and whether some degree of the memorization and recall is necessary to demonstrate achievement of these objectives. This decision must be made in relation not only to the objectives but to the type and amount of information being tested. Thus, if the material is complex and/or highly detailed, the instructor might decide that memorization and recall of specific facts are not important, but that the student should be able to apply formulas, implement procedures, analyze situations and develop solutions, recommendations or judgments. In this case, the instructor might allow a textbook to be consulted. Such resource determinations also include whether students are allowed to use calculators, word processing, or other aids.

More complex demonstrations of learning, such as identifying, organizing, analyzing, and discussing information to demonstrate comprehension of a body of literature and/or the ability to use critical thinking skills, can be assessed through individual student assignments (term papers, literature reviews, discussion or concept papers, research proposals, etc.). These assessments are vulnerable to plagiarism which can be detected in many cases by using a tool such as Turnitin.com and through the instructor's own knowledge of the field and of the individual student's other writing and thinking accomplishments.

Administration Conditions

Besides determining whether and what types of outside resources are permitted, instructors also need to consider the conditions of test administration. These considerations involve

• time limits
• dates of testing
• whether testing is done at one time for all students (synchronous) or
• whether students may test during a "window" of time (asynchronous)
• and where students are located

Regarding time limits for the test, the instructor should consider any test administration restrictions, such as needing to have students be able to complete the test within 50 minutes in a classroom. Or if the test is being administered in the FSU Center for Assessment and Testing, should the student be potentially allowed to stay until the Center closes? This decision depends on instructor judgment of how much time to reasonably allow students and also whether the learning objective demands that the student should be able to perform within a certain period of time. Time limits for paper pencil tests administered within the classroom are usually the rule, while time limits for online tests offered in Blackboard can be set, but are not enforced. That is, the student will be alerted when the time is up, but the test will not close. If the student continues, the instructor will see that the student went over the time limit, but will not know which items were completed within the time limit and which were answered after the deadline.

Test administration conditions also involve whether students are tested at the same time and in the same place, or at different times and, sometimes, even on different dates and in different places. These conditions of when and where testing will occur has implications for whether the instructor should be using different versions of the same test during any one test administration. (Good testing practice advocates that instructors use different versions from one semester to another.) When students are tested in the same time and place, usually in a classroom, they can all receive the same version of the test because they will not be able to pass on knowledge of the test to another student testing at a later time or on a different date. However, if students are seated close together in a classroom, secure test practice encourages instructors to have either different versions drawn from an overall pool of items or different versions that vary the sequence of the items and/or the response choices. If students are being tested in a more extended time-frame, then having different versions and/or varying item sequence and response choices become more critical for ensuring that students are demonstrating their own knowledge and skills rather than knowledge passed on by students who had tested earlier.

Finally, test administration conditions also include whether students are proctored while testing or not. Because of the effects of high-stakes testing on course outcomes for students, instructors should consider using proctors when tests require

1. verification of student identity.
2. demonstration of recall without any outside resources or only limited ones furnished by the instructor.
3. time limits to be strictly enforced.

Proctored testing for online students and for some very large lecture courses that meet face-to-face (and test using Blackboard or LON CAPA) can be provided by the FSU Center for Assessment & Testing. Their services include testing students in the FSU Test Center as well as vetting and facilitating proctored testing for students not in the Tallahassee area. For online students, this includes tests that are paper/pencil and tests given through Blackboard.

Summary

For high-stakes tests that impact student grades for a course and progress through a program, it is important that instructors seek to make their tests valid and reliable in terms of learning objectives or expected outcomes; that they consider the type of tests they are giving (high-stakes versus low-stakes), and determine resource and administration requirements that are appropriate for the learning objectives, the students, and the test administration situation.

 

More News, Events and Articles