Web of Deceit  

Online assessments and tests save employers time, money and effort. But with all the ways for candidates and employees to cheat -- and with so much at stake -- can employers trust the results?

By Louis Greenstein

Call it human nature. If you develop a way to test people's skills, knowledge, aptitude or behavior, someone will figure out how to fake optimal results. There's a belief among many -- from teachers to pupils, from industrial psychologists to training managers -- that the more precautions you take against cheaters, the harder they'll work to cheat, turning to increasingly inventive tactics.

And with the Web, who even needs inventive tactics? Considering all the "brain dump" sites and easy ways for test takers to inflate scores and misrepresent themselves, are unproctored online pre-employment assessments worth the silicon they're digitized on? Can employees be trusted to take certification exams on their home computers?

Even in a secure, proctored environment, can we reasonably expect online assessments to be accurate when we know the answers are hovering just beyond a firewall?

Brian Wilkerson, who heads up the talent management practice at Washington-based HR consultancy Watson Wyatt Worldwide, says yes, these tests can generally be trusted -- as long as employers don't rely too much on them. Psychologists, test designers and industry observers interviewed for this story agree that online assessments can play a valuable role in the selection process, as long as certain precautions are taken.

The Biggest Risk

All tests are not created equal: Some are harder than others to cheat on, whether administered in a proctored or unproctored environment. "To me, the biggest risk is where there are right and wrong answers," says psychologist Matthew O'Connell, co-founder and executive vice president of Select International, a Pittsburgh-based provider of assessment systems for employee selection and development.

O'Connell says makers of large-scale cognitive tests -- SAT, GRE, LSAT -- do "a phenomenal job keeping the tests fresh and not so cheatable." But most businesses that conduct employee assessments, he believes, are less diligent than their peers in the educational market. According to O'Connell, unproctored tests are only a good initial screening tool.

And it isn't only cheating that employers should worry about: "Some people are better test takers than others," he says. "Just because you can answer multiple-choice questions about Word or Excel, you may not be able to actually use the program. So, for the second screening, say, 'Here's a document. Edit it.' "

The "right answers" for many certification tests are easily obtained from any number of "brain dump" Web sites that sell answers to certification exams.

In an e-mail correspondence, a representative of TestKing.com said its practices were lawful and that there were no outstanding legal actions against them. When asked about a recent case involving answers to an Oracle database administrator exam, the representative said the test covers 95 percent of Oracle's current exam. "If the exam changes," says the e-mail, "we change our product accordingly."

In August of 2006, Microsoft filed suit against TestKing in Federal District Court in Seattle for copyright infringement. Microsoft says the lawsuit (Microsoft Corporation vs. John Doe, DBA TestKing) was settled, but neither party will discuss specifics.

Calls and e-mails to Oracle, SAP and Cisco for this article were not returned.

Microsoft responded to our inquiries in an e-mail: "Microsoft takes seriously the importance of protecting the investment Microsoft-certified professionals make in achieving a professional certification. For that reason, Microsoft has a team of people focused on preventing and addressing exam fraud, piracy and any other illegal exam and certification activities. All violations of the rules for the certification program are reviewed and addressed. Given that there are currently matters pending, Microsoft is unable to provide further comment at this time."

Most experts familiar with assessment testing believe that employers relying on technical certification tests for incumbent workers should administer the tests in a proctored environment -- either at the office or at a testing center. "It's tough to trick employers who are well-prepared," says Rich Milgram, CEO of King of Prussia, Pa.-based job network Beyond.com.

But even in a proctored environment, cheating happens. "Proctors get lazy," says Don Sorensen, vice president of marketing for Salt Lake City-based Caveon Test Security, a firm founded to work on the problem of test fraud. "Have him or her walk around," he advises. And have the test takers read the test-use agreement that comes with most tests. These agreements spell out rules of conduct -- and the consequences for violating them.

Caveon's newsletter includes humorous tales of unscrupulous test takers, such as the folks who wrote answers on rubber bands, wore the rubber bands around their wrists to the test site, and when the proctor wasn't watching, stretched them out to reveal the answers.

Sorensen recommends asking vendors how their tests are developed and how often answers change. He says employers should tell vendors to withhold scores until after an analysis that looks at factors such as how long it took to complete a test, unusual answer patterns and other anomalies.

Nick Hallwood is senior vice president of product management for SHL, a London-based assessment developer that uses Caveon for test security. "Most of the cheating at the moment is in the high-stakes tests that revolve around licensure," he says.

Like others interviewed for this article, Hallwood distinguishes between ill-gotten test answers and legitimate practice tests. "Clearly, there are advantages to practicing a test in advance," he says. "We recommend that candidates do that. But if it's a fixed test and you know the answers, you'll pass." Many test vendors offer sample tests candidates can use for practice. The questions are similar to those on the actual exam, but not the same.

Precautions to Take

One way test makers keep actual exam questions and answers out of the hands of test fakers is called "computer-adaptive technology." This process randomly selects a moderately difficult first question from a question pool about 10 times larger than the actual test questions; and offers subsequent questions that are easier or harder to answer, based on each of the test taker's responses.

PreVisor, a pre-employment testing company headquartered in Roswell, Ga., uses computer-adaptive technology so that "each test is a unique experience for each test taker," according to Chief Marketing Officer Kurt Ballard.

Noting that other vendors use computer-adaptive technology, but not for pre-employment assessments, Ballard says, "We are the only company doing this in our space, and we really haven't found a hole in this technology yet."

As Michael Fetzer, vice president of product development, puts it, "we are adding additional bricks in the wall to keep cheaters out."

Allan Schweyer, executive director and senior vice president of research at Washington's Human Capital Institute, a research, training and consulting organization, offers a number of easy precautions employers can take against cheating on assessment tests.

"Online testing is a good way to reduce the cost of testing candidates," he says. "It gives interviewers a short list of people who scored best." But like others, he advises organizations not to rely solely on online assessments.

Interestingly, psychologist Jeff Weekley found incumbents are less likely to embellish responses in behavioral assessments (where there are no right or wrong answers) than candidates. "Research suggests that we all give ourselves the benefit of the doubt," says Weekley, a senior research director for recruitment and retention specialists Kenexa, in Wayne, Pa. "The majority don't go crazy with it, but we find differences between applicants and incumbents. Applicants are likely to pump the answers," he says, "because they want the job." Weekley recommends using online assessments as first screens and using other methods to confirm initial results.

According to him, when a test that assesses cognitive skills is administered in an unproctored environment, test takers "must be asked to re-demonstrate those cognitive skills" in front of an interviewer.

Terry Gudaitis is the cyber intelligence director at Cyveillance, an Arlington, Va.-based Internet security firm that scans the Web looking for test questions that have been released, solicitations by professional test takers and questions/answers for sale.

"We don't see hacking or penetration of production servers," says Gudaitis. "We see people who are getting hold of answers and selling them."

The Oracle database administrator test available from TestKing, for example, has 50 multiple-choice answers. Five people can take the test and each one memorizes 10 answers. Afterwards, they convene, pool the answers and sell them online. The only response a company has is to change their test.

"The cost of redoing an exam is quite pricey," says Gudaitis. When exam answers are made public, she says, "it causes pain."

When sending employees to an outside testing center, SHL's Hallwood says, HR should ask the personnel there a few questions: Do they check IDs (preferably two photo IDs)? How do they make sure you are who you say you are? Can people see one another's screens while the testing is taking place? Can people open their cell phones and access a Web site for answers during the test? Ask what happens with test takers' data. How are the answers secured once they are logged? Ask to see their policies. Is there a written policy for "electronic devices?" If so, how is the policy enforced?

Gudaitis and Todd Bransford, Cyveillance's vice president of marketing, say employers should also take a number of steps on their own to reduce cheating on assessments. For one, they should make sure their blogging and Internet policies spell out the rules -- and the consequences for breaking them. It may be perfectly acceptable for one employee to coach another prior to an exam.

But sharing answers should be off limits. Employers may also forbid employees from blogging about the testing facilities. "I've seen companies with detailed regulations about what employees may put on their blogs or personal Web sites," says Bransford.

Verify the Results

Beyond.com's Milgram says his company gives technical employees verification tests in person. "We ask them to design stuff right in front of us to assess their skills," he says. "Resumes and certifications only get you so far. They'll get you in the door, but it's your performance that matters."

Milgram believes in assessing someone on the job through a 90-day trial. And when you test online: "Verify. Call the person up after the test," he says. "Ask [him or her] a few of the questions again, but reworded."

SHL's Hallwood points to the huge convenience for employers when people take tests remotely. But he acknowledges the downside: "How do I know it was Nick Hallwood who took the test and not his friend, who is a rocket scientist?" he asks.

One way to increase the certainty, he says, is to tell the test taker that scores will be forwarded to the employer and the next step will be a short verification test. "If there's a serious discrepancy," says Hallwood, "it's flagged. That way, you get the best of both worlds -- the process and cost improvements of online remote tests and the confidence of a second verification."

Caveon offers a service called Web Patrol that searches the Web for tests for sale. "We use real people," says Sorensen, "not bots or spiders to search the Web." When Web Patrol spots a company selling answers that are too close to the actual test to be a legitimate study guide, Caveon alerts its client and sends a cease-and-desist order to the seller.

Watson Wyatt's Brian Wilkerson says the best learning management systems have elements of adaptive technology -- question pools and randomness. "I think [test developers] have taken a big bite out of cheating," he says. "But every time there's a new tool, cheaters find a way. They will mimic the logic, or send multiple test takers."

Wilkerson believes in validating assessment results during behavioral interviews -- and by completing more effective reference checking and background screening. He recommends using social and business networking sites.

"LinkedIn has a reference request tool. You can check references other than what the candidate gave you." As for online assessments, Wilkerson says, "don't rely on just the testing. It's a great part of the process, but not the process itself."

Based on the incidents he's seen in which applicants exaggerate on their resumes, Human Capital Institute's Schweyer believes that "eight out of 10 [candidates] probably would cheat if they think they could get away with it." But that figure, he estimates, drops to less than 1 in 10 if you tell people that test results will be verified.

"It's funny," he says, "what some people will do for a $10,000 raise and an increase in title."