Should computers decide who gets a job? That’s a question recently asked by Government Executive Magazine which looked at a study of hiring practices and how much hiring recommendations generated by standardized testing differed from managers who “went with their gut” in terms of eventual employee performance and retention. The results were, at least as I read it, a bit muddled.

First of all, the test subjects they looked at were all applicants for low skill service jobs. (Such as you’d find for new hires at call centers or data entry firms.) They were all given standardized tests and those results were passed on to client firm managers who could either hire the top scoring applicants or use their own impressions and disregard the test results.

The tests included not only questions about technical and cognitive skills, but also questions to assess personality and overall fit for the firm. The companies hired HR managers and gave them the test results (coded as green for best scores, yellow for moderate, and red for lowest) but also gave them the ability to factor in other qualities that they thought might make for good hires. Essentially giving them the go ahead to pick candidates with lower scores if they thought that they’d in fact be good candidates. That’s when things went off the rails.

According to the study, when managers used their discretion to override the hiring order implied by the test results (by hiring an applicant who perhaps had a score in the yellow range, when one in the green range was available) the outcomes, of both tenure and productivity, were worse.

These results suggest, at least in part, that using a hiring algorithm – in essence turning the decision primarily over to a data management system – could produce better results. But even the authors of the study acknowledge that it’s just not as simple as all that.

But on the other hand relegating people—and the firms they work for—to data points focuses only on the success of firms in terms of productivity and tenure, and that might be a shallow way of interpreting what makes a company successful. Firms that are populated only by high-achieving test takers could run the risk of becoming full of people who are all the same—from similar schools, or with the same types of education, similar personality traits, or the same views. And that could potentially stall some of the hard work being done in the effort to introduce more diversity of every kind into companies all over the country. And that type of diversity, too, has been proven to be an increasingly important factor in overall firm performance.

As with most things in the business world, these are decisions with long term impacts which can vary significantly based on where you are in the food and power chains. If you’re hiring somebody to do little more than punch numbers into a spreadsheet or assemble fast food orders, some basic skills testing might indeed be the most predictive ruling factor. After all, you wouldn’t want to hire someone based on their winning smile and assurances of being a hard worker only to find out that they’ve never used a keyboard other than the one of their Android phone. But even then there are other factors to consider. Is the person a dedicated worker who shows maturity and responsibility? Are they interested in not only meeting but exceeding expectations in the hopes of advancing their career? You won’t get that from a test, and sometimes it’s only the eye and ear of a skilled interviewer who picks such things up.

When you move up the food chain to senior employees and middle managers, such tests are essentially useless. You’re going to be looking at the resume and their references in addition to your own gut instincts. If they weren’t capable of performing the basic functions of the job they never would have risen far enough to be considered for a job with greater responsibilities. (With some exceptions for government work, granted.) Sometimes you have to meet a person, look them in the eye and shake their hand before you get a sense of what you’re really getting. Sure, there are some top notch actors out there who may fool you, but that tends to be found out rather quickly once they’re on the firing line.

Algorithms and computers may be a fine addition for low level entry work, but a human is going to be able to leave them in the dust for important jobs. Then again, if we’d let HAL select the astronauts for the 2001 mission we might never have found the obelisk, so…