How to Conduct a Traditional Judgmental Evaluation as An Employer

Performance Appraisals: Traditional Judgmental Evaluations

The most common method of performance appraisal is the traditional judgmental evaluation. This is a loose term when it comes to formal methodology and is more accurately described as an overall collection of different methods. In a judgmental evaluation, the appraisal is completed by raters, such as immediate supervisors, higher level supervisors, and/or human resources employees. Although there may be some part of a judgmental evaluation that includes an objective measure, the vast majority of information gathered and delivered in a judgmental performance evaluation is subjective. As covered previously, this particular method is subject to numerous possible errors. Unfortunately, many jobs cannot be accurately evaluated using only objective measures; thus, rater training becomes vital to ensure the integrity and accuracy of a judgmental evaluation.

The most common errors that arise from judgmental evaluations tend to be issues dealing with social cognition and our predisposition to evaluate others based on varying levels of personal experience. That is to say that although the rater may try to be as objective as possible in their assessment of an employee, they are unlikely to be able to achieve a level of complete objectivity. Rater training can help minimize the likelihood of certain errors, such as the halo and horns effects. The halo effect is characterized by an individual's tendency to rate someone higher in areas that may not be entirely accurate, because they are especially strong in a different area; the horns effect is the opposite. This is where a rater might score an individual lower than deserved in some areas because of a deficiency in a single activity. Obviously, there are other potential areas where poor evaluations may be made, such as anytime the rater and the employee have strong personal feelings for one another, whether those feelings are positive (such as friendship) or negative.

Another common error that may result from judgmental evaluations occurs when numerous raters are evaluating different employees using the same standard, but who have not received sufficient rater training. Imagine the following scenario: Within one company, there are six different departments with six department supervisors. Imagine that each department supervisor is to perform an appraisal of each of their employees using a standard form; this form is composed of 100 questions that are all answered on a Likert scale (such as 1 to 5). Imagine that on this form an answer of three indicates the employee meets expectations; a a five indicates they consistently excel in that area and exceed expectations, while a one indicates the opposite. Without proper rater training, you may easily end up with one department head who is more severe in their appraisals and one department head who is very liberal with their praise. In this case, it is likely that two employees who are similar in terms of fulfilling expectations for any given particular area may still have very different scores. Alternatively, imagine that a particular employee has a supervisor who is less effusive with praise and then transfers departments or has a new supervisor who performs appraisals more harshly; this employee may be performing at exactly the same level but suddenly looks on paper as though they are not, or that they have become deficient in some way. When promotions and raises are on the line based on performance evaluations, it is easy to see how this type of inconsistent rating can be a big problem.

Rater training has been shown to be helpful in minimizing the likelihood of some of these social cognition errors. Unfortunately, full scale rater training can be time-consuming and expensive and is most beneficial for behavioral analysis. That said, even small businesses and organizations may receive some benefit from in-house or small-scale training to help raise the awareness of managers to practice objective thought processes prior to performing appraisals, as well as to establish a company-wide baseline so the different managers are more likely to be operating on the same page when it comes to the standards of performance.

There are other concerns, as well, when it comes to judgmental evaluations. Just as many departments -- or even entire businesses -- are expected to produce certain results, some managers are expected to meet or provide certain types of evaluations, which may affect their motives when completing performance appraisals. In some companies, it may be that a manager feels that their department needs to stand out and will inflate scores as a result. Sometimes management may be aware that there are going to be cutbacks and wants to ensure that their department (or certain employees in their department) are more likely to be safe if they have outstanding performance evaluations. Sometimes a manager may rate someone more favorably if they view them as being needier in some way, such as being a single parent or having economic troubles.

Another challenge with traditional judgmental appraisals are accidental paired comparisons. Comparing two or more employees, rather than rating each individually against pre-established criteria, can be considered a legitimate performance appraisal method. This approach can help minimize or eliminate central tendency and leniency errors; it also allows for a more realistic type of evaluation to occur as there's no expectation for some pre-designated perfection standard that is impossible to fulfill.

This performance measure, however, is not without its own flaws -- and that's before we get into the accidental employee comparison challenges. The halo/horns effect is still a concern using this method, as the inclination toward allowing personal preference for one employee over another may result in inaccurate evaluations. Typically the best use of employee comparison methodology occurs when the supervisor identifies the two most high-performing or "best" employees in their department for each dimension being evaluated. All other individuals within the department would then be ranked according to that standard for each performance to mention. One of the more obvious challenges with this particular method is when your team is aware of what employee is being used as the standard, especially if the same employee sets the standard in the majority of dimensions. Depending on the type of work environment you have, employees may respond positively to this method (encouraging them to engage in goal setting), or negatively, by creating a negative or hostile work environment for the employee who excels, particularly if that individual is not among the more popular staff members socially.

Interested in learning more? Why not take an online Job Performance Appraisals - A How To Guide course?

Another method of judgmental performance appraisal is the utilization of behavioral scales. Behaviors, while not quite as objective as numerical values, are still more definite than traits and are more easily measured objectively. Essentially, supervisors determine particular traits they find to be indicative of good or bad job performance and then rate each employee according to whether or not they demonstrate these behaviors while they work. There is more than one method of utilizing behavioral scales, each with its own benefits and challenges. This type of evaluation, overall, can be highly effective when performed correctly; the most critical step to ensure success using this type of evaluation is to carefully select the criteria and, whenever possible, have more than one individual track employees' demonstration of these good and bad characteristics to prevent any subconscious bias on the rater's part.

Again, this type of evaluation can be tremendously effective, particularly when determining an achievement plan for the individual being evaluated. The most important thing an employer can do to ensure that a judgmental evaluation is accurate and helpful is to train their raters thoroughly to avoid some of these issues. Of course, it is also critical that a good measurement tool is being used to help stifle the subjectivity of a judgmental evaluation.

Peer and Self-Evaluations

Some companies and organizational psychologists advocate for the use of peer evaluations and/or self-evaluations to help combat some of the issues with judgmental evaluations performed by supervisors (as detailed in the last lesson). While these types of evaluations do negate some of the issues with traditional judgment evaluations, they are still prone to some of their own flaws.

Peer assessment is usually completed by co-workers on the same hierarchical level as the person being evaluated. As these individuals may easily be people with whom the employee interacts most on a day-to-day basis, it is understandable why their opinion may be sought during the performance appraisal process. There are three common methods of peer assessment, all of which may include horizontal and vertical relationships with potential raters. Put simply, horizontal relationships are those with whom the employee is on the same level, performing similar functions as the rater. With vertical peer relationships, evaluations are performed by someone who is not higher up in the company hierarchy than the individual being assessed, but may be in a position where they serve a different function, particularly one that relies upon, or works closely together with, the person being assessed.

First, peer rating is one method of peer evaluation whereby each member of a group rates the other members (or the individual being assessed, if they are the recipient of a sole assessment) based on a set of performance dimensions. In this way, this type of peer assessment is similar to a judgmental evaluation, except that it occurs at the hands of the individual's peers; this, theoretically, helps prevent some of the issues present in traditional judgmental evaluations by a sole manager or supervisor -- especially one with whom the employee may not spend much time.

Another method of peer evaluation is peer ranking. In this process, each member of the group ranks their fellow group members from best to worst, based on specific dimensions of performance. Obviously, this type of assessment works best when performing evaluations of multiple group members at a time, if only because it is more efficient to complete it all at once.

A third common method of peer evaluation is known as peer nomination. With this type of evaluation, each group member nominates the individual they believe to be the best (out of the group) on various levels of performance dimensions. One of the most obvious limits with this type of method is that ranking does not allow for appropriate appreciation of people who are closely skilled, where one individual barely surpasses the other. For example, imagine a department of six individuals who work together on a regular basis and are performing peer evaluations for all members of the group for a single dimension of job performance. Let's assume that persons A and B clearly excel, compared with the rest of the group, but person A is slightly better at that particular dimension than person B. Persons C, D, E, and F are significantly worse at this dimension and are also all on approximately the same level. When forced to rank these individuals, the list will not demonstrate the accomplishments of person B compared with person A and will also imply a greater variation of skill between persons C-F. In this way, peer nomination may not accurately reflect the strengths and weaknesses of each member of the team.

Obviously, there are pros and cons to each of these types of peer evaluation. The limits of peer assessment begin with the difficulty in adapting it to an individual assessment, rather than the assessment of several employees are members of the group at once. There is also likely to be limited understanding regarding the job functions of the employee being evaluated (as their peers may not always know or understand all of their job functions). And just as management motivation can produce inaccurate evaluations, the motivation of peers can also be questionable. There may exist conflicts between different members of the team that are based on personal issues, rather than job function. Likewise, if promotions and/or raises are tied to evaluations, employees may be motivated to score others lower in order to improve their own likelihood of receiving a raise or promotion. Another issue is whether the peer evaluation is done in such a way that the individual being evaluated is made aware of what their peers are saying about them. Negative comments and scores may create friction within the department.

When it comes to self-evaluations, this type of evaluation method has most often been used as part of a larger performance appraisal process. Typically speaking, employees perform a self evaluation using a list or form outlining various dimensions of performance that commonly fits on a graphic rating scale, similar to what many managers use during a judgmental evaluation. In fact, when self-evaluations are used as one component of a larger performance evaluation (usually along with a judgmental evaluation), it is common for the employee to actually use the same form or graphic scale as their supervisor. While some managers will have an employee complete a self-evaluation and consider that to be sufficient for performance appraisal, is much more common for supervisors to require the employee to discuss their self-evaluation, sometimes comparing it directly to the manager's or supervisor's judgment evaluation.

The biggest problem with self-evaluation as the sole aspect of a performance appraisal is quite obvious - the vast majority of employees are going to rate themselves highly on the scale for most (if not all) of the dimensions of performance. Not only does this process often negate the entire purpose of a performance appraisal (if no real appraisal is being done), but it becomes even trickier when you have an employee that actually takes their self-evaluation seriously and provides genuine answers and identifies weaknesses. When you have one staff member who does this among other staff members who rate themselves very highly (whether it is due to purposeful manipulation of the appraisal or if the individuals genuinely believes they are virtually without flaws in their job performance), an employer ends up with inconsistent evaluations that essentially mean nothing.

One limitation of self-evaluations may be a surprising, but legitimate, concern. Many employees who are asked to perform self-evaluations, which are then discussed with their supervisor, sometimes report higher stress levels than they would have had based solely on a judgmental evaluation. Many employees fear that they will be judged more harshly if they believe themselves to be better at a particular performance dimension than their supervisor believes them to be. Thus, the process can become more stressful than necessary and may result in an inaccurate appraisal.

One concern employers should have when it comes to peer evaluations and self-evaluations is the same as in judgment evaluations - the accuracy and clarity of the measurement tool. If a graphic scale is being used, it is absolutely imperative that the questions or dimensions of performance measurement are as clear as possible to promote accurate results. For example, imagine there is a question on an evaluation form that relates to the number of absences an employee has had. If the question is posed as such: "Employee attends work regularly without any unnecessary absences," there may be inconsistent answers. The employee himself may believe that his absences have been necessary and have not been extensive; a co-worker who has had to pick up the slack of the employee whenever they were absent may produce a radically different answer. However, if the evaluation form instead says: "The employee has not exceeded the annual number of paid leave days allotted," both the employee and their peers now have a much clearer understanding of what the questionnaire is trying to determine and are likely to produce a more accurate appraisal.

There are obviously many limits and concerns inherent in both peer and self-evaluations. Nevertheless,they may be able to play an important role in the performance appraisal process if handled correctly.