Better by the Bunch: Evaluating Job Candidates in Groups

The key to avoiding gender stereotyping in the hiring process lies in evaluating job candidates as a group, rather than one at a time. So says new research by Iris Bohnet, Alexandra van Geen, and Max H. Bazerman.
by Maggie Starvish

New research suggests that organizations wishing to avoid gender stereotyping in the hiring or promotion process-and employ the most productive person instead—should evaluate job candidates as a group, rather than one at a time.

“The three of us have produced one of the most exciting papers that I have worked on"
—Max Bazerman

A recent collaboration between Harvard Business School and the Harvard Kennedy School (HKS) reveals that you're much less likely to stereotype by gender if you apply an "evaluation nudge"—an intervention aimed at overcoming biased assessments, in which people are evaluated jointly rather than separately regarding their future performance.

These findings are presented in the working paper When Performance Trumps Gender Bias: Joint versus Separate Evaluation, coauthored by HKS Professor and academic dean Iris Bohnet, HKS doctoral student Alexandra van Geen, and HBS Professor Max H. Bazerman.

"This project integrates my prior work on using joint decision-making to create more ethical decisions with Iris's long-standing expertise in developing strategies to promote gender equality," says Bazerman. "Together, the three of us have produced one of the most exciting papers that I have worked on."

The idea for the paper grew from a conundrum that Bohnet faced when she became faculty director of the Kennedy School's Women and Public Policy Program (WAPPP). Although she'd built a career on using behavioral economics to help others improve their decision-making, she couldn't figure out how to apply those insights to gender issues.

"It took me a few months," says Bohnet,"and then all of a sudden it became very obvious to me."

Employing The Behavioral Economics Toolbox

What became obvious, Bohnet says, was that she needed to look at gender stereotyping as just another heuristic that people employ during the decision-making process. Such heuristics are often subconscious, and though they help people make decisions faster, they don't necessarily help them make better ones.

This article is part of a continuing series on faculty research and teaching commemorating the 50th anniversary of the first women to enter Harvard Business School's two-year MBA program.

"A lot of discrimination happens because of implicit biases where people like you and me —who don't want to do the wrong thing, who don't want to discriminate, or don't want to treat people differently based on their demographic characteristics—fall into these stereotype traps because stereotypes have something to do with how the world is," says Bohnet. "As long as we don't see women in leadership positions, for example, we don't associate leadership with women. I thought if that is true, then I can treat that mistake the way I treat other mistakes people make."

This meant applying insights from the behavioral economics toolbox. The research team was interested in how organizations can change to accommodate what are often natural and subconscious individual behaviors, making it easier for a person to make the right choice. The 2008 book Nudge, written by Richard H. Thaler and Cass R. Sunstein, piqued their interest. (The book also inspired the term "evaluation nudge.")

"The contribution of the book was to show that we can start changing environments to make it easier for our minds, which aren't perfectly rational and don't absorb information perfectly, to succeed in life," Bohnet explains.

A familiar scene to job seekersWhile the book's authors applied their theory to policy changes—such as the Pension Protection Act of 2006, which mandated in part that employees be automatically enrolled in a pension plan upon hiring and must choose to opt out rather than opt in—Bohnet, Bazerman, and van Geen decided to apply the theory to evaluation decisions at organizations.

"Max had thought of the particular question that we look at—joint versus separate evaluation—a long time ago," Bohnet says, adding that the idea for the paper came from "a combination of Max's earlier work thinking about joint versus separate evaluation in the domain of product evaluation, Alexandra's expertise in experimental economics and her passion for gender, and my interest in working on closing gender gaps."

The Experiment

The trio's research included a two-stage experiment conducted at the Harvard Decision Science Laboratory involving some 654 male and female participants. Of those, 554 played the role of an employer selecting an employee for a job assignment, while 100 played the role of employee.

In stage one, the "employees" completed two rounds of a math or verbal task for which they were paid based on performance; they also completed a brief demographic questionnaire that included, most importantly, their gender. The nature of the tasks were deliberate: "Most studies that measure explicit gender attitudes find that females are believed to be worse at math tasks and better at verbal tasks than males," the paper explains.

In stage two, each "employer" was asked to choose an employee among the pool of participants, with the knowledge that employers would be paid according to that employee's round-two performance. The employers were informed of the employees' gender and past performance. (For this stage of the experiment, the researchers deliberately narrowed the employee pool down to participants whose stage-one task performances were similarly mediocre, such that there was no obvious star performer to choose.)

In some cases the employer was performing a joint evaluation, in which he or she was presented with the round-one results of both a male and a female employee. In others, the employer was presented with the round-one results of only one male or female employee—a separate evaluation.

"This paper … has insights that could be implemented by organizations tomorrow if they so choose"
—Iris Bohnet

The results were striking. In cases of separate evaluation, the employers were more likely to choose male employees for the math tasks and females for the verbal tasks. But in cases of joint evaluation, stereotypes did not seem to matter at all. The researchers attribute their findings to frame of reference.

"Our hunch is that the mechanism works something along the following lines: if you look at one pair of shoes, it's hard to evaluate the quality of those shoes," Bohnet explains. "You will be much more likely to go with stereotypes or heuristics or rules of thumb about shoes. But if you have several pairs of shoes available, you're much more likely to be able to compare different attributes of the shoes."

Next Steps

The researchers are currently testing their theory at a large company. Instead of submitting promotion recommendations to upper management individually, supervisors from several departments are evaluating their promotion recommendations together, as well as submitting the recommendations jointly.

Bohnet has also presented the working paper at the World Economic Forum, on Capitol Hill, and at the Harvard Faculty Club.

"It's a paper that's very close to my heart," she says. "This paper has potential in academics and for the world of practice because it has insights that could be implemented by organizations tomorrow if they so choose."

The researchers encourage businesses to consider the benefits of joint evaluation not only to close the gender gap, but also because of the potential positive effects on a company's bottom line. "By basing decisions on performance rather than uninformative stereotypes," Bohnet says, "instead of hiring who you think is the most productive person, you'll hire who actually is the most productive person."

For Bazerman, the paper is a great illustration of the collaborative process among the various schools at Harvard. "Working with Iris Bohnet and Alexandra van Geen is a good example of this," he says. "Iris is a well-recognized scholar on the role of gender in decision-making and public policy, and Alexandra is a spectacular doctoral student. Like HBS, HKS and WAPPP care deeply about how to improve decision-making and find solutions to our bounded ethicality."

Readers: The researchers have a question for you. How does your organization hire and promote people, and how are jobs assigned—by explicitly comparing potential candidates or by simply going with the obvious ones? We would love to hear from you.

Share your thoughts in the comment section of this story, or get in touch with them directly at;; or

About the Author

Maggie Starvish is a writer based in Somerville, Massachusetts.

Post A Comment

In order to be published, comments must be on-topic and civil in tone, with no name calling or personal attacks. Your comment may be edited for clarity and length.
    • J.D.Bailey
    • TIA, Adelo
    I agree, hiring and promotions have become biased towards candidates flexible social engineering skills. This is good for sales and marketing, but carries far less production value for all other organizations (especially management level).

    As a group of equals, I think, there is a subtle peer-review occurring and personnel (managers, HR, SME) doing hiring and promotions are more conscious of looking week or foolish to colleagues. IOW: Self-image is at threat, which is very important in any work environment.

    I hope .Gov, .Mil, and .Com catch on to group review soon. There are far to many person-centric career-managers, and far too few mission-true manager-successes.
    • Kapil Kumar Sopory
    • Company Sopory, SMEC(India) Private Limited
    Unhealthy competition has to be discouraged. Cut-throat competition is still worse. Demands of achieving business goals are important but reaching milestones dubiously is a practice which leads to disaster sooner than later.
    One practice which is observed is the matching reduction in price of a commodity with its quality/measure (weight, size,etc.) so cleverly that it would not be discernible easily and people would easily be duped. Some airlines reduced their fares but downgraded the passenger services, the latter coming to light only after boarding. They did reap some moneatary benefits but ultimately their " Business Competition" harmed the society. It meant loss of reputation as well.
    • Kapil Kumar Sopory
    • Company Secretary, SMEC(India) Private Limited
    Hiring and promoting, by whatever means done, lack absolute objectivitty.
    In our organisation, recruitment is a ticklish issue as we, being technical consultants, have to find skilled people meeting strict criteria for each individual position fixed by our clients which leads to frantic search and fixing up the one found to fulfil the criteria. Thus, only hard skills are generally focused upon for we do not want to let go the one "traced out" after perspirative efforts. At times, some people in junior positions are required but not many at a time and hence evaluation as a group as advocated by Maggie Starvish is not applicable. The idea is proper nevertheless.
    Promotion/ annual increment is based on assessments via a procedure which involves advising by the subject his over-the-year performance to his reporting authority and the latter making his comments and recommendations. However, here too, the decision at times is not data/performance-based but is based on subjectivity.
    Ideally, such decisions must not be based on whims and fancies but it is easier said than done.
    • Peter Lee
    • Mg Consultant, RDS Remuneration Data Specialists Pte Ltd
    Yes, I agree. The findings are not surprising.

    This applies to almost all aspects of work. My experience is that when organisations that are already successful have difficulty in increasing their productivity further, it is usually their inability to improve their work collaboration processes i.e. people do not trust one another fully to share and collaborate effectively.

    The issue is that most organisations hit this bottle-neck after they have exhausted low hanging fruits of management.
    • Lucy Christie
    • Group GM Talent, Transfield Services
    It is good to see research emerging to confirm what we have been adopting as good talent practice to promote gender (and other demographic) equity. A training exercise we use from time to time is to present on the job performance reporting and assessment data with a gender agnostic name and review/coach on objective selection. Group discussions have worked well for us where the composition of the group is healthy. These discussions do not work well if the group convenes to reinforce a shared bias - either conscious or unconscious.
    • Saoda Rassul
    • Associate Professor, KazNTU (Kazakh National Technical University) Kazakhstan Almatu
    To evaluate and hire candidates for the job, as a decisive factor for the organization itself and for the recruitment of staff as a whole is professionalism, skill qualification, competence, literacy, scholarship, integrity and culture. It would seem from these basic qualities as honesty and organizational culture of staff and management is not only prosperity, but also the future of any company.