Yale/ETS Team Design Behavioral Assessment for Grad Admissions

By David Lariviere

For the past five years, ETS research scientists have been working closely with Yale University’s School of Management on developing a noncognitive online assessment to better determine how applicants will perform once they arrive on campus.

“We were particularly interested in going beyond what the GRE and GMAT tests measured,” explained Laurel Grodman, the school’s Managing Director of Admissions. “We wanted more confidence to assess applicants who don’t perform as well on traditional measures while also looking at those who perform very well on the tests and wind up underperforming in school.”

This year about 1,000 Yale School of Management applicants are taking a behavioral assessment, an online admissions tool developed by the Research division at ETS. Yale is the first and only school to explore this tool for candidate evaluation.

The results have been a success so far. “We were looking to see incremental predictability on top of the measures we already have and we feel we did find some value in terms of determining who will perform effectively in the program,” said Bruce DelMonico, Assistant Dean of Admissions.

Patrick Kyllonen, Distinguished Presidential Appointee for ETS, and leader of research on this assessment tool, says this behavioral assessment “could be a complement” to the GRE test because it measures the interpersonal and intrapersonal attributes the GRE test is not designed to capture.

We were looking to see incremental predictability on top of the measures we already have and we feel we did find some value in terms of determining who will perform effectively in the program.

~ Bruce DelMonico

The test features a forced-choice design which presents applicants with two statements and requests they choose the one that is most attractive to them. An example would be selecting between, “I work well with other people” and “I work hard,” according to Kyllonen. Test takers have approximately 20 to 25 minutes to select 120 preferences so there’s not a lot of time to make a decision.

“There are no right and wrong answers,” said Grodman. “We researched a host of other instruments and they were all gameable.” By “gameable” Grodman means it was obvious what the preferred response should be. The Yale team found that the ETS behavioral assessment was not gameable because there was no obvious preferred response.

There is no need for applicants to prepare for the test and they don’t require specialized knowledge or background to complete it.

For Yale, the appeal of this assessment was largely the idea that it could help in addressing diversity and inclusiveness concerns. “That was definitely in the forefront of our thinking,” DelMonico said. “It’s still too early in the process to determine whether we’ll achieve that but I’m optimistic we will once we have more data points.”

I think we’re in a very good place right now and we’re looking for additional information.

~ Patrick Kyllonen

The ultimate goal is to connect what is learned in the assessment to the student experience inside the classroom at Yale. “That would be hugely beneficial to everyone involved here,” said DelMonico.

Everyone involved in this behavioral assessment is happy with its progress. “I think we’re in a very good place right now and we’re looking for additional information,” explained Kyllonen.

Grodman said “the biggest hurdles have been overcome” and all that remains is how to seamlessly put it into operation.

Grodman was complimentary of the assistance ETS has provided Yale. “It really is a productive partnership. ETS has been great in working out all the intricacies and giving us very responsible research.”