Virtual understanding

SHE HAS MICROSOFT on speed dial. She regularly collaborates with billion-dollar gaming companies. Only 11 years removed from receiving her PhD, she’s a leader in human-computer interaction (HCI), one of the world’s most rapidly evolving disciplines.

Given all this, it’s perhaps easy to conjure images of Regan Mandryk as a computing prodigy, a precocious young programmer who spent much of her adolescence immersed in the worlds of Super Mario and Zelda. But one soon learns that this associate professor of computer science isn’t concerned with stereotypes or convention. It’s her job, in fact, to defy them.

“I wasn’t a gamer at all,” says Mandryk, recalling her childhood in Winnipeg as the youngest of three children. “I was very much into board games, though. My family played board games quite often, and I was definitely competitive.”

Even as a university student, computer science and gaming weren’t initially on her radar. After receiving her undergraduate degree in math and physics at the University of Winnipeg, she enrolled in a kinesiology master’s program at Simon Fraser University (SFU).

Mandryk’s master’s studies focused on modeling human movement in virtual environments. She began wading, almost by accident, into the worlds of virtual reality and computer science.

“It was the 90s. What can I say?” says Mandryk with a laugh as she explains her work in virtual reality.

This interest followed Mandryk into her PhD research, which she also conducted at SFU. One of the labs there had a large tabletop computer display, she remembers, and “people were always looking for things to do with it.” While others focused on using the display to create single-player games—think Pac-Man or Dungeons and Dragons—Mandryk saw the potential in using computers to enhance how people interact with others. So her first PhD research project was, naturally, to create a computerized board game.

“I like board games because people are interacting with each other through the game, and not just with the game itself,” she explains. “So I thought that would be a great idea, to see if we could leverage all the advantages of board games but do it through a computer which has its own advantages, such as simulation, graphics—all that kind of stuff.”

That project inspired the path of her PhD program, which focused on how emotions are influenced within interactive-play environments.

“I created this approach that involved hooking people up to physiological sensors and using mathematical models to measure their experience using a computer game,” Mandryk explains. “It’s basically like hooking someone up to a lie detector machine and asking, ‘Are you having fun?’ You can look at the data in real time, and see a graph that represents how much fun or frustration someone is experiencing.”

Firmly planted in the world of HCI, Mandryk started looking for faculty positions across Canada. At the top of her list was finding a mentor, she says. And at the top of that list was Carl Gutwin.

Gutwin is one of the world’s best-known researchers in HCI who, during his nearly two decades at the U of S, has established one of North America’s most highly regarded HCI research programs. 

“Carl was the reason I applied here in the first place,” Mandryk says. “He’s not only one of the best HCI researchers in Canada, but also just a prince of a human being.”

She received offers from every university she interviewed at, but the decision on which one to accept didn’t take long. 

“This was the last interview I did, and I was just twitterpated when I left. I was so immediately enamoured by the place.”

Computers with empathy

After joining the university, Mandryk was interested in finding less intrusive ways to expand on her PhD research. Hooking research subjects up to sensors can potentially influence emotion regardless of the technology they are using, so Mandryk and one of her grad students began to look at other ways to gauge users’ emotions.

They analyzed keystroke data by using the timing and pressure of a subject’s typing tendencies to determine how they responded to different kinds of technology. They also examined pupillometry, which measures pupil dilation in response to stimuli, and extracted heart rate data using webcam images of a subject’s face.

It wasn’t until she began working with PhD student Kathrin Gerling, though, that Mandryk started to immerse herself in what’s known as the serious games movement—using video games to help address real-world issues. Gerling was interested in researching the benefits of motion-based games for elderly people in care homes. Working with the Saskatoon Health Region, Mandryk and Gerling conducted studies that led to numerous papers and the development of guidelines on how to safely integrate full-body motion into games designed for seniors.

Mandryk next moved into another sphere within the serious games movement: developing technology to help children with neurological disorders regulate their emotions. Working with families of children with conditions such as autism and fetal alcohol spectrum disorder, she developed a biofeedback system where the game experiences changed depending on the child’s reaction. In racing games, for example, the screen would begin filling with mud splatter when emotions reached a certain level. In hockey games, the screen would fill with frost, while in jungle adventure games, spiders would obscure the child’s view.

Working with children and their families was rewarding to Mandryk, but she also felt tremendous pressure to succeed.

“When you’re working with families directly, you’re giving them hope. So you don’t want to create anything that won’t work. You don’t want to give them false hope.”

The system Mandryk developed can be used with any off-the-shelf video game to help children learn how to self-regulate emotions. It was patented by the U of S and although it hasn’t been commercialized yet, it is something Mandryk would like to pursue in the not-too-distant future.

Like everything, it’s a matter of finding time in a discipline where time is in perpetual short supply.

A focus on people

After only nine years as a faculty member, Mandryk’s track record already outpaces that of many senior researchers. She has nearly 200 publications, has been appointed to leadership roles on national HCI programs and committees, and was recently elected as an inaugural member of the Royal Society of Canada’s College of New Scholars, Artists and Scientists.

“I spend a lot of my time turning down opportunities, and sometimes I don’t even have the time to do that,” she says with a laugh. “It’s a challenge. I’m young enough and well-known enough now that people expect me to be really active. So one of the keys for me is having the right team around me.”

The relationships Mandryk forges with colleagues and students, however, are hardly one-way streets.

PhD student Max Birk travelled from Germany to Canada solely for the opportunity to work with Mandryk. He says that aside from her sterling reputation in scholarly circles, she has also become well-known in the industry as an accommodating and supportive mentor.

An avid gamer, Birk was interested in examining the role personality plays in gaming experiences. While that wasn’t an area in which Mandryk was particularly well-versed, she put in a lot of time to get herself up to speed.

“She’s very collaborative and willing to adapt or expand her skill set to meet the needs of her students,” says Birk. “She doesn’t expect students to just conform to her area of expertise and she really didn’t hesitate to build up her knowledge, particularly around the theoretical branch of psychology that aligns with my research interests.”

Having worked in HCI for more than a decade now, Mandryk sees the inherent value in collaborating with students like Birk.

“Ultimately this is a software industry—applications are software. But they’re being used by people,” she explains. “So it’s very important that people who are working in this area understand both the technology and the person using it.”

The future of interaction

Working in HCI is certainly not for the faint of heart, but opportunities in the field are seemingly boundless.

“Virtually all of our graduates can get jobs in the field they want; there are just so many opportunities,” says Mandryk, adding that Canada has the third-largest video game industry worldwide, behind only the United States and Japan. “This is a super-hot area right now. Research is really active, departments are hiring, student enrolments are growing. It’s an exciting place to be.”

Mandryk’s career mirrors the industry in which she works, and is in a constant state of evolution. The latest shift came this April when she was awarded nearly $2.5 million in funding through the Natural Sciences and Engineering Research Council of Canada for a new research initiative. Over the next six years, Mandryk will study ways of understanding and improving user experiences in video games, training new generations of graduate students along the way.

Currently, she is wading into the world of social media, working with Microsoft and Sesame Workshop to expand the capabilities of technology such as Skype.

While Skype works well for business meetings and simple conversations, it doesn’t allow grandparents to interactively connect with grandchildren. Mandryk is looking at ways to integrate games and other interactive components into web-based communications technologies.

So when grandma is in Tennessee and her grandchild is in Toronto, they can build online connections that go beyond mere conversations.

“It really all goes back to the philosophy that we need to build technology that adapts to humans,” says Mandryk, “and not expect humans to adapt to the technology.”