News & Events


Touchscreen menu innovation faster, simpler

Posted on 2018-03-12 in Science & Technology, Students & Campus Life

Student Mohammad Sami Uddin uses his new, faster HandMark-Finger menu developed for smart TVs, computer desktops and tablets. (Photo: David Stobbe)

By Federica Giannelli

This article first ran as part of the 2018 Young Innovators series, an initiative of the U of S Research Profile and Impact office in partnership with the Saskatoon StarPhoenix.

In a world where people learn how to use iPads before they learn to walk, finding simpler ways to interact with touch devices is a must.  

University of Saskatchewan PhD computer science student Mohammad Sami Uddin has tackled the problem by developing a new multi-touch menu called HandMark-Finger. The interface, which is designed for large touchscreens such as tablets, makes it easier and about 30 per cent faster to input commands.

“This research has the potential to make everyday interactions with touch devices simpler,” said Carl Gutwin, computer science professor and Uddin’s supervisor.

“People are beginners for the first few hours they use their device, but are experts for the rest of the time,” he said. “We are looking at ways to make touchscreen menus fast for experts.”

Mohammad Sami Uddin
Mohammad Sami Uddin (photo: David Stobbe)

The researchers have found that by speeding up human interaction with devices and by fitting in more commands, HandMark-Finger may have real-life applications in devices such as in tablets, computer desktops, and smart TVs. Uddin has both published his results and presented them at national and international conferences.

To use HandMark-Finger, users have to touch the screen palms down. Different sets of commands then pop up around the fingers. Like “digital pianists,” they use all the fingers to interact with the screen, instead of the one- or two-finger tapping and swiping done on devices now.

Unlike with familiar interfaces, the HandMark-Finger comes with around 40 commands. Since the menu is built around users’ hands, it helps them remember the location of commands, with fingers serving as memory cues.

“I got the idea from the way people use their fingers to play the piano,” said Uddin. “People have intimate knowledge of their hands, so we used them as a memory device by associating commands to fingers.”

Uddin has installed the HandMark Menu on a large touchscreen device and tested how quickly and accurately people could find and select commands on the screen. His results from 13 users show that the time needed for learning how to use the menu is similar to current devices, as are the error rates.

Uddin has also developed a second 160-command interface he calls HandMark-Multi. In this version, sets of commands appear between the thumb and index finger, depending on how many fingers touch the screen. Unlike with HandMark-Finger, users make more mistakes and are slower to learn the menu.

“We played with different HandMark versions to see their performance and whether they could fit a large number of commands,” said Uddin. “People in our study preferred HandMark-Finger over regular menus and over HandMark-Multi because it provided clear reference points for the location of commands.”

Uddin and Gutwin agree that more research and re-design is needed before HandMark Menus reach commercial devices. The researchers will now focus on how best to group commands on the screen, and perhaps conduct a large-scale study.

The project is funded by the federal agency NSERC and is part of the U of S Interaction Lab, which has been achieving success in advancing computer interfaces, computer games, and systems used to visualize information.

Federica Giannelli is a graduate student intern in the U of S research profile and impact unit.


Back to News Listing

Related Articles

StarPhoenix obituary: College of Arts & Science remembers Professor Emeritus

Posted on 2018-03-20

Leonard Gordon Miller served as head of the Department of Philosophy

Statement from the College of Arts & Science on Reconciliation, Decolonization and Indigenization at the University of Saskatchewan

Posted on 2018-03-13

The College of Arts & Science is publicly reaffirming its commitment to reconciliation efforts and to supporting Indigenous students

Hope, shipped to your door

Posted on 2018-03-09

Third-year women and gender studies student Shawn Clouthier created Courage Box, a subscription service to support mental health

From South America to Saskatoon

Posted on 2018-03-09

She hasn’t even officially started her own career yet, but Mayra Samaniego is already working on two different continents to inspire the next generation of young women in the field of computer science

Social media's incorrect astronomy assertions

Posted on 2018-03-08

Why critical thinking and facts are important in the 21st century