News & Events


Touchscreen menu innovation faster, simpler

Posted on 2018-03-12 in Science & Technology, Students & Campus Life

Student Mohammad Sami Uddin uses his new, faster HandMark-Finger menu developed for smart TVs, computer desktops and tablets. (Photo: David Stobbe)

By Federica Giannelli

This article first ran as part of the 2018 Young Innovators series, an initiative of the U of S Research Profile and Impact office in partnership with the Saskatoon StarPhoenix.

In a world where people learn how to use iPads before they learn to walk, finding simpler ways to interact with touch devices is a must.  

University of Saskatchewan PhD computer science student Mohammad Sami Uddin has tackled the problem by developing a new multi-touch menu called HandMark-Finger. The interface, which is designed for large touchscreens such as tablets, makes it easier and about 30 per cent faster to input commands.

“This research has the potential to make everyday interactions with touch devices simpler,” said Carl Gutwin, computer science professor and Uddin’s supervisor.

“People are beginners for the first few hours they use their device, but are experts for the rest of the time,” he said. “We are looking at ways to make touchscreen menus fast for experts.”

Mohammad Sami Uddin
Mohammad Sami Uddin (photo: David Stobbe)

The researchers have found that by speeding up human interaction with devices and by fitting in more commands, HandMark-Finger may have real-life applications in devices such as in tablets, computer desktops, and smart TVs. Uddin has both published his results and presented them at national and international conferences.

To use HandMark-Finger, users have to touch the screen palms down. Different sets of commands then pop up around the fingers. Like “digital pianists,” they use all the fingers to interact with the screen, instead of the one- or two-finger tapping and swiping done on devices now.

Unlike with familiar interfaces, the HandMark-Finger comes with around 40 commands. Since the menu is built around users’ hands, it helps them remember the location of commands, with fingers serving as memory cues.

“I got the idea from the way people use their fingers to play the piano,” said Uddin. “People have intimate knowledge of their hands, so we used them as a memory device by associating commands to fingers.”

Uddin has installed the HandMark Menu on a large touchscreen device and tested how quickly and accurately people could find and select commands on the screen. His results from 13 users show that the time needed for learning how to use the menu is similar to current devices, as are the error rates.

Uddin has also developed a second 160-command interface he calls HandMark-Multi. In this version, sets of commands appear between the thumb and index finger, depending on how many fingers touch the screen. Unlike with HandMark-Finger, users make more mistakes and are slower to learn the menu.

“We played with different HandMark versions to see their performance and whether they could fit a large number of commands,” said Uddin. “People in our study preferred HandMark-Finger over regular menus and over HandMark-Multi because it provided clear reference points for the location of commands.”

Uddin and Gutwin agree that more research and re-design is needed before HandMark Menus reach commercial devices. The researchers will now focus on how best to group commands on the screen, and perhaps conduct a large-scale study.

The project is funded by the federal agency NSERC and is part of the U of S Interaction Lab, which has been achieving success in advancing computer interfaces, computer games, and systems used to visualize information.

Federica Giannelli is a graduate student intern in the U of S research profile and impact unit.


Back to News Listing

Related Articles

Graduate student receives Women of Distinction Award for trades work

Posted on 2018-06-20

Devin West honoured for her work in the trades and for activism for women

Shanghai summer

Posted on 2018-06-18

Economics student shares her experiences living and working in China

Talking tech

Posted on 2018-06-08

Computer science professor Ian Stavness is helping to develop the next generation of speech synthesis

Innovation in horse rehabilitation

Posted on 2018-06-08

After the loss of her horse Cash, Samatha Steinke is trying to find a better way to treat injured horses

Clone wars: finding buggy code copies

Posted on 2018-06-08

Bad code can have 'a massive impact,' says computer science professor Chanchal Roy

U of S political studies student heading to Harvard Law School

Posted on 2018-06-08

Areeb Salim says he's confident his studies at the U of S has prepared him well for Harvard