News & Events


Touchscreen menu innovation faster, simpler

Posted on 2018-03-12 in Science & Technology, Students & Campus Life

Student Mohammad Sami Uddin uses his new, faster HandMark-Finger menu developed for smart TVs, computer desktops and tablets. (Photo: David Stobbe)

By Federica Giannelli

This article first ran as part of the 2018 Young Innovators series, an initiative of the U of S Research Profile and Impact office in partnership with the Saskatoon StarPhoenix.

In a world where people learn how to use iPads before they learn to walk, finding simpler ways to interact with touch devices is a must.  

University of Saskatchewan PhD computer science student Mohammad Sami Uddin has tackled the problem by developing a new multi-touch menu called HandMark-Finger. The interface, which is designed for large touchscreens such as tablets, makes it easier and about 30 per cent faster to input commands.

“This research has the potential to make everyday interactions with touch devices simpler,” said Carl Gutwin, computer science professor and Uddin’s supervisor.

“People are beginners for the first few hours they use their device, but are experts for the rest of the time,” he said. “We are looking at ways to make touchscreen menus fast for experts.”

Mohammad Sami Uddin
Mohammad Sami Uddin (photo: David Stobbe)

The researchers have found that by speeding up human interaction with devices and by fitting in more commands, HandMark-Finger may have real-life applications in devices such as in tablets, computer desktops, and smart TVs. Uddin has both published his results and presented them at national and international conferences.

To use HandMark-Finger, users have to touch the screen palms down. Different sets of commands then pop up around the fingers. Like “digital pianists,” they use all the fingers to interact with the screen, instead of the one- or two-finger tapping and swiping done on devices now.

Unlike with familiar interfaces, the HandMark-Finger comes with around 40 commands. Since the menu is built around users’ hands, it helps them remember the location of commands, with fingers serving as memory cues.

“I got the idea from the way people use their fingers to play the piano,” said Uddin. “People have intimate knowledge of their hands, so we used them as a memory device by associating commands to fingers.”

Uddin has installed the HandMark Menu on a large touchscreen device and tested how quickly and accurately people could find and select commands on the screen. His results from 13 users show that the time needed for learning how to use the menu is similar to current devices, as are the error rates.

Uddin has also developed a second 160-command interface he calls HandMark-Multi. In this version, sets of commands appear between the thumb and index finger, depending on how many fingers touch the screen. Unlike with HandMark-Finger, users make more mistakes and are slower to learn the menu.

“We played with different HandMark versions to see their performance and whether they could fit a large number of commands,” said Uddin. “People in our study preferred HandMark-Finger over regular menus and over HandMark-Multi because it provided clear reference points for the location of commands.”

Uddin and Gutwin agree that more research and re-design is needed before HandMark Menus reach commercial devices. The researchers will now focus on how best to group commands on the screen, and perhaps conduct a large-scale study.

The project is funded by the federal agency NSERC and is part of the U of S Interaction Lab, which has been achieving success in advancing computer interfaces, computer games, and systems used to visualize information.

Federica Giannelli is a graduate student intern in the U of S research profile and impact unit.


Back to News Listing

Related Articles

The Discover Arts and Science Instagram Contest

Posted on 2018-09-20

Enter on Sept. 25 to win a $100 Shop usask gift card

Chemistry professor to make pitch during Dragons’ Den season premiere

Posted on 2018-09-19

Excir Works develops innovative process for extracting gold from electronic waste

College of Arts and Science launches Fall Events Guide

Posted on 2018-09-17

Featuring lectures, exhibitions, concerts and galas happening on campus and in Saskatoon

Free Pancake Breakfast

Posted on 2018-09-14

All College of Arts and Science students, faculty and staff are invited

A lifetime of achievement

Posted on 2018-09-14

What continues to power Professor John Tse’s passion for discovery?

Canadian paleontologists coming to campus

Posted on 2018-09-14

U of S researchers will be joined by other prominent paleontologists from across the country when the Department of Geological Studies hosts the prestigious Canadian Paleontology Conference