Smartwatches can now track your finger in mid-air using sonar
smartwatches grow smaller, it gets tougher for
people to interact with screens the size of a
matchbook.
That could change with a new sonar technology
developed by University of Washington computer
scientists and electrical engineers that allows you to
interact with mobile devices by writing or gesturing
on any nearby surface -- a tabletop, a sheet of
paper or even in mid-air.
FingerIO tracks fine-grained finger movements by
turning a smartphone or smartwatch into an active
sonar system using the device's own microphones and
speakers.
Because sound waves travel through fabric and do
not require a line of sight, users can even interact
with a phone inside a front pocket or a
smartwatch hidden under a sweater sleeve.
In a paper to be presented in May at the
Association for Computing Machinery's CHI 2016
conference in San Jose, California, the UW team
demonstrates that FingerIO can accurately track
two-dimensional finger movements to within 8mm,
which is sufficiently accurate to interact with
today's mobile devices. The work was recognized
with an honorable mention award by the conference.
"You can't type very easily onto a smartwatch
display, so we wanted to transform a desk or any
area around a device into an input surface," said
lead author Rajalakshmi Nandakumar, a UW
doctoral student in computer science and engineering.
"I don't need to instrument my fingers with any
other sensors -- I just use my finger to write
something on a desk or any other surface and the
device can track it with high resolution."
Using FingerIO, one could use the flick of a finger to
turn up the volume, press a button, or scroll through
menus on a smartphone without touching it, or even
write a search command or text in the air rather
than typing on a tiny screen.
FingerIO turns a smartwatch or smartphone into a
sonar system using the device's own speaker to emit
an inaudible sound wave. That signal bounces off the
finger, and those "echoes" are recorded by the
device's microphones and used to calculate the finger's location in space.
Using sound waves to track finger motion offers
several advantages over cameras -- which don't
work without line-of-sight when the device is
hidden by fabric or another obstructions -- and
other technologies like radar that require both
custom sensor hardware and greater computing
power, said senior author and UW assistant professor
of computer science and engineering Shyam
Gollakota.
"Acoustic signals are great -- because sound waves
travel much slower than the radio waves used in
radar, you don't need as much processing bandwidth
so everything is simpler," said Gollakota, who directs
the UW's Networks and Mobile Systems Lab. "And
from a cost perspective, almost every device has a
speaker and microphones so you can achieve this
without any special hardware."
But sonar echoes are weak and typically not
accurate enough to track finger motion at a high
resolution. Errors of a few centimeters make it
impossible to differentiate between writing individual
letters or subtle hand gestures.
The UW researchers employed a type of signal
typically used in wireless communication -- called
Orthogonal Frequency Division Multiplexing -- and
demonstrated that it can be used to achieve high-
resolution finger tracking using sound. Their algorithms
leverage the properties of OFDM signals to track
phase changes in the echoes and correct for any
errors in the finger location to achieve sub-
centimeter finger tracking.
To test their approach, the researchers created a
FingerIO prototype app for Android devices and
downloaded it to an off-the-shelf Samsung Galaxy
S4 smartphone and a smartwatch customized with
two microphones, which are needed to track finger
motion in two dimensions. Today's smartwatches
typically only have one, which can be used to track
a finger in one dimension.
The researchers asked testers to draw shapes such
as stars, squiggles or figure 8s on a touchpad next to
a smartphone or smartwatch running FingerIO.
Then they compared the touchpad tracings to the
shapes created by FingerIO's tracking.
The average difference between the drawings and
the FingerIO tracings was 0.8 centimeters for the
smartphone and 1.2 centimeters for the
smartwatch.
"Given that your finger is already a centimeter
thick, that's sufficient to accurately interact with
the devices," said co-author and electrical
engineering graduate student Vikram Iyer.
Next steps for the research team include
demonstrating how FingerIO can be used to track
multiple fingers moving at the same time, and
extending its tracking abilities into three dimensions
by adding additional microphones to the devices.
Story Source:
The above post is reprinted from materials provided
by University of Washington . The original item
was written by Jennifer Langston.
Comments
Post a Comment