new Gesture Controls Can Be Used In Sports Apps

I have been using the next generation of Apple Watch software and there is a unique new feature that WILL become more widespread in sports watches even though it is not intended as a sports feature per se…even higher-level athletes using Garmins would find this useful.

There is a video of it in action further above.

In a nutshell:

You can use hand gestures instead of taps, presses and swipes to control your watch.

For controlling fitness usage that will mean adding laps/segments and scrolling screens.

How it works

Gesture Controls are configurable. In the example I describe here, I used a fist clench gesture which the Apple Watch converted into an action to page through screens. Other gestures and watch actions can be used.

Q: Does it work well?

A: Yes! Gesture recognition is great but gestures are limited in scope right now and the actions you can assign to them have limitations.

Background

For many years, there have been several methods available to control your watch whilst exercising. How well each method works varies depending on personal preference at your level of exertion. Let’s take a specific example of how we can currently scroll through screens on our watches/apps.

• Buttons – Perhaps the best and most widely used method. You know what pressing a button is like! You also know that 4 or 5 buttons are needed for most sports watches to be treated as truly credible…at least for now.
• Touch – taps & swipes on a watch screen or heart rate monitor have been used in the past to trigger various actions. Usually, a swipe on your touchscreen will switch between the screen of your sports profile/app and many years ago Polar had the Heart Touch feature working with an H7 and V800.
• Audio Control – This works by voice recognition and as such only smartwatches have voice recognition through Wear OS, watchOS or similar, advanced firmware.

Some of these methods require you to bring your arms together or to interact with a specific part of a touchscreen and that can be non-trivial if you are exerting yourself and using a touchscreen swipe. For a cyclist and perhaps an XC runner, these interactions can add a small amount of danger to the interaction!

Are buttons perfect? A: No. Sometimes you might have to contort your right hand to press the bottom on your left wrist and then you might find that your laggy Garmin has made you do an extra press and you have to page back.

How Gesture Control Works

You’ll need watchOS 8 and iOS 15.

Currently, 4 gestures are recognised by the Apple Watch 6/SE and these are fist clench, double fist clench, pinch and double pinch. Once you have followed the instructions from the Apple Watch to recognise your gestures, you assign each gesture to an action on the watch.

Open the Watch app on your iPhone and go to Accessibility>Asistive Touch>On.

These screens take you through the steps and you can see on the second screen where the double pinch is assigned the backwards action (usually back to the previous screen)

 

Great, you’re done…kinda.

You can now use the Apple Watch and control it to open apps and navigate the screens and options within the apps. It’s fairly powerful but its usefulness depends on which gestures you assign to which actions. The normal actions during workouts would be some combination of a basic screen tap, left/right screen swipes, hardware button press and a crown press/rotate.

But now your problems start.

Every app you use is subtly different and each app will interpret actions differently. For example, the Apple Workout app requires a double tap to take a lap, whereas the iSmoothRun app requires a simultaneous press of the two hardware buttons. The Stryd apple Watch app scrolls up and down rather than left to right and lets the side BUTTON (not the crown) take a lap.

So now you can see why I chose the example of switching between screens during your workout. Perhaps taking a lap would be more usefully done with a hand gesture but you can’t because none of the apps I looked at could be configured to allow a lap to be taken with the actions currently supported by gesture control.

This will eventually be resolved with new gestures recognised by Apple like finger click or pinching other fingers together. Then the hand gesture actions themselves need expanding to cover double screen taps and various button combinations. Finally, app developers need to modify how their apps work with the available gesture actions.

What I got to work

My goal was to start an app as normal (fingers) and then use one gesture to take laps and another to scroll screens

iSmoothRun would take a screen tap as a way to scroll screens as would the Apple Workout app as shown in this video.

I couldn’t get LAP functionality to work at all with any gestures, although it might be possible on other apps.

Take Out

Apple intends gestures to be used as Accessibility Features. Which is great.

However, it also shows to me that these features can work really well in some sports scenarios. So well, that they WILL be used by watch and app developers in the near future starting on the Apple Watch 7 / watchOS 8 in September and indeed Samsung introduced some gestures to Galaxy Watch 3 in 2020.

The use of gesture control will allow sports watch designs with fewer than 4/5 buttons. That could be a boost to the sporting credibility of the traditional 2-buttoned smartwatch.

I got the screen scrolling to work well but couldn’t quite get my apps to take a lap. However, this feature is only in Beta right now so we are at the early stages and its scope will grow over time.

What I would like to see

• More gestures recognised – such as pinching a different finger
• More watch actions supported – such as a watch shake or screen double-tap.
• Enable app developers to create a gesture profile that only works within their app rather than at a system level.

Interesting!