Cars have turned into computers-on-wheels. The modern car has dozens of computer systems taking care of engine control, safety systems or networked services. The number of these systems will only increase in the future, promising us safer, better, more enjoyable voyage.

But: how can we make sure that all these computer systems do not steal the driver’s attention? How to design interaction with car systems to feel effortless, intuitive, and non-distracting? How to give all the necessary information to drivers without overloading their brains? How to make it all safe?

The MIVI project seeks answers to these questions, combining research findings with industrial R&D solutions.

What’s the problem?

Modern car computer systems mostly work at the background, without need for driver attention. However, when something unusual happens, the driver is asked to quickly make sense of the situation and take appropriate control. In traffic, such situations can arise very quickly, within seconds.

Driver distraction is one problem that plagues car traffic. The more entertainment and navigation or commercial services are vying for the driver’s attention, the less time is spent in the primary task, driving. Switching attention between tasks is hard for all humans, and particularly troublesome for drivers.

Semi-autonomous cars do not completely remove these problems. They take over some driving tasks, but still rely on the human to act in critical situations. Unfortunately, it is difficult for humans to maintain interest in traffic if the car handles it most of the time. It is doubly difficult for humans to quickly understand a new situation and take over control as needed.

More displays…?

The en vogue trend in car interior design is to add larger and larger displays, as many as possible. However, interior displays may require drivers to take their eyes off the road and focus at (often quite complicated) user interfaces. This results in shifting the driver’s attention from the primary task (i.e. driving and traffic status) to a secondary task (e.g. navigation via maps or traversing a music library). Such a shift can be extremely dangerous as research shows that focusing back onto the road may take even 20 seconds (Lu et al.. 2017).

How about speech or haptics?

Speech-based user interfaces do not distract the driving task unless the information mediated is complex (e.g. Kun et al. 2007). However, noise inside the car can mask some of the interaction (Hoggan et al. 2009; Harvey & Pointon, 2018).

The use of haptics and touch-actuated or gesture-based interfaces has often been considered as an option to overcome shortcomings of both visual and auditory-based information mediation. Our research in the field (Farooq et al., 2014; Pakkanen et al., 2014, Lylykangas et al., 2016) shows that by using haptics in current in-vehicle infotainment (IVI) systems, it is possible to reduce the visual workload and decrease reaction time for driving-related tasks.

However, at the moment, most haptic interaction methods produce vibration based feedback, which can be prone to signal distortion (loss) and integration (change) in a haptically noisy environment (Hoggan et al. 2009).

Multimodal interfaces

We seek novel multimodal in-vehicle interaction and information presentation (MIVI) methods. “Multimodal” means UIs that make use of several methods simultaneously to interact with people, to combat the shortcomings of any single method.

Information and warnings may be given via displays, audio or haptics; commands to the car may use speech, touch surfaces, gestures, or gaze direction.

Consider a navigation system. Instead selecting options from a touchscreen to setup a navigator, speech input is activated by squeezing a steering wheel. Basic direction information (e.g. turn right or left) is then given via haptic actuators implemented in the driver’s seat, while more complex information is mediated with a standard text-to-speech synthesizer. A gesture towards the navigation system activate a function that reflects the map to a head-up display (HUD) so that there is no need to take the eyes off the road at any point.

Driver monitoring

An adaptive car actively monitors and adapts to the driver and the environment. Interaction can change based on personal preferences, environment information (e.g. noise level inside car, traffic or weather outside car, selected route) or even driver’s state (e.g. focus of attention in terms of direction of the gaze). Such systems open up novel possibilities for redesigning in-car interaction.

MIVI Project work

All of the above requires development of hardware, software systems, system architecture, AI infrastructure and studying the functionality of these new systems with human participants in laboratory settings and actual driving situations.

Our main implementation steps:

• Selecting the best existing HW solutions and components for multimodal IVI systems
• Creating technology prototypes that are completely novel or not presently available
• Developing IVI software platforms able to utilize multimodal interaction
• Developing AI methods for analyzing visual and auditory information
• Integrating hardware, software, and AI infrastructure
• Testing the new IVI systems with drivers
• Sharing, disseminating, and publicizing of the results

For more details, please look at our publications and the news.