Usage based insurance solutions where smartphone sensor data is used to analyze the driver’s behavior are becoming prevalent these days. However, a major shortcoming of most solutions on the market today, is the fact that trips where the user was a passenger, e.g. taxi trips, get included in the user’s risk profile.

Accelerometer and gyroscope data can be used to estimate a user’s driving behavior.

At Sentiance, we developed a deep learning based solution that gradually learns to model the user’s driving behavior, and that can be used to detect and remove those trips where the user was actually a passenger.

The input to our deep neural network is raw accelerometer and gyroscope data. A convolutional neural network is used to learn low-level features that describe the user specific driving characteristics, whereas an LSTM layer is stacked on top of the convolutional block to model long-term temporal dependencies within the sensor data.

The goal of this project was to let the neural network learn a feature space in which trips with a similar driving behavior appear close to each other, while the cosine distance between trips with distinct kinds of driving behavior should be large. Since it is difficult and expensive to gather large amounts of labelled data in this case, we applied a clever transfer learning trick; The network was initially trained with thousands of userIDs as output labels, and was forced to learn to classify which trip originated from which user. Once convergence was reached, we chopped of the top soft-max layer of the network, and used the dense layer directly as our feature space.

Transfer learning was used by training the network to perform a related task, and then using one of the dense layers directly as our feature space for the real task at hand.

Basically, this allowed the network to learn a metric space that we can then use for clustering or outlier detection, even on trips for new users that never appeared in our training data.

Check out our latest blog post at Sentiance that describes the model and approach in details: Applying deep learning to distinguish drivers from passengers using sensor data

The post Hybrid deep learning for modeling driving behavior from sensor data appeared first on Computer vision for dummies.

At Sentiance, we use machine learning to extract intelligence from smartphone sensor data such as accelerometer, gyroscope and location. We’ve been doing this for quite a while now, and are very proud on our state-of-the-art results regarding sensor based activity detection, map matching, driving behavior, venue mapping and more.

The obvious next step is to go from simply detecting what you are doing, to predicting what you will be doing in the future. Knowing your near-term future allows us the explain the intent of your current situation. For example, if we detect you are currently running, and we can predict that you will be on a train followed by a visit to your work location, then we can immediately explain why you are running; you’re obviously not just being sportive today!

Deep learning to the rescue

To be able to come up with long-term predictions, we started out with a simple Markov Chain like approach and ended up turning to deep learning. We trained a Long Short-Term Memory (LSTM) recurrent neural network on several thousands of event timelines. The network learns to encode general human behavior and surprisingly is able to quickly adapt to specific user habits. The following figure illustrates the architecture of our deep learning pipeline:

LSTM architecture used for event prediction. The input consists of a sequence of your last 128 events, while the output is a prediction of your next event, together with a duration estimate of the current event. The network is implemented using TensorFlow

I’m extremely proud of the results we achieve, mainly because of the following:

1. The network is often able to predict events that a human observer would not even think of. This sometimes feels like magic, even for the geekiest of our data scientists – and we all like magic!
2. We started out with simpler Bayesian models (Markov based) and gradually moved to more advanced solutions
3. Deep learning in this case actually solves a real problem instead of just following the hype
4. Our deep learning pipeline actually runs in production for millions of users. We put a lot of effort in making it scalable, reproducible and maintainable.

Check out our technical blog post that outlines the details on how we trained and tested our models, and how they actually work: http://www.sentiance.com/2017/04/25/predictive-analytics-applying-deep-learning-on-mobile-sensor-data/

Check out the videos of our prediction results (Links to the original blog post)

The post Deep learning for long-term predictions appeared first on Computer vision for dummies.