Predicting where you'll look in a 360° video

Researcher, UMass Amherst · 2018 · Archived

A sequence-to-sequence LSTM that predicts head movement, so only the part of a 360° video you're about to look at streams in full quality.

I trained it in PyTorch on a public head-movement dataset from 57 participants and wrote the quaternion math for head rotation by hand. It also reads precomputed saliency maps, which mark where each frame draws the eye.

Stack

Languages
Python
Models and agents
PyTorch