Temporal Factorization Vs. Spatial Factorization
L. Zelnik-Manor and M. Irani
The traditional subspace-based approaches to segmentation (often referred to as multi-body factorization approaches) provide spatial clustering/segmentation by grouping together  points with consistent motions. We are exploring a dual approach to factorization, i.e., obtaining temporal clustering/segmentation by grouping together frames capturing consistent shapes. cuts are thus detected at non-rigid changes in the shape of the scene/object. In addition it provides a clustering of the frames with consistent shape (but not necessarily same motion). For example, in a sequence showing a face which appears serious at some frames, and is smiling in other frames, all the ``serious expression'' frames will be grouped together and separated from all the ``smile'' frames which will be classified as a second group, even though the head may meanwhile undergo various random motions.

ECCV'04 Paper in pdf

Full length version (submitted to review)

Some example results:
These videos show results of temporal clustering of frames. Frames are labeled according to recovered temporal clusters.

Single sequence:
Brave Heart (1.1M avi)
Lord of the Rings (7.5M avi)
Hand (3.6M avi)
Facial Expressions (1.8M mp4)

Across sequences:
Expression Recgonition (21.8M avi)
Pose Recognition (15.5M avi)