Fast video segmentation using encoding cost data

This paper presents a simple and effective pre-processing method, developed for the segmentation of MPEG compressed video sequences. The proposed method for scene-cut detection only involves computing the number of bits spent for each frame (encoding cost data), thus avoiding decoding the bitstream. The information is separated into I-, P-, B- frames, thus forming 3 vectors, which are independently processed by a new peak detection algorithm, based on overcomplete filter banks and on joint thresholding, using a confidence number. Each processed vector yields a set of candidate frame numbers, i.e., 'hints' of positions where scene-cuts may have occurred. The 'hints' for all frame types are recombined into one frame sequence and clustered into scene cuts. The algorithm was not designed to distinguish among types of cuts, but rather to indicate its position and duration. Experimental results show that the proposed algorithm is effective in detecting abrupt scene changes, as well as gradual transitions. For precision- demanding applications, the algorithm can be used with a low confidence factor, just to select the frames, which are worth being investigated by a more complex algorithm. The algorithm is not particularly tailored to MPEG and can be applied to most video compression techniques.