Anomaly detection in traffic video is a basic computer vision task, which has been paid more and more attention due to its importance in the intelligent transportation systems. Faced with such problems as the complexity of traffic scene, lack of data, inaccurate definition of abnormal behavior, dense and chaotic traffic flow affecting the video quality of real-time traffic feeding, it is still a challenging problem. The vehicle abnormal behavior detection algorithms based on surveillance video proposed in recent years are summarized in this paper. Firstly, the vehicle detection framework and tracking framework used in current algorithms are introduced. Then the vehicle abnormal behavior detection methods are introduced and analyzed from two aspects of abnormal feature extraction and behavior learning modeling methods. Finally, commonly used datasets are presented, and future directions are discussed.