Since the non-contact full-field vibration measurement technique is characterized by low cost and high accuracy, exploring its application for offshore structural damage identification is of great significance for establishing a health monitoring system. Existing offshore structure pile damage identification is usually based on vibration sensor signals, structural modal and derived features for damage identification, and there is a lack of research on overall feature extraction and damage identification for continuous full-field displacement response of pile foundations for offshore structures. Based on the deep learning unsupervised autoencoder method, the continuous 3D-DIC(three-dimensional digital image correlation) displacement field under environmental excitation was used as the input, the spatial dimension features of the displacement field were adaptively extracted, the continuity of the temporal dimension was maintained, and the displacement anomaly index and damage detection index were constructed, so as to realize the identification of local damages to the offshore structure. The applicability and effectiveness of the proposed method are also verified by an experimental model of a high pile wharf. The analytical and experimental results show that the deep learning unsupervised autoencoder method proposed can effectively realize the damage identification of pile foundation, which provides a new strategy for the adaptive damage identification of continuous full-field displacement response.