The displacement response of a bridge is identified as a fundamental condition for the structural health monitoring and safety assessment of bridge structures. In order to fully leverage the advantages of machine vision for measuring structural displacements and to enhance its applicability, a structural displacement monitoring method based on the pinhole camera model was proposed. Sparse optical flow was employed to track structural feature points, achieving sub-pixel level image displacements. By employing the optical geometric relationships of camera imaging, an analytical solution was established between real-world displacements and image displacements. The image displacements were then substituted into algebraic relationships to obtain the true structural displacements. In the indoor experiments on dynamic load displacement identification of a simply supported bridge model, compared to the measurements obtained using a linear variable differential transformer displacement meter, the proposed method achieves a maximum error within 6% for pitch angles up to 30° and yaw angles up to 35°, thereby meeting the application requirements for monitoring the displacement deformations of bridge structures.