The suppression of load swing angle of tower crane system is crucial for its safe operation. The existing anti-swing control methods have achieved good results in restraining the swing angle. However, these anti-swing control methods do not consider the buffeting of steel wire rope and load, which poses potential safety hazards. To solve this problem, a method to suppress the swing angle of tower crane system considering wire rope buffeting was proposed. Specifically, firstly, the nonlinear buffeting mechanism model was established for the tower crane in a horizontal motion, and then the nonlinear buffeting torque transmitted to the main shaft of the motor after the buffeting of the system was calculated, the equivalent reverse of the buffeting torque was used as the compensating torque. Secondly, the notch filter was used to filter out other disturbances apart from the compensating torque. After that, the compensating torque was superposed with the control torque calculated by a sliding mode anti-sway control law to control the output torque of the drive motor. Finally, the simulation and experimental results demonstrate that the proposed method achieves better control effectiveness in the pendulum angle suppression.