Compressed air energy storage (CAES) caverns undergo frequent loading and unloading cycles during operation, leading to deterioration of the surrounding rock properties and, consequently, affecting the cavern’s seal integrity. This paper focuses on the granite strata at the Yue yang County Compressed Air Energy Storage Demonstration Plant. As Hunan Province’s first underground energy storage project utilizing granite host rock, this project is characterized by complex geological conditions, high host rock hardness, and low initial permeability. Based on cyclic loading and unloading tests, an in-depth analysis of the evolution patterns and mechanisms of permeability was conducted. First, granite loading and unloading tests were conducted with varying numbers of cycles (0, 20, 40, 60, and 80). Using ultrasonic testing technology, a nonlinear relationship was established between damage variables and the number of cycles. Second, permeability tests were conducted on granite specimens after cyclic loading and unloading under various pore pressures and stress conditions, providing a quantitative description of the influence of the number of cycles on permeability. Finally, CT scanning was employed to investigate the internal structural damage characteristics induced by cyclic loading and unloading, elucidating the mechanism of permeability evolution in granite. These research findings provide a crucial basis for evaluating the evolution of surrounding rock damage and the stability of pressure storage caverns under cyclic injection and extraction conditions, and offer valuable reference for underground energy storage projects in the southern granite geological context.