The quantification of permeability under particle accumulation is of great significance for evaluating the water resistance performance of goaf collapse zones, fault fracture zones, and reservoir stress sensitivity. A particle packing permeability evolution model was established, stress sensitivity and main controlling factors under different particle arrangement structures were analyzed and mechanism of stress sensitivity between body deformation and structural deformation is explored based on Hertz deformation. Results show that if only considering the deformation of the body, the stress sensitivity of permeability is inversely proportional to the number of combined particles, and the elastic modulus of particles is the dominant factor in stress sensitivity in loading and unloading tests. The stress sensitivity of particles undergoing structural deformation is much higher than that of the body deformation, and is directly proportional to the degree of structural deformation (particle combination structure adjustment). The stress sensitivity of granular structures is the result of the combined effects of structural deformation, equivalent elastic modulus, and structural adjustment. The rationality of stress sensitivity of different particle structures has been numerically verified, and the adjustment of permeability at different positions is the main factor leading to non-uniform distribution of pore pressure and changes in permeability. It is believed that low-permeability reservoirs (coal seams) have strong stress sensitivity and are closely related to rock and fluid properties.