The filling quality of rock-fill dams is considered important for ensuring the safety and stable operation of dam projects. The traditional water-filling method is recognized as time-consuming, labor-intensive, inefficient, and slow in feedback. A new and efficient solution is provided by three-dimensional (3D) laser technology for the accurate detection of rock-fill compaction quality. An auxiliary device with adjustable scanning distance and angle is designed. This device is combined with a handheld 3D laser scanner to achieve 3D reconstruction and precise volume measurement of inspection pits with uneven walls and varying sizes. To reduce the influence of uneven pit walls on volume calculation, a method for identifying uneven wall areas based on point cloud distribution is proposed. This method is applied to achieve accurate volume calculation for inspection pits with irregular surfaces. In the application case of the 315 m high Shuangjiangkou rock-fill dam, this method is successfully used for compaction quality detection of different dam materials. Compared with the traditional water-filling method, the maximum relative error of 12 sets of compaction indicators is only 3.08%. The accuracy and efficiency of this method are thus demonstrated.