The filling quality of rock-fill dams is considered important for ensuring the safety and stable operation of dam projects. The traditional water-filling method was recognized as time-consuming, labor-intensive, inefficient, and slow in feedback. A new and efficient solution was provided by three-dimensional (3D) laser technology for the accurate detection of rock-fill compaction quality. An auxiliary device with adjustable scanning distance and angle was designed. This device was combined with a handheld 3D laser scanner to achieve 3D reconstruction and precise volume measurement of inspection pits with uneven walls and varying sizes. To reduce the influence of uneven pit walls on volume calculation, a method for identifying uneven wall areas based on point cloud distribution was proposed. This method was applied to achieve accurate volume calculation for inspection pits with irregular surfaces. In the application case of the 315 m high Shuangjiangkou rock-fill dam, this method was successfully used for compaction quality detection of different dam materials. Compared with the traditional water-filling method, the maximum relative error of 12 sets of compaction indicators is only 3.08%. The accuracy and efficiency of this method are thus demonstrated.