A large number of soft metamorphic rocks are developed in Hanzhong area of Shaanxi province, which are easy to soften under the condition of rainwater infiltration, causing slope instability and damage. Phyllite of the wedge-shaped block of Lueyang Cai Zi Ba is taken as the research object, and rock direct shear tests under different soaking conditions are designed to reveal the fracture form and deterioration effect of phyllite under the action of water rock. The results show that the rock mechanical parameters decrease with the increase of soaking time under the soaking condition, and change slightly at 14 days and gradually become stable. In the early stage of soaking, the particles rub against each other, resulting in more rock powder. In the middle and late stage of soaking, the particles produce rounding effect under the aqueous solution, and the sliding between the particles causes the rock powder to decrease obviously. The order of influence of water-rock action on the mechanical properties of phyllite is cohesion > peak stress > internal friction angle. According to SEM observation, it is concluded that rock softening in water immersion is a process from micro deformation to macro failure. Phyllite structural surface roughness is seriously affected by aqueous solution, which is the important factor inducing landslide and stabilizing failure of soft metamorphic rocks in southern Shaanxi.