To address the problems of high complexity and low efficiency in the cooperative pursuit of maneuvering targets by multiple unmanned vehicles in continuous and dynamic environments, a counterfactual multi-agent policy gradient algorithm based on curriculum experiential learning was proposed. The cooperative pursuit efficiency of unmanned vehicles under non-stationary and partially observable conditions was thereby improved. First, a pursuit environment with dynamically moving targets and a dynamic model of unmanned vehicles was constructed, along with the criteria for successful target capture. Then, the cooperative pursuit problem of unmanned vehicle formations was modeled as a decentralized partially observable Markov decision process (Dec-POMDP). A counterfactual policy gradient algorithm within a curriculum learning framework was proposed to guide unmanned vehicles to complete the pursuit task step by step. The complex cooperative pursuit task was decomposed into three progressively difficult stages: target approaching, target capturing, and target encircling. For each stage, corresponding step-wise and terminal rewards were designed to progressively guide the agents’ learning process. Simulation results demonstrate that the proposed method, based on curriculum learning and counterfactual policy gradients, can adapt to complex environmental changes and autonomously adjust formations and path planning. In scenarios involving dynamic targets with varying maneuverability, the multi-agent cooperative pursuit strategy exhibits strong robustness and generalization ability.