In order to solve the problems of failing to accurately distinguish polyps from the surrounding mucosa and the limited generalization ability in clinical polyp segmentation algorithms, a new polyp segmentation network that combines cross-layer feature fusion and boundary guidance was proposed. It primarily consists of three modules: the multiscale feature extraction module was used to extract multiscale information and polyp details from the encoder’s shallow output. The cross-layer feature fusion module was designed to integrate the cross-layer features through collaborative attention and adaptive channel calibration strategies, achieving dynamic calibration and fusion of these features. The boundary extraction guidance module involves the use of morphological operators for the extraction of boundaries and polyp regions from the initial segmentation map. Features within the internal polyp region were leveraged to enhance those near the boundary, resulting in accurate segmentation outcomes. The experimental results show that the model’s Dice coefficients on the CVC-ClinicDB and Kvasir datasets are 94.24% and 92.41%, respectively. Compared with the existing mainstream methods, the proposed method not only segments multiple types of polyps but also improves the segmentation accuracy and generalization ability.