The urban underground pipeline system is a critical infrastructure of cities, for which regular inspections for pipeline defects are required to ensure operational safety. However, defects in urban underground pipelines are often complicated by factors such as complex backgrounds, large-scale changes, diverse morphologies, and noise. To achieve efficient semantic segmentation of underground pipeline defects, an improved model, AME-DeepLabV3+, was proposed based on the DeepLabV3+ framework. The main improvements include the adaptive graph channel attention (AGCA) mechanism was introduced to optimize the feature representation in the channel dimension and enhance the model’s adaptability to complex scenarios. A multi-branch parallel convolution (MPConv) was designed to process low-level features, enhance edge details, and reduce interference from backgrounds and noise. An efficient selective feature fusion (ESFF) module was designed to achieve better feature fusion while reducing the number of parameters. Experimental results show that the improved DeepLabV3+ model increases mIoU by 4.76% and mPA by 1.69% compared to the baseline model, while maintaining real-time performance. Compared with other high-performance segmentation models such as SegFormer, certain advantages are also demonstrated by the proposed model.