Semantic segmentation of remote sensing images plays a crucial role in agriculture production, urban planning, and other fields. However, due to factors like imaging distance, lighting conditions, objects, and environment, there is a problem of semantic ambiguity in remote sensing images, which leads to uncertainty in segmentation. A multi-scale context attention (MSCA) method that combined pyramid pooling with attention mechanisms to better utilize contextual information was proposed for this problem. Additionally, this method significantly reduced the computational complexity and memory usage of attention methods. Experimental results on the ISPRS Potsdam dataset demonstrate that the MSCA method achieves superior segmentation performance for target classification with ambiguous semantic information in remote sensing images while almost not increasing memory consumption and maintaining consistent inference speed.