The hole segmentation of core images is important for oil exploration. In order to solve the problems of discontinuous hole edge segmentation, low segmentation accuracy and large number of parameters, an improved model based on UNet network MSHAM-UNet was proposed. Firstly, aiming at the problem of semantic information loss caused by the hopping connection of UNet models to feature maps of different scales, a multi-scale hybrid fusion attention module (MSHAM) combined with a hybrid attention mechanism (HAM) was designed, which performed attention feature fusion on shallow feature maps with spatial information and deep feature maps with semantic information, so as to enhanced the ability of the network to aggregate information at different scales. Secondly, the GP-bneck module was used to replace part of the ordinary convolution, which can reduce the number of model parameters and deepen the network, and enhance the network feature extraction ability. The experimental results show that the F1-score, intersection over union (IoU) and mean intersection over union (MIoU) of the MSHAM-UNet network on the core hole dataset reach 87.35%, 77.27% and 90.21%, respectively, which are 5.29%, 4.02% and 4.84% higher than the original model, and are also improved compared with the mainstream semantic segmentation model, which provides a new idea for core pore research.