As a typical heterogeneous rock, the macroscopic fracture behavior of conglomerate is significantly influenced by its mesoscopic structure. Accurate prediction of crack propagation paths is of vital importance for evaluating the stability of deep underground engineering. In this study, semi-circular bend(SCB) test and phase-field simulations were combined to systematically investigate the mechanical properties of conglomerate and the influence of its mesoscopic structure on macroscopic mechanical behavior and crack propagation paths. Based on digital image processing and random generation algorithms, a multiphase mesoscopic model was established, which realistically reflects the spatial distribution, particle size, and volume fraction of gravels, enabling a cross-scale characterization from mesoscopic structures to macroscopic mechanical responses. The results indicate that the load–displacement curve of conglomerate exhibits four characteristic stages: compaction, linear elasticity, plastic deformation, and brittle failure. The fracture toughness increases with increasing gravel content and particle size, but decreases with increasing prefabricated notch length. The crack propagation path is significantly affected by the spatial distribution of gravels. Hard gravels hinder crack growth and induce deflection, branching, or arrest of crack paths. When the gravel volume fraction exceeds 40%, crack propagation is dominated by bypassing around gravels. Compared with circular particles, polygonal particles are more likely to induce stress concentration, resulting in a higher peak load of the model. The reliability of the phase-field method in simulating the failure behavior of heterogeneous conglomerates is validated through comparison with experimental results, providing a theoretical basis for the optimization of hydraulic fracturing in conglomerate reservoirs.