The Chishui Danxia landform in China is characterized by bright red-bed sandstone, yet the quantitative factors controlling its color diversity remain insufficiently understood. The study was aimed to investigate the mechanisms of sandstone color variation under environmental influences to support color matching in ecological restoration. A multidimensional sampling strategy, including planar–vertical isolation and single-factor control, was applied. Color parameters were measured using the CIE-Lab system. Mineral compositions were analyzed by XRD and SEM-EDS, and iron speciation was determined spectrophotometrically. Three main findings are obtained. (1) Hematite and clay minerals jointly control the dominant red tone through Fe3+-induced coloration and surface adsorption. (2) Microstructural analysis shows that sandstone in strongly weathered zones exhibits a looser fabric and more fractured grains; the b-value increases by 5.91 compared with weakly weathered zones, corresponding to a brighter surface appearance. (3) A temperature increase of 10 °C decreases the b-value by 0.38, while moisture affects brightness through variations in the L-value. Controlled experiments confirm these environmental effects. This study establishes a quantitative framework linking mineral composition, microstructural evolution, and environmental factors to color expression, providing a scientific basis for color-compatible restoration materials in Danxia landscapes.