To alleviate wind power fluctuations and improve the energy allocation performance of hybrid energy storage systems (HESS), an energy management method integrating an improved triangulation topology aggregation optimizer-variational mode decomposition (ITTAO-VMD) with an adaptive mode control strategy was proposed. First, opposition-based learning initialization and adaptive Gaussian mutation were introduced to improve the TTAO, thereby enhancing its optimization performance for VMD parameters. Second, in the power decomposition stage, a composite fitness function was constructed and a modal energy hierarchical method was proposed, which decomposed the HESS power into a primary energy layer and a high-frequency fluctuation layer. These components were allocated to the hydrogen-battery subsystem and the supercapacitor, respectively. Finally, an adaptive cooperative control strategy for hydrogen and battery storage was designed to absorb the primary energy while maintaining a reasonable state of charge. The simulation results indicate that the energy-type advantages of hydrogen and battery storage are fully exploited, the capacity burden of the supercapacitor is reduced, and complementarity among different storage types is achieved.