
科学技术与工程

中国技术经济学会
Survey on Multimodal Data Visualization Analysis Methods
Wei HE , Yu-fei WEI
Science Technology and Engineering ›› 2026, Vol. 26 ›› Issue (2) : 433 -449.
With the advancement of information acquisition technology, multimodal data such as text, images, video, and audio, has become a significant area of research. A more comprehensive perspective for understanding complex data is provided by the complementary nature of this information, overcoming the limitations of single-modality studies. However, a key challenge is the lack of interpretability in current analysis methods based on machine learning and pre-trained models. Consequently, the underlying data context and model principles are often difficult for users to comprehend. To address this, visualization techniques were employed for the interactive analysis of complex multimodal data, enabling the in-depth exploration of hidden information and patterns. In this review, relevant concepts were first introduced, and commonly used public multimodal datasets were summarized. The process of multimodal data analysis was then described and analyzed in detail. Furthermore, a systematic review of visualization analysis methods and their characteristics was also conducted, covering stages from data exploration and modality representation to alignment and application. Finally, existing challenges and future research directions in multimodal data visualization analysis were addressed.
multimodal / visual analytics / modal representation / modal alignment
/
| 〈 |
|
〉 |