Survey on Multimodal Data Visualization Analysis Methods

Wei HE , Yu-fei WEI

Science Technology and Engineering ›› 2026, Vol. 26 ›› Issue (2) : 433 -449.

PDF
Science Technology and Engineering ›› 2026, Vol. 26 ›› Issue (2) : 433-449 DOI: 10.12404/j.issn.1671-1815.2501030
review-article
Survey on Multimodal Data Visualization Analysis Methods
Author information +
History +
PDF

Abstract

With the advancement of information acquisition technology, multimodal data such as text, images, video, and audio, has become a significant area of research. A more comprehensive perspective for understanding complex data is provided by the complementary nature of this information, overcoming the limitations of single-modality studies. However, a key challenge is the lack of interpretability in current analysis methods based on machine learning and pre-trained models. Consequently, the underlying data context and model principles are often difficult for users to comprehend. To address this, visualization techniques were employed for the interactive analysis of complex multimodal data, enabling the in-depth exploration of hidden information and patterns. In this review, relevant concepts were first introduced, and commonly used public multimodal datasets were summarized. The process of multimodal data analysis was then described and analyzed in detail. Furthermore, a systematic review of visualization analysis methods and their characteristics was also conducted, covering stages from data exploration and modality representation to alignment and application. Finally, existing challenges and future research directions in multimodal data visualization analysis were addressed.

Graphical abstract

Keywords

multimodal / visual analytics / modal representation / modal alignment

Cite this article

Download citation ▾
Wei HE, Yu-fei WEI. Survey on Multimodal Data Visualization Analysis Methods. Science Technology and Engineering, 2026, 26(2): 433-449 DOI:10.12404/j.issn.1671-1815.2501030

登录浏览全文

4963

注册一个新账户 忘记密码

References

Funding

PDF (32120KB)

23

Accesses

0

Citation

Detail

Sections
Recommended

/

〈 〉