This book offers a systematic and in-depth investigation on Cross-Modal Communication Technology. In chapter 1, the authors present the background and motivation of this domain. In chapter 2, the authors outline multi-modal services and a general cross-modal communication architecture, including typical service categories, communication requirements, architectural details, challenges, and design principles. In chapters 3, 4, and 5, the authors respectively propose three key techniques within the cross-modal communication architecture: cross-modal coding, cross-modal transmission and cross-modal signal recovery. For each technique, the authors provide an in-depth presentation of the representative algorithms and the related experimental results. chapter 6 describes an established database and the three developed cross-modal communication prototype systems. Chapter 7 introduces extended techniques, such as cross-modal semantic communications, cross-modal communications in extremely low-resource scenarios. Finally, the authors provide a summary of the entire book and discuss future research directions in chapter 8. The readers will gain a deep understanding of the development and technical details of this new communication paradigm for the emerging multi-modal services.

Currently, haptic information is gradually being integrated into traditional audio-visual-dominant multimedia services, forming multi-modal services. Multi-modal services have been considered killer applications in the B5G and 6G eras, including remote industrial robotic grasping, tele-surgery and diagnosis, online immersive shopping and gaming, etc. To support the thriving multi-modal services, the cross-modal communication paradigm is proposed. Compared with traditional multimedia communications, the cross-modal communication paradigm is characterized by the collaborative transmission and processing of audio, visual, haptic signals and streams. By fully exploring the potential correlations among audio, visual, and haptic modalities, it enhances natural human-machine interactions and immersive experiences in multi-modal services, while meeting the requirements of low latency, high reliability, and high throughput.

This book targets graduate and undergraduate students majoring in the areas of wireless communications, computer science and engineering, and electrical engineering as well as researchers working in this field. Professionals seeking multimedia communications for the emerging multi-modal services will also want to purchase this book.

Autorentext

Xin Wei is a professor at Nanjing University of Posts and Telecommunications, China. He earned his Bachelor's degree from Nanjing University of Posts and Telecommunications in 2005, and obtained his Ph.D. from Southeast University, China, in 2009. From 2010 to 2011, he conducted postdoctoral research at Zhejiang University in China. His research interests include the areas of multimedia communications, educational technology, smart buildings. In recent years, he has published numerous academic papers in journals like IEEE JSAC, IEEE TCOM, IEEE TCSVT, IEEE TMM, IEEE WCM, Computers & Education, Interactive Learning Environments, Building and Environment, Energy & Buildings. He won the IEEE TMM Best Paper Award in 2024.

Dan Wu is professor in Army Engineering University of PLA, China. She earned her Bachelor's, Master's, and Ph. D. degrees from the Institute of Communications Engineering, PLA University of Science and Technology, China in 2006, 2009, and 2012, respectively. Her research interests are in the areas of multimedia communications and wireless communications.

Liang Zhou is a professor and vice president at Nanjing University of Posts and Telecommunications, China. He earned his Bachelor's and Master's degrees from Nanjing University of Posts and Telecommunications in 2003 and 2006, respectively, and obtained his Ph.D. from Paris-Saclay University, France, and Shanghai Jiao Tong University, China, in 2009. Following this, he conducted postdoctoral research at ParisTech in France and the MIT in the USA. From 2010 to 2011, as a Humboldt Scholar, he carried out research at the Technical University of Munich in Germany. He has been honored with titles such as Distinguished Professor by the Ministry of Education (2020), National Excellent Young Scientist Fund (2013); His research primarily focuses on multimedia communications and intelligent communications. In recent years, he has published numerous academic papers in journals like Nature Communications, IEEE JSAC. He has led key projects funded by the National Natural Science Foundation of China, as well as industry-academia collaboration projects with companies like Huawei and ZTE. Currently, he serves as the Chair of the Multimedia Communications Technical Committee of the IEEE Communications Society, Director of the Professional Committee of the China Institute of Communications, and is an editorial board member for academic journals such as IEEE Transactions on Communications, IEEE Transactions on Vehicular Technology, IEEE Internet of Things Journal, IEEE Wireless Communications, and IEEE Network. He is an IEEE Fellow/IET Fellow/BCS Fellow/AAIA Fellow.