This book introduces human reconstruction technologies in smart homes leveraging mmWave signals. It begins by presenting the overview of human reconstruction and the development of mmWave sensing technology. The authors introduce a mmWave sensing-based human posture reconstruction approach, which exploits mmWave signals to sense and track multiple users' postures as they move, walk or sit. It also presents a facial expression reconstruction system that reconstructs 3D human faces and continuously exhibits facial expressions using mmWave. The authors describe a hand gesture reconstruction system that utilizes mmWave signals to generate 3D hand skeletons and reconstruct hand meshes. A thorough investigation of state-of-art research work is further presented covering human reconstruction, mmWave sensing and mmWave-based human reconstruction. Finally, conclusions and the direction of future research are highlighted for this book.

Human and the cyberworld are moving towards high synchronization as more IoT devices are integrated in smart homes. By reconstructing human postures, facial expressions, and hand gestures in the cyberworld, a virtual human avatar can map the state of human into the physical world and deeply integrate reality and virtuality. The process strongly energizes emerging applications such as virtual reality, augmented reality, meta-universe, immersive games, etc.

This book targets graduate-level students majoring in the computer science and engineering, and electrical engineering. Professionals working in wireless signal-based human sensing will also find this book a valuable resource.

Discusses the newly emerging needs for human reconstruction using RF signals in AIoT environment Investigates three primary sensing problems in human reconstruction Discusses wireless sensing, ubiquitous computing and human computer

Autorentext

Dr. Hao Kong is currently an assistant professor in School of Computer Engineering and Science, Shanghai University. He received his Ph.D. degree in the Department of Computer Science and Engineering of Shanghai Jiao Tong University, in 2023. From February 2022 to February 2023, he visited the Broadband Communications Research (BBCR) Lab, Department of Electrical and Computer Engineering, University of Waterloo, Canada. His research focuses on Internet of Things, wireless sensing and mobile computing. He has published over 10 papers on high-reputation journals and conferences, including IEEE TMC, IEEE/ACM TON, IEEE COMST, ACM MobiSys, ACM MobiHoc, IEEE INFOCOM, etc., and one monograph.

Dr. Jiadi Yu received the PhD degree in computer science from Shanghai Jiao Tong University, China, in Jun. 2007. After that, he was at Shanghai Jiao Tong University, China, (from Jul. 2007 to Jan. 2009) and Stevens Institute of Technology, USA, (from Feb. 2009 to Dec. 2011), respectively, as Postdoctoral research associate in computer science. And then, he joined Dept. of Computer Science and Engineering, Shanghai Jiao Tong University as assistant professor. He is currently an Associate Professor in Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, China. Dr. Yu is an active and accomplished researcher in the field of Mobile Computing and Sensing, Cyber Security and Privacy, and Internet of Things. He has published more than 100 refereed paper including ACM MobiCom, ACM MobiSys, ACM MobiHoc, ACM UbiComp, IEEE INFOCOM, IEEE ICDCS, IEEE TMC, IEEE ToN, IEEE TPDS, etc. His works were reported in IEEE Xplore Innovation Spotlight, 2017, and IEEE Spectrum, 2019. He was Publicity Co-Chair of IEEE DySPAN 2019 and ACM WiSec 2019, and was TPC Member of IEEE INFOCOM(2016-2022), AAAI 2022, IEEE ICDCS 2022, IEEE ICPADS(2014, 20192020), etc.

Inhalt

Chapter 1 Introduction.- 1.1 An Overview of Human Reconstruction.- 1.2 Development of mmWave Sensing.- 1.3 Organization of Monograph.- Chapter 2 mmWave-based Human Posture Reconstruction.- 2.1 Background and Motivation.- 2.2 System Overview.- 2.3 Multi-User Detection and Separation.- 2.3.1 User Detection on mmWave Signals.- 2.3.2 User Separation on mmWave Signals.- 2.4 Single-User Posture Reconstruction.- 2.4.1 Posture Feature Representation.- 2.4.2 3D Posture Reconstruction.- 2.5 Multi-User 3D Posture Tracking.- 2.5.1 Posture Mapping with Point Cloud.- 2.5.2 3D Posture Tracking.- 2.6 Evaluation.- 2.6.1 Evaluation Setup.- 2.6.2 Overall Performance.- 2.6.3 Quantitative Result.- 2.6.4 Localization Performance.- 2.6.5 Performance in Different Scenarios.- 2.6.6 Comparison with SOTA Systems.- 2.6.7 Impact of Factors.- 2.7 Summary and Further Reading.- Reference.- Chapter 3 mmWave-based Facial Expression Reconstruction.- 3.1 Background and Motivation.- 3.2 System Overview.- 3.3 Facial Feature Extraction.- 3.3.1 Feature Representation Extraction.- 3.3.2 Geometric Feature Extraction.- 3.4 3D Facial Expression Reconstruction.- 3.4.1 Facial Shape Reconstruction.- 3.4.2 Facial Expression Reconstruction.- 3.4.3 3D Avatar Generation.- 3.5 Evaluation.- 3.5.1 Evaluation Setup.- 3.5.2 Overall Performance.- 3.5.3 Comparison with SOTA Systems.- 3.5.4 Facial Expression Recognition.- 3.5.5 Impact of Mask.- 3.5.6 Impact of Distance and Orientation.- 3.5.7 Impact of Background Environment.- 3.5.8 In-the-Wild Evaluation.- 3.6 Summary and Further Reading.- Reference.- Chapter 4 mmWave-based Hand Gesture Reconstruction.- 4.1 Background and Motivation.- 4.2 System Overview.- 4.3 Hand Joint Regression.- 4.3.1 Hand Detection on mmWave Signals.- 4.3.2 Hand Feature Extraction.- 4.3.3 3D Hand Joint Regression.- 4.4 Mesh Reconstruction.- 4.4.1 3D Hand Model.- 4.4.2 Hand Mesh Reconstruction.- 4.5 Evaluation.- 4.5.1 Evaluation Setup.- 4.5.2 Overall Performance.- 4.4.3 Comparison with SOTA Systems.- 4.4.4 Impact of Distance and Angle.- 4.4.5 Impact of Human Body.- 4.4.6 Impact of Gloves and Handheld Object.- 4.4.7 Impact of Background Environment.- 4.6 Summary and Further Reading.- Reference.- Chapter 5 State-of-Art Research.- 5.1 Human .-Pose Reconstruction.- 5.2 mmWave Sensing in Smart Homes.- 5.3 mmWave-based Human Sensing Applications.- 5.4 Summary of Existing Research.- Reference.- Chapter 6 Conclusion and Future Work.- 6.1 Conclusion of the Book.- 6.2 Future Work.- Reference.