Radio astronomy is irreversibly moving towards the exabyte era. In the advent of all-sky radio observations, efficient tools and methods to manage the large data volume generated have become imperative. This book brings together the knowledge of several different research fields to present an overview of current state-of-the-art methods in data-intensive radio astronomy. Its approach is comprehensive and data-centric, offering a coherent look at the four distinct parts of the data lifecycle:

• Data creation, storage and archives
• Data processing
• Post-processing and data analysis

• Data access and reuse Large data management has been the topic of discussion within the astronomical community for decades. Some relevant areas explored in this volume are: ongoing technological innovations in interferometers and computing facilities; difficulties and possible solutions for the huge processing demands of radio telescope projects such as LOFAR, MeerKat, ASKAP; concepts for reliable and fast storage for archiving; and more. Written by experts across astrophysics, high-energy particle physics, data science, and computer science, this volume will help researchers and advanced students better understand the current state of data-intensive radio astronomy and tackle the major problems that may arise from future instruments.

  A comprehensive overview of data-intensive radio astronomy Written by experts across astrophysics, high-energy particle physics, data science, and computer science Follows the data lifecycle to present a coherent flow of content

  Autorentext

Eleni Vardoulaki is an astrophysicist working as coordinator for data-intensive radio astronomy at the Thüringer Landessternwarte Tautenburg (TLS). She obtained her Bachelors in Science in Physics from the University of Ioannina, Greece, in 2002, specializing in astrophysics. She obtained her Doctor of Philosophy in Astrophysics from the University of Oxford in 2010. She has worked as postdoctoral researcher at Centro de Astrofisica da Universidade do Porto (CAUP) in Porto, Portugal, as a research fellow at the University of Crete (Greece), as a visitor researcher at the California Institute of Technology (Caltech) in Pasadena (USA), as a postdoctoral researcher at the Argelander-Institut für Astronomie (AIfA) in Bonn (Germany) and at the Max-Planck-Institut für Radioastronomie (MPIfR) in Bonn (Germany). Her expertise is on extragalactic radio sources associated with active galactic nuclei (AGN). She is interested in the physical properties of AGN, their host galaxies and relation to environment, disentangling AGN and star formation, as well as machine learning and artificial intelligence. She is a science communicator (Rogue Astrophysics, Astronomy on Tap) and TEDx speaker.

Marta Dembska adopts an interdisciplinary approach, engaging in fundamental research that encompasses radio astronomy, experimental science, and laboratory experiments, complemented by a proficiency in data science. Currently, she focuses her research on topics particularly relevant for advancing scientific workflows. This includes Research Data Management, data provenance and FAIR principles for scientific data management. She graduated with honours, attaining a BSc in technical physics specialized in computers in physical measurement in 2007, followed by an MSc in computer physics in 2010, both from the University of Zielona Góra. She subsequently earned her PhD in radio astronomy in 2013 at the same university, with her dissertation receiving honours. She's been a research fellow at the German Aerospace Center (DLR) since 2014. Her scholarly interests encompass (meta)data management and data analysis, where she utilises her extensive expertise to bridge diverse interdisciplinary domains.

Alexander Drabent, born in 1986 in Leinefelde (Germany), obtained his doctorate at the Friedrich Schiller University in Jena (Germany), in physics on diffuse radio emission in merging galaxy clusters. Since 2012 he has been a research fellow at the Thüringer Landessterwarte in Tautenburg in Germany and has specialized in the field of radio astronomy. He works on software development and data processing for the Low Frequency Array (LOFAR), in particular the regime of calibration and data reduction pipelines.

Matthias Hoeft leads the radio astronomy group at the Thüringer Landessternwarte (TLS). He is in particular interested in radio galaxies, clusters of galaxies and the large-scale structure in the universe. The TLS operates one station of Low Frequency Array (LOFAR), which allows researchers to study the radio sky at very low frequencies with unprecedented resolution and sensitivity. Hoeft leads the processing of LOFAR data on supercomputers of the Forschungszentrum Jülich. Moreover, Hoeft participates in the PUNCH4NFDI consortium.

Inhalt

Preface.- Introduction - Concepts and challenges of data-intensive radio astronomy.- Part 1. DATA CREATION, STORAGE AND ARCHIVES.- CHAPTER 1: From the data generation to the archive.- CHAPTER 2: Storage and archives.- CHAPTER 3: Computing infrastructure.- Part 2. DATA PROCESSING.- CHAPTER 4: Challenges of radio data processing (big radio data processing).- CHAPTER 5: Implementations for specific radio observatories.- CHAPTER 6: Co-design and software architecture.- CHAPTER 7: Lesson learned from SKA pathfinders regarding processing.- PART 3. POST-PROCESSING AND DATA ANALYSIS.- CHAPTER 8: Continuum Source extraction and identification.- CHAPTER 9: Other types of source extraction and identification.- CHAPTER 10: Using AI for radio (big) data.- CHAPTER 11: Visualisation for analysis.- PART 4. DATA ACCESS AND REUSE (accessibility, VR/VO FAIR).- CHAPTER 12: Exploitation platforms & Virtual Observatory.- CHAPTER 14: Data documentation beyond provenance: metadata, Research Data Management (RDM), FAIR.- Epilogue and future outlook.- Glossary.- Index.