Speaker
Description
The EUCLID project, which is part of the M-class ESA Cosmic Vision program, aims to map the geometry of the dark universe through two independent probes: weak gravitational lensing and galaxy clustering. It will conduct a survey of the extragalactic sky in the visible and near-infrared bands, resulting in the generation of up to 30 petabytes of data. The architecture of the Euclid Science Ground Segment (SGS) has been designed to distribute computation and data among 9 Science Data Centres (SDCs) and minimize data transfers between them. Key architectural components of the SGS include the Euclid Archive System for metadata inventory, a Distributed Storage System providing a unified view of the data stored in the SDCs, an Infrastructure Abstraction Layer for workflow management, and a Common Orchestration System for a balanced distribution of data and processing. Containerization, in conjunction with the CernVM-FS software distribution service, facilitates software deployment across all SDCs. The processing pipelines within the SGS primarily follow the HTC paradigm, with GPU acceleration being essential for a critical analysis step in production. Resource estimation is conducted based on simulations and periodic end-to-end tests, guiding the procurement strategies of institutes and agencies.
This presentation will provide an overview of the Euclid SGS computing and software infrastructure, highlighting the rationale behind certain design choices and the lessons learned during the software development and integration phases.
