2024-10-03, 13:30–13:50, Theatre Hall (Conference Center Laxenburg)
xcube is a mature and capable Python software package and framework for EO data ingestion, processing, analysis, visualization, and dissemination. In the scope of the Open Earth Monitor project, xcube is being updated and expanded to support new data sources, improve on-demand cluster processing capabilities, and run seamlessly on the new Copernicus Data Space Ecosystem. Recent work also focuses on providing a maximally preconfigured turnkey distribution of xcube, increasing its suitability as a drop-in compute engine for cloud infrastructures such as CDSE. xcube’s features are complemented by the new zappend tool, which provides robust creation and updating of large, slice-structured Zarr datasets.
This talk will describe and demonstrate a typical large-scale processing workflow using the xcube framework in the CDSE ecosystem – running the gamut through data ingestion from multiple sources through the xcube data store subsystem, data cube construction and normalization, data synthesis and processing to export, dissemination, and seamless visualization via the server and viewer components. Scalability and big data capability is accounted for throughout through approaches such as object storage, parallelization, on-demand cluster processing, dataset pyramidization, and lazy computation. The newly implemented components and improved integration make xcube an ideal tool for the realization of typical Open Earth Monitor workflows.
Open-Earth-Monitor Cyberinfrastructure (Grant agreement ID: 101059548)
Pontus Lurcock is a software engineer at Brockmann Consult GmbH, with a strong focus on geodatacubes and analysis-ready earth observation data. He has extensive experience of working at the interface between informatics and geosciences, and holds an MSc in Computer Science and a PhD in Geology.