Melissa Latella
Sessions
Earth System Models and Earth Observations are crucial for studying the Earth, providing scientific insight into fundamental dynamics and valuable predictions about Earth’s future. However, they generate huge amounts of data, at different temporal and spatial scales, so it becomes of paramount importance to access them in a seamless and efficient way for scientific analysis. Usually Earth Science datasets are represented with hundreds or thousands of files that can introduce a lot of burden to the user in terms of management, since it requires the user to set up computational and storage resources for accessing and retrieving data and writing code to load and prepare data into in-memory data structures for analysis.
In this talk, we describe in detail the architectural design, implementation and deployment of a data management and analytics system in order to facilitate cataloguing, accessing and processing Earth Science data. The system has been designed using a cloud-native architecture, based on containerized microservices, that facilitates the development, deployment and maintenance of the system itself. It has been implemented by integrating different open source frameworks, tools and libraries and has been deployed using the Kubernetes platform and related tools such as kubectl and kustomize.
The Data Platform consists of different components that will be introduced and described together with the related technologies adopted: (a) the Catalog, based on Intake and MongoDB for cataloguing and indexing the datasets published and managed in the system, (b) the Analytics Engine, based on the geokube and dask Python libraries: geokube is used for specialised geospatial operations (such as extracting a bounding box or a multipolygon) according to different types of geoscientific datasets and dask for parallel and distributed processing; (c) the Broker implemented using RabbitMQ framework for managing the user workload requests; finally, (d) the Rest APIs and the OGC standard interfaces (i.e., WPS) to access data and to submit analytics workflows.
An instance of the Data Platform has been deployed in production at Euro-Mediterranean Centre on Climate Change (CMCC) for the delivery and analysis of data produced by the CMCC Research Divisions. In this talk, we will showcase different Use Cases, related to sectors such as climate change and wildfire management, that demonstrate how the system has been used at CMCC, within different projects and initiatives, for building downstream products and services that need to access, analyse and process Earth Science data.
The United Nations (UN) 2030 Agenda aims at promoting sustainable development at environmental, social and economic level. The definition of the Sustainable Development Goals (SDGs) and of the associated Global Indicator Framework represent a data-driven effort, helping countries in evidence-based decision-making and policies. SDG indicators’ monitoring and reporting across countries can benefit from substantial use of Earth Observation (EO), including satellite and in-situ networks, and of their processing through data analytics and numerical modeling approaches, making the 2030 Agenda implementation robust, viable and faster, both technically and financially.
This talk introduces SDGs-EYES, a major new European initiative aiming at boosting the European capacity for monitoring the UN SDGs. SDGs-EYES addresses current gaps in the UN SDGs monitoring by exploiting data and information coming from the European Copernicus Programme and by providing a scientific and technological platform for building indicators through the integration of EO data, advanced numerical modeling, data analytics and Machine Learning approaches. Furthemore, SDGs-EYES aims to build a portfolio of decision-making products and services for the assessment and monitoring of SDG indicators whose trends could impact the environment and the society from an inter-sectoral perspective, aligning with the EU Green Deal priorities and challenges.
The SDGs-EYES scientific approach and framework are introduced and described with particular reference to three interconnected SDGs, specifically on climate (SDG13), ocean (SDG14) and land (SDG15). These SDGs are mostly focused on the biosphere as foundation of prosperity, development and co-benefits in the society and economy, but also relevant due to their nexus with additional SDGs, targets and indicators related to socio-economic and (geo)political factors (e.g., human health, environmental crimes, water and food insecurity, poverty, conflicts, displacements, migrations).
Five Pilots (encompassing EU and extra-EU regions) will be used to demonstrate and validate the SDGs-EYES approach and results, that is application-oriented scientific products, technological solutions and user-tailored services.