Garin Smith
Sessions
The ‘Exploitation Platform’ concept derives from the need to access and process an ever-growing volume of data. Many web-based platforms have emerged - offering access to a wealth of satellite Earth Observation (EO) data. Increasingly, these are collocated with cloud computing resources and applications for exploiting the data. Rather than downloading the data, the exploitation platform offers a cloud environment with access to EO data and associated compute and tools that facilitate the analysis and processing of large data volumes. The Exploitation Platform benefits users, data providers and infrastructure providers. Users benefit from the scalability & performance of the cloud infrastructure, the added-value services offered by the platform – and avoid the need to maintain their own hardware. Data hosted in the cloud infrastructure reaches a wider audience and Infrastructure Providers gain an increased cloud user base.
Users are beginning to appreciate the advantages of exploitation platforms. However, the market now offers a plethora of platforms with various added value services and data access capabilities. This ever-increasing offer is rather intimidating and confusing for most users. In order to fully exploit the potential of these complementary platform resources we anticipate the need to encourage interoperation amongst the platforms, such that users of one platform may consume the services of another directly platform-to-platform.
EOEPCA (EO Exploitation Platform Common Architecture) is a European Space Agency (ESA) funded project with the goal to define and agree a re-usable exploitation platform architecture using standard interfaces to encourage interoperation and federation between operational exploitation platforms - facilitating easier access and more efficient exploitation of the rapidly growing body of EO and other data. Interoperability through open standards is a key guiding force for the Common Architecture: platform developers are more likely to invest their efforts in standard implementations that have wide usage; off-the-shelf clients and software are more likely to be found for standards-based solutions.
The EOEPCA system architecture is designed to meet a set of defined use cases for various levels of user, from expert application developers to consumers. The architecture is defined as a set of Building Blocks (BBs), exposing well-defined open-standard interfaces. These include Identity and Access Management, Resource Discovery, Data Access, Processing Workflows, Data Cube Access, Machine Learning Operations, and more. Each of these BBs are containerized for Kubernetes deployment, which provides an infrastructure-agnostic deployment target.
The exploitation platform is conceived as a ‘virtual work environment’ where users can access data, develop algorithms, conduct analysis and share their value-adding outcomes. The EOEPCA architecture facilitates this through a Workspace BB that provides a user-centric platform experience in which the standard discovery, visualisation and access interfaces are re-used for user-owned resources maintained within the platform - including data, applications, added-value products (from processing), etc. This is supported by an Application Hub building-block that provides interactive web-tooling for analysis, algorithm development, data exploitation and provides a web dashboard capability through which added-value outcomes can be showcased.
Our presentation will highlight the generalised architecture, standards, best practice and open source software components available.
The EarthCODE (Earth Science Collaborative Open Development Environment) vision provides an integrated, cloud-based, user-centric development environment which can be used to support the European Space Agency’s (ESA) science activities and projects. Building on activities that developed the European EO open-source ecosystem and the Open Earth System Science community (e.g. EOEPCA - Exploitation Platform Common Architecture, DeepESDL - Deep Earth System Data Lab, openEO Platform, ESA Euro Data Cube, etc.), ESA is implementing EarthCODE as a collaborative platform for conducting Earth System Science sustainably and adhering to FAIR and Open Science Principles. EarthCODE will enable the long-term persistence of research outputs from science activities.
EarthCODE looks to maximise reproducibility, reuse, and consumption of research outputs by the wider community, promoting a flexible and scalable architecture developed with interoperable open-source blocks, with a long-term vision evolving by incrementally integrating industrially provided services from a portfolio of the Network of Resources. EarthCODE platform collaborators will participate in creating integrated architecture, with interoperable solutions and federated capabilities.
EarthCODE will use EOEPCA Open Standards to help support Open Science, and help drive these standards. Open science principles are increasingly being embraced in the field of Earth Sciences, promoting transparency, collaboration, and accessibility of research. This is being done by promoting open access publications, preprints and open review processes, sharing data/methodologies for verification, reproducibility and reuse. In software development, these principles allow inspection, modification, and code contribution, encouraging collaboration among researchers through various platforms (i.e. GitHub, GitLab, etc.). Sharing of educational resources openly allow for global audience, and involvement of the public through citizen science for scientific research.
EarthCODE will provide an Integrated Development Platform, giving developers the tools needed to develop high quality workflows that allow experiments to be executed in the cloud and the reproduced by other scientists, following Open Science principles. Our solution is built around existing open-source solutions and building blocks, primarily the Open Science Catalogue, EOxHub and EOEPCA. With it’s adopted federated approach, EarthCODE will have the capability to facilitate processing on other platforms, i.e. DeepESDL, ESA EURO Data Cube, Open EO Cloud/Open EO Platform and AIOPEN/AI4DTE.
Collaboration and Federation are at the heart of EarthCODE. As EarthCODE evolves we expect providing solutions allowing allow federation of data and processing. EarthCODE has ambition to deliver a model for a Collaborative Open Development Environment for Earth system science, where researchers can leverage the power of the wide range of EO platform services available to conduct their science, while also making use of FAIR Open Science tools to manage data, code and documentation, create end-to-end reproducible workflows on platforms, and have the opportunity to discover, use, reuse, modify and build upon the research of others in a fair and safe way. EarthCODE thus aims to make possible the eight enabling elements of the EO Open Science and Innovation vision: open data, open-source code, linked data & code, open access documentation, end-to-end workflows reproducible on platforms, open science resources, open science tools, and a healthy community applying all the elements in their practice.