As data becomes the centre of innovation in modern economy and society, we start to face new challenges and limitations. Although tremendous progress has happened over the past several years on increasing productivity for data processing over commodity systems and providing new services with Big Data and Cloud technologies, the projected data deluge brings business, consumers, and the society in general to a new frontier.
EVOLVE is a European Innovation Action funded by the European Union's Horizon 2020 Research and Innovation programme. It aims to build a large-scale testbed by integrating technology from High Performance Computing (HPC), Big Data and the Cloud. The project is composed by 19 specialised partners from 11 European countries, including Sunlight, DDN Storage, Bull, IBM, Forth Foundation for Research and Technology, Institute of Communication and Computer Systems (ICCS), memoscale, Web Lyzard Technology, Loba, Thales, Space, CybeleTech, Neurocom, MemEx, Tiemme, Virtual Vehicle, AVL, BMW and Koola.
Find out more: https://www.evolve-h2020.eu/
Creating new data-intensive services in terms of dataset size and data processing is an onerous and costly process that requires deep expertise. It requires:
However, most organisations today lack these resources and the associated expertise.
Sunlight brings management capabilities for processing massive and demanding datasets without requiring extensive IT expertise. The EVOLVE solution focuses on:
partners solving the big data challenges of the future
EVOLVE’s testbed is based on:
Advanced Computing Platform - The main aspects of EVOLVES's hardware platform are its large scale, fast interconnect and memory. EVOLVE core architectural contribution is harnessing accelerators. The testbed will support accelerated nodes by GPU, FGPA and specialised processors.
Storage Subsystem Architecture - In EVOLVE the storage is envisioned as a tiered architecture. The storage subsystem
uses a shared Infinite Memory Engine (IME) and fast local “non-volatile memory express” storage devices. Storage will be extended with advanced data protection, compression an encryption features.
Safety & Ease of Deployment, Access & Use: EVOLVE will provide shared access to the testbed for improving productivity and Total Cost of Ownership (TCO). Cloud native technology will be used for the deployment of containerised high performance applications. End-to-end encryption will ensure safety and privacy.
End-to-end Workflows - EVOLVE is using end-to-end workflows that express full data-processing pipelines, including data ingest from external sources with time constraints. The Extract-Transfom-Load process will be fully supported for all pilot applications.
Versatile Software Stack - To realize workflows, EVOLVE is providing a versatile software stack that employs existing data processing engines that have proven flexibility and breadth of applicability
At the centre of EVOLVE lies an advanced HPC-enabled testbed that is able to process unprecedented dataset sizes and to deal with heavy computation, while allowing shared, secure, and easy deployment, access, and use.
Agri Production - to optimize yield using numerical models and massive historic data, improve productivity by allowing easy deployment, and use of the testbed by domain experts in agri analytics.
Maritime Surveillance - to use more sophisticated (and resource-demanding) algorithms for more accurate detection and identification, reduce response-time, better quality of service and larger area coverage.
Sentinel-2 (S2) satellite images - The repetitiveness (every few days) and the free access of Sentinel-2 (S2) satellite images result in massive datasets. The EVOLVE testbed will allow better accuracy and improved response times.
Public Transport (PT) services improve the competitiveness and social cohesion of cities. Recent developments in ICT/ITS have had a strong impact on bus services management and improve overall city mobility planning.
Mobility services - to develop and validate complex vehicle routing, car passenger matching, and fleet management algorithms based on simulated car sharing or, in general, on-demand mobility data.
Predictive Vehicle Maintenance - To allow developments in predictive maintenance both at the levels of model training and diverse dataset analysis. This will lead to the creation of innovative services.
Automotive Services - to appropriately manage and use all the local and global information from vehicle fleets to evaluate and validate correct operation in all possible situations.
In all cases, experts are working on models that provide accurate predictions, data processing and validation techniques over massive datasets. These have the potential to improve the efficiency of existing services or introduce new services in the respective domains.