Speakers
Dr
Arpad Szomoru
(JIVE)Dr
Rob van der Meer
(ASTRON)
Description
ASTRON has developed and is operating the LOFAR telescope facility both in the Netherlands and as part of the ILT. The facility collects a large amount of data of which, after correlation and first reduction, about 7 PB per year user data products are stored in the Long Term Archive (LTA).
ASTRON has focused on providing software programmes and pipelines to create user ready data products in the archive. The archive is distributed over three sites. Users have access to the data through one portal, but user site data reduction still involves local clusters and therefore transporting high volumes of data. The focus for optimisation has mostly been on the facility side. With the EOSC Pilot Science Demonstrator for LOFAR we aim to improve the user experience both for power users and non-power users. We will build on existing knowledge and combine existing tools to show the complete path from facility to user, to demonstrate it can be done and to demonstrate the capacity and shortcomings, or challenges for the future.
I wil also present a pilot from the JIVE institute, to make the European VLBI Network (EVN) data accessible through the cloud. The EVN archive has everything that a large archive has, except the size. Providing a cloud version would be a good test case for storage and data reduction in the cloud.
Third point on my list will be the AENEAS goal to design a European Science Data Center for the SKA. Building on the existing infrastructure and using knowledge and requirements of current large archives and compute facilities, and mapping a scale increase of one to two orders of magnitude, we will stretch the capacity of any cloud or existing infrastructure to the limit. It is therefore very important that the design of the ESDC runs parallel to the emergence of the EOSC and on the way learning from each other.
Primary authors
Dr
Arpad Szomoru
(JIVE)
Dr
Rob van der Meer
(ASTRON)
