Notes and Actions from the WP4 Kick-off meeting
1 March 2017
We had 2 WP4 meetings and a joint meeting with WP3 after lunch. The main points and actions were:
Network level Actions
-
Make a list of network level tests – which protocol stack and tool (we already have basic agreement)
-
Agree a set of data moving tools gridftp, bbcp, jive5ab (uses udt), aspera access from AARNet, S3-like REST, BitTorrent (for ceph storage)
-
Start defining European sites for tests Onsala, Jodrell, ASTRON/SARA, Julich, Bologna
People noted the AARNet-GEANT 10Gigs
Storage level
Input from RAL (STFC) the LHC Teir1 in UK on what is now possible using ceph and I swift.
They will use gridftp to move data but aiming for S,3 like AARNet 3 site replication storage and data moving are not separated.
Julich and RAL (new partner) will work together.
Replica manager level
The who, what, why, level of detail for the SKA “data replica manager” most unclear.
Agreement it needs to be SKA wide and users should not need to know where the data files are.
Data placement:
Again details of SKA requirements are quite uncertain
-
Agree the SDP will be in charge of scheduling and sending the data out of the telescopes
-
Agree Telescopes will only send the data out once – cost of submarine cable
-
Agree that for the European ESDC the data products will be distributed across the sites that form the ESDC
-
NOT clear that all data products will go to one regional site and then be re-transmitted
-
NOT clear that some products will go to individual regional centres and all users will be able to send compute jobs to all sites to see all the data
-
Suggestion that ESDC will have all data from both Telescopes
Software and Hardware Platforms
Some think it will all be based on OpenStack – and OpenStack will rush to help SKA
Others think SKA will have to invest heavy manpower in o OpenStack
Agreement that OpenStack will have its own agenda – need not be in phase with what SKA needs
Some sites use bare linux for data transfer nodes
Others use Containers
WP3-WP4 joint milestones etc
General agreement that the timing is about right
Willingness to work together – that’s good
WP4 internal communications Conclusions
AARNet will give us access to zoom – excellent
Slides to go on the meeting page indico
Agreement to have VC/telcon 1 per month – day & time to be fixed
Agreement to have extended VC to discuss bit & bytes of results every 3 months
Agree to have ½ day workshop at each AENEAS F2F which will be every 6 months.
Manpower
We would like to add Tim Rayner to the AARNet team (have a look at the slides for the others)
All to confirm their participation and FTE in the WP4 Tasks
Tasks
-
T4.1 and T4.4 PoC continue the setting up and work in progress – draft technical notes on the results.
-
T4.2 draft questions to get the technical info needed, introductions with WP2 made so we work with them on whick sites to approach and when
-
T4.3 Need a person to start on DMZ
There are minutes attached to this event.
Show them.
