Radio Astronomy is a field that generates huge volumes of data. Stuart Weston, of the Institute for Radio Astronomy and Space Research at AUT, needed a better way to manage that data.

Stuart’s research is conducted using data from the AUT’s Warkworth Radio Astronomical Observatory. Collected from two antennae at the Observatory, the many terabytes of data must be processed to correlate the signals. Radio Astronomy research centres overseas use high performance computing (HPC) clusters to perform this function. 

Such clusters must be purchased, housed, powered, monitored, upgraded and maintained at considerable expense – far out of reach for New Zealand scientists. The nearest existing cluster is in Pawsey, Western Australia but AUT wants to start processing the data here in New Zealand. So, Stuart used some standard servers provided by AUT over a regular data link. This method works, but can be slow and cumbersome.

What Stuart needed was a completely new solution to manage big data in New Zealand. We’re delighted to be part of that solution.

The solution

We partnered with Research and Education Advanced Network New Zealand (REANNZ) to create the Long White Cloud Correlator (LWCC). REANNZ provides a dedicated, fast data link from the antenna to our Catalyst Cloud, so that the data can be sent to us and processed in our data centre, before being sent back to Stuart, ready to be used for further science.

Using the new REANNZ link and the Catalyst Cloud, Stuart’s data is processed five times faster than before, at the same speed as a standard HPC cluster. Stuart now doesn’t have to worry about server maintenance and upgrades or data links, as the LWCC takes care of all of this for him.

Stuart is very pleased with the solution REANNZ and Catalyst have brought together.