Correlation of radio astronomy data in real time

I’d recommend Catalyst Cloud! I think it’s the way to go for more research groups to provision compute quickly and easily, and scale.
Stuart Weston

Stuart Weston

Institute for Radio Astronomy & Space Research (IRASR)

5 / 5

Stuart Weston's Catalyst Cloud rating

Radio Astronomy is a field that generates huge volumes of data. Stuart Weston, of the Institute for Radio Astronomy and Space Research at AUT, needed a better way to manage that data.

Stuart’s research is conducted using data from the AUT’s Warkworth Radio Astronomical Observatory. Collected from two antennae at the Observatory, the many terabytes of data must be processed to correlate the signals. Radio Astronomy research centres overseas use high performance computing (HPC) clusters to perform this function. 

Such clusters must be purchased, housed, powered, monitored, upgraded and maintained at considerable expense – far out of reach for New Zealand scientists. The nearest existing cluster is in Pawsey, Western Australia but AUT wants to start processing the data here in New Zealand. So, Stuart used some standard servers provided by AUT over a regular data link. This method works, but can be slow and cumbersome.

Creating nodes on demand, doing your processing, tearing it all down in the end, and not having it running and incurring costs is quite advantageous. It certainly helps with my budget.

Stuart Weston Institute for Radio Astronomy & Space Research (IRASR)

What Stuart needed was a completely new solution to manage big data in New Zealand. We’re delighted to be part of that solution.

The solution


We partnered with Research and Education Advanced Network New Zealand (REANNZ) to create the Long White Cloud Correlator (LWCC). REANNZ provides a dedicated, fast data link from the antenna to our Catalyst Cloud, so that the data can be sent to us and processed in our data centre, before being sent back to Stuart, ready to be used for further science.

Using the new REANNZ link and the Catalyst Cloud, Stuart’s data is processed five times faster than before, at the same speed as a standard HPC cluster. Stuart now doesn’t have to worry about server maintenance and upgrades or data links, as the LWCC takes care of all of this for him.

Stuart is very pleased with the solution REANNZ and Catalyst have brought together.