Get Regular Updates!
|How big data is transforming the way Aussie astronomers unravel mysteries



NASA/STScI Digitized Sky Survey/Noel Carboni

How big data is transforming the way Aussie astronomers unravel mysteries

How big data is transforming the way Aussie astronomers unravel mysteries

The universe is huge, and astronomers are dealing with phenomenal sets of data in their search to uncover the secrets of the cosmos.

How big data is transforming the way Aussie astronomers unravel mysteries

Buried in the massive scale of data are nuggets of astronomical gold waiting to be discovered.

That’s where Ryan Ridden-Harper comes in.

As part of his PhD at the Australian National University, Ryan developed a computer program to sift through archived data from the Kepler space telescope, which was retired in 2018.

He’s already found a ‘vampire star’, but he’s most interested in extragalactic transients.

Extragalactic transients include the explosive death of massive stars or objects colliding that cause supernovae.

“Things that blow in galaxies far away from us,” explains Ryan.

View Larger
Image|NASA and L. Hustak (STScI)
Vampire star


Using a computer to search through big data wasn’t as easy as a few clicks.

“The real challenge was that we had no idea what, where or when in the data we were going to find something,” says Ryan.

Ryan had to set parameters for what counted as something of interest.

Like a house hunter might limit their search to cost or location, Ryan narrowed his search to light signals of a particular brightness and length.


Image|Ryan Ridden-Harper
Kepler data showing the vampire star

Ryan’s computer program code can detect a wide variety of phenomena.

They include explosions, stars, asteroids, stellar flares and anything that causes an increase in brightness.

“At the end of this project, we should have a complete list of all those transient things that occurred in Kepler, which will be useful to astronomers across the world,” says Ryan.

Big, big data

Kepler’s K2 mission alone produced upwards of 10 terabytes of data. That’s more than 10 jam-packed iMacs.

If you imagine 10 iMacs full of high-resolution images, Ryan isn’t glancing at each picture. He’s combing through each pixel.

“I’ve had access to very powerful computers at the Australian National University but unfortunately it’s only going to get worse,” Ryan says.

“The next set of data we want to look at is significantly bigger.”

Big impact on astronomy

While the scale of the data gathered by the Kepler telescope is vast, the Square Kilometre Array (SKA) is described as “the largest science facility ever built by mankind”.

The SKA is an array of telescopes that add up to a single lens the size of a square kilometre. Such a big lens can see a lot.

Video|Square Kilometre Array
A new observatory is born

Andreas Wicenec, Professor of Data Intensive Research at the International Centre for Radio Astronomy Research, says the SKA will gather a “phenomenal” amount of data.

“When it is fully operational, it could potentially bring in 100 terabytes of data per second,” says Andreas.

“Even the currently planned Phase 1 will already collect about 1 terabyte per second.”

View Larger
Dish antennas belonging to CSIRO’s Australian Square Kilometre Array Pathfinder

The archived data of Phase 1 of the SKA could amount to 300 petabytes a year. That’s enough data to fill 300,000 iMacs.

“The SKA will be looking for multiple things, but many discoveries could hide in 300 petabytes of data,” says Andreas.

Projects like Ryan’s are crucial to furthering mankind’s understanding of the universe.

“Archived data has the potential to yield more discoveries than the initial goals of an observation project,” says Andreas.

Meanwhile, back on Earth …

Unfortunately, Ryan’s next project won’t necessarily be any easier than the Kepler project.

View Larger
The Kepler spacecraft on a stand for fuelling on 3 February 2009

Each set of data has its own quirks and flaws that need to be resolved before he can trawl through the data, but he’s enthusiastic about the benefits.

“Being able to find these things is pretty exciting and valuable for science,” says Ryan.

“Hopefully this is just the start … we’ve already noticed some other interesting objects in the Kepler data that we’re still following up on.

“I’ve got my fingers crossed that there will be some crazy object floating around in the Kepler data for me to find.

“The vampire star will hopefully be just the first of many.”

Particle Puns


Creative Commons Logo

Republishing our content

We want our stories to be shared and seen by as many people as possible.

Therefore, unless it says otherwise, copyright on the stories on Particle belongs to Scitech and they are published under a Creative Commons Attribution-NoDerivatives 4.0 International License.

This allows you to republish our articles online or in print for free. You just need to credit us and link to us, and you can’t edit our material or sell it separately.

Using the ‘republish’ button on our website is the easiest way to meet our guidelines.


You cannot edit the article.

When republishing, you have to credit our authors, ideally in the byline. You have to credit Particle with a link back to the original publication on Particle.

If you’re republishing online, you must use our pageview counter, link to us and include links from our story. Our page view counter is a small pixel-ping (invisible to the eye) that allows us to know when our content is republished. It’s a condition of our guidelines that you include our counter. If you use the ‘republish’ then you’ll capture our page counter.

If you’re republishing in print, please email us to let us so we know about it (we get very proud to see our work republished) and you must include the Particle logo next to the credits. Download logo here.

If you wish to republish all our stories, please contact us directly to discuss this opportunity.


Most of the images used on Particle are copyright of the photographer who made them.

It is your responsibility to confirm that you’re licensed to republish images in our articles.


All Particle videos can be accessed through YouTube under the Standard YouTube Licence.

The Standard YouTube licence

  1. This licence is ‘All Rights Reserved’, granting provisions for YouTube to display the content, and YouTube’s visitors to stream the content. This means that the content may be streamed from YouTube but specifically forbids downloading, adaptation, and redistribution, except where otherwise licensed. When uploading your content to YouTube it will automatically use the Standard YouTube licence. You can check this by clicking on Advanced Settings and looking at the dropdown box ‘License and rights ownership’.
  2. When a user is uploading a video he has license options that he can choose from. The first option is “standard YouTube License” which means that you grant the broadcasting rights to YouTube. This essentially means that your video can only be accessed from YouTube for watching purpose and cannot be reproduced or distributed in any other form without your consent.


For more information about using our content, email us:

Copy this HTML into your CMS
Press Ctrl+C to copy

We've got chemistry. Want something physical?