The Amateur Data Scientist and Her Projects

With so much data available for free everywhere, and so many open tools, I would expect to see the emergence of a new kind of analytic practitioner: the amateur data scientist.

With so much data available for free everywhere, and so many open tools, I would expect to see the emergence of a new kind of analytic practitioner: the amateur data scientist.

Just like the amateur astronomer, the amateur data scientist will significantly contribute to the art and science, and will eventually solve mysteries. Could the Boston bomber be found thanks to thousands of amateurs analyzing publicly available data (images, videos, tweets, etc.) with open source tools? After all, amateur astronomers have been able to detect exoplanets and much more.

Also, just like the amateur astronomer only needs one expensive tool (a good telescope with data recording capabilities), the amateur data scientist only needs one expensive tool (a good laptop and possibly subscription to some cloud storage/computing services).

Amateur data scientists might earn money from winning Kaggle contests, working on problems such as identifying a Bonet, explaining the stock market flash crash, defeating Google page-ranking algorithms, helping find new complex molecules to fight cancer (analytical chemistry), predicting solar flares and their intensity. Interested in becoming an amateur data scientist? Here’s a first project for you, to get started:

First project: Do large meteors cause multiple small craters or a big one?

If meteors usually break up into multiple fragments, or approach the solar system already broken down into several pieces, they might be less dangerous than if they hit with a single, huge punch. That’s the idea, although I’m not sure if this assumption is correct. Even if the opposite is true, it still worth asking the question about frequency of binary impact craters.

About to hit Earth

Eventually, the knowledge that meteorites arrive in pieces rather than intact could change government policies and priorities, and maybe stop spending money on projects to detect and blow up meteors (or the other way around).

So how would you go about estimating the chance that a large meteor (hitting Earth) creates multiple small impacts? And how many impacts on average: 2, or 3? An idea consists in looking at Moon craters and checking how many of them are aligned. Yet what causes meteors to explode before hitting (and thus creating multiple craters) is Earth’s thick atmosphere. Thus, Moon would not provide good data. Yet Earth’s crust is so geologically active that all crater traces disappear after a few million years. Maybe Venus would be a good source of data? Nope, even worse than Earth. Maybe Mars? Nope, just like Moon. Maybe some moons from Jupiter or Saturn would be great candidates.

Double impact, seen million years later

Once a data source is identified and the questions answered, deeper questions can be asked, such as: when we see a binary crater (two craters, same meteor), what is the average distance between the two craters? This will also help better assess population risks, and how many billion dollars NASA should spend on meteor tracking programs.

In any case, as a starter, I did a bit of research and found the following data, with a map showing impact craters on Earth.

Visually, with the naked eye, it looks like multiple impacts (e.g. binary craters), and crater alignment, is the norm, not the exception. But the brain can be very lousy at detecting probabilities. So a statistical analysis is needed. Note that the first step consists in processing the image to detect craters and extract coordinates, using some software or writing your own code. But this is still something a good amateur data scientist could do, I’m sure you can find the right tools on the Internet.

A better idea might be to use some public data I published a while back: it is more comprehensive, already in Excel format, and also has dates attached to craters, so that binary craters created on the same year are likely to be true twins (that is, coming from a single piece of space rock that broke into two fragments sometimes in the past, before hitting Earth). 

Google ‘binary craters on Earth’ for additional info.