Data Mining & Scraping

March 29 2016

An interesting link containing the main open sources of data, including some that are pre-processed for machine-learning.

In addition, Google offers an interesting depository of all available sources of Public Data and even makes it easy to set up comparisons and different viewing styles.

October 30 2014

The EA needs data, and this data needs to come from various places around the web. It will mine it and scrape it from a specific set of sites. The specifics of the process are a bit fuzzy round the edges to me at the moment – this is one lead I have ( However there is a rich source of data coming out of NYC, and considering I have a few weeks till the Alumni Show it seems sensible to make life as uncomplicated as possible – no new coding tricks required please!


Ultimately this part of the project is not about visualising Big Data, but instead about visualising the evolving emotional algorithm of the EA’s coded interior. The same principles apply though, which is why I need to do some work examining styles and conventions of infographics.

These are some interesting depositories of links that might be useful: