March 29 2016

An interesting link containing the main open sources of data, including some that are pre-processed for machine-learning.

http://www.forbes.com/sites/bernardmarr/2016/02/12/big-data-35-brilliant-and-free-data-sources-for-2016/#78ccecaa6796

In addition, Google offers an interesting depository of all available sources of Public Data and even makes it easy to set up comparisons and different viewing styles.

October 30 2014

The EA needs data, and this data needs to come from various places around the web. It will mine it and scrape it from a specific set of sites. The specifics of the process are a bit fuzzy round the edges to me at the moment – this is one lead I have (http://scrapy.org/). However there is a rich source of data coming out of NYC, and considering I have a few weeks till the Alumni Show it seems sensible to make life as uncomplicated as possible – no new coding tricks required please!

• BBC Article – How NYC is releasing Big Data back to the public
• http://www.bbc.co.uk/news/technology-24505860

• Source of NYC Big Data
• https://data.cityofnewyork.us/

These are some interesting depositories of links that might be useful:

• http://www.quora.com/Data/Where-can-I-find-large-datasets-open-to-the-public 
• http://www.scaleunlimited.com/datasets/public-datasets/