Wednesday 6 May 2020


 
Secondary Data
A Kingston University Geography, Geology and Environment Guide


David Kidd, Department of Geography, Geology and the Environment, Kingston University, May 2020.



Secondary data is data collected by someone else for another purpose, rather than being collected yourself.

A huge amount of secondary data (and documents that contain useful data) are available online. These data are varied in subject, scale and degree of aggregation, geographical coverage, as well as their completeness, quality, permitted usage and file format the data is supplied in. The ability to locate, understand, evaluate secondary data, and then to undertake suitable analysis and visualisation to address a particular question, is an essential practical and academic skill known as ‘data literacy’. This document provides a brief introduction to secondary data and information resources for students of geography and environmental science.

Introduction

The amount and variety of data available for an area of interest depends on how large it is and where it is in the world. In general, more small-scale ‘global data’ is available, than large scale ‘local’ data, however, local data is available for certain areas (the UK in particular through academic licensing). Some global datasets are now of sufficient resolution that they may be used for regional or even local application, but always consider error as it may be significant.

The Kingston GGE List of Secondary Data Sources is a collation of some secondary data sets and other information resources that are regularly used by GGE students for their assignments and dissertations. Be aware that there is a lot more data available on the WWW, so do use search engines to look for other data if is not listed. 

Before using any data set you must evaluate whether it is ‘fit-for-purpose’ to be used to answer your question (aim). However, to answer this, you must first specify your ‘data needs’ which are determined by the question(s) being asked of the data, geographical area of interest, and the analytical approaches to be employed – these should be outlined in your project proposal. Once you have a good understanding of your data needs then you can begin to look for data that meets your criteria.

When you collect primary data you have control (within your resource limits) over what data is collected, where and how. You have no such control over secondary data, so can only work with the materials (data) that are available. This may result in a mismatch between data requirements and data availability.

If you cannot find the exact data you need, then you should consider whether you could make what is required from the data that is available? For example, I may have to create a UK data set I might need to combine separate data for England, Scotland, Wales and Northern Ireland. If you are going to combine or analyse data from different sources always consider their relative geographical scale. In general, ‘down-scaling’ data to the lower is simpler and more appropriate than ‘up-scaling’ to higher resolutions.
You may also find that the data is sufficient to partially, but not entirely, address your aim. Whether you can do enough for your dissertation will depend on how much possible, so you are advised to talk to your supervisor if this is a concern.

When evaluating data, be flexible and consider whether you could adapt your aim and methodology to the data. Perhaps data is available for you to ask the same question of a different organism, organisation or place; or, maybe it could be answered at a continental or global scale, but not the scale of regions within a country? Try and think laterally, ask yourself, ‘what questions close to your original aim are answerable with the available data?’

Ultimately the ‘fitness-for-purpose’ of the data you locate is judged on the following in relation to the aim and method:

  1. covers the area of interest;
  2. extends over the required time period;
  3. has the required attributes;
  4. is sufficiently complete, with known bias and error;
  5. the data format it is supplied in (e.g. csv, shapefile, NetCDF, Web API);
  6. the spatial, temporal and attribute resolution;
  7. its provenance (who collected it, for what purpose and how); and
  8. if the data license permits the intended use.

The  Data List

The Secondary Data List contains links to approximately 200 secondary data and information resources that may be of use in assignments and dissertations.
  • The list is divided by topic, but do look across topics as many sources are cross-disciplinary.
  • Look across scales, perhaps there is a high-resolution global data that can be clipped to your local area?
  • The list is biased towards the Kingston/Surrey, London, the UK and global sources.
  • A lot of the resources relate to GIS data, but there are also sites with other quantitative and qualitative data including surveys of public opinion.
  • Always remember that there is a lot more data available that listed here. Use Internet search engines and Data Portals (e.g. data.gov.uk) to find other data that might be more appropriate for your aim.

If know of any resources that you think should be added to this list then please email them to David Kidd (david.kidd@kingston.ac.uk).


1.          General
1.1.           Data from publications. Some journals require researchers to deposit their data in a data repository or as appendices to the publication as a condition of publication. If data is archived, then this will be detailed in the publication (or online page for the paper). The Dryad repository was set up to archive biological data sets, but is open for use by other communities as well. Figshare has a wide set of users and varied content that includes unpublshed figures and conference posters.
1.2.           The London Datastore (Mayor of London) supplies a very wide range of information and data on the capital. Look here first!
1.3.           UKMap a modern, highly detailed, feature-rich mapping database of Greater London. Its unique, innovative design offers users a flexible choice of integrated map features within a single geographic information source. Available through Digimap Piolet.
1.4.           Mapping London is UCL's ongoing efforts to map London.
1.5.           The European Data portal allows you to find data on the EU and constituent countries.
1.6.           The NSF Arctic Data Centre has a map based catalog of Arctic datasets.. 
1.7.           Quantarctica is a GIS package of data for Antarctica for QGIS. 
1.8.           The British Antarctic Survey  provides various datasets. 
1.9.           SCAR maintain the Antarctic Digital Database Map Viewer that has metadata links to data sources.
1.10.       Credo Research Encyclopaedia – search for general information on a topic across encyclopaedia and other reference books.
1.11.       ESRI Living Planet and ESRI Online data. ESRI provide a wide range of data through their Living Atlas project as well as data published by ESRI Online users. Use the three dots on the right of a data set to open it in ESRI Online or ArcGIS desktop. Layers can also be added directly to ArcGIS Desktop or ArcGIS Pro.
1.12.       UNData – portal on UN databases.

2.          Basemaps and Places
2.1.           The Ordnance Survey supplies digital maps at scales from 1:1250 to 1:50,000. OS Open Data can be directly downloaded, additional data (e.g. large-scale maps) can be downloaded from Digimap. Digimap also provides access to historical OS maps allowing you to see change through time.
2.2.           OS Code-Point Open locates all Great Britain (not NI) postcodes as point locations. The geographical area covered by each postcode is a commercial product, however OpenDoorLogistics provide a free Reconstructed UK postcode boundaries dataset.
2.3.           The OS Points of Interest® dataset is the most comprehensive, location-based directory of all public and privately-owned businesses, education and leisure services in Britain. Available through Digimap.
2.4.           OpenStreetMap (OSM) is a crowd-sourced effort to create a open global street level mapping. Shapefiles of OSM data can be downloaded from various sites. Alternatively, the OSM pbf format can be opened directly in ArcGIS/ArcPro with the ESRI Interoperability Extension (request a licence through the KU Service Desk Portal). 
2.5.           Natural Earth is a global public domain map dataset of human and natural features available at 1:10m, 1:50m, and 1:110 million scales. Suitable for country to global scale maps. Vector and raster layers.
2.6.           DIVA-GIS provides base data from various sources by country. 

3.          Elevation and Bathymetry
3.1.           OS DEMs at 5m and 50m gridded DTMs and contours may be downloaded from Digimap.
3.2.           The UK Environment Agency have 0.25m0.5m1m and 2m  DSMs for approximately 60% of the UK.
3.3.           The most detailed global elevation models available are the 30m ASTER and SRTM DEMs.
3.4.           NOAA supply a 1km resolution global DEM
3.5.           The General Bathymetric Chart of the Oceans is a 15-arc second global elevation and bathymetry model.
3.6.           Open Topography is a data sharing application for elevation data sets. Mostly US, but elsewhere as well.

4.          Aerial Photography and Satellite Imagery
4.1.           Aerial Digimap provides access to some of the highest quality aerial photography available for Great Britain.
4.2.           Britain from Above has images from the Aerofilms collection. The collection includes 1.26 million negatives and more than 2000 photograph albums dating from 1919 to 2006.
4.3.           Google Earth time slider allows you to view earth imagery going back to 1980’s.
4.4.           The Surrey Air Survey is a mosaic of aerial photography of Surrey taken in 2007-8.
4.5.           The USGS EarthExplorer provides access to wide range of satellite imagery, elevation models and land use data.
4.6.           The Earth Online portal provides access to European Space Agency imagery.
4.7.           The Copernicus Open Access Hub provides free and open access to Sentinel-1, Sentinel-2, Sentinel-3 and Sentinel-5P products.

5.          Population, Demographics and Social Sciences
5.1.           The UK 2001 and 2011 Census are available through Infuse. You must download the attributes you want and their census geographies separately and then join them together in your GIS.
5.2.           Casweb provides access to Aggregate UK Census data. 
5.3.           I-CeM provides access to standardised and integrated historical census microdata for Great Britain 1851 to 1911
5.4.           WorldPop supply global population and demographic variables at scales from 100m resolution.
5.5.           Deprivation data for the UK is available from the UK Census Data Service.
5.6.           UK Data Service provides access to a wide range of social and economic data sets, including surveys.
5.7.           The UK Data Archive is home to the UK's largest collection of social, economic and population data.
5.8.           European Social Survey - https://www.europeansocialsurvey.org/data/
5.9.           IBSS: International Bibliography of the Social Sciences. Discover interdisciplinary research’ anthropology, economics, politics, and sociology
5.10.       International Encyclopedia of Human Geography [via iCat]
5.11.       Migration to New Worlds: Data on two centuries of mass migration

6.          Cultural Artefacts
6.1.           Culture Grid is the closest thing the UK has to a national aggregator for museum collections information. It brings together three million object records from over a hundred collections, and makes them available to re-use in more interesting ways elsewhere.
6.2.           English Heritage provide data on Listed Buildings, Scheduled Monuments, Battlefields and other protected cultural assets.
6.3.           UNESCO World Heritage Sites.

7.          Environment and Biodiversity
7.1.           The UK Meteorological Office provides various UK meteorological and climate data.
7.2.           MAGIC is DEFRA's portal onto geographic information about the natural environment from across UK government. The information covers rural, urban, coastal and marine environments across Great Britain. It is presented in an interactive map which can be explored using a standard web browser. Links are provided to the sources of data.
7.4.           UK National Land Cover datasets are available through Digimap.
7.5.           The NBN is a portal to UK species distribution records collected by many different organisations.
7.7.           The CEDA Archive is the Natural Environment Research Council's Data Repository for Atmospheric Science and Earth Observation data.
7.8.           Environment Agency river and flood maps and more available through data.gov.uk. The EA also hosts the Water Quality Archive. 
7.9.            EU Water Framework Directive related, UK data
7.10.       The Center for Ecology and Hydrology is a UK Natural Environment Research Council research organisation that provides data on terrestrial and freshwater environments.
7.11.       Natural England is the body that oversees UK natural environment conservation designations, schemes and the National Historic Landscape Characterisation. 
7.13.       ESPON have several datasets concerning the extent and characteristic of urban areas in Europe.
7.14.       Norpil supply the World database of large urban areas, 1950-2050.
7.15.       Protected Planet is the most up to date and complete source of information on protected areas. Updated monthly, it is managed by the United Nations Environment World Conservation Monitoring Centre (UNEP-WCMC) with support from IUCN and its World Commission on Protected Areas.
7.16.       The Environmental Data Explorer is the authoritative source for data sets used by UNEP and its partners in the Outlook (GEO) report and other assessments. More than 500 different variables, as well national, subregional, regional and global statistics or as geospatial data sets (maps), covering themes like Freshwater, Population, Forests, Emissions, Climate, Disasters, Health and GDP. Display them on-the-fly as maps, graphs, data tables or download the data in different formats.
7.17.       PALSAR-2 is an annual 25m-1km resolution forest/non-forest raster data set. 
7.18.       Global Forest Watch provide a viewer on to forest cover and a lot related global data including land use and biodiversity.
7.19.       London Air hosted by Kings College supplies data on air quality in the city.
7.20.       European Environment Agency data and maps.
7.22.       The London Tree Map is an interactive map of >75,000 of the estimated 8 million trees in London. Mostly street trees and some in parks.
7.23.       The USGS HydroSHEDs project supplies global hydrological data and models derived from the 90m global SRTM elevation model.
7.24.       NCAR provide a comparative list of Sea Surface Temperature and related oceanographic data.
7.25.       The Atlas of the Biosphere is a collection of global maps and data sets concerning global land use, ecosystems, water resources and human impact.
7.26.       NOAA provide a range of weather, climate and ocean data, much of it global.
7.27.       The IUCN provide range maps for many (‘higher’) animals with an emphasis on Red List species. 
7.28.       GBIF is a global portal onto more than 1.3 billion species records supplied by museums and other publishers.
7.29.       OpenTraits have a list of species trait databases.
7.30.       TRY Plant Trait database
7.32.       Living Planet Index – a database of species population records used to calculate change in global biodiversity.
7.33.       Google Earth Engine is a free cloud-based environment that combines a multi-petabyte catalog of satellite imagery and geospatial datasets with planetary-scale analysis capabilities and makes it available for scientists, researchers, and developers to detect changes, map trends, and quantify differences on the Earth's surface.
7.34.       NERC Earth Observation Data Acquisition & Analysis Service. Data from the latest satellite passes from a number of sensors such as OLCI, VIIRS and AVHRR are downloaded each day and processed into a range of products including sea surface temperature.

8.          Weather and Climate
8.1.           WorldClim is a set of global gridded climate layers with a spatial resolution of about 1 km2.  It also has some future and past (Mid-Holocene and the Last Glacial Maximum) climate models.  
8.2.           Andreas Hamann provides a wide range of climate and derived variables aggregated to various temporal units for North America, South America, and Europe gridded at a resolution of 1km-4km. 
8.3.           UEA Climate Research Unit publish palaeoclimate records and future and past climate simulation models.
8.4.           Climate change scenarios can be downloaded in GIS formats from the US National Center for Atmospheric Research (NCAR).
8.5.           EUMETSAT European Meteorology and Climate data.

9.          Geosciences
9.1.           A 1:625,000 scale geological map of the UK and some other spatial data sets are freely available from the British Geological Survey.  The Geology of Britain Viewer provides view only access to BGS 1:50,000 geological maps.
9.2.           The Soilscapes Viewer provides view only access to UK soil maps.
9.3.           British Geological Survey Soil property data (web map service)
9.4.           OneGeology is a community based effort to create a global geological map with links to data sources.
9.5.           GeoScienceWorld is a comprehensive resource of eBooks for researchers in the Earth Sciences

10.       Hazards, Conflict, Disaster Management and the Emergency Services
10.1.       Download data on crime and policing, the Fire Service (via data.gov.uk) and Ambulance call outs (via data.gov.uk).
10.3.       NOAA NCEI/WDS Global Significant Earthquake Database, 2150 BC to Present  with damage information.
10.4.       The USGS publishes a global earthquake catalog.
10.5.       The UNISDR Global Risk Data Platform is a multi-agency effort to share spatial data information on global risk from natural hazards. Users can visualise, download or extract data on past hazardous events, human & economical hazard exposure and risk from natural hazards. It covers tropical cyclones and related storm surges, drought, earthquakes, biomass fires, floods, landslides, tsunamis and volcanic eruptions.
10.6.       EM-DAT provides information on the human impact of disasters, such as the number of people killed, injured or affected, as well as economic damage estimates and disaster-specific international aid contributions.

11.       Food and Health
11.1.       EarthStat serves geographic data sets that help solve the grand challenge of feeding a growing global population while reducing agriculture’s impact on the environment.
11.2.       The UN Food and Agriculture Organisation Geonetwork portal provides access to many global data sets.

12.       Transport
12.1.       Transport for London Open Data
12.2.       OpenFlights is a database of airports, ports, flights and more.
12.3.       A global map of Travel time to major cities is provided by the European Commission's Joint Research Center.

13.       Energy
13.1.        UKERC Energy Data Centre, near comprehensive coverage of the UK energy research scene.
13.2.       UN Energy Statistics Database.
13.3.       International Energy Authority data and statistics.
13.4.       World Wind Atlas
13.5.       Global Solar Atlas

14.       Historical
14.1.       A Vision of Britain through Time provides an interface onto UK historical records, administrative areas, population statistics and other cultural information. 
14.2.       Historical OS maps can be viewed through the Digimap Historic Roam Viewer
14.3.       ConnectedHistories (British History Sources 1500-1900) brings together a range of digital resources related to early modern and nineteenth century Britain with a single federated search that allows sophisticated searching of names, places and dates.
14.4.       The Museum of London has an interactive version of The Archaeology of Greater London (2000).
14.5.       Layers of London is a map-based history website developed by the Institute of Historical Research. Users can access free historic maps of London and contribute stories, memories and histories to create a social history resource about their area. 

15.        Government and Economics
15.1.       Data.gov.uk is the main portal onto UK government information and data, some of which is spatial. It is not the friendliest system so do persevere.
15.2.       The Combined Online Information System (COINS) is a database about UK Government expenditure, collecting financial data from across the public sector, but not down to transaction level.
15.3.       Kingston Data allows you to explore information about the London Borough of Kingston upon Thames aggregated to different administrative units. View only, no download. 
15.4.       Surrey-i is Surrey County Council's information and data portal.
15.5.       The World Bank Data Catalog provide a variety of data on different countries of economic relevance (power, transport, soils, water, etc.).
15.6.       IBISWorld: Industrial and market data.
15.7.       International Monetary Fund supply global economic data and forecasts.
15.8.       Mintel: Consumer data and market research and analysis
15.9.       Thomson ONE: Company and share price data; mergers and acquisitions, new issues, repurchases.
15.10.   UN Conference on Trade and Development publish international development and foreign direct investment reports.