new and easier ways of working with aggregate data and geographies from uk censuses justin hayes uk...
TRANSCRIPT
![Page 1: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/1.jpg)
New and easier ways of working with aggregate data and geographies from UK censuses
Justin HayesUK Data Service Census Support
![Page 2: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/2.jpg)
Overview
• The UK Data Service• UK Data Service Census Support
• Census background• Aggregate data• Geographies• Dissemination challenges
• InFuse• Data model• Geography model• Live demonstration!
![Page 3: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/3.jpg)
The UK Data Service
• An ESRC initiative integrating several previous resources
• A single, comprehensive and integrated point of access to a wide range of social science data
• Access beyond traditional academic audience where possible
• Support, training and guidance• ukdataservice.ac.uk
![Page 4: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/4.jpg)
UK Data Service Census Support (CS)
• A specialist unit of the UK Data Service
• Access to, and support for use of data from the last five UK censuses (1971 – 2011)
• Add value by making census outputs easy to find, understand and use
• Extend audience beyond ‘experts’• Long history of innovation• census.ukdataservice.ac.uk
![Page 5: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/5.jpg)
ukdataservice.ac.uk
![Page 6: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/6.jpg)
![Page 7: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/7.jpg)
• Aggregate component of census outputs
CS@Mimas
Justin Hayes
Rob Dymond-Green
Richard Wiseman
![Page 8: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/8.jpg)
• Aggregate component of census outputs
CS@Mimas
Justin Hayes
Rob Dymond-Green
Richard Wiseman
![Page 9: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/9.jpg)
UK Censuses
• Decennial questionnaire surveys• Entire UK population every ten years* since 1801• Questions about people and households• 2011 Census cost ~ £500m
• Primary evidence for government policy and spending• Wide range of high quality demographic and socio-economic
characteristics• What? - Detailed combinations of characteristics• Where?- Small areas• When? - Long history
• Rich secondary source of information• Open Government License!
![Page 10: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/10.jpg)
UK 2011 Census
• 27 March 2011• Three UK census agencies (ONS, NRS, NISRA)• New questions and variables• Targeted enumeration• Online and postal completion• Sophisticated quality assurance• Best census ever!
![Page 11: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/11.jpg)
Census aggregate outputs
• Counts of people and households* with particular combinations of characteristics for particular geographical areas• Females aged 16-74 in employment in associate professional
and technical occupations and usually resident in wards in the County of Devon
• Derived from unit-level questionnaire responses• Variables and categories• Sex - Male and Female and All• Age - single and multiple years• Ethnicity and Occupation – standard classifications
• Traditionally specified by tables combining one or more variables
![Page 12: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/12.jpg)
Aggregate specification tables
![Page 13: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/13.jpg)
Census aggregate data
Age : Age 16 to 74 - Economic activity : in employment the week before the census - Occupation : 3. Associate professional and technical occupations - Sex : Female - Unit : Persons
Age : Age 16 to 74 - Economic activity : in employment the week before the census - Occupation : All categories\ Occupation - Sex : Female - Unit : Persons
![Page 14: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/14.jpg)
Census aggregate data
![Page 15: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/15.jpg)
Aggregate data
![Page 16: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/16.jpg)
2011 Census geographies
• Subdivisions of the UK into smaller areas• Sets of similar areas called geographies
• Functional and statistical geographies• Local government districts• Wards and electoral divisions• Expecting around 100 different geographies
• Hierarchies of geographies with nesting areas• Administrative• Statistical• Health, Electoral, Postcode, etc
![Page 17: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/17.jpg)
UK administrative geographies
![Page 18: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/18.jpg)
UK statistical geographies
![Page 19: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/19.jpg)
Dissemination challenges
• Size and complexity of planned outputs• Ongoing releases from three different agencies• Inconsistencies in definitions
• Categorisation differences within and between countries• Table universes• Inconsistent labelling
• Incomplete geographical availability of data• Disclosure control• Lower Threshold (LT), Higher Threshold (HT) and other data
• Thousands of separate datasets• Restricted global operation and understanding
![Page 20: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/20.jpg)
InFuse
• Live service with 2001 census data since 2012• 2011 data since 2013• Tip of the iceberg!• Data model• Geography model
![Page 21: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/21.jpg)
InFuse data model
• Single multidimensional dataset• Deconstruction, rationalisation and re-integration of
variables and categories • All UK table specifications processed• Integration of table universes as variables• Enforce consistency across dataset• Library of variables and categories to describe all counts
• Re-insertion of counts into model• Retain original cell identifiers
• Attachment of metadata
![Page 22: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/22.jpg)
2011 census variables
• 97 variables and counting!
![Page 23: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/23.jpg)
InFuse geography model(s)
• Raw geography model• All original geographies and their areas• Direct and indirect hierarchical relationships
• Simplified geography model• Combinations of equivalent geographies into geography sets
with UK coverage where possible• Condensed standard/merged geographies in England
• Selections of areas across the UK• Multiple geographies in one operation• Geography jumps in interface
• Currently administrative and statistical geographies• More to follow
![Page 24: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/24.jpg)
Raw geography entities and relationships
![Page 25: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/25.jpg)
InFuse administrative and statistical geographies
![Page 26: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/26.jpg)
![Page 27: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/27.jpg)
![Page 28: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/28.jpg)
![Page 29: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/29.jpg)
![Page 30: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/30.jpg)
![Page 31: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/31.jpg)
![Page 32: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/32.jpg)
![Page 33: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/33.jpg)
InFuse features
• Open access• All data is open via Open Government Licence
• Global search across entire UK dataset• Variable combinations• No tables!
• Guide users to find data• Populated variable combinations• Available geographies
• More data for more geographies• All LT and HT data available for all areas above LT
• Improved contextual information• No data fast!
![Page 35: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/35.jpg)
What’s next?
• Big data release imminent!• Progressive release of UK 2011 outputs
• Scottish and Northern Ireland
• Integrated boundary data in GIS formats• Interface design and features• More contextual information
![Page 36: New and easier ways of working with aggregate data and geographies from UK censuses Justin Hayes UK Data Service Census Support](https://reader036.vdocuments.site/reader036/viewer/2022062423/56649ef35503460f94c060fa/html5/thumbnails/36.jpg)
After that?
• Integration of multiple censuses• Non-census data• External access to API for application development• Development of data and geography models• Continued engagement with NSIs
• Data production using multidimensional approach• Automated disclosure control• No all or nothing table constraints