Use Microdata Samples (PUMS) are files containing records for samples of
individuals and households. PUMS allow data users to conduct their own analyses
of the the American Community Survey (ACS); the files contain the full range of
population and household responses collected on individual ACS questionnaires.
USA is a highly comprehensive website and visualization engine for public US
American Community Survey (ACS) is an ongoing survey that provides vital
information annually about America and its people. The survey generates data
that help determine how more than $400 billion in federal and state funds are
distributed each year.
data platform that helps people work together to solve problems faster by
creating new ways to discover, prepare, and collaborate with data. data.world can
ingest surveys in new formats to increase visibility and usability for new
cohorts of users.
The purpose of StatsUserNet is to stimulate
engagement between users, and between users and producers of official
statistics. The site carries news of interest to users, materials relating to
relevant events, discussion forums and other facilities.
ONS has launched a Data Science Campus. This will act as a hub for the
whole of Government to gain practical advantage from the wider investment in
data science research and help cement the UK’s reputation as an international
leader in data science.
Charles Bean, Professor of Economics at the London School of Economics and
formerly Deputy Governor for Monetary Policy at the Bank of England, with the
support of a small team of officials from HM Treasury, ONS and the Bank of
England, conducted an independent review of UK economic statistics to assess
the UK’s future (economic) statistics needs.
a five year strategy for the UK official statistics system. The strategy covers
three principal elements of the UK official statistics system for which the UK
Statistics Authority has oversight: (1) GSS, (2) ONS, and (3) independent
regulation; setting out the mission, values and context for the work for
It would be natural to imagine that geodemographics has now become obsolete – in view of the big data that companies can access on individual consumer purchases, transactions and online browsing behaviour.
The 2017 CGG Seminar will explore how geodemographics has adapted itself for the digital age, and is increasingly being recognised as a valuable source for online marketing, alongside its more traditional off-line applications.
Archive of Market and Social Research is being created to provide a national resource
on the development of the research industry, historic research studies, key
techniques and ground-breaking contributions by industry personnel.
This overview highlights the key themes of the General Data Protection Regulation (the GDPR) to help organisations understand the new legal framework in the EU. It explains the similarities with the existing UK Data Protection Act 1998 (the DPA), and describes some of the new and different requirements.
Ipsos MORI was commissioned by theEconomic and Social Research Council
(ESRC) and the Office for National Statistics
(ONS) to conduct a public dialogue examining the public’s views on using linked administrative data for research purposes, and to explore the operational and communications plans of the new ESRC-funded Administrative Data Research Network (ADRN) that was set up in late 2013. The aim was to examine the public’s views on using linked administrative data for research purposes, and specifically to explore the operational and communications plans of the new ESRC-funded Administrative Data Research Network (ADRN) that was set up in late 2013. The dialogue formed part of the widerPublic Attitudes to Science
project, which collected quantitative data about the public’s views on the uses of big data.
UK's largest organisation for funding research on economic and social issues. We support independent, high quality research which has an impact on business, the public sector and civil society.
Natcen is Britain’s largest independent social research agency. For the last 40 years Natcen has worked on behalf of government and charities to find out what people really think about important social issues and how Britainis run.
UNECE is one of five regional commissions of the United Nations. They recently held a UNECE Expert Group meeting on
International Censuses in Geneva. Please find a link below to the papers and presentations as well as a formal report summarising the main issues discussed.
ADAN - the Advanced Analytics Network - aims to help
identify, develop and disseminate research industry best practice in the
general area of advanced analytics.
On Thursday 10th
November 2016 ADAN will be holding its third symposium, (a 2 hour evening
mini-conference to be held in London),
which will build on the success of the previous two events.
The Registrar General for Scotland has
published Scotland's Census 2021 – Topic Consultation Report on 15
August 2016 and it is available from the Scotland's Census website. The success of the 2021 census relies on a comprehensive understanding of user needs and concerns, as well as the support and participation of the general public.
The UK Data Service is funded by the Economic and Social Research Council (ESRC) to meet the data needs of researchers, students and teachers from all sectors, including academia, central and local government, charities and foundations, independent research centres, think tanks, and business consultants and the commercial sector. They produced a series of upcoming events and webinars including:
This is one of a series of reports prepared
under the provisions of the Statistics and Registration Service Act 2007 . The
Act gives the Statistics Authority power to re-assess whether the Code of
Practice for Official Statistics continues to be complied with in relation to
official statistics already designated as National Statistics. The report
covers the following population estimates and projections (population) statistics
produced by the Office for National Statistics (ONS) for the UK and its
The Code of Practice for Official Statistics promotes
the production and dissemination of official statistics that inform decision
making, and supports the continuous improvement of those statistics. It is a
concise and specific statement that requires sound judgement and
DataShine COWZ-EW created by Chris Gale and Oliver O’Brien allows users to explore COWZ-EW data visuallly on a map.
COWZ-EW is a geodemographic classification of Workplace Zones (WZs) for England and Wales. It is based on the characteristics of workers and their workplaces, as recorded in the 2011 Census. COWZ-EW is an interim product for
England and Wales, produced by the University of Southampton in
collaboration with ONS. It was released in response to strong user
interest whilst a full UK-wide classification is being finalised. The
dataset is made available under the Open Government Licence v3.0. Users
may re-use this information free of charge, in any format or medium
under the terms of the licence.
See the COWZ-EW userguide here: http://cowz.geodata.soton.ac.uk/download/
The Ministry of Defence (MOD) have created an online tool to create
maps. They have a
collection of existing templates saved, but the public can also import maps as well.
The MOD are collecting
the information about where these maps go so that once they are published they
can release them as open date files
Access to administrative data for research is a positive public good. The de-identification and linking of these records, collected in the course of business by governmental and other bodies, creates a whole new potential for social and economic research and policy evaluation, which up until recently has remained largely unexplored.
The Administrative Data Research Network (ADRN) is a UK-wide initiative, funded by the Economic and Social Research Council (ESRC) in 2013, to facilitate access to linked de-identified administrative data for social and economic research aimed at benefiting society. The results from the Network's earliest projects will be presented.
The Key and Quick Statistics tables for the United
Kingdom bring together data from the 2011 Censuses conducted in England and Wales,
in Scotland, and in Northern Ireland, to provide UK harmonised data for areas
throughout the United Kingdom. These UK tables were released by ONS in 2014 as
downloadable files, and have now been added to the wide range of Census data
available through nomis
The Office for National Statistics is the UK's largest independent producer of official statistics, covering the UK's economy, society and population, including the UK Census. It is the recognised UK National Statistical Institute.
The new complete redesigned ONS website has been launched, following extensive feedback from the July 2015 Beta version. The search and navigation tools have been significantly improved, as has the overall performance and general appearance. Nearly 150,000 redirects enable users to find existing content and much of the older content that has been moved to the National Archives
ONS is retaining a full time team to regularly maintain and enhance the website. More about development of the new website is available in the Digital Publishing Blog.
O2 signed deals with Starbucks and L’Oréal to be the first brands to undertake a six-month trial of its large-scale location-based mobile marketing platform in the UK.
Geodemographics for Marketers provides readers with the know-how to leverage geodemographics as
an effective research tool to identify locatable segments for highly targeted
marketing. International in scope and
impartial in its approach, the book demonstrates how to implement
geodemographic techniques for practical application in a wide range of sectors,
including retail, financial services and market research.
There has been much talk about dropping the traditional population census and obtaining equivalent estimates from big data and administrative databases. To what extent is this achievable? How much is just hype?
The 2016 CGG seminar will explore big data sources that are available now, or coming imminently, and what they can provide for geodemographics users. The main focus will be on spatially referenced data that can be applied down to small area level.
The largest study of its kind, looking at the attitudes and
behaviours of consumers and citizens in 20 key countries around the world. Ipsos Mori reveals the complex, and sometimes contradictory, findings covering everything
from marriage to migration, from ambition to advertising and from society to
This report describes the
findings from a public dialogue on administrative data commissioned by the
Economic and Social Research Council (ESRC) and the Office for National
Statistics (ONS). The overall objectives were to explore public understanding
and views of administrative data and data linking.
Research for the Royal
Statistical Society carried out by Ipsos MORI reveals that the media, internet
companies, telecommunications companies and insurance companies all come at the
bottom of a “trust in data” league table. Only between four and seven per cent
say they have a high level of trust in these organisations to use data
appropriately, compared with 36% trusting the NHS, and 41% trusting their GP.
The CGG run regular seminars
on the latest developments and uses of Census, geodemographic and
lifestyle data, and data integration in market research and related
fields. The next event will be on 7 March.
Please see the official website of National Records Scotland. This organisation is planning the 2021 Census in Scotland.
National Records of Scotland (NRS) on behalf of the Registrar General for
Scotland, is responsible for conducting a census in Scotland. Planning has begun for Scotland's Census 2021 and will build on the success of the Census in 2011. It will be designed and managed in Scotland, to best meet the needs of its users.
As part of this, National Records Scotland, are now asking their users for their views on the topics to be included on the 2021 Census questionnaire.
Please see below to share your views: http://www.scotlandscensus.gov.uk/consultation-2021
Please see below to view news and events about the Census:
The LAD Data store is a catalogue
of data holdings. You may search a datastore for each local authority district in the UK and the directory provides categories of data sets in terms of Local Authority District.
Weeding out mistakes hidden among billions of data points seems an impossible task. Marco Puts, Piet Daas and Ton de Waal put forward some solutions.
Dugmore explores alternative more
cost-effective options for ‘census taking’ in the future. In this paper, they consider what the options may be, based on approaches and experiences from
other countries, and assess their implications for users.
This paper by Rob Claxton, jon Reades, and Ben Anderson discusses the study of UK regions, comparing the "geographies of talk” to their administrative counterparts, before turning to the ways in which social networks reflect underlying problems of deprivation and of access to opportunity.
The rise of Big Data changes the context in which organisations producing official statistics operate. Big Data provides opportunities, but in order to make optimal use of Big Data, a number of challenges have to be addressed. This stimulates increased collaboration between National Statistical Institutes, Big Data holders, businesses and universities. In time, this may lead to a shift in the role of statistical institutes in the provision of high-quality and impartial statistical information to society. In this paper, the changes in context, the opportunities, the challenges and the way to collaborate are addressed.In this paper by Peter Struijs, Barteld Braaksma, Piet JH Daas, the changes in context, the opportunities, the challenges and the way to collaborate are addressed. The collaboration between the various stakeholders will involve each partner building on and contributing different strengths.
The paper explores this potential through preliminary analysis of a
‘smart meter-like’ dataset, and when set alongside the limited
literature to date, the results suggest that aggregated household load
profiles may reveal key household and householder characteristics of
interest to census users and national statistical organisations.Much of the attention has been focused on the (re)use of
governmental administrative datasets but, we argue that data held by
commercial organisations may offer considerable additional value as a
supplement to familiar census or survey and administrative data."
ONS, the UK's largest producer of official statistics, is interested in
understanding the impact that big data may have on their statistical
processes and outputs. They have established a project to investigate the
potential advantages of using big data, to understand the challenges
with using these data sources and to establish a longer term strategy
for big data within official statistics.
The ONS report discussion therefore concentrate on temporal patterns of power demand that may be able to more robustly distinguish between household types due to the potentially different timings and intensities of their everyday habits and routines.
The Economic and Social Research Council (ESRC) is delighted to announce the 20 successful grants of the first Transformative Research Call, a pilot for 2012/2013.
The call aims to provide a stimulus for genuinely transformative and groundbreaking research ideas at the frontiers of social sciences.
The goal of OpenPopGrid is to provide an OpenData dataset to improve the spatial representation of the published ONS population dataset. OpenPopGrid achieves this using a dasymetric mapping approach - using additional data to restrict the redistribution of the population to specific areas, i.e. residential buildings.
Geodemographics for Marketers is being
published by Kogan Page early in 2016.The book has been written to provide marketing
professionals and students with the know-how to leverage geodemographics
techniques, and demonstrates their many applications and benefits. The author is Barry Leventhal, and the book
includes articles and case studies contributed by a wide range of experts.
The investment of time
and resources in a national census, or similar system of data collection, can
only be justified if the results are accessible to users and meet their needs.
This involves wide consultation in accordance with the principles and practices
set out in the Code of Practice for Official Statistics. This is the home page for consultations on
the 2021 Census.
Are you a user of Census data? Or do you think Census data could be useful to you?
On 22 July 2-5pm, the Royal Statistical Society is hosting an event 'Planning the 2021 Census: Recent Developments', at which ONS representatives and Chris Skinner (LSE) will talk about the work so far on the next Census. It will include talks on the current topic consultation and the use of administrative data.
The event provides the opportunity to be involved in the planning early on, and consider what you need from the next Census outputs.
You can see the agenda here: http://www.statslife.org.uk/events/eventdetail/475/-/planning-the-2021-census-recent-developments
If you'd like to attend, email email@example.com to book your place.
paper provides the initial view of
the Office for National Statistics (ONS)
on the content of the 2021 Census
questionnaire for England and Wales. It
forms the starting point of consultation on
the topics and sub-topics to be
included. The initial view is informed by
evaluation of the success of the 2011
topics and questions, as well as
evidence about user requirements from recent
consultations for Beyond 2011. It is
also informed by ONS's current understanding of
affordability and by its aspirations for
using administrative data.
UK Statistics Authority has a statutory function under section 8 of the
Statistics and Registration Service Act 2007 to monitor the production and
publication of official statistics. The Authority will publish reports under
this section of the Act and will issue these as "Authority Monitoring
Reports". They will be produced with the involvement of external experts
and user interests, and will reflect the independent views of the Authority.
Andy Dickinson, teaches Digital and Online Journalism at the Divison of Journalism at the University of Central Lancashire and is part of the Media Innovation Studio team. He wrote a blog summarising his thoughts on what the general election results will mean for us in terms of OpenData.
Now that nearly all census outputs are out and
being used, the UK Data Service are hosting a two day conference to celebrate the UK censuses. This conference is scheduled for the 16th - 17th of July 2015, and will be held at the Humanities Bridgeford Street
Building, University of Manchester.
The conference will include three plenary
The value of the Census
Making use of the Census
Challenges for the future
Danny Dorling and Bethan Thomas
Office for National Statistics
After three decades without a Population
and Housing Census, a census enumeration was conducted in the Republic of the
Union of Myanmar from 30 March to 10 April 2014. To support transparency of the
census and to better understand the way data was collected, the Ministry of
Immigration and Population invited an independent Census Observation Mission to
observe the census. The Mission was made up of 47 experienced observers; 23
international and 24 Myanmar nationals. These experts were statisticians,
census experts, demographers or social scientists. This document is a summary
of the findings of the Observation Mission. Additional information, not from
the Observation Mission, has been included to provide readers with a clearer
understanding of information that may need further explanation.
report provides a brief overview of the implementation of the 2010 World
Population and Housing Census Programme, including information on national
participation in the 2010 census round, successes and challenges and a summary
of activities carried out by the Statistics Division of the Department of
Economic and Social Affairs in support of the 2010 World Programme. The report
also describes the preparatory activities for the 2020 World Population and
Housing Census Programme. The Statistical Commission is invited to express its
views regarding the information provided in the report. In particular, the
Commission is invited to approve the draft resolution on the 2020 World
Programme, review and endorse the draft Principles and Recommendations for
Population and Housing Censuses: the 2020 Round, Revision 3, and provide
guidance on the proposed activities of the Division for the 2020 World
The website of the 2010 World Population
and Housing Census Programme facilitates the international exchange and sharing
of knowledge and information on census taking, as well as provision of guidance
to countries, and monitoring progress on the implementation of the
Programme. The website presents
practical information and guidelines on census methodology and on best
practices to help countries plan and carry out a census, as well as an
up-to-date account of national census taking activities and of provision of
support to countries in the implementation of the census round. The website
also provides information on national, regional and international activities
related to the 2010 World Programme. A major component of the website is the
census knowledge base that is a repository of documents on census methodology,
including reports on what has been done by countries during their censuses.
‘portal' featuring geodemographic products created with open methods, data and
tools by researchers from the University of Liverpool and University
College London. The
products and the underlying data are free to download and use, including
commercial applications. The researchers add that methods are
transparent and reproducible, with products supported by an
active user community, and available to modify and augment. The
website currently features the UK Output Area Classification (OAC) built in
partnership with the ONS and created entirely from the 2011 census
data, the bespoke London Output Area Classification (LOAC), and the
Temporal Output Area Classification (TOAC) exploring geodemographic change
between 2001 and 2011. A linked website has interactive and searchable maps
of each classification down to very local levels.
Nigel Shadbolt, Professor of Artificial Intelligence at Southampton University,
believes in the power of open data. With Sir Tim Berners-Lee he persuaded two
UK Prime Ministers of the importance of letting us all get our hands on
information that's been collected about us by the government and other
organisations. But, this has brought him into conflict with people who think
there's money to be made from this data. And open data raises issues of
privacy. Nigel Shadbolt talks to Jim al-Khalili about how a degree in psychology
and philosophy led to a career researching artificial intelligence and a
passion for open
Lab of the University of Tartu, Estonia focuses on various research topics such
as activity spaces, travel behaviour, tourism, segregation, ICT use and
environmental impacts for pursuing a deeper understanding of spatial
mobility. In order to gain the best possible
understanding of spatial mobility the research group applies both quantitative
and qualitative data. One of its main interests lies with the novel methodology
based on mobile telephone use, and active and passive mobile positioning data.
The latter enables researchers to give a more comprehensive insight into
individuals’ activity spaces and spatiotemporal regularities than most of the
traditional data sources.
ONS are reviewing the criteria and
process used to grant Approved Researcher status, and the safeguards used to
ensure the confidentiality of all personal data held by ONS, and are proposing
a number of possible changes, which would address issues already identified. This is being carried out through a public
The Administrative Data Research Centre for England is led by the University of Southampton, and runs in collaboration with University College London, the London School of Hygiene and Tropical Medicine, the Institute for Fiscal Studies and the Office for National Statistics. The Centre combines exceptional physical facilities with high performance computer platforms to provide knowledge and evidence to help develop, implement and evaluate public policy and future research
New Techniques and Technologies for
Statistics (NTTS) is an international biennial scientific conference series,
organised by Eurostat, on new techniques and methods for official statistics,
and the impact of new technologies on statistical collection, production and
dissemination systems. The purpose of
the conference is both to allow the presentation of results from currently ongoing
research and innovation projects in official statistics, and to stimulate and
facilitate the preparation of new innovative projects (by encouraging the
exchange of views and co-operation between researchers - including the possible
building of research consortia) with the aim of enhancing the quality and
usefulness of official statistics and to prepare activities related to research
in statistics within the European Framework Programme for Research and
Development (Horizon 2020).
On 8th January 2015, the ONS held an Outputs Evaluation workshop to collect feedback from users on the 2011 Census.
MRS CGG Chairman, Barry Leventhal, with input from members of the Association of Census Distributors and the Demographics User Group, presented the business sector's views on outputs from the 2011 Census. here.
This January 2015 paper sets out the research strategy and questions for the design and prototyping phase of the Beyond 2011 programme in 2015 and 2016, a high-level testing strategy for this period, and the main testing phase of the programme in 2017.
This paper from ONS details the
recommendations for the 2021 census:
• An online census of all households and
communal establishments in England and Wales in 2021 as a modern successor to
the traditional, paper-based decennial census. ONS recognises that special care
would need to be taken to support those who are unable to complete the census
• Increased use of administrative data and
surveys in order to enhance the statistics from the 2021 Census and improve
annual statistics between censuses