Stats NZ has a new website.

For new releases go to

www.stats.govt.nz

As we transition to our new site, you'll still find some Stats NZ information here on this archive site.

  • Share this page to Facebook
  • Share this page to Twitter
  • Share this page to Google+
Non-Sampling Error in the Measurement Phase of the Survey Process

Introduction

The measurement phase is the second of the three major phases where we can reduce non-sampling error. The importance of this area could easily be overlooked because it is the most mechanical phase in the survey process. However, all our assumptions and conclusions are based on the data collected and processed during this phase. Therefore the effort we put into controlling the effects of non-sampling error, by ensuring that the responses are accurate and that the data is processed successfully, will have significant benefits in improving overall data quality.

Learn more about the sources of error in these areas

Select an area of interest from the three areas below to find more detailed information on the sources of non-sampling error. Learn why they can be a problem, what their impact is, what Statistics New Zealand currently does to minimise the associated error and what we will be doing in the future. The sources of error encountered in each area are presented according to their priority.

top

Interviewing

Interviews, whether face to face or over the phone, are commonplace for household surveys, but are not generally carried out for our economic surveys. However there is direct personal contact via the telephone for follow-ups and additional data collection, and now all business questionnaires contain a toll free telephone number to ring for assistance. One of the overall objectives and indeed requirements of Statistics New Zealand is that it is impartial in the production of national statistics. It is therefore fundamental that in being as helpful as we possibly can, we never influence the responses of a business. We want the interviewer to avoid asking questions, whether intentionally or not, that are different to what is on the questionnaire.

A respondent should answer the same way irrespective of the interviewer they get, and ultimately regardless of whether they filled in a paper form or were interviewed. This is not currently considered to be a significant source of non-sampling error since our surveys are primarily postal surveys. We have a strong training system in place to ensure interviewers and helpline staff all meet an equally high standard. In addition, this source of non-sampling error is not considered a major issue for the future at this time due to the minimal face to face interviewing carried out for economic surveys. However the potential still exists for this to be a problem, so we are always monitoring our procedures to ensure we maintain our high standards.

More detailed information on the sources of error from Interviewing.

top

Respondent error

The errors made by respondents can be the result of a number of problems, such as misunderstanding and misinterpretation of the questions, or misreporting of information due to perceived privacy issues. Errors can also occur when the information sought relates to a time period long ago, the information is not collected by the business, or is not directly compatible with the collection methods carried out by the business.

It is by being proactive in this area that Statistics New Zealand is able to reduce the associated non-sampling error. We target the components that cause these errors, for example, by improving the questionnaire, reducing respondent load, allowing different ways to respond and educating respondents about the critical importance that we place on confidentiality. We value the time and effort put in by respondents and try to make this process as simple, effective and efficient as possible.

More detailed information on the sources of error from Respondent error

top

Survey processing system

The processing of the survey data that has been collected involves data capture, cleaning and editing. At each of these stages there is the potential to introduce non-sampling error. For example, data may captured incorrectly, classified inaccurately, or mistakes may be made inadvertently when editing the data. Once the data has been entered mistakes will not be picked up if there are inadequate checking and quality management strategies in place. The single most important action to control non-sampling error at this stage is to have an integrated and standardised processing system that is still flexible enough to incorporate the idiosyncracies of our many different surveys.

At Statistics New Zealand we have invested significant resources in this area so that we now have a processing system that is of a very high standard. In addition, we have a substantial array of checks in place to identify and fix most (but not necessarily all) errors. These include comparing values in the current period to data in previous periods to highlight punch errors when a unit is very different from the trend, range checks (no negatives where there should not be), logical checks (do the individual components sum up to the totals?), and having one person responsible for each industry in the survey.

Currently we control these sources of non-sampling error well, and this has significant benefits at later stages. For example, if we pick up unusual or incorrect values at the processing stage, then we can eliminate the need to make revisions to published data due to incorrect values.

More detailed information on the sources of error in the Processing system

  • Share this page to Facebook
  • Share this page to Twitter
  • Share this page to Google+
Top
  • Share this page to Facebook
  • Share this page to Twitter
  • Share this page to Google+