Stats NZ has a new website.

For new releases go to

www.stats.govt.nz

As we transition to our new site, you'll still find some Stats NZ information here on this archive site.

  • Share this page to Facebook
  • Share this page to Twitter
  • Share this page to Google+
Non-Sampling Error and the Inference Phase of the Survey Process

Introduction

The inference phase is the third major phase to target for the control and minimisation of non-sampling error. The sources of error in this phase occur in the process of turning raw data into finished datasets available for publication and further analysis. Errors introduced in this area can still have a significant negative impact on the quality of data provided for analysis. Therefore, despite the methods implemented to control non-sampling error in the first two phases, we must continue to work hard to control the non-sampling error in this phase as well. This will allow us to build on the successes of the first two phases and produce final datasets of very high quality.

The major areas of non-sampling error concern in the inference phase

Non-response, data adjustment and misinterpretation.

Select an area of interest from the three areas below to find more detailed information on the sources of non-sampling error. Learn why are they can be a problem, what their impact is, what Statistics New Zealand currently does to minimise the associated error and what we will be doing in the future. The sources of error encountered in each area are presented according to their priority.

top

Non-response

Non-response is quite literally when a selected business does not return our survey form. There are two main types of non-response, unit non-response when the business does not return the questionnaire at all or returns it too late, and item non-response when only some of the questions are answered. Non-response is always a problem in economic surveys and while we would like to achieve a 100 percent response rate, the contact and follow-up is done with a constrained budget and timeframe. A higher response rate could often be achieved with more money and time, although it is probably unrealistic to ever expect a 100 percent response rate.

A more detailed explanation of non-response is given in non-response as a systematic error. Essentially though, if the non-respondent's values are quite different, for example much higher or much lower, than what the respondents gave, our final estimated value is not going to be correct. This source of error must be minimised since it directly effects just how good our final estimate is, and the larger the non-response the larger the bias could be. The impact of non-response therefore depends on two things, the amount of non-response and the difference in characteristics between those who did respond and those who did not respond. The difficulty is that we can not measure just how different the non-respondent's values would have been. If however we can minimise the number of non-respondents, any bias that does exist will have less effect.

Non-response is a very good example of the type of error that can be reduced in the initial design phase. Response rates can be increased by making it easier to respond. For example, by improving our Business Frame information so the questionnaire goes to the right person, by improving questionnaire design and the ways in which businesses can respond and by reducing the overall load placed on businesses. There are still other ways we can address non-response directly as and once it occurs, and these are discussed in detail via the non-response link below.

More detailed information on the sources of error from Non-response.

top

Data adjustment

There are a variety of in-house unit record adjustments made by Statistics New Zealand including trading day adjustment, GST adjustment and adjustment of the time period.

Some adjustments, such as a GST adjustment, involve removing the GST component from a reported value when we want the final values to be GST exclusive. Another type of adjustment that is made to the overall data (not at unit record level) is seasonal adjustment, which involves identifying seasonal components and removing them to find the underlying trend. It is very important to note that we make these adjustments in an open and transparent way with the sole intention of improving the final data quality.

More detailed information on the sources of error from Data adjustment.

top

Misinterpretation

The misinterpretation of results can occur if survey output users are not aware of certain factors that influence the characteristics of the variables under investigation. All users should be aware of the methodology of any given survey, and at the very least, should know what the survey limitations are. This will allow informed judgements to be made when using the data. Potentially, misinterpretation could also occur if users were presented with inaccurate graphs or insufficient documentation.

This area has not had a great deal of attention traditionally, especially in comparison with the other sources of non-sampling error identified in previous sections. However, there has been a move to better understand user misinterpretation. Quality initiatives are being undertaken to ensure that the users of our data gain the complete picture and do fully understand our survey outputs. While this source of error does not directly impact on the output statistics, it is a source of error that affects the interpretation of
Statistics New Zealand estimates, and has therefore been included as the final part of the survey process. Statistics New Zealand is a national statistical agency, and aims to provide for informed decision-making, so we do need to understand and control this source of error too. In order to achieve this, users of the data need to know how it was obtained, how the results are derived and what limitations, if any, exist in the data.

More detailed information on the sources of error from Misinterpretation

  • Share this page to Facebook
  • Share this page to Twitter
  • Share this page to Google+
Top
  • Share this page to Facebook
  • Share this page to Twitter
  • Share this page to Google+