Stats NZ has a new website.

For new releases go to

www.stats.govt.nz

As we transition to our new site, you'll still find some Stats NZ information here on this archive site.

  • Share this page to Facebook
  • Share this page to Twitter
  • Share this page to Google+
Non-Sampling Error and the SNZ Business Frame (BF)

Business Frame

Delays in birthing businesses

New businesses (births) are identified from a client registration file provided each month by the Inland Revenue Department (IRD) (the taxation office in New Zealand). We record new additions and send out our monthly frame update survey, where we also find out more information about the structure of the business. Occasionally births are identified through other sources such as the media, Government Actuary or survey feedback.

This source of error has a high priority within the organisation since big delays in the birth of businesses could have a number of significant impacts. There is a potential for displacement of the recording of the economic effect of one-off events, such as large (for New Zealand) sporting events held here such as the America's Cup sailing regatta. There could also be lags in how quickly we pick up economic growth and subsequent underestimation of level estimates for a particular period.

We have significantly reduced the time taken to add new births, but there is still some
delay present. This is due to late goods and services tax (GST) and pay as you earn (PAYE) IRD registrations, and also the time it takes to record new businesses, post out and receive the results from our initial update survey and then add the businesses to our Business Frame. Recent initiatives, such as incorporating a key word strategy to automatically code the industry of some known non-employing businesses (a single taxi driver), and a new strategy to improve the priority rules we have for birthing, have helped to reduce the size of our update survey. This has further reduced the time taken to update the Business Frame.

In the future we are planning to implement procedures for smarter incorporation of tax data, for example, expanding the use of the key word strategy to automatically code the industry for a wider range of businesses. These procedures, and the continuing development of our birthing rules, will further eliminate the need in a majority of cases for an initial frame update survey to be posted out. There will still be a need for some direct contact with businesses, but the more automated we can make this process, the
quicker we can birth new businesses onto our Business Frame.

top

Delays in ceasing businesses

If a business stops trading then (in general) it will be 'ceased' on our live Business Frame and will no longer be eligible to be surveyed. We primarily identify ceases in two ways, either from our Annual Frame Update Survey (AFUS) when a business tells us it has ceased trading, or when a business de-registers for GST which is identified from changes to the IRD file. Delays occur because of the time it takes to send out, receive and then process the AFUS, as well as from a business not responding to the survey. Another delay can occur when a business may have ceased trading but still files GST returns as it sells off its assets, and so this information alone is not enough to cease the business. We also get feedback from our regular surveys, through which a business lets us know it will soon cease trading.

Any significant delays in the ceasing of businesses could lead to an overestimation of level estimates if we are imputing for units that we consider to be non-respondents but which have actually ceased trading. Consequently, there is a potential for displacement of the economic effect of (negative) one-off events or lags in how quickly we pick up economic decline in the economy.

Currently, in addition to the direct identification from GST data or the AFUS, we use a number of fixed rules to determine if a unit is to be ceased on our Business Frame. In certain cases this improves the time it takes to cease a business by eliminating the need to send out an AFUS survey. For example, certain enterprises are ceased if their GST returns are zero for an entire year (although a record is kept in case the business does become active again).

top

Industries not actively updated

Traditionally, some industries have not been updated as regularly as others on the Business Frame. For example, for agriculture surveys we have used other sources of information to create an up to date frame specifically for these surveys (this can potentially have its own problems; see frames created using other sources). Agriculture is not part of our other industry-wide surveys such as the Annual Enterprise Survey, and since it was expected that a specific frame would be created each year, it was also not updated directly in the Annual Frame Update Survey (AFUS). There are some industries where there are a number of non-employing businesses, and some industries that are GST exempt, so these are not updated as well as we would like. Finally, there is a known but as yet unmeasureable undercoverage of businesses that evade tax, the 'black economy'.

The lack of updating for certain industries may then cause problems when we do run industry-wide surveys using our Business Frame. There is a potential to bias overall economic measures towards trends in what is happening in the more well covered and updated industries, and potentially underestimate what occurs in the more outdated industries.

Our current aim is to ensure that all industries are treated similarly, and this will be achieved to a large extent by changing the focus for Business Frame updating. Two major initiatives are under way to ensure that how often a business is updated, and how often this occurs, will depend on the economic significance of the business (determined by GST turnover), not what industry the business is in.

The first initiative is the smarter and more complete use of tax data, as a frame universally sourced from tax data will ensure that all industries are treated the same. Building on this, the second initiative will result in an improvement in the way we target our AFUS. The most economically significant businesses, as determined by GST turnover, will be surveyed every year while the less economically significant businesses will either be surveyed over a rotating three-year cycle or be covered/updated by using their relevant tax data. The largest and most economically significant businesses will get the greatest attention, but we will still ensure coverage of smaller business through tax data - and this will all happen regardless of the industry a business is in.

top

Inaccurate classification

The classification of business information relates to items such as industry, institutional sector, size (GST and/or number of employees), storetype, and business type. These items are often used in the survey design for stratification and selecting the sample for a survey. The impact of inaccurate classification is that it could potentially lead to units being placed in the wrong stratum and contributing to the wrong industry (or even survey), units being sent the wrong questionnaire, and/or imputation being based on incorrect information. If a large business is misclassified, this can add volatility to the outputs, and the business will often need to be specially treated. Finally, if corrections have to be made due to fixes of inaccurate classification, this would result in revisions to previously published statistics.

Currently the Business Frame maintenance uses the GST registration information and the Annual Frame Update Survey (AFUS), as well as respondent details, to assign a business its initial size and classification information. These provide a good level of information, but in addition, regular sample monitoring processes ensure most outlier effects are addressed. Regular and timely sample redesigns help ensure that any incorrectly stratified sample units can be re-stratified.

top

Outdated information

Outdated or inaccurate information for non-classification variables such as business structure, contact name, phone number and address have the potential to be a significant problem. The problem can be caused by the delay between changes to businesses and the time at which changes are made to the Business Frame. Errors can also be made which would result in the real world information being different to the information on the Business Frame. Not identifying a birth or a death is an extreme example of outdated information.

The impact of outdated information includes potential difficulty when contacting the business by phone, for example, not being able to contact the right person. There can also be problems with apportioning and imputation if the number of geographic units that make up an enterprise are unknown. Further, being unable to contact the business due to incorrect details adds to the amount of non-response and therefore imputation required.

Current sample maintenance procedures aim to update the information as quickly as possible within set standards and guidelines. The regular updating of the Business Frame will help to reduce this source of error. We currently update almost all businesses once a year either through IRD tax data or the Annual Frame Update Survey (AFUS), and also incorporate feedback from surveys and other sources whenever possible to try keep information up to date.

top

Data collected using external sources

We are becoming more reliant on external sources to supplement, and in some cases replace, the need for us to survey businesses for data. There are advantages to using external sources for acquiring data, such as reducing costs, but perhaps the most significant is the reduction of the load on respondents by not having to ask them to fill in a survey.

The reliance on external sources can introduce new errors if not managed well. For example, external sources may use different structures for recording businesses to those used by Statistics New Zealand, use different definitions to determine what information is contained in a given group, not collect information over certain ranges, or have data that contains some errors. This adds some cost as we regroup data, but also, if differences/errors were not identified, could add variability to our survey statistics and outputs.

We are currently working to increase the use of externally sourced data in our economic surveys, particularly the use of tax data already collected by the New Zealand IRD. We are developing strong relationships with our partner organisations, giving us better control of our use of externally sourced data. In this way we can realise all the advantages of using this data, while making sure that we do not introduce new sources of error into our survey process. This is still a fairly new area for us, and in the future we will be working to more fully understand the relationships between the data sources and the structures of our Business Frame, to ensure that we maximise the benefit of using external sources of data.

top

Frames created using other sources

The 1999 Agriculture Production Survey used a frame other than our Business Frame, while the 2000 survey used a combination of our Business Frame and other sources to create the survey frame. Specialist surveys run by Statistics New Zealand, such as the Biotechnology Survey, often use frames specifically created for the survey from other information sources. This has many benefits, such as ensuring that the population of interest is fully covered by the frame, and conversely, that the frame does not include businesses which are not in the population, for example, are not involved in biotechnology or not involved in agriculture.

Relying on other sources for information to construct the frame does introduce new risks, as quality can be difficult to determine, and there may still be problems of duplications, undercoverage and incorrect information. It is also more difficult to manage respondent load, as we can not monitor and reduce the overlap of businesses being selected into multiple surveys as we do when using only our Business Frame.

A lot of work is carried out to minimise the error in frames created from other sources.
These include validation studies, to investigate the quality (coverage and accuracy) of external information sources, and specific frame matching exercises where we compare, when possible, the likely frame information with partial information obtained from a number of other sources.

top

Inappropriate link of collection unit to statistical unit

The statistical unit is the business or part of the business that we require data from, and actually select to sample. This might be the geographic unit (physical location) of which there can be many across New Zealand, for example, each shop in a chain of retail stores. The collection unit is the unit that we actually have contact with and send the questionnaire(s) to. This may be, for example, the head office of the chain of retail stores.

Depending on the systems in place in the business, the enterprise may not be able to provide information for all the stores at the levels that we require. For example, it may only be able to provide aggregated data, not the data for each type of store in each region. If we are producing estimates at these levels, then we may have to divide in some way the total value across the selected stores. This is done in standard and documented ways, but is still unlikely to be an exact representation of the true situation.

Currently this is not a significant problem, but the potential exists for this to cause some problems in the collection of data. Our current procedure of having all businesses recorded on the Business Frame with the same structural hierarchy for consistency, and using consistent methods for dealing with data provided at different levels, will ensure that when this problem occurs, the effect should be small. These standards also make it easy to understand where changes had to be made, and therefore recognise where, if at all, any limitations exist in the data.

 

  • Share this page to Facebook
  • Share this page to Twitter
  • Share this page to Google+
Top
  • Share this page to Facebook
  • Share this page to Twitter
  • Share this page to Google+