Measuring regional labour markets in South Africa: How robust are sub-national estimates from the Quarterly Labour Force Survey?

ABSTRACT

This article explores the limitations of using data from the Quarterly Labour Force Survey (QLFS) to estimate robust sub-national labour market indicators in South Africa. The precision of labour market indices in the QLFS is very sensitive to which geographic scale is examined – national, provincial or metro – and in each instance the size of the population of the province or metro. The results show that errors from sampling at regional and city levels may be prohibitively large although in some instances broad patterns between regions can be identified notwithstanding a fairly blunt instrument. The findings highlight the imperative to be transparent about sampling errors and to foster sensitivity within government, business and the public in general. This exercise is instructive for generating other regional socio-economic indicators that are based upon similar household sample surveys such as the General Household Survey, Living Conditions Survey and Income and Expenditure Survey.

KEYWORDS:

• Labour market
• regional development
• sub-national indicators
• South Africa
• quarterly labour force survey

JEL CODES:

• C83
• R12
• J21

1. Introduction

Sustainable Development Goal 11 (creating safe, inclusive, safe, resilient and sustainable cities) commits governments with the international research community to measure and monitor progress across cities and regions. Improving governance of Africa’s rapidly expanding cities depends on policy that is responsive to the local context (Collier & Venebles, 2017). There is growing momentum behind a set of International Organisation for Standardisation (ISO) metrics for cities that will facilitate more robust cross-country comparisons (World Council on City Data, 2017). However accurately assembling data at a local and regional scale is a challenge for many African countries and cities (United Nations, 2017; van Belle, 2017).

Sample surveys are a common tool for collecting information on many different issues over regular periods of time and at reduced cost (Rao, 2017). The national statistical agency in South Africa, Statistics South Africa (StatsSA), undertakes a wide array of regular national household sample surveys that are designed to be representative at provincial (regional) and metropolitan (city) levels. The inherent limitation in measuring local progress using sample surveys is that smaller geographies imply a smaller sample size and therefore a loss of accuracy in measurement.¹

This paper explores the limitations of using the Quarterly Labour Force Survey (QLFS) to generate provincial and metropolitan (metro) level labour market indicators in South Africa. The analysis focuses on measuring changes in the rate of unemployment for the period 2008 to 2014, but the findings are instructive for other socio-economic indicators (such as poverty, inequality and living conditions) derived from StatsSA surveys which share a similar sample size, sampling frame and survey methodology such as the General Household Survey (GHS), Living Conditions Survey (LCS) and Income and Expenditure Survey (IES). We show that margin of error in estimating the rate of unemployment for many regions and metros is relatively large and therefore the scope for sub-national analysis is fairly limited in the absence of mitigating interventions such as small-area modelling.

The paper begins by reviewing the literature and moves on to review the data and explain the methods. Section 4 presents the extent of errors in sample measurement at regional and city levels by estimating the unemployment rate and other labour market indices for the period 2008 to 2014. Section 5 explores how labour market information is disseminated by the media, with attention on the packaging of the results by StatsSA. The paper concludes with a summary of the key findings and some reflections for scholars and practitioners.

2. Literature

The challenge of reliably estimating statistics for smaller sub-populations (or domains) is well documented in the literature on ‘small area estimation’ (see Rao & Molina, 2015; Rao, 2017). There is an inverse relationship between the sample size and the margin of error which means that smaller geographies (or any smaller domain) may suffer from prohibitively large standard errors (Hanneman et al., 2012; Rao, 2017). There are variety of techniques that attempt to mitigate against imprecision in the measurement of smaller samples (Ghosh & Rao, 1994; Singh et al., 1994; Jiang & Lahiri, 2006; Pfeffermann, 2013; Rao, 2017). One approach is to make use of indirect estimators which ‘borrow’ strength from related domains to increase precision. These are based on either implicit (such as ‘synthetic’ or ‘composite’ estimators) or explicit models (such as ‘hierarchical Bayes’ or ‘empirical best’ estimators) that are related to the sample of interest through supplementary administrative or other survey data (see Rao & Molina (2015) for an overview).

Most research focused on the small area level in South Africa rely on direct estimators from the Population Census or the larger sample Community Survey to measure income poverty or composite indicators of well-being (Rossouw & Naude, 2008; Noble & Wright, 2013; Sartorius & Sartorius, 2016; Burget et al., 2017; David et al., 2018). Alderman et al. (2002) use survey-to-census imputation models to predict poverty rates for magisterial districts in South Africa using the 1996 Population Census combined with IES/October Household Survey (OHS) data. Demombynes & Ozler (2005) apply the same approach to explore income inequality and rates of crime at the neighbourhood level. Both studies focus on improving the Census data as opposed to bolstering the strength of smaller sample surveys. Morudu (2016) advocates applying Ziph's rule (which is an empirical occurrence, rather than a theoretical model) to predict population and GDP estimates for years in between the Population Census.

Unfortuantely, the Population Census is not comprehensive in assessing labour market conditions and the questionnaire design has changed in ways which make comparisons over time problematic (Klaasen & Woolard, 1999; Kingdon & Knight, 2001, 2004; Casale et al., 2004). The 1994–99 OHS, 2000–07 Labour Force Survey (LFS), and 2008-onwards QLFS are the principal vehicles for collecting labour market information in South Africa on a regular basis for the respective periods, although there are comparability issues in methodology, questionnaire design and sampling frame between them (Casale et al., 2004; Burger & Yu, 2006; Yu, 2007; 2009). Some studies of the South African labour market include sub-national estimates or controls for the labour market, typically between urban and rural areas, but few focus on representivity at the local level between provinces or metros (Kingdon & Knight, 2006; Festus et al., 2016; Bhorat & Khan, 2018). Statistics South Africa regularly report on provincial labour market indicators with each release of the QLFS (StatsSA, 2015).

Demand for sub-national data in South Africa is evident from the presence of a commercial market for modelled regional data. Reliance upon commercial sources is ubiquitous within local, provincial and national government and it is not uncommon for such commercial data to be cited in academic research, even though these for-profit companies do not release their methodology (Rogerson, 2010; Morudu, 2016). The reliability of commercially curated data is ultimately dependent on the quality of their data modelling (which is not released) and upon the quality of their secondary sources (which presumably depends heavily on StatsSA surveys). Administrative records can provide detailed information at the local level but suffer from gaps in coverage, restrictions in access, and are in many cases not sufficient codified (Woolfrey, 2014).

3. Data and methods

The purpose of this article is to interrogate the usefulness of drawing on data from the QLFS to generate reliable regional labour market statistics. This is intended to be instructive for estimating other socio-economic indicators that can be generated from similar household surveys and which share the same methods and sampling frame such as the GHS, LCS and IES.

StatsSA household sample surveys are based upon a master sample from the Population Census which is designed to be representative at national, provincial, and metropolitan municipality (metro) levels. Sample weights adjust for original selection probabilities; sub-sampled, segmented or excluded primary sampling units; weight trimming; non-response; and final weights are benchmarked to demographic population estimates (StatsSA, 2015). Imputation is also applied to instances of item non-response and edit failures. The QLFS sample is designed as a rotating panel with 1/4 of the sample being replaced after each round. This has the benefit of minimising covariance between panel observations although the sample is fully replaced after one year (Nijman et al., 1991). There are approximately 30 000 households and 85 000 individuals in each QLFS. South Africa’s population was roughly 54 million in 2014 hence the QLFS sample size is roughly 0.16% of the population.

South Africa has one of the highest rates of unemployment in the world and changes in labour market conditions are under regular scrutiny in the local press. We analyse patterns and changes in the expanded rate of unemployment at the national, provincial and metropolitan scales for the period 2008 to 2014 using the fourth quarter from each year. This avoids the need to adjust for seasonality and the period aligns to the same Census 2011 master sample frame.

The simplest method in assessing the accuracy of regional estimates is to generate standard errors for each statistic. Standard errors can be expressed as confidence intervals – the bandwidth within which the true population estimate will fall at a certain level of confidence – and displayed visually in the figures as error bars. Error bars are shown at a level of ‘twice the standard error’ which corresponds to a 95% level of confidence. This provides the reader with an easy interpretation of where the true population statistic is likely to fall 95 percent of the time (Hanneman et al., 2012).

We can extend the intuition of the confidence intervals to calculate how much the unemployment rate would need to change on average for the shift to be statistically significant between two periods. A common misconception is that if the confidence intervals of two independent means overlap by even a small degree the null hypothesis (that the estimates are the same) cannot be rejected (Schenker & Gentleman, 2001). As discussed by Cumming & Finch (2005), a better rule of thumb is that the error bars (lower and upper tails respectively) should overlap by roughly no more than half (or 0.58 to be more precise) to be confident that there is a statistically significant difference between the estimates (see Figure 1). We can apply this rule of thumb to the 95% confidence intervals for the rate of unemployment and estimate the size of the change that is required for the unemployment rate to change significantly.² In other words, this calculation represents the average proportional shift that is needed such that the (lower and upper) error bars do not overlap by our rule of thumb of 0.5. This allows us to produce an estimate for the average size of a change that is needed for unemployment to be judged to have changed significantly for each geography at a 95% level of confidence.

Figure 1. Overlapping confidence intervals. Source: Taken from Cumming and Finch (2005).

4. Confidence in labour market indicators from the QLFS

Figure 2 illustrates changes in the expanded rate of unemployment over the period 2008 to 2014.³ The y-axis for each geography is fixed at 15 percentage points so that comparisons across regions can be made easily by eye. Dashed lines are fitted where the change between two consecutive years is not statistically significant at a 95% level of confidence. The provinces (Gauteng and the Western Cape) and metros (and Cape Town, Johannesburg and eThekwini) depicted in the figure are the largest in South Africa (with relatively smaller sampling errors) and depict an upper-bound to the level of accuracy of sub-national estimates.

Figure 2. Rates of unemployment by selected provinces and metros, QLFS 2008–2014. Source: Quarterly labour force survey; own estimates.

The figure shows that the South African economy has stagnated since the 2008 global recession, with the rate of unemployment rising from 28.7% in 2008 to 34.6% by the end of the period. How robust is this apparent trend of rising unemployment both at national and sub-national levels?

At the national level, the error bars in the figure are fairly small meaning that the rate of unemployment for South Africa can be measured with reasonable precision, with changes greater than 1 percentage point typically statistically significant. In comparison, the error bars in Gauteng and the Western Cape are roughly double the size than at national and roughly three times the size in the Cape Town and Johannesburg metros. In the case of eThekwini, the level of accuracy has deteriorated so much that the error bars are nearly four times the size of the national-level. Practically, this makes identifying sub-national trends difficult, and if anything, unemployment for these regions follows the same pattern as South Africa (with the possible exception of eThekwini). Point estimates for the provinces and metros show an uncomfortable amount of year-on-year fluctuation – a telling sign of the loss of precision in measuring unemployment at sub-national levels.

Table 1 lists various measures of the average size of the sampling error for the rate of unemployment for all provinces and metros in South Africa. The first column shows the average number of observations per region over the period. The second column calculates the average standard error and the third column reflects this as a factor of the national estimate. As seen in the table, the standard error for measuring the rate of unemployment ranges from a low of 0.42 for South Africa and a high of 3.91 for Tshwane metro. This pattern naturally corresponds to the average number of observations, with larger samples having lower standard errors and vice versa. As a factor of the standard error of South Africa, provincial standard errors are at least double to triple the size (and 4.7 times the size in the case of the Northern Cape). The metros show greater variation amongst each other with the standard error ranging from 2.7 times the size of the national sample error for Cape Town, and more than 7 times the size for Buffalo City or Tshwane.

Table 1. Standard errors for the unemployment rate at national, provincial and metro levels, QLFS 2008 to 2014.

		Average no of obs	Average standard error (SE)	SE as factor of RSA	Bandwidth of confidence interval @95%	% point change to be significant*
	South Africa	33168	0.42	n/a	1.66	1.2
Provincial	Gauteng	6217	0.86	2.0	3.35	2.4
	Western Cape	4377	0.86	2.0	3.69	2.6
	KwaZulu-Natal	5186	1.09	2.6	4.26	3.0
	Limpopo	3393	1.42	3.4	5.55	3.9
	Free State	3028	1.30	3.1	5.10	3.6
	Eastern Cape	3275	1.35	3.2	5.28	3.8
	Mpumalanga	3132	1.42	3.4	5.56	4.0
	North West	3009	1.35	3.2	5.27	3.7
	Northern Cape	1552	1.99	4.7	7.79	5.5
Metro	Cape Town	2928	1.13	2.7	4.44	3.2
	Jo'burg	2522	1.29	3.0	5.06	3.6
	Ethekwini	2042	1.63	3.9	6.41	4.6
	Ekurhuleni	1511	1.76	4.2	6.91	4.9
	NMBM	899	2.44	5.8	9.54	6.8
	Mangaung	872	2.12	5.0	8.31	5.9
	Buffalo City	534	2.99	7.1	11.69	8.3
	Tshwane	314	3.91	9.3	15.22	10.8

Source: Quarterly Labour Force Survey; own estimates.

*This is the predicted percentage point change required for such a change to be statistically significant at a 95% level of confidence by applying the rule of thumb (see explanation in section 3).

What does this mean about making accurate judgements of the rate of unemployment at different geographic-scales? How much does the rate of unemployment need to change for us to be confident about this movement? Table 1 column 4 calculates the average bandwidth of the confidence interval for the rate of unemployment in each region at a 95% level of confidence. Applying our rule of thumb to the size of the confidence interval shows just how much the unemployment rate needs to change on average to be statistically significant at a 95% level of confidence (see discussion in section 3). Column 5 of Table 1 reveals that the rate of unemployment for South Africa would need to increase/decrease by at least one percentage point for the change to be statistically significant. In Gauteng (the province with the smallest standard error) the unemployment rate must fluctuate by more than two percentage points for anything to be confidently concluded. For the Gauteng metros of Johannesburg and Ekurhuleni this must be greater than 3.5 percentage points and more than 10 percentage points in the case of Tshwane. Two out of the nine provinces and six out of the eight metros need a change of greater than 4 percentage points to conclude anything reliably about fluctuations in the rate of unemployment.

In summary, without due consideration for fairly large errors in sampling at the sub-national level, an estimated change in the rate of unemployment across time may well be spurious. Users of the data need to be circumspect, particularly for smaller provinces and metros. But it would be premature to conclude that there is no merit in trying to measure statistics at the sub-national level using the QLFS. Larger differences between any two point estimates may allow for robust comparisons even where measurement is fairly imprecise. For example, comparisons across heterogeneous groups or involving longer time-horizons may facilitate robust conclusions.

Figure 3 demonstrates how clear patterns can be identified by making comparisons across regions (instead of time) which are significant at a 95% level of confidence despite the large error bars. The Western Cape has a significantly lower level of unemployment than other provinces at roughly 25%, followed by Gauteng at just under 30%. The remaining provinces have rates of unemployment ranging between 35% and 45%, but the error bars are staggered such that any distinct differences between them are inconclusive. Examining the metros shows that Cape Town, Johannesburg and eThekwini have significantly lower levels of unemployment in comparison to other metros at roughly 25%. Unfortunately, the error bars for the remaining metros are prohibitively large and not much further can be concluded. Hence, where there are large differences in the point estimates between regions, this allows for some meaningful comparisons notwithstanding the lack of accuracy.

Figure 3. Rates of unemployment across provinces and metros, QLFS 2014 Q4. Source: Quarterly labour force survey 2014 Q4; own estimates.

There may be other descriptive filters that are applied on top of the rate of unemployment at a regional level. For example, researchers may be interested in the rate of unemployment by race or by education in each province. This constrains the sample even further with obvious implications for the size of the sample error. Figure 4 demonstrates the effect by showing the rate of unemployment for South Africa and Gauteng broken down by race and over time⁴. At national level, there is a clear distinction between race groups (ranked from lowest to highest as White, Indian, Coloured and African), and a general trend of rising unemployment over time (with the exception of the Indian population). If the same exercise is done for Gauteng – the province with the largest regional sample – the estimates become very blunted. Identifying significant differences in the rate of unemployment between African and Coloured populations in Gauteng is troublesome for certain years and trends over time are difficult to identify, although the general pattern resembles the national picture. Applying filters at the regional level will further divide an already limited sample, compounding the impact of errors from sampling.

Figure 4. Rates of unemployment by race in South Africa and Gauteng, QLFS 2008–2014. Source: Quarterly labour force survey; own estimates.

Figure 5 displays another important labour market indicator: the distribution of employees per industry. Some meaningful comparisons between regions can be made although these are with broad brush strokes and beyond the reach of smaller metros. At a national level a robust structural pattern of employment by sector can be identified. The Eastern Cape reflects a similar pattern to the national distribution although the mining and financial services sectors contribute proportionately less to total employment in the province whilst community services contribute proportionately more. This makes sense considering large under-developed and rural parts of the Eastern Cape and the lack of a mining economy in the region. The eThekwini metro is an important logistics base for South Africa and has a relative strength in manufacturing. Examining the pattern of employment in eThekwini confirms this narrative showing comparatively lower proportions of employees in agriculture and community services, and higher proportions in manufacturing and transport. When it comes to employment by sector within the Tshwane metro (the smallest metro), the standard errors are simply too large to draw any useful conclusions whatsoever.

Figure 5. Employment shares by broad industry category, QLFS 2014 Q4. Source: Quarterly labour force survey 2014 Q4; own estimates.

In summary, the reliability of labour market indices in the QLFS is very sensitive to which geographic scale is examined – national, provincial or metro – and in each instance the size of the population of the province or metro. The QLFS is reasonably accurate in measuring changes in the rate of unemployment at the national level but unemployment would need to change by more than 2.4 percentage points for Gauteng and the Western Cape and more than 4 percentage points for most provinces and metros for anything legitimate to be concluded. Reliable descriptive patterns can be identified at the regional scale in some instances such as unemployment between regions (rather than over time) or differences in the distribution of employment by sector. However these are cases where the point estimates are significantly different between regions and the survey instrument is understandably blunt.

5. Reporting on QLFS results by statsSa and distribution by the media

The findings demonstrate the need to be upfront about the accuracy of the data. For academic purposes it is imperative that researchers report on the size of the standard errors when estimating labour market statistics. For non-academic purposes, more should be done to sensitise the media and government officials to the reliability of labour market statistics particularly for smaller regions.

StatsSA release a descriptive report with each QLFS for dissemination to the public which is an important source for responsible reporting. The report includes a lengthy appendix which provides an extensive list of labour market indicators (including regional labour market indices) with accompanying sampling errors for each statistic. Unfortunately, the main findings and analysis make no mention of sampling errors contained in the appendix. The narrative seems to ignore the fact that some of the findings could be the result of errors from sampling.

Table 2 demonstrates how this is potentially problematic by reproducing a summary table of ‘Key Labour Market Statistics’ taken from page v of the StatsSA statistical release for the QLFS 2014 Quarter 4 (StatsSA, 2015). The original table from the StatsSA report includes no indication of errors from sampling (these must be assembled by searching through the appendix). As seen in Table 2, only one of the fourteen labour market indicators changed significantly between 2014 Quarter 3 and 2014 Quarter 4 at a 95% level of confidence, and this is at a national level. If a less stringent level of confidence of 85% is applied then the total number of labour market indicators which are statistically significantly rises from one to five out of fourteen. No caution is given in the written discussion of the results in the StatsSA report.

Table 2. Key labour market indicators; QLFS 2014: Q3 & Q4.

	Jul-Sep 2014	Oct-Dec 2014		Statistically significant change at a level of confidence of:
	Thousands		Absolute change	95%	90%	85%	80%	75%	P-value
Population aged 15–64	35 489	35 643	155						0.70
Labour force	20 268	20 228	−40						0.80
Employed	15 117	15 320	203			✓	✓	✓	0.15
Formal sector (non-agricultural)	10 843	10 911	68						0.60
Informal sector (non-agricultural)	2 407	2 448	41						0.61
Agriculture	686	742	56						0.35
Private households	1 180	1 219	38						0.45
Unemployed	5 151	4 909	−242		✓	✓	✓	✓	0.08
Not economically active	15 221	15 415	194					✓	0.21
Discouraged job-seekers	2 514	2 403	−111						0.37
Other (not economically active)	12 707	13 012	305	✓	✓	✓	✓	✓	0.02
Rates (%)
Unemployment rate	25.4	24.3	−1.1		✓	✓	✓	✓	0.06
Employment/population ratio (absorption rate)	42.6	43.0	0.4						0.33
Labour force particpation rate	57.1	56.8	−0.4						0.41

Source: QLFS 2014: Q3 & Q4; Statistics South Africa (2015).

Figure 6 displays changes in formal-sector employment by industry between Quarter 3 and Quarter 4 2014 and replicates the figure from page x of the StatsSA report, however error bars at a 95% level of confidence are now included. As seen by the size of the error bars, none of the changes in employment numbers are statistically significant (again this is at a national level). If the total magnitude of the change in employment is greater, which is generally the case over a longer time horizon, than the analysis would be more worthwhile.

Figure 6. Change in formal-sector employment by industry, QLFS 2014: Q3 & Q4. Source: QLFS 2014: Q3 & Q4; Statistics South Africa (2015).

Lastly, Table 3 reproduces regional employment numbers between 2013:Q4 and 2014:Q4 taken from page ix of the report but measurements of errors from sampling are now included. The North West experiences a significant jump in the number of employees over the period. Only if we accept a low level of confidence of 70% can we conclude that the Western Cape and the Limpopo also experienced significant changes in their levels of employment. Evidently, some of the discussion contained in the official statistical release of the 2014 Quarter 4 QLFS report is based upon fairly imprecise statistics and the report makes no attempt to sensitise the reader to such problems. Inaccuracies are only amplified when analysed at the regional level.

Table 3. Changes in employment by province, QLFS 2013:Q4 & 2014:Q4.

	Oct-Dec 2013	Oct-Dec 2014		Statistically significant change at a level of confidence of:
	Thousands		Absolute change	95%	90%	85%	80%	75%	70%	P-value
South Africa	15 177	15 320	143							0.33
Western Cape	2 235	2 170	−65						✓	0.28
Eastern Cape	1 332	1 336	4							0.95
Northern Cape	329	320	−9							0.65
Free State	746	772	26							0.34
KwaZulu-Natal	2 527	2 520	−7							0.91
North West	869	948	80	✓	✓	✓	✓	✓	✓	0.02
Gauteng	4 823	4 881	58							0.50
Mpumalanga	1 149	1 138	−11							0.80
Limpopo	1 168	1 235	67						✓	0.27

Source: QLFS 2013:Q4 & 2014:Q4; Statistics South Africa (2015).

Concern over the accuracy of labour market indicators is not an exercise in academic abstraction. Research has found that announcements in the media may influence financial markets and have a real impact on the economy (Dougal et al., 2012; Engelberg & Parsons, 2011; Garcia, 2013; Solomon et al., 2014). Table 4 highlights online coverage by the South African media following the release of the QLFS 2014 Quarter 4 report which was released on the 10th February 2015. The main findings were disseminated widely by all major media houses even though the improvement in the national rate of unemployment was a modest fall of 1 percentage point (which is borderline in terms of statistical significance). In another instance, the unemployment rate in Quarter 3 of 2015 increased by just 0.5 percentage points between Quarter 2 and Quarter 3 (a statistically insignificant change even at a 75% level of confidence) but generated nearly three times the number of media releases on the internet. The point here is that unemployment trends are monitored closely in the South African media with little apparent sensitivity to the reliability of the results. This is problematic in light of the way in which StatsSA currently package their reporting.

Table 4. Internet news headlines by media houses following the release of the QLFS 2014 Q4.

News headline	Source
10-Feb
SA employment stats improve slightly, unemployment still huge issue	All4Women
Unemployment decreases in 4th quarter	SANews.gov.za
Construction industry biggest contributor to y/y increase in employment	SA Construction News
Unemployment rate eases	Moneyweb
South Africa's jobless rate eases to 24.3 percent in Q4 2014	Sharenet
Unemployment falls in fourth quarter	BDlive
Jobless rate eases to 24.3%	News24, Fin24
SA’s jobless rate dips	IOL
South Africa's jobless rate eases to 24.3% in Q4 2014	Engineering News, Polity.org
SA's unemployment rate dips	iAfrica.com
S.Africa's jobless rate eases to 24.3% in Q4 2014	CNBC Africa
Rand extends losses as commodity currencies remain under pressure	BDlive
Unemployment rate in South Africa declines 24.3% in fourth quarter 2014	eProp
11-Feb
Unemployment rate drops to 24.3%	CapeArgus
Survey: Drop in joblessness	The Witness
Unemployment rate dips to 24.3%	Daily News
12-Feb
EC bucks Q4 trend and loses jobs	Daily Dispatch

Source: Meltwater media monitoring database.

6. Concluding remarks

This article has explored the limitations of making use of QLFS data in generating sub-national statistics for South Africa. The results show that robust measurement of regional labour market trends over time is constrained by the wide margin for error in sub-national samples, although broad patterns between regions can be identified where there is large variation between point estimates. The implication is that household survey data (even at a national level) must be accompanied by relevant representations of the sampling error.

For practitioners and scholars who are focused on urban and regional development, the findings should encourage a healthy scrutiny of sub-national statistics from sample surveys. The apparent neglect (or ignorance) of basic standard errors is ubiquitous in public policy documents and policy formation is vulnerable to error. Similarly, the media popularises labour market statistics (and particularly negative sentiment) without regard for its accuracy, which may give wrong impressions on the true state of the economy and can have a real impact on investor sentiment. Commercial sources of modelled data should be triangulated against what is known from more transparent official sources.

In the case of the QLFS in South Africa, it is recommended that StatsSA include error bars or confidence intervals in the main analytical section of their statistical reports and point out explicitly in the narrative where labour market indicators are less robust. This will make the findings more transparent and encourage sensitivity in public consumption. The Population Census (and the corresponding Community Survey) remains an important point of reference for sub-national estimates, notwithstanding its limitations in assessing labour market conditions. There is scope for further research which tests various methods of small area estimation and on the merits of pooling consecutive quarters of the QLFS to bolster the sample size.

Disclosure Statement

No potential conflict of interest was reported by the author.

Notes

1 There is a distinction between administrative boundaries of a region or city as opposed to functional boundaries (Todes, 2008; van Huyssteen et al., 2009; Schmidt & du Plessis, 2013). For the purposes of this study we use administrative municipal and provincial boundaries as defined within the QLFS.

2 In the case of comparing differences between two proportions, rather than means, Cumming (2009) demonstrate that the rule of thumb applies provided that the upper and lower tails are roughly symmetrical and the proportions are not near the limits of zero or one.

3 The expanded definition of unemployment includes discouraged job-seekers (see Posel et al. (2014) for a discussion).

4 Gauteng provides the best case scenario for generating robust estimates of unemployment by race as the province with largest sample.