Letter to the editor: SIA reviewers (in Australia) need different guidelines

In 1995 an Interorganizational Committee on Principles and Guidelines for Social Impact Assessment in the United States published a set of SIA guidelines (The Interorganizational Committee on Guidelines and Principles for Social Impact Assessment 1995). Ten years later the Committee published an update based on experience with the first set of principles, noting, in summary.

SIA provides information to agencies and communities about social and cultural factors that need to be considered in any decision; provides a mechanism for incorporating local knowledge and values into the decision; and can help a decision-maker identify the most socially beneficial course of action for local, regional, and national interests. (The Interorganizational Committee on Principles and Guidelines for Social Impact Assessment 2003, p. 232)

However, neither publication provides explicit guidance to help decision makers assess the SIAs they receive. A similar approach is taken in more recent publications (Vanclay 2003; Esteves et al. 2012; Vanclay et al. 2015) and in New South Wales, the Department of Planning & Environment [DPE]’s SIA guideline for State significant mining, petroleum production and extractive industry development (DPE 2017).

Two of these recent publications (Vanclay et al. 2015; DPE 2017) include a list of review questions; the DPE guidelines at Appendix D, and Vanclay et al. in a chapter titled Review Criteria for checking Social Impact Assessment Reports and Social Impact Management Plans. This chapter provides questions to which ‘peer reviewers and/or community groups’ should have regard when reading an SIA. The list has 91 review questions, among which are:

Were the methods used to predict the impacts adequately described and appropriate?

Was the process to establish significance of the impacts described and was it reasonable?

Was there a discussion of the limitations of the methodology and of the SIA in general?

Was there evident adequate awareness of social research methods and appropriate reference to the literature on the methods of SIA and social research generally?

Was there a justification provided for each social indicator?

Is there a discussion about and use of existing (secondary) sources of baseline data?

(Vanclay et al. 2015, p. 66–69)

There is nothing inherently wrong with these questions which can also act as quick (if exhaustive) checklist reminders for SIA authors. Most SIA guidelines are directed to these authors, the people preparing initial SIAs for project proponents. Ideally, these authors have a social science background.

However, SIA authors are not the only assessors of social impacts as several NSW laws make clear. These include:

• 1. The NSW Environmental Planning and Assessment Act, 1979

Section 4.15 of this Act requires a consent authority to take account of the likely social and economic impacts of a proposed development.

• 2. The NSW Local Government Act, 1993

Section 8A(2) requires local government authorities to consider social justice principles and the long term and cumulative effects of actions on future generations.

• 3. The NSW Liquor Act, 2007

Section 48(5) states that the Authority

must not grant a licence, authorisation or approval to which a relevant application relates unless the Authority is satisfied… that the overall social impact of the licence, authorisation or approval being granted will not be detrimental to the well-being of the local or broader community.

• 4. The NSW Gaming Machines Act, 2001

Depending on the number of gaming machines applied for and the area in which the premises are located, section 36 of the Act requires the Authority to be satisfied that an increase in the number of gaming machines will either provide ‘a positive contribution’ to the local community, or ‘have an overall positive impact on the local community’.

Planning consent authorities in NSW can be local government authorities, planning panels, or the Independent Planning Commission. The liquor and gaming authority is called The Independent Liquor and Gaming Authority [ILGA].

Some decisions can be appealed on their merits. In the case of planning, decisions can be appealed to the NSW Land and Environment Court [LEC]. Some decisions of ILGA can be appealed to the NSW Civil and Administrative Tribunal (NCAT). These appeals may canvass the merits of likely social impacts and deal with the adequacy of the way in which social impacts were considered at the consent stage. A merit appeal involving social impacts means that the court appealed to is the final social impact assessor.

Thus, the SIA prepared for a proponent is only the first of several assessments. In each of the decision-making processes required by these pieces of legislation, there is a potential hierarchy of four assessments, namely:

i the SIA lodged with an application by a proponent.

ii an assessment by a public servant working in the organisation which has received the application. This person reviews the proponent’s SIA and prepares a report for the consent authority.

iii an assessment by the decision makers in the consent authority, for example, a planning panel or ILGA. The decision makers review the SIA and the SIA assessment review report and make a decision.

iv an assessment by the person hearing an appeal (in either the LEC or NCAT). New SIAs may be prepared by both the appellant(s) and the respondent(s). The Judge or Commissioner reviews and assesses all documents and may hear from SIA expert witnesses.

As this list shows, social impacts information relevant to a decision builds as the process progresses. The initial SIA is only the first input and there can be three stages of review and social impact assessment thereafter – more if an application involves liquor and gaming as well as planning authorities.

The problem with many guidelines lies with the fact that many reviewing assessors and decision-makers (at stages ii, iii and iv above) are not peer review social scientists nor members of a locally affected community group. As non-social scientists, these reviewers and decision makers are unlikely to wade through 91 questions which have been prepared for a different purpose.

Further, SIA guidelines and the review questions state what should happen in social impact assessment. They do not help decision-makers to identify the many ways in which compliance with due process and good social science may appear to have been achieved when in fact it has not. The review questions exhaustively list what should be done but not what is commonly done. As a result, these guidelines do not address the specific requirements of the hierarchy of assessors and decision makers. While reviewing assessors and decision makers should know what a competent SIA looks like, they should also be able to recognise common mistakes and sources of bias that render SIAs partial, incomplete or misleading.

SIA guidelines present best practice, but the reality is that most SIAs are less than perfect. Initial SIAs are usually prepared and paid for by a proponent. Proponents do not submit SIAs which say that their project should not go ahead. As well, writing an SIA requires the SIA author to focus on the impacts that matter, rather than on all possible impacts. As the Interorganizational Committee noted, a good SIA should ‘Focus the assessment – Deal with issues and public concerns that “really count”, not those that are “easy to count”’ (The Interorganizational Committee on Guidelines and Principles for Social Impact Assessment 1995, p. 35). An author’s choice of impacts that matter may be influenced by the proponent’s interests.

By contrast, the SIA reviewer is always looking for both issues that matter and mistakes and errors that matter. While what matters vary from place to place and proposal to proposal, there are some mistakes and errors that occur frequently and usually matter.

Many of these mistakes and errors are standard inclusions in discussions of social science research methods. A list of these would include: unsubstantiated assertions (there is a pre-determined demand for alcohol in any community), physical determinism (the building will create social cohesion), misuses of social data – rates, indicators and comparators, selective referencing, poorly designed surveys and biased reporting of consultation feedback. The list would also include mistakes that often feature in guidelines on economic analysis such as confusing before and after with ‘with and without’, double counting and liberties taken with consumer surplus, contingent valuation and multivariate analysis. None of these are problems unique to SIA, but many decision-makers, the final SIA assessors, are likely to be unfamiliar with them.

There is also the serious problem of omissions. Most SIA guidelines emphasise distributional equity of impacts across different population groups, and it is widely recognised that social impact assessment is an exercise in precaution. However, many SIAs neither mention nor address either of these concepts. Decision makers need to be alert to the presence and effect of these omissions. For example, consideration of public health is often omitted (not the same as lists of medical services and a few morbidity rates). Research documenting the relationship between relative (distributional) inequality and public health is well-established in the epidemiological literature, including in very accessible summaries for the lay reader (Wilkinson and Pickett 2012, 2018, and the many publications of The Equality Trust, UK). Increasingly distributional inequalities which undermine public health can also be found in mapped data, such as maps of crime rates, census and social benefits data. Nonetheless, if the SIA does not make these connections and explain their relevance, it is unlikely that the decision makers will do so.

This omission is significant. Public health profoundly affects the quality of social life and reflects social well-being. Social well-being, in turn, depends on effective public policy (Wilkinson and Marmot 2003). SIA is a public policy tool.

Finally, a set of guidelines for reviewers would address the question of adequacy when considering mitigations, for example, whether proposed mitigations are within the capacity of the proponent to deliver, are likely to be effective, can be enforced as a condition of consent, and are likely to last.

Although it is possible for reviewing assessors and decision makers to extrapolate some of this guidance from existing SIA guidelines, many reviewing assessors have neither the time nor the professional background to do so efficiently, nor to identify the errors and omissions that matter. Guidelines for SIA reviewers and decision makers are needed.

Disclosure statement

No potential conflict of interest was reported by the author.