Framing an independent, integrated and evidence-based evaluation of the state of Australia's biophysical and human environments

Figures & data

Table 1. Australia's SoE 2011 assessment themes and their main focus.

Theme	Focus
Atmosphere	Climate, greenhouse gases, ambient air quality, indoor air quality, stratospheric ozone.
Inland Water	Inland river, lake, wetland and aquifer environments and ecosystems.
Land	Soil, vegetation and land use systems.
Marine Environment	Marine environments, ecosystems and biodiversity.
Antarctic Environment	Environments, ecosystems and biodiversity of Australia's Antarctic Territory and sub-Antarctic islands.
Biodiversity	Species, environments, native vegetation, ecological communities and ecosystem services.
Heritage	Natural heritage and reserved lands and waters, Indigenous heritage, historic heritage places.
Built Environment	Liveability and efficiency associated with human-made physical structures and the environmental regulating, provisioning and cultural services that support these.
Coasts	Pressures that impact the coast assets and values.

Table 2. The Common Assessment and Reporting Framework (CARF) for Australia's SoE 2011 report: a scale-independent project logframe.

Phase	Step sequence of activities		Objective intended achievements	Policy Driver legislative mandate and requirements	Actors organisations and players involved	Output the desired product	Purpose* attributes of the project serviced	Performance Measure indicative step-wise KPI
Preparatory	1a	Establish purpose for assessment and reporting	Confirm the policy mandate and scope for assessment and reporting, and identify target audiences	Interpret and operationalise the legislative mandate	Australian Environment Minister; DSEWPaC executive; other Australian Government agencies	Whole of government commitment to process	Clients, Relevance, Coordination & Facilitation	Summary statement of purpose; confirmed with key actors
	1b	Identify the set of key stakeholders, and their specific interests	Establish context of the process, specific sectors with an interest in the findings, and specific points of engagement	Ensure that key informants and stakeholders are effectively engaged to build ownership of process and outcomes	SoE community, national and state government agencies, resource users, NGOs, tertiary science/knowledge base	Stakeholders comitted to process	Clients, Relevance, Credibility, Communication, Coordination & Facilitation	Summary statement of stakeholders and their specified interests; confirmed with key actors
	1c	Audit existing data, reports, and benchmarks; secure additional topic reviews where needed	Establish the nature and extent of existing knowledge; fill key gaps in knowledge	Ensure data and knowledge base is current and best available for assessment	SoE community, national and state government agencies, resource users, NGOs, tertiary science/knowledge base	Inventory of data and knowledge, agreed with key actors	Clients, Relevance, Credibility, Communication, Coordination & Facilitation	Summary statement of existing knowledge and sources; confirmed with key actors
Consultation	2a	Set boundaries, be clear about what is in and out of scope; eg spatial and boundaries for the area(s) to be assessed and any sub-divisions that might be needed	Spatially bound the assessment and reporting process and its broad components, for clarity about what areas/assets/values are included/excluded; secure agreement with key stakeholders	Implement the mandate for national environment assessment and reporting consistent with ecosystem based management and reporting	National and state government agencies, NGOs, tertiary science/knowledge base subset (societies, experts, workshop participants)	Bounding the biophysical, issues and governance systems covered by the report	Relevance, Credibility, Communication	Maps, process structure and boundaries agreed with key actors
	2b	Parameters and decision model: identify and agree on typology - aspects to be assessed, the grading scale to be deployed, the baselines/reference points, rules for grading decisions, and form of final reporting	Match the reporting capacity/expectation to the biophysical structure of assets and values; clearly establish the distinction between assessment of the environment vs assessment of sustainability	Operationalise the reference/benchmark framework for assessment decisions relative to policy needs/mandate	Tertiary science/knowledge base subset (experts, workshop participants)	Agreed set of metrics to be covered by the report, and rules governing assessment process decisions	Credibility, Communication	Typology and decision model agreed with key actors
Data capture	3	Prepare information base, assign scores/grades through workshops where specific findings can be tested with peers, iterate workshop outcomes for verification and confirmation	Assign credible and defendable grades to the ecosystem assets and values with a known level of accuracy and confidence	Produce robust assessment decisions in a report that is credible and defendable	Tertiary science/knowledge base subset (experts, workshop participants)	Credible and defendable assessment decisions	Credibility, Relevance	Raw workshop outputs
Synthesis, interpretation	4	Conduct a detailed scoring analysis to support report card output	Synthesise and summarise the assessment findings	Produce assessment decisions in a report that is accessible and effective for the target audiences	Lead agency; supporting facilitator	Credible, accessible and effective assessment report	Credibility, Communication, Coordination & Facilitation	Synthesised workshop outputs
Production	5a	Peer review	Secure quality peer review of findings, independent of theme authors	Produce report that is credible and defendable	DSEWPaC, independent reviewers with qualifications and experience at least equal to those of the theme authors	Report of peer review, response from theme authors	Credibility, Relevance	Reviewer reports, authors response reports, consequent updating of draft report
	5b	Production	Range of accessible products that engage with target audiences	Report that provides strategic decision support and is effective for national-level policy making	Lead agency; supporting facilitator; communications team	Draft SoE report	Communication, Relevance, Coordination & Facilitation	Draft SoE report
Release, outreach	6	Outreach with report card findings and policy implications	Direct engagement with target audiences	Report that provides strategic decision support and is effective for national-level policy making	Australian Environment Minister; national government agencies; NGOs, media team	Issue discussion amongst policy makers and the broader Australian community	Communication, Relevance, Coordination & Facilitation	Level of public engagement
Feedback	7	Formal evaluation of report process and effectiveness	Performance-based feedback on effectiveness, and to guide future improvements for the design and implementation of future national SoE reports	Estimate the effectiveness and impact of the process and the report, relative to the mandate, and contribute to ‘return on investment’ and ‘value for money’ public funds expenditure assessment	Australian Environment Minister; lead agency; other national government agencies	Guidance for incremental improvement of the assessment and reporting process	Communication, Relevance, Coordination & Facilitation, Impact	Public domain report of formal evaluation process
Iteration	8	Repeat comparable process, 5 yearly cycle	Enable adaptive ecosystem-based management through time series analysis of environment condition and trends	Interpret and operationalise the legislative mandate	Australian Environment Minister; DSEWPaC executive; other national government agencies	Whole of government commitment to process	Clients, Relevance, Coordination & Facilitation, Impact	Summary statement of purpose; confirmed with key actors
*Project Attributes and		Clients: ensure that all interested and contributing actors are identified, have the opportunity to input, and have access to outputs
their defining features		Relevance: Maintain the focus of the process and the assessment on the primary environment issues and matters that relate to an objective, robust and independent assessment with accessible outputs
		Coordination/Facilitation: there is an effective set of arrangements with capacity to ensure a high level of coordination is maintained by the process, and facilitation of client consultation is effective
		Credibility: the assessment process is transparent, independent, evidence-based, is conducted to a high technical level of robustness and independently peer reviewed, and issues findings bounded within an appropriate envelope of uncertainty and confidence
		Communication: the interim and final products of the assessments are appropriately framed into accessible language and concepts, and made available to the community and the actors in a programmed manner to enable informed and respectful analysis and public feedback
		Impact: the demand for print and electronic SoE products is tracked, and there is an assessment of citations in the various technical journals and policy development domains that may indicate uptake and the type and level of policy response