Application of the MERIT survey in the multi-criteria quality assessment of occupational health and safety management

Abstract

Occupational health and safety management systems apply audit examinations as an integral element of these systems. The examinations are used to verify whether the undertaken actions are in compliance with the accepted regulations, whether they are implemented in a suitable way and whether they are effective. One of the earliest solutions of that type applied in the mining industry in Poland involved the application of audit research based on the MERIT survey (Management Evaluation Regarding Itemized Tendencies). A mathematical model applied in the survey facilitates the determination of assessment indexes WOP_i for each of the assessed problem areas, which, among other things, can be used to set up problem area rankings and to determine an aggregate (synthetic) assessment. In the paper presented here, the assessment indexes WOP_i were used to calculate a development measure, and the calculation process itself was supplemented with sensitivity analysis.

Keywords:

• audit
• management of occupational health and safety
• multi-criteria assessment
• sensitivity assessment

1. Introduction

While in the 1970s the concept of occupational safety, especially in the mining industry, was primarily focused on ensuring appropriate technical means, determining reasons and circumstances of accidents and defining human share in their causing, the beginning of the 1980s gave rise to a new approach where more attention was paid to the social and organizational aspect.[1–5] The notion of human factor error was defined as an action which deviates from the required standards and requirements of a situation. The said deviations occur when one such element as information, proper tools, knowledge, physical capabilities or incentives (motivation) to carry out work tasks in the right way is defective or is missing.[6–10] The level of safety is increasingly frequently treated as the resultant of the employees’ attitude, their behaviour and surrounding environment. The methodology of a behaviour-based system is becoming popular along with other concepts, involving the participation of employees in the problems of occupational safety improvement (sharing responsibility for the level of occupational health and safety [OHS]) as well as the concept of OHS management. Since a great variety of measures are undertaken in the field of health and safety management, this concept is often interpreted as all actions involving the problems of OHS, which, in turn, leads to a falsely understood opinion that everything which is being offered now has been already done before. However, according to David Paterson (creator of the philosophy of safety management), ‘dangerous actions, dangerous conditions and accidents are observable facts or events which are symptoms of some dysfunctions in the management system’.[11] Therefore, we can conclude that according to the principle of limited trust (i.e., one of the principles of safety management), there is no totally secure (safe) realm of human activity, and we should always predict potential disruptions (which can be manifested, e.g., as breakdowns).

The necessity to implement the principles of the occupational safety management system in Polish companies is being enforced by a limited efficiency of the currently applied methods of safety analysis (methods which are based principally on grouping and balancing the accidents which have been reported, whereupon respective models assessing the reasons and circumstances of these accidents are applied) and by the legal, organizational and economic requirements. The problem of occupational safety can be viewed from the perspective of economic development: in economically developing Poland, the level of accepted risk is expected to be decreasing. There is a commonly accepted regularity which states that the more a country is economically developed, the greater attention is paid to the problems of occupational safety and to the implementation of most rational methods aiming to prevent hazards and risks.

1.1. MERIT survey as a research method

The most important issues which should be covered in each OHS management system involve among others the following:[12]

• prevention of accidents at work;

• minimizing the probability of occupational diseases;

• constant efforts to improve OHS in the organization;

• incessant perfection of OHS operations;

• ensuring appropriate resources and means facilitating the implementation of OHS policy;

• improving the qualifications of employees and involving them in OHS activity.

The implementation of each system of OHS management involves the use of a certain instrument which facilitates its monitoring and which can be applied to carry out the efficiency assessment of the undertaken actions. Such a function is fulfilled by the audit of the OHS management system.

In the realm of Polish coal mines, the MERIT survey (Management Evaluation Regarding Itemized Tendencies) was a diagnostic instrument used for the first time in the assessment process of decision making quality in the field of health and safety management, developed by the US National Institute for Occupational Safety and Health (NIOSH). The survey, which was an element of the total safety management (TSM),[13,14] was adapted in the mid-1990s to Polish conditions within the framework of the program Partners in Economic Reforms.

The MERIT survey contains 29 questions grouped into nine problem areas:[15,16]

1. Planning actions in the field of occupational safety management.

2. Investigation of accidents.

3. OHS control and inspection.

4. Observation and analysis of the way of work task realization.

5. Personal protection.

6. OHS regulations in the company.

7. Information provided on the condition of OHS.

8. Promotion of OHS.

9. Personal evaluation of OHS conditions.

The survey has a formal character, i.e., each questions is answered by selecting one of five options scored from 0 to 4 (0 = fail, 4 = ideal). The respondents are expected to choose an answer which in their opinion reflects the realization phase of the actions undertaken in the above mentioned problem areas in the best way. The respondents are selected through a simple dependent draw (without returning).

Basing on the completed survey sheets, an assessment index WOP_i is determined individually for each problem area:[13–16] (1) where i = number of area subjected to assessment; j = awarded assessment grade; c_j = number of answers with the assessment grade j; p = number of questions within the problem area; n = number of respondents taking part in the research.

Using the values of assessment indexes WOP_i we can work out comparative rankings of the problem areas, whereby it is possible to define strong and weak points involving the OHS management system, and hence to highlight the areas which need to be changed.

2. Multi-criteria assessment with the use of a development measure: essence, applicability

In the widely understood assessment process, a progressively greater importance is being attributed to synthetic measures determined by the application of multi-dimensional statistics methods. Through the application of these measures, we can replace a whole set of features which describe an object (partial assessments) with one variable which is an aggregate (synthetic) quantity. Such methods can be exemplified by the method where a so-called development measure is applied. In this method, in order to determine assessment criteria (goodness-of-fit criteria), the following elements are defined:

• an abstract point P_o being the reference solution of the coordinates {x_o1, x_o2,  …  , x_om} satisfying the conditions:[17] (2) (3)

where S = a set of stimulants (stimulants – assessments whereof the increments of absolute values are assessed as positive); D = a set of destimulants (assessments whereof the increments of absolute values are assessed as negative);

• points P_i being the graphic interpretation of objects subjected to assessment.

The distance between points P_i and the point P_o is determined with the following dependence: (4) where  = normalized coordinates of the point P_i;  = significance (rank) of the jth partial feature determined on the basis of the survey of experts’ opinions.

In order to determine the above mentioned measure, the output variables have to be normalized. The aim of the normalization is to make the variables which have different denominations comparable and to unify the character of the features. To achieve the above objective, we have to single out the features which are stimulants, destimulants or nominants. For the nominants features, we define intervals in which they behave like stimulants and intervals in which they behave like destimulants. In the normalization process, we can apply the standardization of variables, quotient transformations or unitarization.[18] In the application, quotient transformations were applied in compliance with Equations 5–6: (5) (6)

If there are quality features in a set, they should be first quantified (they should be ascribed numerical values).

The value of the development measure m_i was calculated from the dependence: (7) where m_i  = the object is more developed the more the value of its measure is approaching the value of 1.

3. Sample application of a development measure in the multi-criteria quality assessment process of the OHS management at the department level of a coal mine

The following departments were involved in the survey studies carried out at the coal mine in the first and second quarter of 2013:

• all mining departments: G-1, G-2, G-3, G-4, G-5;

• all departments of preparatory works: GRP-1, GRP-2, GRP-3;

• both armouring-liquidation departments: GZL-1, GZL-2.

These departments were selected for the studies since the production output generated by the departments was crucial in terms of the overall output of the coal mine and their contribution to the preparation stage of the mining front was considerable.

Additionally, the employees of electrical engineering departments took part in the survey (ME-1, ME-2), i.e., the departments providing maintenance works at mining face.

The characteristics of the headings at which the works were carried out in the time period specified above are presented in Table 1.

Table 1. Characteristics of headings where underground works were carried out by the surveyed departments in the time period 2nd – 3rd quarter of 2013.

	Department
Characteristics of headings	G-1	G-2	G-3	G-4	G-5	GRP-1
Heading	• longwall 10b	• longwall 12	• longwall 43c	• longwall 22b	• longwall 7c	• Gallery to trough bottom;
						• Cross-cut at seam 655m
Methane hazard	Category II	Category II	Category II	Category II	Category II	Category III (gallery to trough bottom)
Crump hazard	1st degree	1st degree	1st degree	1st degree	1st degree	1st degree
Dust hazard	Class B	Class B	Class B	Class B	Class B	Class A
Water hazard	–	–	–	–	–	1st degree (gallery to trough bottom)
	Department
	GRP-2	GRP-3	GZL-1	GZL-2	ME-1	ME-2
Heading	• cross-cut XXII;	• testing incline I to the west;	• armouring of longwall 915;	• liquidation of haulage plane 14;	• area of longwall 22b;	• area of longwall 10b;
	• water gallery III at seam 782m	• gallery B-1/2;	• armouring of longwall 245	• liquidation of cross-cut to gallery 1z;	• area of longwall 43c	• area of longwall 12
		• dip-heading D-1
				• armouring of haulage dip-heading at seam 509
Methane hazard	Category I	Category I	Category I (only longwall 245)	Category I	Category II	Category II
Crump hazard	1st degree	1st degree	1st degree	1st degree	1st degree	1st degree
Dust hazard	Class A	Class B	Class B	Class B	Class B	Class B
Water hazard	2nd degree	1st degree	–	–	–	–

The number of participants in respondent groups taking part in the survey was around 40–50% of the total staff of the departments (a total of 342 surveys were carried out).

Based on the survey results, the values of relative variability measure (values of variability indexes) were determined (see Table 2). The best fitting of respondents’ opinions was in the case of area C assessment ‘OHS control and inspection’ carried out at department GZL -2 (V(x) = 9.1%), the lowest fitting was for the assessment of area F assessment ‘OHS regulations in the company’ carried out at department G-1 (V(x) = 50.0%).

Table 2. Values of variability indexes V(x) (%) in individual problem areas.

	Department
Problem area	G-1	G-2	G-3	G-4	G-5	GRP-1	GRP-2	GRP-3	GZL-1	GZL-2	ME-1	ME-2
A	18.8	26.7	23.5	22.0	20.4	17.0	30.6	19.6	22.8	39.1	39.6	45.4
B	22.4	23.2	28.5	28.2	25.4	24.5	22.4	24.3	22.2	17.9	18.2	19.6
C	15.2	28.4	26.3	29.1	24.0	23.7	17.5	22.8	18.3	9.1	32.3	26.4
D	21.5	17.5	18.5	12.6	23.9	24.9	24.3	24.5	19.8	13.9	16.7	17.3
E	34.1	34.2	40.5	39.5	31.4	38.9	34.0	35.8	36.0	35.2	40.8	37.2
F	50.0	37.7	39.9	39.2	35.9	34.8	25.0	36.1	33.9	32.0	20.2	20.4
G	34.8	32.1	34.4	32.8	34.6	17.0	34.6	20.8	27.8	31.3	32.9	26.9
H	40.1	39.3	47.6	34.8	34.0	29.4	32.0	31.1	36.0	39.7	47.0	47.0
I	26.5	23.4	44.5	29.5	21.4	25.1	16.5	17.9	25.3	16.4	16.6	17.2

The obtained values of WOP_i indexes can be treated in two ways: as the final assessments within single-criterion tasks or as partial assessments within multi-criteria tasks. When the WOP_i values are treated as final assessments within single-criterion tasks, the highest assessments were given to the solutions involving:

• planning actions in the area of OHS management (area A): actions undertaken by department G-1 (WOP_A = 3.31);

• accident investigations and practical conclusions effected by such investigations (area B): actions undertaken by department GZL-2 (WOP_B = 3.22);

• the quality of inspections carried out by the supervising staff of the coal mine and external entities responsible for the supervision of work conditions (area C): actions undertaken by department GZL-1 (WOP_C = 3.29);

• the quality of the supervision of work tasks execution (area D): actions undertaken by department G-4 (WOP_D = 3.67);

• the use of individual protection measures (area E): actions undertaken by department G-5 (WOP_E = 2.91);

• the access to OHS regulations and the quality of disciplinary system regarding the observance of OHS regulations (area F): actions undertaken by department GRP-2 (WOP_F = 3.12);

• OHS training (area G): actions undertaken by department GRP-1 (WOP_G = 3.15);

• the promotion of safe execution methods of work tasks (area H): actions undertaken by department G-4 (WOP_H = 2.67);

• the attitude of the supervisors to the OHS issues (area I): actions undertaken by department GZL-2 (WOP_I = 3.30).

The overall set of survey results is presented in Table 3 (non-normalized values of assessment) and Table 4 (normalized values of assessment).

Table 3. Partial assessments obtained from the MERIT survey: non-normalized values.

	Department
Problem area	G-1	G-2	G-3	G-4	G-5	GRP-1	GRP-2	GRP-3	GZL-1	GZL-2	ME-1	ME-2
A	3.31	3.20	3.19	3.30	3.24	3.04	2.94	2.57	2.68	2.37	2.48	2.58
B	2.86	2.69	2.73	2.90	3.18	3.15	2.94	3.18	3.00	3.22	3.14	3.21
C	2.77	2.89	2.46	2.67	3.03	3.08	3.27	3.04	3.29	2.93	3.14	3.16
D	3.57	3.63	3.50	3.67	3.49	3.37	3.21	3.39	3.50	3.59	3.57	3.53
E	2.63	2.46	2.27	2.47	2.91	2.67	2.70	2.57	2.71	2.63	2.52	2.42
F	2.40	2.49	2.35	2.57	2.67	3.04	3.12	2.96	3.00	3.00	3.10	3.05
G	2.71	2.60	2.50	2.53	2.52	3.15	2.52	2.39	2.50	2.56	2.48	2.59
H	2.49	2.47	2.27	2.67	2.64	2.56	2.49	2.32	2.00	2.30	2.19	2.16
I	3.14	3.20	2.23	2.90	2.97	2.78	2.82	2.79	3.11	3.30	3.24	3.26

Note: MERIT = Management Evaluation Regarding Itemized Tendencies.

Table 4. Partial assessments obtained from the MERIT survey: normalized values.

	Department
Problem area	G-1	G-2	G-3	G-4	G-5	GRP-1	GRP-2	GRP-3	GZL-1	GZL-2	ME-1	ME -2
A	0.83	0.80	0.80	0.83	0.81	0.76	0.74	0.64	0.67	0.59	0.62	0.65
B	0.71	0.67	0.68	0.73	0.80	0.79	0.74	0.80	0.75	0.81	0.79	0.80
C	0.69	0.72	0.62	0.67	0.76	0.77	0.82	0.76	0.82	0.73	0.79	0.79
D	0.89	0.91	0.88	0.92	0.87	0.84	0.80	0.85	0.88	0.90	0.89	0.88
E	0.66	0.61	0.57	0.62	0.73	0.67	0.67	0.64	0.68	0.66	0.63	0.61
F	0.60	0.62	0.60	0.64	0.67	0.76	0.78	0.74	0.75	0.75	0.77	0.76
G	0.68	0.65	0.63	0.63	0.63	0.79	0.63	0.60	0.63	0.64	0.62	0.65
H	0.62	0.61	0.57	0.67	0.66	0.64	0.62	0.58	0.50	0.57	0.55	0.54
I	0.79	0.80	0.56	0.73	0.74	0.70	0.71	0.70	0.78	0.82	0.81	0.82

Note: MERIT = Management Evaluation Regarding Itemized Tendencies.

In the case of multi-criteria assessment, each of the surveyed departments can be presented as point P_i in the nine-dimensional space described with the indexes WOP_i (the values of WOP_i indexes are interpreted as the coordinates describing the location of point P_i). In this case, the reference solution is made up by the point P_o of the coordinates {4; 4; 4; 4; 4; 4; 4; 4; 4}, which are the maximum obtainable assessment values WOP_i.

Although the works are carried out in similar mining-geological conditions (Table 1), there is a big diversity between the values of the development measure obtained for the investigated departments. The values of the development measure were determined both for non-diversified significance of the assessed problem areas (Table 5) and for diversified significance (Table 7); the accepted significance diversification of the assessed problem areas is presented in Table 6.¹ In both cases, the object (department) GRP-1 had the highest assessment score (the highest development level). Based on the carried out sensitivity analysis it can be observed that the values of the development measure m_i for this department are 0.277 (with non-diversification of the significance of the assessed problem areas) and 0.285 (with their diversification). In the case of department GRP-1, the closest to the optimal (maximum) coordinate is the coordinate D – ‘Observation and analysis of work task realization’ (the distance from the ideal coordinate is 0.630) as well as B – ‘Investigation of accidents’ and G – ‘Information provided on the condition of OHS’ (in both cases the distance from the ideal coordinate is 0.852). At the same time, the longest distance with respect to the maximum coordinate was found in the case of coordinates H – ‘Promotion of OHS’ (1.444), E – ‘Personal protection’ (1.333) and I –’Personal evaluation of OHS conditions’ (1.222). While in the case of areas H and E, the values of WOP_i index obtained by department GRP-1 do not differ considerably from the results obtained by the remaining departments (none of the departments obtained the value of WOP_i index equal to at least 3.0), in the case of area I, the difference between department GRP-1 and the best result (obtained by department GZL-2) is as high as 0.518. In the case of department GRP-1 the improvement of the quality of the actions involving the promotion of the safe work task realization, ensuring a satisfactory amount and quality of individual protection measures and appropriate shaping of work space (undertaking actions aiming to reduce physical and mental effort at workstations) should be viewed as main directions of reparatory measures to be undertaken by the departmental supervising staff (with respect to the promotion of the safe work task realization and access to an appropriate amount and quality of personal protection measures, the above recommendation applies to all investigated departments).

Table 5. Distances (C_io) and measures (m_i) of goodness of fit (for ).

	Department
Parameter of assessment	G-1	G-2	G-3	G-4	G-5	GRP-1
Cio	0.887	0.915	1.089	0.906	0.812	0.787
mi	0.185	0.160	0.000	0.168	0.255	0.277
	GRP-2	GRP-3	GZL-1	GZL-2	ME-1	ME-2
Cio	0.858	0.936	0.909	0.899	0.907	0.899
mi	0.212	0.141	0.166	0.174	0.167	0.175

Table 6. Significance of problem area.

Problem area involving the assessment of the quality of occupational health and safety management	Significance (rank) of problem area
A. Planning actions in the field of occupational safety management	0.62
B. Investigation of accidents	0.87
C. OHS control and inspection	0.79
D. Observation and analysis of the way of work task realization	0.91
E. Personal protection	0.86
F. OHS regulations in the company	0.79
G. Information provided on the condition of OHS	0.81
H. Promotion of OHS	0.64
I. Personal evaluation of OHS conditions	0.83

Note: OHS = occupational health and safety.

Table 7. Distances (C_io) and measures (m_i) of goodness of fit (for diversified significance of problem areas).

	Department
Parameters of assessment	G-1	G-2	G-3	G-4	G-5	GRP-1
Cio	0.784	0.810	0.968	0.806	0.716	0.692
mi	0.190	0.163	0.000	0.168	0.260	0.285
	GRP-2	GRP-3	GZL-1	GZL-2	ME-1	ME-2
Cio	0.756	0.816	0.784	0.775	0.783	0.777
mi	0.219	0.157	0.190	0.200	0.191	0.197

The second department, after GRP-1, having the highest development index was department G-5 whereof index m_i was respectively 0.255 (with non-diversification of the significance of the assessed problem areas) and 0.260 (with their diversification). With respect to department G-5, the closest to the maximum coordinate is the coordinate D –’Observation and analysis of work task realization’ (the distance from the ideal assessment is 0.515), A – ‘Planning actions in the field of occupational safety management’ (0.758) and B – ‘Investigation of accidents’ (0.818). For the coordinate G (wrong transfer of information on the condition of OHS), the distance from the reference point coordinate is 1.485, and for the coordinate H (poor promotion of OHS) the said distance is 1.364.

As to departments G-1, G-2, G-4, GRP-2, GRP-3, GZL-1, GZL-2, ME-1 and ME-2 the values of development measures have a similar level, but not lower than that recorded for departments GRP-1 and G-5.

By far the longest distance to the reference solution was recorded for department G-3. In this case the distance between the coordinates of point P_o and the coordinates describing the point P_i, being the graphical interpretation of the department's localization is oscillating within the range from 0.500 (coordinate D – ‘Observation and analysis of the way of work task realization’) to 1.769 (coordinate I – ‘Personal evaluation of OHS conditions’).

4. Conclusion

According to the assumptions of TSM, the investigation of hazard potential, or the research on the quality of OHS management, should be viewed as an integral part of a manufacturing process at all stages. Dangerous incidents usually show the areas of insufficient control over the work environment and as a rule lead to the generation of loss. Such events connected both with the physical conditions of a workplace and with human behaviour can be identified among others by the application of audit examination. The MERIT survey is an example of such formalized heuristic methods of information acquisition in the realm of OHS management. The survey allows us to determine assessment indexes which can be applied to follow the tendencies and changes involving the quality of undertaken actions pertaining to the realm of OHS. The acquired index values can serve to create ranking setups, covering not only the compared problem areas but also the entities subjected to assessment (coal mines, departments, workstations). It is equally important that the realization method of the MERIT survey can facilitate the acquisition of a wide range of information based on the professional knowledge and experience of a great number of employees who, with the help of the survey, can contribute to the identification of the existing deficiencies and dysfunctions.

The determined assessment indexes based on the MERIT survey can be treated in two ways: as final assessments within single-criterion tasks (in the case of the MERIT survey, there are nine problem areas, nine single-criterion tasks and hence nine final assessments) or as partial assessments (nine partial assessments) within one multi-criteria task.

The second case can be understood as a typical task leading to the acquisition of many objectives, i.e., a simultaneous optimization of more than one criterion (e.g., the quality of the carried out accident investigations, the quality of the offered personal protection equipment, or the quality of the organized OHS training). For the solution of a multi-criteria task understood in that way, this paper offers the application of a development measure, and the survey of experts’ opinions was used not only to assess each of the problem areas but also as a diversification tool defining the significance of each of them (in the case of the MERIT survey alone, the problem areas are not subjected to valuation). In the author's opinion, the assessments of problem areas WOP_i and the values of development measure m_i for each of the departments can be helpful in determining the directions of corrective measures to be undertaken in the field of OHS management (identification of strong and weak sides of OHS management), and they can also be applied to activate the employees to take action towards the improvement of work safety (the results of the audits viewed as a competition factor between individual departments).

Disclosure statement

No potential conflict of interest was reported by the authors.

Notes

1. The respondents diversified the significance of the assessed problem areas from 0.0 to 1.0.