Methodological Issues in the Measurement of Workplace Bullying

Morten Birkeland Nielsen, Guy Notelaers, and Stäle Valvatne Einarsen

Methodological Issues in the Measurement of

Workplace Bullying 236

Assessing Workplace Bullying 236

The Self-Labelling Method 238

Advantages and Disadvantages of the Self

Labelling Method 239

Behavioural Experience Method 241

Operational Criteria to Estimate the Prevalence

of Bullying 244

ROC Analysis to Estimate Cut-Off Scores 245

Latent Class Cluster Analysis to Estimate the

Number and Different Groups of Targets 247

Advantages and Disadvantages of the Behavioural

Experience Method 248

Witnesses and Perpetrators 250

Using Witnesses as Informants 250

Relying on Perpetrator Reports 252

Empirical Findings 253

Targets and Victims 254

Conclusion: A ‘Best Practice’ Approach for

Measuring Workplace Bullying 256

Bibliography 258

Methodological Issues in the Measurement of Workplace Bullying

Our understanding of a given phenomenon is dependent on how successful we are in defining, operationalizing and measuring the concept (Robson, 1993). To generate valid knowledge about workplace bullying, we need to use sound and reliable measurement instruments and assessment methods. However, as workplace bullying has been assessed with a range of different methods and study designs throughout 30 years of research on the topic, it is quite likely that methodological issues have influenced our understanding of the phenomenon. The use of different methodological approaches to its measurement may limit both the internal and external validity of research findings (Neall and Tuckey, 2014; Notelaers and Van der Heijden, 2019). To make valid interpretations of the nature, frequency, antecedents and outcomes of workplace bullying, it is necessary to know and understand how methodological issues affect our results.

This chapter will shed light on and discuss important methodological challenges in this field and issues that we need to take into consideration when conducting research on workplace bullying. First, we will provide an overview of the different measurement methods used to assess workplace bullying, as well as the strengths and limitations of these different methods. Second, we will present some issues related to study designs and explain how different approaches to surveying bullying may lead to different findings that cannot easily be compared across studies. In order to reduce methodological problems in future research, we will suggest a best practice recommendation for the measurement of workplace bullying. This chapter thereby aims to function as a tool kit that can assist researchers and practitioners in selecting the best measurement methods for assessing workplace bullying while also providing guidelines for how results from various studies best can be understood.

Assessing Workplace Bullying

As discussed elsewhere in this book, workplace bullying is a complex phenomenon comprising several facets. Using the theoretical definition by Einarsen et al. (2011) as a basis, there are at least three characteristics that somehow should be taken into consideration in order to provide a valid measure of the bullying phenomenon (see Figure 6.1). First, identify whether there exists

FIGURE 6.1 Objective and subjective aspects of workplace bullying.

an actual exposure to or portrayal of negative and unwanted behaviours, depending on whether the focus is on the target or the perpetrator. In doing so, we must establish whether a given employee has been the target or perpetrator of systematic and prolonged instances of such behaviours. Second, take into consideration the target’s subjective interpretation of a power imbalance with regard to the perpetrator (Hewett etal., 2018; Nielsen, Gjerstad etal.. 2017). Third, assess other parties (the alleged bully, witnesses, etc.) and their account of the situation, the behaviours involved and the potential power imbalances involved. Hence, theoretically, bullying has both objective features (the actual behaviours and exposures) and subjective perceptions (the interpretations). However, most studies on workplace bullying tend to address the latter (Neall and Tuckey, 2014), that is, more or less subjective perceptions of being exposed to bullying.

Following theories on measurement (Cozby, 2001; Notelaers and Van der Heijden, 2019), a satisfactory overlap between the theoretical and operational definition of the concept is necessary to achieve valid findings on bullying. That is, the measurement instrument should operationalize the central characteristics of the conceptual definition. With regard to the objective part of the bullying concept, a valid assessment should therefore assess the actual exposure to negative acts and the regularity and persistency of these acts, as discussed in the chapter on investigating complaints in the present book by Hoel and Einarsen. With regard to the subjective parts, an instrument would need to consider the power imbalance between target and perpetrator as well as how the involved parties perceive, report and interpret the given and involved events.

Yet, when reviewing the different measurement methods applied in research on workplace bullying, it becomes obvious that these do not necessarily capture all the characteristics of the theoretical definition (Ciby and Raya, 2015; Nielsen et al., 2010). Rather, most have either assessed: (1) the respondents’ overall perception of being victimized by bullying (often referred to as the self-labelling method), (2) the respondents’ perception of being exposed to a range of specific bullying behaviours (often referred to as the behavioural experience method), or (3) a combination of the two methods. Hence, as the vast majority of studies of workplace bullying has been limited to examining the phenomenon from the perspective of those exposed (Neall and Tuckey, 2014), our knowledge to date about the phenomenon mainly reflects the targets’ subjective interpretations of the events rather than an understanding based on a full 360-degree assessment that includes all involved parties.

However, as it would be highly difficult to gather information from all relevant sources in a research study, it is significant to acknowledge that information solely based on the perceptions of those exposed is still useful and important and therefore a valid area of research on its own. Furthermore, any interrater overlap between the target and an alleged bully, or even with witnesses, should not be expected to be high. Many of the incidents involved in a bullying process are not consistently observed by a given witness, and actor-observer differences in behavioural ratings are well known. Hence, one would need multiple witnesses to provide a valid outsider perspective, and would still not have the full picture.

Anyhow, for many applied purposes it is still interesting and valuable to know both antecedents and risk factors of subjectively reported claims of bullying as organization always should try to prevent such experiences. For many practitioners, be it HR managers, health and safety personnel, counsellors or clinicians, it is likewise important to know both individual, group and societal outcomes when employees perceives to be exposed to workplace bullying as well as the underlying mechanisms creating these outcomes (Nielsen and Einarsen, 2018).

The Self-Labelling Method

While most studies from the last 10-15 years has used the behavioural experience approach (Neall and Tuckey, 2014), the

'Bullying takes place when one or more persons systematically and over time feel that they have been subjected to negative treatment on the part of one or more persons, in a situation in which the person(s) exposed to the treatment have difficulty in defending themselves against them. It is not bullying when two equally strong opponents are in conflict with each other’

According to this definition, have you been subjected to bullying at the workplace during the last six months

• 1. No
• 2. Yes, once or twice
• 3. Yes, now and then
• 4. Yes, about once a week
• 5. Yes, many times a week

FIGURE 6.2 Self-labelling method as presented by Einarsen and Skogstad (1996, pp. 190-191).

self-labelling approach was the most frequently used method for assessing perceived exposure in studies on workplace bullying in the 1990s and early 2000s (Nielsen et al., 2010). When applying this method, participants are usually given a singleitem question asking whether or not they have been bullied within a specific time period. In some studies, the respondents are offered a theoretical definition of bullying before being asked whether or not they have experiences in the workplace which corresponds to the presented definition (e.g., Einarsen and Skogstad, 1996; O'Moore et al., 2003). In other studies, the question about bullying has been asked without a preceding definition (e.g., Lewis, 1999; Rayner, 1997). An example of the self-labelling method with definition is displayed in Figure 6.2.

Response categories also vary between studies. While some studies use a simple yes/no response alternative (Finne et al., 2011), other studies employ a five- to seven-point frequency scale usually ranging from ‘never bullied’ to ‘bullied daily’ (Cowie et al., 2002).

Advantages and Disadvantages of the SelfLabelling Method

A clear advantage of the self-labelling method is that it does not take up much place in a questionnaire w'hile also being easy to administer. Furthermore, as the method explicitly asks respondents whether they are exposed to bullying, the face validity of the method is convincing, i.e„ that the method is subjectively viewed as covering the concept it purports to measure (Cozby, 2001). The method may in addition have high construct validity if the respondents are presented with a precise and easy-to-grasp theoretical definition that explains the concept. Despite these strengths, the self-labelling method does not come without some flaws and difficulties. From a psychometric point of view, the use of single-item measures is often discouraged as this kind of measures are said to suffer from reliability issues. For instance, few studies have examined the test-retest reliability, that is, the consistency in the measure over time. In the studies that have used restest measures, the stability is rather low. For instance, in a prospective study that examined selflabelled exposure to bullying over a two-year timeframe, the stability in bullying was only .23 (Nielsen and Knardahl, 2015). A pending question is then how much the variation in reporting that is caused by actual change and how much that is due to unreliability in the measure.

The method does not offer any insight in the nature of the behaviours involved. Hence, by using this approach gained information is restricted to whether the respondents perceive themselves as victims of bullying, whereas information on how the bullying was conducted is ignored. Consequently, as people may have different personal thresholds for labelling themselves as bullied, the self-labelling method is a very subjective approach in which personality, emotional factors, cognitive factors, and misperceptions may figure as potential biases. For instance, labelling oneself as a victim of bullying has been found to be associated with feelings of shame which may even last after the bullying has ended (Felblinger, 2008; Lewis, 2004). Because of shame, some targets may also find it threatening to their self-esteem to admit to victimization, and they will therefore avoid labelling themselves even though their experiences corresponds with the formal definition of workplace bullying (Nielsen, 2009; Out, 2005). Related to this, perception of masculinity may also affect the thresholds for self-labelling as research has shown that men label themselves as victims of bullying to a lesser degree than do women (Salin, 2003). Thus, it may be that more men try to avoid disclosing personal information that might make them appear weak or vulnerable. Furthermore, Vie and colleagues (2011) showed that exposure to bullying behaviours were related to mental health complaints quite irrespectively of any self-labelling in those exposed.

The subjectivity of the self-labelling method may be especially important to take into consideration when the respondents are not offered a definition of the bullying concept. In such cases, some may not endorse a self-label item simply because the particular situation they experienced does not conform to their personal definition of, and experiences with, bullying (cf. Magley et al., 1999). In a comparison of formal academic definitions and lay definitions of workplace bullying, lay definitions excluded central elements found in academic definitions, while they at the same time included elements that were not found in any of the academic definitions of bullying (Saunders et al., 2007). Although most lay persons included perpetration of negative behaviour as a defining characteristic, only 15.2% of the respondents included power imbalance, and only 14.7% included the persistency criterion found in the formal definition. Furthermore, many lay definitions also included themes of fairness and respect that are not found in the formal definitions.

Hence, if respondents are not presented with a definition of workplace bullying, there is a risk of a large discrepancy between how the researchers and the respondents perceive the phenomenon under investigation. Of course, even when a precise definition is presented, there is no guarantee that respondents actually read and digest the definitions they are provided with, or that they do not simply stick to and apply their own definition of the bullying concept when answering the selflabelling question. Yet, in their meta-analysis of prevalence rates of workplace bullying, Nielsen and colleagues (Nielsen et al., 2010) showed that self-labelling with definition studies yielded far lower estimates of bullying than self-labelling studies without definitions. Consequently, when a definition of the bullying concept is presented for the respondents, it seems to be read and taken into consideration.

In sum, the self-labelling method is an approach that is easy to apply in research. Yet, because of the subjectivity bias, and the fact that it does not provide any information on the nature and content of the bullying, the method has important limitations. For instance, the self-labelling with definition approach does not guarantee against respondents who are exposed to a high level of negative and unwanted behaviour but who, for some reason, do not label themselves as victims of bullying. Because of such shortcomings, the behavioural experience method has been proposed as an alternative approach when assessing exposure to workplace bullying.

Behavioural Experience Method

When applying the behavioural experience method, respondents are usually presented with an inventory that includes various types of unwanted and negative behaviour that may be labelled bullying if occurring repeatedly over time. The respondents are then asked to report how frequently they have been exposed to the different behaviours listed in the inventory within a given time period. An array of different inventories has been developed to assess the behaviours involved in workplace bullying (see Escartin et al., 2019 for a complete overview). Some of these have only been used in one single study, whereas others, such as the Leymann Inventory of Psychological Terror (LIPT; Leymann, 1990a), the Negative Acts Questionnaire (NAQ/ NAQ-R; Einarsen et al., 2009; Einarsen and Raknes, 1997) and the Workplace Aggression Research Questionnaire (Harvey and Keashly, 2003) have been employed in a range of studies. Yet, the NAQ seems to be by far the most utilized inventory. As an illustration of the behavioural experience method, the full nine-item version of the NAQ (the Short-NAQ/S-NAQ) is presented in Figure 6.3.

The following behaviours are often seen as examples of negative behaviour in the workplace. Over the last six months, how often have you been subjected to the following negative acts at work?

Please tick answer that best corresponds with your experience over the last six months:

do not get along with

FIGURE 6.3 Behavioural experience method: the negative acts questionnaire—short (Notelaers et al., 2018).

In their review of studies on the prevalence of workplace bullying, Nielsen and colleagues (Nielsen et al., 2010) found that altogether, 41% of the included behavioural experience studies employed a variation of this assessment instrument. In another meta-analysis that examined outcomes of workplace bullying, the NAQ was employed in 56% of the studies employing a behavioural experience method (Nielsen and Einarsen, 2012).

A clear limitation of many of the behavioural experience inventories used in research on workplace bullying is that they have not been thoroughly tested and validated in well-designed validation studies. Hence, little is known about the accuracy, factor structure and trustworthiness of the different instruments across samples and cultures. A notable exception from the rule is the aforementioned NAQ, which has been found to have good psychometric properties across several studies, organizational settings and countries (see Table 6.1 for an overview

Table 6.1 An Overview of Validation Studies for the Negative Acts Questionnaire

of validation studies). The most commonly used version, the NAQ-R, investigates the frequency and persistency of the respondent’s exposure to 22 different types of unwanted and negative behaviour that range from subtle and indirect acts such as gossiping, to more direct behaviours such as threats of physical abuse. In order to assess specific harassing behaviours without priming the respondents with regard to bullying, all items are formulated in behavioural terms, with no reference to the word ‘bullying’. Based on their experiences at the workplace during the last six months, respondents are asked to indicate how often they have been exposed to the 22 negative acts using a response scale ranging from ‘Never’ to ‘Daily’. In a study on the psychometric properties of the NAQ-R, the instrument was found to comprise three underlying factors labelled as personal bullying, work-related bullying and physical intimidating forms of bullying. It has also been shown that the instrument may be used to differentiate between groups of employees with regard to levels of exposure to bullying, ranging from infrequent to severe exposure to harassment (Leon-Perez et al., 2014; Nielsen et al., 2009; Notelaers et al., 2006). Hence, the NAQ may also be used to identify all forms of psychological harassment within the range of one-off incidences, such as incivility, to more severe and systematic harassment, such as exposure to ongoing and severe workplace bullying.

Data based on the behavioural experience inventories may be used in several different ways. The most common approach has been to compute an overall continuous sum or average score on the basis of the individual items to indicate the degree of exposure to bullying behaviours without considering whether the respondent actually can be considered as a target of bullying.

This overall score may then be applied in correlational based analyses and so on in order to determine associations between the degree of exposure and relevant predictor or outcome variables. A problem here is that few respondents report high exposure, and the measure is highly skewed with the problems following from such a distribution (see Notelaers and van Heijden, 2019, for more details). However, if one needs to distinguish between targets and non-targets of bullying, there are several ways to analyze data based on the behavioural experience approach that are not affected by such problems.

Operational Criteria to Estimate the Prevalence of Bullying

By using the operational criterion method (cf. Solberg and Olweus, 2003), the researcher applies a predefined threshold or cut-off value, often developed on the basis of theoretical assumptions about bullying, to decide when the exposure is frequent and systematic enough to be considered as bullying. Leymann (1990b), for instance, stated that one has to be exposed to at least one negative act per week over a period of at least six months to be reckoned as a target of bullying. Based on empirical data, Mikkelsen and Einarsen (2001) claim that two negative acts per week over a six months period are required to classify the experience as bullying, whereas Agervold (2007) proposes that the said episodes should take place at least three or four times a week and continue for at least six months.

While an operational criterion method is relatively easy to use and to interpret, the decision that targets only comprise those respondents who are subjected to a specific number of behaviours with a given frequency (e.g., on a weekly basis) is questionable (Notelaers et al.. 2006). First, the use of operational criteria reduces bullying to a strict ‘either-or’ phenomenon rather than reflecting its escalating process. Second, by using the operational criterion approach, the number of reported targets may actually be a function of the number of items included in the inventory. That is, an inventory which includes a wide range of behaviours may identify more targets than a shorter inventory. Thirdly, when using a cut-off approach based on for instance the Leymann criterion. employees exposed to a wide range of specific behaviours each occurring only now and then, are not regarded as targets of bullying even though they actually are exposed to many negative behaviours regularly. Fourth, the operational criterion approach implies that all kinds of negative acts are equally important when distinguishing between targets and non-targets, even though it may actually be that some behaviours are more harmful than others.

ROC Analysis to Estimate Cut-

Off Scores

As all of the above-mentioned operational criteria are rather arbitrary, concern has been expressed about their validity. Statistically developed cut-off criteria have therefore been proposed as a more reliable alternative for analysing behavioural experience data (Björkqvist et al.. 1994; Ölafsson and Johannsdottir, 2004). One approach for developing statistical criteria is the use of a receiver operating characteristics curve (ROC). ROC curves are useful tools for showing the connection or trade-off between clinical sensitivity and specificity for every possible cut-off for a test, and it is thereby a method used to assess the accuracy of a test. It works by giving a plot of the sensitivity of a test versus its false-positive rate for all possible cut-off points (Obuchowski, 2003). By creating a sum-score of the behavioural experience items and combining this score with another indicator of bullying or an outcome of bullying in an ROC. it is possible to determine at which sum-mative level of exposure to bullying behaviours corresponds with perceived victimization of bullying or a specific health problem and thereby set a cut-off value for when the exposure reaches the critical threshold. For instance, by including behavioural experience data and data from the self-labelling method in a ROC-analysis, and thereby combining information about frequency of exposure with information about perceived victimization, one can determine at which level of exposure to bullying behaviour a worker self-labels as a victim of bullying (see Hutchinson et al., 2017).

By examining the overlap between behavioural exposure and self-labelling in ROC-analysis of the short nine-item version of the NAQ (range 9-45) employing a large sample of Danish workers, it was found that respondents with a score of >12 were occasionally bullied, whereas those with a score of >16 were victims of bullying (Conway et al., 2018). In another study which performed the ROC-analysis on the original 22 item NAQ-R (range 22-110) scale in a representative sample of Norwegian employees, Notelaers and Einarsen (2013) found that respondents with a score below 33 on the NAQ-R scale should not be labelled as bullied. Respondents with a score between 33 and 44 were classified as being occasionally bullied, whereas respondents with a score of 45 or higher were classified as victims of severe workplace bullying. However, evidence suggests that the thresholds for bullying may differ across countries and cultures. In a study from Serbia that replicated and extended the study by Notelaers and Einarsen, it was found that a cut-off of 34 on the 22-item NAQ-R indicated being bullied occasionally, whereas a score of >81 indicated victims of bullying (Petrovic et al., 2017). According to Hofstede’s (2001) cultural dimensions research, Serbia is a country with much higher power distance compared to Norway. This suggests that Serbian employees to a larger extent accept a hierarchical order in which everybody has a place and which needs no further justification, where subordinates expect to be told what to do, and where the ideal boss is autocratic (Hofstede, 2001). Hence, it may be that Serbian employees have a different threshold for interpreting events at the workplace as bullying. Employing an Indian sample, cut of scores were set at 40 for occasional bullying and 56 for ‘severe bullying’ (Gupta etal., 2017). However, it may be difficult to ascribe the divergence between the different

ROC threshold values to cultural variations alone. Another explanation is that a particular ‘golden standard’ for evaluating cut-off points relies on the researcher. In medical science, the gold standard is often objective in nature such as having a disease or not. In psychology, however, the standard has been argued to be closer to tin or lead than to gold because one usually has to rely on more subjective data in order to set the thresholds (Streiner and Cairney, 2007). As highlighted in this chapter, research on workplace bullying and harassment lacks an objective and universal standard for when an employee is bullied and the different choices made by scholars will thereby result in different ‘golden standards’ and consequently in different cut-off values.

Latent Class Cluster Analysis to Estimate the Number and Different Groups of Targets

To avoid the limitations with the operational criterion, employing a Latent Class Cluster analysis (LCC) has also been proposed as a method that may overcome many of the shortcomings of the operational criterion method (Nielsen, 2009; Notelaers et al., 2006). LCC is a statistical method for identifying subtypes of related cases (latent classes) from multivariate categorical data. As opposed to the operational criterion method which only classifies respondents into targets and nontargets, the LCC approach distinguishes empirically between several mutually distinct groups of respondents on the basis of the nature and frequency of their reported exposure to bullying behaviour (Giang and Graham, 2008; Notelaers et al., 2006). Rather than relying on some arbitrary cut-off value or a more or less arbitrary ‘golden standard’, the number of groups and the prevalence within the different exposure groups are here explored and tested empirically.

The LCC method has been applied in several studies on workplace bullying (Galanaki and Papalexandris, 2013; Leon-Perez et al., 2014; Notelaers et al., 2011). For instance, in a study among Belgian employees, six different groups of respondents were identified based on their exposure to negative behaviours (Notelaers et al., 2006). The respondents in the first group (cluster), which comprised 35% of the total sample, reported low or no exposure to all negative behaviours and were consequently labelled as ‘not bullied’. The respondents in the second cluster did also report relatively low exposure to negative behaviours. However, some negative acts related to the respondents’ work performances occurred more frequently. The cluster was therefore labelled as ‘limited work criticism'. This cluster was the second largest group and covered some 28% of the respondents. The third cluster was characterized by exposure to a limited amount of work-and person-related criticism. The cluster was labelled as ’limited negative encounters’, and nearly 17% of the respondents belonged to this group. Respondents in the fourth cluster were subjected to a range of negative acts, yet mainly ‘now and then'. The cluster was labelled as ‘sometimes bullied’, with 9% of the sample belonging to this cluster. The fifth cluster comprised respondents who reported exposure to various work-related negative acts, yet on a frequent basis. The cluster was therefore labelled as ‘work-related bullying’. Eight percent of the respondents were classified in this cluster. The last cluster was characterized by a high probability that they were subjected to negative behaviours at least on a weekly basis. The cluster, which comprised about 3% of the sample, was labelled as ‘victims of bullying’. By comparing the different clusters, it becomes clear that the LCC method includes information about both the frequency and the nature and severity of the behaviours the respondents have been exposed to. A noteworthy aspect of these findings is that the differences in the exposure to bullying between the six clusters correspond with the view of workplace bullying as a gradually escalating process that involves a set of stages with a distinct nature. Supporting the internal and external validity of the LCC method, similar cluster solutions to the one described above have also been obtained in later studies from Norway (Nielsen et al., 2009) and the UK (Einarsen et al., 2009).

Advantages and Disadvantages of the Behavioural Experience Method

The behavioural experience method has been considered to be a more ‘objective’ method than the self-labelling approach as it does not require the respondents to label their experience as bullying, and as the decision about whether or not someone is bullied rests with the researcher through the use of an operational criterion or through statistical analysis (Frese and Zapf, 1988; Notelaers et al., 2006). Consequently, findings should have a lower risk for being influenced by cognitive and emotional processes. Yet, one may also question the ‘objectivity’ of the behavioural experience method (Agervold, 2007; Einarsen, 1996). ‘Objective bullying’ may refer to a situation in which actual external evidence of bullying is found (cf. Brodsky, 1976). In other words, to be considered as an objective measurement, the reported unwanted and negative treatment must be confirmed by third parties (Agervold, 2007) or supported by objective evidence (see Hoel and Einarsen, this volume).

Moreover, in resemblance with the self-labelling method, perceptual bias may also affect the reporting of exposure to specific negative behaviour. As all perceptions may be influenced by one’s attitudes, personality, and affective states (Bower et al., 1981; Lazarus, 1982), respondents may perceive and experience the same behaviour differently.

Whereas an important strength of the behavioural experience approach is that the method takes the nature, frequency, and duration of the unwanted behaviours into consideration, at least as seen from the perspective of those exposed, a limitation is that the power distance between target and perpetrator is not explicitly measured (Nielsen, 2009). This lack of overlap between the theoretical and operational definition of the bullying concept may contaminate the construct validity of the method since it is likely that targets able to defend themselves against bullying react differently compared to targets not able to defend themselves. That is, that it cannot exclude those cases where bullying targets show as much negative behaviours against the other party as they receive. However, in a Norwegian study that aimed to determine whether the perceived ability to defend oneself moderated the association between exposure to bullying at work and symptoms of anxiety the findings suggested that bullying may be detrimental even when targets perceive to have the ability to defend themselves against the mistreatment (Nielsen, Gjerstad et aL, 2017). The study, which comprised 1,608 respondents from a national probability sample of Norwegian employees, showed that the ability to defend only had a protective effect on the relationship between exposure to bullying behaviours and anxiety in cases of very low exposure. In cases of high exposure, there was a stronger increase in levels of anxiety among employees able to defend themselves than among those who generally felt unable to defend. The authors explained this surprising finding by building on the situational-congruence model (Diener et al., 1984). This model proposes that individuals will experience a heightened negative effect in situations that are incompatible with their personality characteristics (Hies et al., 2011). With regard to workplace bullying, it is therefore likely that a negative outcome will emerge as a response when the individual experiences an incongruence between self-concept (‘I am able to defend myself’) and external exposures (systematic exposure to bullying behaviours) as this creates an imbalance between the targets own perception of him-/herself and actual life experiences. Hence, the findings of this study suggest that it is the magnitude and frequency of the exposure that constitutes the menace rather than the perceived power differences between target and perpetrator. Alternatively, it may be that the power imbalance is actually manifested in the very reported exposure, and that this imbalance has a more profound impact on the target as compared to the subjective perception of being able to defend oneself (Nielsen, Gjerstad et al., 2017). No one who is in fact able to defend themselves would accept such a high exposure over such a long time. Hence, this theoretical and conceptual limitation of the behavioural experience method may in practice not play such a significant role anyway.

A related limitation of the behavioural experience methods is that, by only asking the respondents about the frequency and duration of the behaviour, we do not know' w'hether the respondents actually would label the behaviour as bullying. In some cases, it may be that behaviour w'hich most people would perceive as potential acts of bullying is seen by other employees as a part of their job. An item from the NAQ-R (Einarsen et al., 2009) may serve as an example. While most people would experience ‘threats of violence or physical abuse’ as potential negative workplace behaviour, behaviour of this kind is something many prison warders or nurses in psychiatric hospitals experience as a part of their jobs on an almost daily basis (cf. Ireland and Snowden, 2002; Maghan, 1999). Hence, in some settings, there is a risk of labelling regular and ‘normal’ wwk exposures as bullying w'hen using the behavioural experience method, even though the behaviours may not necessarily be perceived as such by the targets. This could especially be a problem when employing an operational approach such as the Leymann criterion that does not differentiate between the nature and severity of the various behaviours. Another problem with most of the inventories used to measure exposure to bullying behaviours is that they do not differentiate between different actors of the bullying. Yet, this can easily be solved by adding questions about the perceived perpetrators as is the case when employing the NAQ (Einarsen et al.. 2009)

Witnesses and Perpetrators

Using Witnesses as Informants

As discussed previously in this chapter, a complete assessment of workplace bullying requires that one should also include the experience of other parties (the bully and bystanders/wit-nesses) in the measurement, i.e., peer nominations. However, an important limitation of research on workplace bullying is the lack of such verification of bullying incidents (Coyne et al., 2003; Einarsen, 2000; Hoel et al., 1999). That is, most studies have assessed the phenomenon from the target perspective without obtaining any information to verify the behaviour or without obtaining the views of other parties, such as the bully or other employees (Neall and Tuckey, 2014). In theory, a target’s experience of bullying could in part be verified by gathering information on observed bullying from colleagues in work groups, as done when conducting a full and applied investigation of a complaint of workplace bullying (see Hoel and Einarsen, this volume). According to Agervold (2007), the assessment of witnesses should come closest to an objective assessment of bullying, and school based research on bullying has often used the peer nomination method to identify both victims and bullies (cf. Cowie et al., 2002; Solberg and Olweus, 2003). Yet, within a workplace setting there are practical and ethical problems of using this approach (Coyne et al., 2003). One problem with the third-party approach is that bullying at the workplace, in contrast to school-related bullying, usually takes the form of subtle and indirect behaviours such as withholding information and slander. Due to its subtle nature, such events may not be directly observable by third parties, and they may therefore not be able to report the bullying even when there actually is a true case. A related issue is that third parties are unlikely to be present at all time and they will therefore only observe snapshots of the interaction between the perpetrator and the target, rather than the full pattern of the mistreatment. Consequently, third parties may not receive enough information about the bullying, and are therefore unable or unwilling to perceive and label something as bullying until it has reached the stage of directly observable aggression (Einarsen, 1996). Moreover, economic dependency could also prevent people from being honest in their assessment due to fear of retaliation (Bjorkqvist et al., 1994). This would be particularly true in peer nominations where one is asked to assess one’s superiors or people in formal positions of power. Furthermore, bystanders are not necessarily neutral and objective observers, but may rather be colluders or otherwise being in conflict with the targeted colleague. In a study in a large heterogeneous UK sample, Hoel et al. (2010) showed that witnesses to bullying reported different leadership styles for the immediate superior than did those who were victims of bullying. While victims of bullying perceived their immediate leader as inconsistent and high on non-contingent punishment, witnesses to bullying perceived their leader as more authoritarian.

Relying on Perpetrator Reports

Using perpetrators to assess bullying may also be somewhat problematic. Perpetrators may not perceive themselves as such, or they may not understand or wish to admit that their behaviour can be considered as bullying. Supporting this argument, a study from a representative sample of Norwegian employees which examined individual situational predictors of perpetrated bullying found that self-reported victims of bullying (as assessed with the self-labelling method) were 11 times more likely to report themselves as perpetrators when compared to non-victims (Hauge et al., 2009). This suggests that employees with previous experience of bullying, and thereby a first-hand understanding of the exposure, are more likely to also understand that their own behaviour can be experienced as bullying by others. Social desirability—that is. the tendency of respondents to reply in a manner that will be viewed favourably by others—is another potential problem. In one of the few studies which have investigated the social desirability issue among perpetrators of bullying, those who admitted to bully others had low scores on social desirability (Parkins etal., 2006). Yet. the fact that those admitting to have bullied others have low scores on social desirability just confirms the presence of this subset of respondents. Those high in social desirability should of course be expected to deny being perpetrators. In a similar manner, personality characteristics may influence the likelihood of reporting oneself as a perpetrator. In a meta-analysis of studies on enacted workplace aggression, it was established that respondents high in trait anger and negative affectivity were more likely to report being perpetrators than respondents with lower scores on the same traits (Hershcovis et al., 2007). Although these findings may indicate that trait anger and negative affectivity influence the likelihood of actually being a perpetrator, one cannot rule out the possibility that they also provide some sort of self-awareness about being a potential bully.

Taken together, information on witnesses and perpetrators could be useful with regard to complementing reports on bullying from targets, which is a pre-requisite for having a just and fair investigation in applied settings (see Hoel & Einarsen, this volume). Yet, in such an investigation, necessarily also adhering to legal requirements of a due process, one would also rely on other objective evidence such as documents, SMS text messages. minutes, social media, etc. Anyhow, the potential risk of bias in the assessment of witnesses and perpetrators described above suggests that one should be careful with using this kind of information as a form of final validation or verification of a target’s experience. Still, the experiences and perceptions of bystanders, witnesses and perpetrators of bullying is also interesting in itself, something which is shown by several studies on predictors and outcomes of both observed (Nielsen and Einarsen, 2013; Skogstad et al., 2011; Sprigg et al., 2019) and portrayed bullying (Baillien, Rodriguez-Munoz et al., 2011; Balducci et al., 2012; Glasp et al., 2009). Whereas most studies on observers of bullying have used a single item measure (e.g., ‘Have you witnessed bullying of others at your workplace?’), studies on perpetrators have employed either the self-labelling method (Glasp et al., 2007; Hauge et al., 2009) or a behavioural checklist (Baillien, De Cuyper et al., 2011; De Cuyper et al., 2009).

Empirical Findings

The above discussion of the main approaches used to assess workplace bullying suggests that the methods measure somewhat different aspects of the bullying phenomenon. Differences between the methods are also reflected in empirical findings on the prevalence and correlation of workplace bullying. Reviewing findings on the occurrence of bullying, Zapf and colleagues (2003) reported that studies using self-labelling without a preceding definition yielded prevalence rates between 10 and 25%, while self-labelling with a preceding definition resulted in prevalence rates of ol-4% bullying. Supporting the review from Zapf and colleagues (2003), a later meta-analytical study which examined methodological moderators of the prevalence rates of workplace bullying found a mean prevalence rate of 10.6% for self-labelling with definition estimates, whereas a mean rate of 19.8% was found for self-labelling estimates without a definition (Nielsen et al., 2010). The behavioural experience estimates fell in between with a mean rate of 14.7%. The fact that the self-labelling method without definition provides higher estimates of bullying compared to other methods is in line with the study by Saunders and colleagues (2007) which found that lay persons’ definitions of workplace bullying differ from formal definitions of the phenomenon.

The use of measurement methods has also been found to influence the magnitude of associations between exposure to bullying and other variables. A meta-analysis of the associations between workplace bullying and outcome variables which comprised 137 effect sizes and 77,721 respondents showed that the behavioural experience method yielded significantly stronger associations with outcomes than did the self-labelling method (Nielsen and Einarsen, 2012). Furthermore, in a meta-analysis which examined associations between workplace bullying and the Five Factor Model of personality, the self-labelling and the behavioural experience methods provided significant different estimates for four out of five personality traits (Nielsen, Glasp et al., 2017). The relatively large differences between the methods with regard to both prevalence rates and associations with other variables indicate that one should be careful with comparing findings from different studies without taking the measurement method into account. Studies employing latent class cluster analysis shows that relationships with criterion variables (e.g., health and well-being) is much stronger in the victim cluster than in the other target clusters where one experiences systematic yet less frequent exposure (Notelaers et al., 2006), in line with a stricter definition of what comprises workplace bullying.

Targets and Victims

The above discussion of self-labelling and the behavioural experience methods, and the empirical findings on differences between the two methods, may indicate that the two methods investigate overlapping yet somewhat different characteristics of the bullying phenomenon. As mentioned, a main difference between the self-labelling and the behavioural experience method is that the former method includes a cognitive, and thereby highly subjective, evaluation by individuals of whether they feel victimized by the behaviours he/she is exposed to at the workplace. The behavioural experience method, on the other hand, mainly investigates the frequency and persistency of different negative behaviours without taking the victimization aspect explicitly into consideration. To avoid mixing findings from the two methods, it may be useful to apply different labels when referring to bullied respondents. One possibility would be to refer to bullied respondents as victims of bullying when using the self-labelling approach, while referring to targets when the behavioural experience method has been used to identify bullied respondents. Here, the most highly exposed respondents may be referred to as ‘Targets of severe bullying’ more so than ‘victims’. In line with this, we tend to employ the term ‘exposure to bullying behaviours’ when employing sumscores from behavioural checklists.

Although targets and victims have been used interchangeably in the literature on bullying, we might consider treating them as separate constructs. In light of the subjective evaluation of being bullied, a target of bullying would then refer to any employee that experiences exposure to systematic and persistent bullying behaviours at the workplace, regardless of whether they subjectively perceive themselves as being bullied. A victim of bullying, on the other hand, would be a person exposed to equivalent systematic and persistent bullying behaviour and who, in addition, perceives herself or himself as being victimized by this treatment. This view is supported by prior research and theory which suggest that relational powerlessness is a core determinant of victimization (cf. Roscigno et al., 2009), yet did not receive support in the study mentioned above on the role of one's perceived ability to defend oneself as predictor of outcomes of exposure to bullying behaviours (Nielsen et al., 2018). Thus, based on the above reasoning, one could say that all victims are targets of bullying, but targets are not necessarily all victims, a claim also substantiated in empirical research showing that although all victims are targets and as such exposed to a range of bullying behaviours, many targets exist who do not label themselves as victims (Nielsen et al., 2009). At least such a terminology would make clear the methods used to determine who is and is not exposed to bullying.

This distinction between targets and victims also seems to correspond to the view of workplace bullying as a process (Zapf and Gross, 2001). During the early phase of the process, when the bullied employees typically are subjected to behaviour that is difficult to pin down because of its occasional, indirect and discrete nature, one may not yet feel victimized by the bullying. Nor may it be so different from other interpersonal conflicts (see Notelaers et al., 2018). As the treatment gradually develops and escalates to a stage involving more frequent and more direct acts, the nature of the treatment may become clearer. If the target feels that he or she lacks the resources to retaliate in kind against the treatment, a feeling of being victimized may develop. In any instance, the definitional core of bullying rests on the subjective perception that targets experience these behaviours as hostile and humiliating, and that they are directed towards oneself, regardless of how one may label the experience, an argument somewhat in favour of behavioural experience methods. Anyhow, as it is only the bullied person who knows whether or not he or she feels victimized, the self-labelling method also provides unique information about how the bullying is being experienced.

Conclusion: A 'Best Practice' Approach for Measuring Workplace Bullying

In this chapter, we have provided an overview of how workplace bullying has been measured and assessed in survey studies up to this date, as well as strengths and limitations of the different measurement methods. There are strong methodological, theoretical and practical reasons why knowledge about the measurement of workplace bullying is valuable: To achieve reliable and valid findings on workplace bullying, one needs to know how and to what extent different methodological factors actually influence the measurement of the phenomenon. Furthermore, by having knowledge about the reliability and validity of the research methods, one can correct, improve and strengthen the theories in the field (Notelaers and Van der Heijden, 2019). For practical reasons, the use of reliable and valid methods to assess workplace bullying is an important part of the basis for the development and implementation of intervention strategies to prevent workplace aggression (cf. Schat et al., 2006).

As discussed in the introduction of this chapter, workplace bullying is a complex phenomenon with both objective and subjective characteristics and where a full measurement requires information from multiple parties. An example of such an approach to the assessment of bullying can be seen in the applied method "Investigation’ (Hoel and Einarsen, this volume; Merchant and Hoel, 2003). In theory, a valid measurement of workplace bullying should therefore include an objective assessment of the harassing behaviours as well as the subjective interpretations of the event from the target, the alleged bully or bullies, and the bystanders, perhaps also employing more hard evidence from written documents, e-mails, social media etc. In practice, this kind of information would more or less be impossible to acquire in a research project, at least in large-scale surveys from national samples, and research of workplace bullying has therefore mainly examined the phenomenon from the perspective of the targets. That is, we estimate whether respondents were exposed to systematic harassment at their workplace with the behavioural experience method and ask them w'hether they perceive themselves as victimized (by this exposure) with the self-labelling method.

As these two methods assess workplace bullying from somewhat different angles, an implication is that the methods should be regarded as equally useful since both approaches seem to provide valid but complementary information on the phenomenon. One should therefore always integrate both methods in a survey when investigating workplace bullying (e.g., Hewett et al., 2018; Vie et al., 2011). Such a combination has several advantages. First of all, it is in line with the suggestion that more than one operationalization of a given concept should be used in studies on work stressors (Frese and Zapf, 1988). Secondly, by combining the two approaches, we are able to gather information about different groups of respondents with regard to both their persistency of the exposure to negative behaviours and their subjective interpretation of being victimized by the bullying behaviours. Hence, by combining the methods and both measuring respondents’ exposure(s) to persistent bullying behaviour and their perception of being victimized by this behaviour, it is possible to capture all characteristics included in the theoretical definition of workplace bullying. For instance, by combining both methods we can achieve a more detailed understanding of the outcomes of workplace bullying as is shown in studies by Vie and colleagues (2011) and Hewett and colleagues (2018).

When using the self-labelling method, the definitional approach should be the preferred approach. By excluding the definition, one increases the risk that a respondent confirms to have been exposed to workplace bullying without actually having experienced such treatment. However, when a definition is present, this assures a better agreement between the researchers’ and the respondents’ understandings of the concept. An alternative to the self-labelling method would be to extend the behavioural experience checklist with an indicator of power-imbalance. This could be done by directly asking the respondents whether they perceive themselves to be in power imbalance with the person(s) who conducted the behaviours listed in the behavioural experience inventory (Nielsen, Gjertstad et al., 2017) or by developing a separate checklist that reflect power imbalance as a latent construct. This latter approach has yet to be developed in the literature on workplace bullying and upcoming research should therefore examine whether such an approach can be a better indicator of victimization than the self-labelling method.

It should be noted that the use of the measurement methods described in this chapter may represent an ethical dilemma for researchers, practitioners and organizations. As the prevalence rates for bullying vary extensively depending on the operationalization employed, the measurement method can actually be used to manipulate the observed extent of the phenomenon in a given enterprise or survey. For instance, if one wishes to find a relatively low rate of bullying, the selflabelling with definition method can be chosen, whereas the self-labelling without definition or the behavioural experience method could be used if a higher prevalence rate is 'desired'. Correspondingly, if one applies the Leymann criterion when estimating prevalence based on the behavioural experience method, one can obtain a higher rate than what would be done with the three or four act criterion. Hence, researchers and practitioners need to be cautious when investigating bullying in their surveys. However, by both applying the self-labelling and the behavioural experience method in studies on workplace bullying, one decreases this problem as both a ‘high’ and a ‘low’ rate of bullying is acquired, while the latter may also be used to show that there are different degrees of exposure at any given moment.

Bibliography

Abe, K. and Henly, S. J. (2010) Bullying (Ijime) among Japanese hospital nurses. Modeling responses to the revised negative acts questionnaire. Nursing Research. 59(2). 110-118.

Agervold, M. (2007) Bullying at work: A discussion of definitions and prevalence, based on an empirical study. Scandinavian Journal of Psychology, 48. 161-172.

Baillien, E.. DeCuyper. N. and De Witte, H. (2011) Job autonomy and workload as antecedents of workplace bullying: A two-wave test of Karasek’s Job Demand Control Model for targets and perpetrators. Journal of Occupational and Organizational Psychology, 84(), 191-208.

Baillien, E., Rodriguez-Munoz, A., Van den Broeck. A. and De Witte, H. (2011) Do demands and resources affect target’s and perpetrators’ reports of workplace bullying? A two-wave crosslagged study. Work and Stress, 25(2), 128-146.

Balducci, C., Cecchin, M. and Fraccaroli, F. (2012) The impact of role stressors on workplace bullying in both victims and perpetrators, controlling for personal vulnerability factors: A longitudinal analysis. Work and Stress, 26(3). 195-212. doi: 10.108 0/02678373.2012.714543.

Balducci, C., Spagnoli, P., Alfano, V., Barattucci, M., Notelaers, G. and Fraccaroli, F. (2010) Valutare il rischio mobbing nelle organizzazioni: Contributo alia validazione Italiana dello Short Negative Acts Questionnaire (S-NAQ)

[Assessing the mobbing risk in organizations: Contribution to the Italian validation of the Short Negative Acts Questionnaire (S-NAQ)]. Psicologia Sociale. 5(1), 147-167.

Bjôrkqvist, K., Osterman, K. and Hjeltbàck, M. (1994) Aggression among university employees. Aggressive Behavior. 20,173-184.

Bower, G. H., Gilligan, S. G. and Monterio, K. P. (1981) Selectivity of learning caused by affective states. Journal of Experimental Psychology: General, 11.451-473.

Brodsky. C. M. (1976) The harassed worker. MA. Lexington. MA: Lexington Books.

Cakirpaloglu, P, Smahaj. J., Cakirpaloglu. S. D. and Zielina, M. (2017) Bullying at work: Reliability and validity of the Czech version of the negative acts questionnaire-revised. Ceskoslovenska Psychologie. 6/(6), 546-558.

Charilaos, K., Michael, G., Chryssa. B.-T., Panagiota, D.. George, C. P. and Christina, D. (2015) Validation of the negative acts questionnaire (NAQ) in a sample of Greek teachers. Psychology, 6(1), 63-74. http://dx.doi.org/10.4236/ psych.2015.61007.

Chirilà, T. and Ticu, C. (2014) Verifying the psychometric properties of two instruments measuring workplace bullying among Romanian employees Analele Çtiinjiftce ale Universitdfii “Alexandru loan Cuza" din lap. Psihologie, 7(1). 91-109.

Ciby, M. and Raya. R. P. (2015) Workplace bullying: A review of the defining features, measurement methods and prevalence across continents. IIM Kozhikode Society & Management Review, 4(1), 38-47.

Conway, P. M.. Hogh, A., Nabe-Nielsen. K., Grynderup, M. B.. Mikkelsen, E. G„ Persson, R., et al. (2018) Optimal cut-off points for the short-negative act questionnaire and their association with depressive symptoms and diagnosis of depression. Annals of Work Exposures and Health. 62(3), 281-294.

Cowie, H.. Naylor, R. Rivers, I., Smith. P. K. and Pereira, B. (2002) Measuring workplace bullying. Aggression and Violent Behavior. 7, 33-51.

Coyne, !.. Chong, P. S.-L., Seigne, E. and Randall, P. (2003) Self and peer nominations of bullying: An analysis of incident rates, individual differences, and perceptions of the working environment. European Journal of Work and Organizational Psychology, 12(3), 209-228.

Cozby, P. C. (2001) Methods in behavioral research (7th ed.). Mountain View, CA: Mayfield Publishing Company.

De Cuyper, N.. Baillien. E. and De Witte, H. (2009) Job insecurity, perceived employability and targets' and perpetrators’ experiences of workplace bullying. Work and Stress, 23(3), 206-224. doi: 10.1080/02678370903257578.

Diener, E., Larsen. R. J. and Emmons, R. A. (1984) Person x Situation interactions: Choice of situations and congruence response models. Journal of Personality and Social Psychology, 47(3). doi: 10.1037/0022-3514.47.3.580 6491870.

Einarsen, S. (1996) Bullying and harassment at work: Epidemiological and psychosocial aspects. Bergen: University of Bergen.

---. (2000) Harassment and bullying at work: A review of the Scandinavian approach. Aggression and Violent Behavior. 5(4). 379-401.

Einarsen, S„ Hoel. H. and Notelaers. G. (2009) Measuring exposure to bullying and harassment at work: Validity, factor structure and psychometric properties of the Negative Acts Questionnaire-Revised. Work and Stress. 23(1), 24-44.

Einarsen. S„ Hoel. H.. Zapf, D. and Cooper. C. L. (2011) The concept of bullying and harassment at work: The European tradition. In S. Einarsen. H. Hoel, D. Zapf and C. L. Cooper (eds.), Bullying and harassment in the workplace. Developments in theory, research, and practice (2nd ed.. pp. 3-40). Boca Raton: CRC Press.

Einarsen, S. and Raknes. B. I. (1997) Harassment in the workplace and the victimization of men. Violence and Victims. 12(3). 247-263.

Einarsen, S. and Skogstad. A. (1996) Bullying at work: Epidemiological findings in public and private organizations. European Journal of Work and Organizational Psychology. 5, 185-201.

El Ghaziri, M.. Storr. C. L., Simons. S. R.. Trinkoff, A. M.. McPhaul, K. M.. London. M.. et al. (2019) Comparative psychometric review of the Negative Acts Questionnaire-Revised in a unionized U.S. public sector workforce. Work, 62(1). 161— 171. doi:10.3233/WOR-182851.

Escartîn, J.. Vranjes. I.. Baillien. E. and Notelaers. G. (2019) Workplace bullying and cyberbullying scales: An overview. In P. D. D'Cruz and E. Noronha (eds.). Handbooks of workplace bullying, emotional abuse and harassment (pp. 1-44). Singapore: Springer.

Felblinger. D. M. (2008) Incivility and bullying in the workplace and nurses’ shame responses. JOGNN. 37, 234-242.

Finne. L. B.. Knardahl, S. and Lau. B. (2011) Workplace bullying and mental distress—a prospective study of Norwegian employees. Scandinavian Journal of Work, Environment and Health. 37(4). 276-286.

Frese. M. and Zapf, D. (1988) Methodological issues in the study of work stress: Objective vs. subjective measurements and the question of longitudinal studies. In C. L. Cooper and R. Payne (eds.), Causes, coping and consequences of stress at work (pp. 371-411). Chichester: Wiley & Sons. Ltd.

Galanaki. E. and Papalexandris. N. (2013) Measuring workplace bullying in organisations. International Journal of Human Resource Management. 24(11), 2107-2130.

Giang, M. T. and Graham, S. (2008) Using latent class analysis to identify aggressors and victims of peer harassment. Aggressive Behavior. 34(2), 203-213.

Giorgi, G. (2008) The Negative Acts Questionnaire Revised (NAQ-R) in Italy. Prevention Today. October-December, 71-86.

Giorgi, G.. Arenas. A. and Leon-Perez. J. M. (2011) An operative measure of workplace bullying: The negative acts questionnaire across Italian companies. Industrial Health. 49(6). 686-695.

Glas0. L.. Nielsen. M. B. and Einarsen. S. (2007) Personality and interpersonal problems among victims and perpetrators of bullying at work. Paper presented at the 13th European Congress of Work and Organizational Psychology: Sustainable Work: Promoting Human and Organizational Vitality. Stockholm, Sweden.

---. (2009) Interpersonal problems among targets and perpetrators of workplace bullyinq. Journal of Applied Social Psychology, 39(6). 1316-1333.

Gupta, R.. Bakhshi, A. and Einarsen. S. (2017) Investigating workplace bullying in India: Psychometric properties, validity, and cutoff scores of Negative Acts Questionnaire-Revised. Sage Open. 7(2).

Harvey. S. and Keashly. L. (2003) Predicting the risk for aggression in the workplace: Risk factors, self-esteem and time at work. Social Behavior and Personality, 31(8), 807-814.

Hauge, L. J.. Skogstad, A. and Einarsen, S. (2009) Individual and situational predictors of workplace bullying: Why do perpetrators engage in the bullying of others? Work & Stress, 23(4), 349-358.

Hershcovis. M. S.. Turner. N.. Barling. J.. Arnold. K. A.. Dupre. K. E.. Inness. M.. et al. (2007) Predicting workplace aggression: A meta-analysis. Journal of Applied Psychology, 92(1). 228-238. doi:10.1037/0021-9010.92.1.228.

Hewett. R., Liefooghe. A.. Visockaite. G. and Roongrerngsuke, S. (2018) Bullying at work: Cognitive appraisal of negative acts, coping, wellbeing, and performance. Journal of Occupational Health Psychology, 25(1), 71-84.

Hoel. H.. Glas0, L„ Hetland, J., Cooper, C. L. and Einarsen, S. (2010) Leadership styles as predictors of self-reported and observed workplace bullying. British Journal of Management. 21(2), 453-468. doi:10.1111/j.l467-8551.2009.00664.x.

Hoel. H.. Rayner, C. and Cooper, C. L. (1999) Workplace bullying. In C. L. Cooper and I. T. Robertson (eds.), International review of industrial and organizational psychology (pp. 195-230). Chichester: John Wiley & Sons.

Hofstede, G. H. (2001) Culture's consequences. Comparing values, behaviors, institutions, and organizations across nations (2nd ed.). London: Sage.

Hutchinson, M„ Bradbury. J.. Browne, G. and Hurley, J. (2017) Determining the optimal cut-off scores for the Workplace Bullying Inventory. Nurse Researcher, 25(3), 46-50.

Ilies, R.. Johnson. M. D„ Judge, T. A. and Keeney, J. (2011) A within-individual study of interpersonal conflict as a work stressor: Dispositional and situational moderators. Journal of Organizational Behavior. 52(1), 44-64.

Ireland, J. L. and Snowden. P. (2002) Bullying in secure hospitals. The Journal of Forensic Psychiatry, 13(3), 538-554.

Jiménez. В. M.. Munoz. A. R.. Gamarra, M. M. and Herrer. M. G. (2007) Assessing workplace bullying: Spanish validation of a reduced version of the Negative Acts Questionnaire. The Spanish Journal of Psychology, /0(2). 449-457.

Lazarus, R. S. (1982) Thoughts on the relations between emotion and cognition. American Psychologist, 37. 1019-1024.

Léon-Pérez. J. M.. Notelaers. G„ Arenas. A., Munduate, L. and Medina. F. J. (2014) Identifying victims of workplace bullying by integrating traditional estimation approaches into a latent class cluster model. Journal of Interpersonal Violence, 29(7).

Lewis. D. (1999) Workplace bullying—interim findings of a study in further and higher eductaion in Wales. International Journal of Manpower, 20(1/2). 106-118.

---. (2004) Bullying at work: The impact of shame among university and college lectures. British Journal of Guidance and Counselling. 32(3), 281-299.

Leymann. H. (1990a) Handbok for anvdndning av LIPT-formularet for kartlaggning av riskerfôr psykologisk void [Manual of the LIPT questionnaire for assessing the risk of psychological violence at work). Stockholm: Violen.

---. (1990b) Mobbing and psychological terror at workplaces. Violence and Victims, 5(2), 119-126.

Ma, S. C„ Chien, T. W.. Wang. H. H„ Li, Y. C. and Yui, M. S. (2014) Applying computerized adaptive testing to the Negative Acts Questionnaire-Revised: Rasch analysis of workplace bullying. Journal of Medical Internet Research, 16(2), e50. doi: 10.2196/ jmir.2819.

Maghan. J. (1999) Dangerous inmates: Maximum security incarceration in the state prison systems of the United States. Aggression and Violent Behavior, 4(1), 1-12.

Magley. V. J., Hulin. C. L.. Fitzgerald, L. F. and DeNardo, M. (1999) Outcomes of self-labeling sexual harassment. Journal of Applied Psychology, 84(3), 390-402.

Makarem, N. N.. Tavitian-Elmadjian, L. R.. Brome, D.. H amadeh, G. N. and Einarsen, S. (2018) Assessment of workplace bullying: Reliability and validity of an Arabic version of the Negative Acts Questionnaire-Revised (NAQ-R). BMJ Open. 8( 12).

Merchant. V. and Hoel. H. (2003) Investigating complaints of bullying. In S. Einarsen. H. Hoel. D. Zapf and C. L. Cooper (eds.), Bullying and emotional abuse in the workplace. International perspectives in research and practice

Mikkelsen, E. G. and Einarsen. S. (2001) Bullying in Danish worklife: Prevalence and health correlates. European Journal of Work and Organizational Psychology, 10, 393-414.