Codebook for the inter-coder reliability test
Military change post-Musharraf
This refers to the process of military transition and transformation. To what extent is there any observable change in the military institution, in terms of military strategy, doctrine, policy or at an institutional, organisational or operational level? How has this change manifested? Was it a positive or negative change? How does the trajectory of this change look like and what are the possible determinants of this change?
Area of operation (policy sectors or geographic area)
This refers to the policy sector (e.g. development, education, health. Countering Violent Extremism (CVE), security research) in which NGOs operate. Area of operation refers to the geographic region in which NGOs conduct their projects - is a province/area with an instable, volatile security environment, which has a sensitive status, as per government regulations in place? Zones of conflict and high risk of terrorism activity or areas with conservative (tribal) communities are usually defined as sensitive areas.
Non-governmental organisations ’ capacitylimpact
This refers to:
- • Capacity to conduct work which would improve security in Pakistan. Security here is understood broadly, encompassing both human security and national security notions. Capacity can refer to their funding capacity, expertise, professionalism, substantively related to their project. Capacity also refers to NGOs’ ability to assess their impact.
- • Impact on improving security, understood broadly as human security and national security. The impact can be positive, negative, mixed, neutral or weak.
- 180 Inter-coder reliability test
- • NGOs capacity and impact are highly dependent on trust relationships with other actors and NGOs’ credibility.
Result of the Inter-Coder Reliability Test
Node |
Source |
Source size |
Kappa |
Agreement (%) |
A and В (%) |
Not A and not В ('%>) |
Dis agreement (%) |
A and Not В (%) |
В and not A (%) |
Area of operation (policy sectors or geographic area) |
Coding units |
8,579 chars |
0.7747 |
95.62 |
8.67 |
86.94 |
4.38 |
4.31 |
0.07 |
Military change post-Musharraf |
Coding units |
8,579 chars |
0.7202 |
86.42 |
33.99 |
52.43 |
13.58 |
3.33 |
10.25 |
NGO capacity or impact |
Coding units |
8,579 chars |
0.6421 |
82.13 |
38.91 |
43.22 |
17.87 |
9.83 |
8.04 |
Annex 3. Standardised survey questions1