Evolution: Advances in the Project and Methodological Decisions Taken

For more than two decades of activity, PE LA has experienced a clear evolution in the quantity and nature of data collected. This has, of course, given rise to methodological discussions that sought to improve the validity and reliability of the data, as well as new information that might be able to enrich the study of parliamentary elites in the Latin American region.

With the data collected from 1994 to 2016, there has been a substantial increase in the number of interviews: from 933 in the first wave to 1,580 in the third. From that point on. there has been a decrease in the number of interviews, for two main reasons. First, the fieldwork conducted in Venezuela from 2006 was terminated due to difficulties associated with obtaining a representative sample. Second, data from Brazil during the 2011-14 legislative term are still being processed and are therefore not included in these statistics. So, apart from Venezuela, the number of interviews has steadily increased from 1994 to the present. It should be noted that the interviews conducted during the sixth wave (2014-18) have not yet been included, as that round of fieldwork is still in progress.

In conjunction with these data variations, the project has had to confront certain methodological challenges. In this chapter, we place those challenges into four categories: a) questionnaire design; b) unit of analysis; c) fieldwork; and d) data exploitation.

Methodological Decisions in Questionnaire Design

Despite the standardised goal to facilitate comparisons between waves and countries, certain adjustments have been introduced over the course of the project’s development designed to improve the validity and reliability of the data gathered. In this sense, as shown in Table 12.3, early emphasis should be placed on the modification of some question and answer categories that are meant to obtain information or to facilitate its understanding.

Another methodological decision concerned the inclusion of new questions. In that sense, despite problems associated with inclusions that lengthen the questionnaire and consequently complicate fieldwork, it was decided to give priority to present-day subjects of interest to political science and the project itself. Particularly in the most recent waves, some questions related to two specific subfields of research: the study of professionalisation and quality in politics have been included as indicators to potentially measure variables such as political career and leadership;3 while some questions focused on the level of political congruence, the degree of coincidence and any superposition between the attitudes and preferences of citizens and of legislators.

It should be noted that PELA has collaborated with the Barometer of the Americas team at Vanderbilt University in the United States. For nearly three decades this team has spearheaded the Latin American Public Opinion Project (LAPOP),4 a public opinion survey that shares several thematic areas with PELA. Because of this collaboration, changes were made to some questions in order to better fit the approach of the Vanderbilt project.5 Elsewhere, other questions were eliminated because those interviewed found them difficult to understand or because the information they produced did not

Table 12.3 Example of modifications to question and/or answer categories to improve data validity and reliability

Original question

Currently, there is discussion around the role of the government. In this sense, can you please indicate to what extent you either agree or disagree with the following statements?

In a society as complex as this one, the government cannot solve citizens’ problems.

Intervention of the government in socio-economic life is the only way to reduce social inequalities.

The government should focus on a series of specific areas (health, education, justice, etc.) and leave the rest in the hands of individuals.

The government should intervene as little as possible in society and leave private initiative to deal with citizens’ needs.

Revised question

I am going to read some statements about the role of the national government. Please indicate to what extent you either agree or disagree with the following statements using a range where one means ‘strongly disagree’ and seven means ‘strongly agree'.

The government, instead of the private sector, should own the most essential enterprises and industries of the country.

The government, instead of individuals, should be the main agent responsible for the welfare of people.

The government, instead of the private sector, should be the main agent responsible for creating jobs.

The government should implement strong policies to reduce income inequality between the rich and the poor.

The government, instead of the private sector, should be the main agent responsible for providing retirement pensions.

The government, instead of the private sector, should be the main agent responsible for providing health services.

The government, instead of the private sector, should be the main agent responsible for providing university education.

The government should implement strong policies to reduce inequalities between men and women.

The government should implement strong policies to reduce inequalities between different cultural and ethnic groups.

Source: authors' elaboration using PELA.

correspond to that required by the project. Yet more were omitted due to changes in the regional context.6

Another methodological choice discussed over the two decades related to ranges of answers. As Garcia, Mateos and Rivas (2014) argue, a debate is continuing in respect of the use of answer ranges, as in the choice of extension (e.g. 0-10, or 1-7 or the traditional range of 1-10). In the original questionnaire design, a 1-10 range was used, despite a bias effect in which the interviewee considers five to be a ‘medium’ response. The research team chose not to modify this range, so as not to lose comparability with data already gathered.

Methodological Decisions Relating to the Unit of Analysis

The different types of Latin American parliaments presented different levels of fragmentation, with some composed of a large number of parties holding few seats. This involves the challenge of guaranteeing the anonymity of MPs within minority groups (Garcia, Mateos and Rivas 2014), and the challenge of avoiding a plenitude of categories that would make the use and presentation of data less parsimonious. As a solution, it was decided to create the joint category of ‘others’, in which parties with fewer than five MPs would be included.

Another methodological challenge that needed addressing was related to defections by MPs. Changes in the composition of parties following elections often produced changes in the initial sample design. To deal with this situation, it was decided that each sample would be designed according to the composition of legislative bodies in the immediate aftermath of the election, and would ignore any subsequent defections. That is to say, the MPs interviewed are included in the parties to which they were initially attached. This has allowed us to adopt homogeneous criteria for every legislature and to take an initial snapshot of each newly elected parliament.

The third challenge concerns comparisons between countries. Despite the fact that every legislature in every country is taken as an independent unit for analysis, comparative logic is given priority over case studies. In the same way, specific questions are not included for each particular country, in order to improve the comparability and homogeneity of questions and to obtain valid and reliable indicators.

Finally, the time dimension also implies a challenge. During the more than two decades of research, countries in the region have experienced substantive changes in context that may have affected the profiles of MPs as well as their opinions and attitudes. However, as when drawing comparisons between countries, questions relating to specific situations have been avoided in order to facilitate comparisons and standardisation of the questionnaire.

And yet another element - the guarantee of anonymity - prevents us from tracking the ideological evolution of a given MP over their career. It is not, therefore, possible to compare answers given by the same MP at different times. Research ethics towards anonymity are given priority, in order to ensure they contribute to the development of the project.

Methodological Decisions Relating to Fieldwork

Most discussions around this subject have been classified along four axes: a) the moment the questionnaire is applied; b) researchers personally in charge of carrying out the survey; c) questionnaire application by telematic means; and d) cases in which the sample could not be completed.

With regard to when the questionnaire is applied, the research team decided the ideal moment would be at the beginning of the legislative mandate. In this way, we avoid distortions and biases provoked in MPs who are preparing for elections in the near future (especially if they are in mid-campaign), as well as, by contrast, changes in attitude that may be produced later as a consequence of an erosion in the exercise of their duties.

In the second place, from the beginning of the project to the present day, the survey has generally been applied by members of the research team. This involves researchers travelling to the country in question, sometimes with the support of staff and institutions at the location. This presence in the field by members of the team responds to two rationales. The first relates to the need for the fieldwork to be coordinated and supervised by staff already deeply familiar with both the questionnaire and the methodology, in order to avoid doubts or contingencies. Second, this tactic responds to the logic of periodic revisions of the questionnaire to improve the validity and reliability of the indicators used. Application of the questionnaire by members of the team further allows the detection of weaknesses when measuring question and answer categories.

As for telematic application, in recent years there has been an incremental growth of studies using online surveys. However, given the particularities of the sample and the object of study in the PELA project, the survey has so far been applied face-to-face by project researchers. Moreover, to control measurement errors of comprehension and to observe possible inconsistencies that are susceptible to revision, it was decided to not implement the self-administered survey option in order to avoid completion of the questionnaire by a member of the MP’s team, rather than by the MP.

The final methodological decision related to fieldwork was the exclusion of Venezuela as a unit of analysis after the second wave. Despite it being one of the four initial countries included in the project, the fieldwork difficulties following the 2005 elections hindered the collection of a sufficiently representative sample, with no data available for the 2006-11 and 2011-16 periods. Nevertheless, following the 2015 legislative elections, it was possible to resume fieldwork in Venezuela and to obtain a representative sample.

Methodological Decisions Relating to Analysis and Data Exploitation

When exploiting data, the methodological debates have focused on three axes: a) left-right; b) elites-citizens; and c) quality of democracy.

With regard to the first, the use of the terms ‘left’ and ‘right’ in public life in a heterogeneous region such as Latin America has historically proven difficult and ineffective (Alcantara and Rivas 2007). Given this, and despite several authors having argued for the importance of these terms in characterising the behaviour of actors and parties in Western Europe and the United States (Inglehart and Kungemann 1976; Sartori 1976; Kitschelt and Hellemans 1990; Mair 1997), in the particular case of Latin American the impact of other variables (such as populism) should be taken into account in describing the political universe.

However, despite the complexity the left-right axis may present for Latin America, the evolution of democracies in the region has allowed us to capture the characteristics of political and party life along a left-right continuum (Coppedge 1998). In this regard, a number of works have highlighted the notion that, despite differences that may exist, this dimension is observed by Latin American MPs themselves, both for ideological self-placement and for parties and leaders.7 This is associated with specific attitudes and opinions that facilitate and guarantee the validity of the indicator (Alcantara and Llamazares 2006).

As for the second issue, the ongoing collaboration between the Barometer of the Americas team at Vanderbilt University and the University of Salamanca has allowed a deepening in the study of MPs and voters in terms of ideology and programme. In that sense, as Garcia, Mateos and Rivas (2014) show, three objectives are at play: a) grasping the direct relationship between citizens and MPs; b) measuring coincident and divergent interests between them; and c) presenting a global view of coincidence and divergence between the positions of these groups.

The study of congruence between elites and citizens gains special importance in the Latin American context, where a crisis of representation is seen to exist (Mainwaring, Bejarano and Pizarro 2006; Hagopian 1998). As Soroka and Wlezein (2011) argue, it is necessary to delineate the extent to which government action responds to citizen demands and to ask whether errors in the mechanisms of representation involve a source of disaffection and make the consolidation of democracy difficult. In view of this, researchers from PE LA have concentrated their efforts on assessing the level of congruence of opinions and attitudes between MPs and voters from a political, social and economic perspective.

Finally, the relationship between representation and the quality of democracy is a priority area for the research team. The opinions and attitudes of elites play an unquestionable role in the stability of democracy (Diamond 1999), given that they exert a major influence on decision processes compared to other civil society actors. Their professional dedication to politics and greater access to information produce in the governing elites a more elaborate system of beliefs than found among other actors (Dahl 1971). These systems of belief are in turn transferred to society and influence the political culture of the country.

However, the concept of quality of democracy continues to present several methodological challenges due to a high level of abstraction, causing difficulties in operationalisation. In respect of this problem, a debate persists between two positions: one arguing that the level of the quality of democracy can be assessed by citizens; the other connecting it to institutional performance (O'Donnell 1999). Thus, the opinions and attitudes of elites on economic, political and social issues, as well as their positions on policy development, acquire great importance.

 
Source
< Prev   CONTENTS   Source   Next >