Introduction
This essay evaluates key methodological aspects of Zhang and Conrad’s (2012) study on media coverage, public opinion, and presidential agendas in international relations, from the perspective of international development with economics. Focusing on how foreign policy influences national images and economic interactions, it addresses two questions: first, the measurement, validity, and reliability of valence in the policy agenda; second, the internal validity of Hypothesis 5 results. Drawing on concepts from research methods lectures and seminars, the analysis highlights strengths and limitations, emphasising implications for development economics where policy perceptions affect aid and trade.
Measurement of Valence in Policy Agenda: Validity and Reliability
Zhang and Conrad (2012) conceptualise valence in the policy agenda as the positive or negative tone in presidential portrayals of foreign countries, reflecting the executive’s strategic framing to influence public opinion and foreign policy. The policy agenda itself is defined as the prioritisation of issues in official presidential communications, such as State of the Union addresses, which signal economic and developmental priorities (e.g., alliances or sanctions impacting global trade).
To operationalise this, researchers conducted content analysis on 42 State of the Union speeches from 1967 to 2008, coding each mention of a foreign country for valence: positive (e.g., praise for cooperation), negative (e.g., criticism of threats), or neutral. The unit of observation is each individual country mention within a speech, while the unit of analysis is the aggregated valence score per country per year, calculated as (positive mentions – negative mentions) / total mentions.
This measure demonstrates reasonable validity by accurately capturing the conceptual essence of tonal framing in policy agendas, aligning with agenda-setting theory (McCombs and Shaw, 1972). For instance, it validly reflects how positive valence might boost economic partnerships in development contexts. However, limitations include potential construct invalidity, as neutral mentions are not weighted, possibly underrepresenting subtle negativity in economic critiques. Reliability is strong due to inter-coder agreement (Krippendorff’s alpha > 0.8), ensuring consistent coding. Yet, subjectivity in interpreting tone could introduce inconsistency, especially for ambiguous economic references (e.g., “trade imbalance” as negative).
The study links this to public opinion via the Chicago Council on Global Affairs’ surveys, which measure citizens’ favourability towards countries through biennial polls (Smeltz et al., 2016). These surveys use feeling thermometers (0-100 scale) to gauge opinions, enhancing the measure’s external validity by connecting policy valence to public sentiment.
Strengths include comprehensive historical coverage, aiding reliability in longitudinal economic analyses. Limitations, however, involve sampling bias towards U.S.-centric views, potentially unreliable for global development contexts where multilateral agendas differ.
(Word count: 278, excluding references)
Internal Validity of Hypothesis 5 Results
Hypothesis 5 (H5 on page 86, corresponding to H6 on page 82) posits that the valence of a country’s portrayal in the presidential policy agenda positively influences public opinion valence towards that country (Zhang and Conrad, 2012). Internal validity assesses whether observed correlations genuinely reflect causation, free from confounding factors.
The research design employs a time-series analysis with Granger causality tests on panel data from 1967-2008, using lagged variables to infer directionality between policy agenda valence (independent variable, measured as described above) and public opinion valence (dependent variable, from Global Views surveys). This quasi-experimental approach controls for temporality, enhancing internal validity by suggesting policy agendas precede opinion shifts.
However, limitations arise from the correlational design, lacking random assignment, which risks omitted variable bias. For example, media coverage—another variable in the study—might confound the relationship, as sensational news could independently shape opinions (Entman, 2004). Economic factors, such as trade deficits or development aid flows, could also intervene; positive policy valence might correlate with economic benefits, indirectly affecting opinions rather than directly causing them.
Other potential confounders include historical events (e.g., wars) or partisan biases, not fully accounted for in the models. While fixed effects mitigate some issues, the design’s observational nature limits causal claims, potentially overestimating internal validity.
In international development economics, this matters as policy agendas influence investment climates; weak internal validity could mislead on how U.S. portrayals affect global economic perceptions.
(Word count: 242, excluding references)
Conclusion
In summary, Zhang and Conrad’s (2012) measure of policy agenda valence shows sound validity and reliability, though with subjectivity risks, while H5 results have moderate internal validity constrained by design limitations and confounders. These insights underscore methodological rigour’s importance in economics-driven international development research, where accurate measures inform policy impacts on global inequalities. Further experimental designs could strengthen causality claims.
References
- Entman, R.M. (2004) Projections of power: Framing news, public opinion, and U.S. foreign policy. University of Chicago Press.
- McCombs, M.E. and Shaw, D.L. (1972) ‘The agenda-setting function of mass media’, Public Opinion Quarterly, 36(2), pp. 176-187.
- Smeltz, D., Daalder, I., Friedhoff, J. and Kafura, C. (2016) America in the age of uncertainty: American public opinion and U.S. foreign policy. Chicago Council on Global Affairs.
- Zhang, C. and Conrad, C.R. (2012) ‘International coverage, foreign policy, and national image: Exploring the complexities of media coverage, public opinion, and presidential agenda’, International Journal of Communication, 6, pp. 76-95. Available at: https://ijoc.org/index.php/ijoc/article/view/1424.
