2.1 Text Processing and LDA Topic Inference for Each Camp

We selected parties that were on the opposing poles of the political spectrum: two radical right parties and two ethnic parties. We first collected all the documents, generated a ranked list of n-gram keywords and placed the most frequent n-grams on the top of the list.Footnote ⁵ Using the ranked list of n-gram keywords, experts identified grid/group issues (topics) using a classification scheme derived from general social theory (Douglas Reference Douglas1970; Douglas and Wildavsky Reference Douglas and Wildavsky1982).Footnote ⁶

The grid–group approach to the study of politics has been used to study mass political beliefs (Coughlin and Lockhart Reference Coughlin and Lockhart1998), the determinants of dimension dominance (Rehm and Kitschelt Reference Rehm, Kitschelt, Manow, Palier and Schwander2018), party alignments in Western Europe (Rehm and Kitschelt Reference Rehm, Kitschelt, Beramendi, Hausermann, Kitschelt and Kriesi2015), complex political orientations beyond the traditional left–right scale (Grendstad Reference Grendstad2003) and radical right parties (Bustikova and Kitschelt Reference Bustikova and Kitschelt2009). As a classification system, it places political orientations into four categories using the two axes of grid and group: hierarchy, egalitarianism, individualism and fatalism. It offers a more nuanced analytical tool for party classification than the left–right placement and is more versatile than the commonly used traditional versus libertarian distinction used in the Chapel Hill Expert Survey. It does not collapse identity onto one dimension and therefore can account for the fact that ethnic inclusion does not necessarily imply social liberalism.Footnote ⁷

Since our analysis investigates the responsiveness of polar opposites on the so-called “second” axis of party competition (cultural issues as opposed to economic issues), any spike in issues that the party discusses implies both an increase in salience and polarization (Spies and Franzmann Reference Spies and Franzmann2011). To capture different aspects of polarization and more granular action–reaction dynamics, the identity axis needs to be disaggregated. Grid–group allows the analyst to classify attitudes toward state authority as separate from ethnic issues.

We utilize LDA, one of the most popular topic inference algorithms (Blei et al. Reference Blei, Ng and Jordan2002). It assumes that documents represent a mixture of topics, where a topic is a probability distribution over words. In other words, it uses a “bag of words” approach to perform statistical topic modeling and to uncover hidden structures in large text corpora.Footnote ⁸ In our analysis, LDA outperformed two competitive alternatives: vector space and word embedding models.Footnote ⁹ After identifying the grid–group issues using ranked weighted TF–IDF terms, we applied LDA separately on ethnic and radical right parties’ corpora to discover their party-specific latent topics. For each grid–group issue, we determine when an issue is salient for one party (i.e., the issue-specific document volume crosses the threshold and constitutes a spike, as discussed below) and when that leads the other party to respond (with a temporally proximate and topically related spike).

2.2 Detecting Spikes, Similarity and Escalation

We utilize the 68-95-99.7 rule for spike detection (Pukelsheim Reference Pukelsheim1994), also known as the three-sigma rule, which states that in a normal distribution nearly all values lie within three standard deviations ( $\unicode[STIX]{x1D70E}$ ) of the mean ( $\unicode[STIX]{x1D707}$ ). We utilize a fixed-sized sliding window (experimentally determined as 20 weeks) to compute a running average $\unicode[STIX]{x1D707}(20)$ and a standard deviation $\unicode[STIX]{x1D70E}$ for each issue’s weekly volume distribution from each camp. We designate a weekly volume as a spike if the weekly document volume matching an issue exceeds ( $\unicode[STIX]{x1D707}(20)+2\unicode[STIX]{x1D70E}$ ). We tuned this sliding window to 20 weeks because it showed the best performance. When smaller windows (5 weeks, 10 weeks and 15 weeks) are applied, the resultant spikes are noisy. When larger windows are applied (25 weeks, 30 weeks and 35 weeks), the resultant spikes are sparse.

Spikes are categorized into two categories: (1) “escalated” spikes that trigger a reaction from the other camp or (2) “ignored” spikes that lead to no response—based on the relatedness (goodness of fit) of each topic distribution inferred by LDA in consecutive spikes from opposing camps. Spike categorization (escalated/ignored) is a result of shared topics between two consecutive spikes from opposing camps. By matching up consecutive spikes, we capture partisan debates, defined as “formal discussions on a set of related topics in which opposing perspectives and arguments are put forward.”Footnote ¹⁰ To measure the “relatedness” of topics between a pair of consecutive spikes from opposing camps, we utilize the Kullback–Leibler (KL) divergence between LDA topic distributions of consecutive spikes (Kullback and Leibler Reference Kullback and Leibler1951) and then convert it to a similarity measure, scaled between 0 and 1. The KL divergence of the probability distributions $E$ , $R$ on a finite set $X$ is defined asFootnote ¹¹

(1) $$\begin{eqnarray}D(E,R)=\mathop{\sum }_{x\in X}E(x)\log \frac{E(x)}{R(x)}.\end{eqnarray}$$

Given two consecutive spikes from opposing camps, ethnic party spike $S_{E}$ and radical right party spike $S_{R}$ , we first identify latent topics of each spike with their distributions within the documents: $E$ is the distributions of $S_{E}$ topics and $R$ is the distributions of $S_{R}$ topics. For example, when comparing two consecutive spikes related to the issue of “Minorities”—one spike with topics from the ethnic camp such as “minority languages, schools, …etc” versus the following spike from other camp with topics “gypsy problem, schools, …etc”—we measure the distributions of topics in these two spikes with respect to the number of documents matching the “Minorities” topic. We then measure the divergence of topic distributions using the symmetric form of KL divergence (Seghouane and Amari, Reference Seghouane and Amari2007) that measures the divergence of the probability distributions $E$ , $R$ on a finite set $X$ of topics as follows:

(2) $$\begin{eqnarray}D(E,R)=\mathop{\sum }_{x\in X}\bigg((E(x)-R(x))\log \frac{E(x)}{R(x)}\bigg).\end{eqnarray}$$

We normalize this measure with respect to the sum of distributions to be between [0,1] and convert it to a similarity measure as follows:

(3) $$\begin{eqnarray}Sim(S_{E},S_{R})=1-D_{\mathit{normalized}}(E,R),\end{eqnarray}$$

where $Sim(S_{E},S_{E})=1$ means the two distributions of topics across the two spikes are identical. If the similarity of topic distributions between the two consecutive spikes from opposing camps exceeds a certain threshold, then we label the first spike as “escalated”; otherwise, it is considered “ignored.”Footnote ¹²

LDA is used twice. The first time, LDA is applied on the overall corpus (both radical and ethnic corpus) to measure the relatedness of spikes. The second time, LDA is used separately on the radical corpus (radical right parties) and on the ethnic corpus (ethnic parties) to determine issue-specific frames to be exploited as features for the predictive model.

2.3 Models

We experimented with three methods: a baseline vector space model, a word embedding model and an LDA model.

2.3.1 Baseline Vector Space Model

In our baseline model (vector space model), we directly modeled the similarity approach by using the cosine similarity over spikes’ frequent keyword vector representation of $E$ and $R$ , without requiring a lower dimensional space representation of the data, for example, inferring topic distribution LDA or word embedding document to vector as follows:

(4) $$\begin{eqnarray}Sim(S_{E},S_{R})=Cosine(E,R)=\frac{E\cdot R}{\Vert E\Vert _{2}\cdot \Vert R\Vert _{2}}.\end{eqnarray}$$

The similarity measure is a sparse vector representation of frequent keywords. We computed all distances between consecutive spikes using Equation (4) and, by thresholding, determined the labels of spikes whose measure is larger than or equal to the mean, indicating an “escalated” spike from the opposing camp; otherwise, they are labeled as “ignored.”

2.4 Framing Analysis and Predicting Escalation

During a debate on a particular topic, both radical right parties and ethnic parties discuss different perspectives.Footnote ¹⁴ Once escalated and ignored spikes from one camp are determined, we use a sparse-learning framework (Liu et al., Reference Liu, Chen and Ye2009a), with the aim of selecting a subset of discriminating features that can identify and classify contentious (escalatory) spikes as opposed to ignored ones. The following steps describe our algorithm:

1. (1) For each key grid–group issue, run LDA to get latent topics for one camp. Footnote ¹⁵

2. (2) Filter the frame $\times$ spike matrix to include only the top 2,000 terms representing frames from one camp (100 topics (or topic dimensions) each with top 20 terms inferred).

3. (3) Formulate the problem as a logit model streamlined by the SLEP framework (Liu et al., Reference Liu, Chen and Ye2009a) to predict escalated versus ignored spikes. Formally,

(5) $$\begin{eqnarray}\mathbf{min}_{x}\mathop{\sum }_{i=1}^{m}w_{i}\log (1+\exp (-y_{i}(x^{t}a_{i}+c)))+\unicode[STIX]{x1D706}\Vert x\Vert _{1},\end{eqnarray}$$

where $a_{i}$ is the vector representation of the $i$ th spike, $w_{i}$ is the weight assigned to the $i$ th spike ( $w_{i}=1/m$ by default), $A=[a_{1},a_{2},\ldots ,a_{m}]$ is the frame $\times$ spike matrix, $y_{i}$ is the polarity of each spike (+1 for an escalated spike and -1 for an ignored spike), $x_{j}$ , the $j$ th element of $x$ , is the unknown weight for each frame, ( $\unicode[STIX]{x1D706}>0$ ) is a regularization parameter that controls the sparsity of the solution and $|x|_{1}=\sum |x_{i}|$ is the 1-norm of the $x$ vector.

The sparse-learning approach (SLEP) relies on a gradient descent algorithm to solve the above convex and nonsmooth optimization problem (Liu et al., Reference Liu, Ji and Ye2009b). The frames with nonzero values on the sparse $x$ vector yield the discriminant factors for classifying a spike as escalated or ignored based on their polarity (positive or negative). Frames with positive polarity correspond to escalated frames and those with negative polarity to ignored frames.

3.1 The Data Corpus

The corpus comprises 10,597 news and opinion articles downloaded from the official websites of radical right and ethnic political parties in Slovakia from 2004 and 2014. From the ethnic camp, we downloaded all documents from Most-Híd (http://www.most-hid.sk) and from SMK—Party of the Hungarian Coalition (http://www.mkp.sk). From the radical right camp, we downloaded all documents from the SNS—Slovak National Party (http://www.sns.sk) and Slovenská Pospolitost́—The Slovak Brotherhood (https://pospolitost.wordpress.com). The document volume is roughly equal between the two camps.

While the method has broad potential applications for studying a diverse set of cases and topics, a word is in order about why this party system serves as an interesting and important case study to introduce this approach to predicting partisan responsiveness. First, the radical right parties and the ethnic parties in Slovakia are both relatively large compared to some “niche parties” in other countries; so they are politically relevant for coalition formation.

Second, the political space in Slovakia has been characterized by a high degree of variation in the extent to which it is polarized on issues of national identity, and this variability allows us to track a truly dynamic process of contestation. Finally, over the past two decades, the political scene in Slovakia has been quite stable in terms of the actors that anchor both political poles. This provides consistency over time in the analysis since the actors are identifiable with transparent profiles and reputations that have been established over a relatively long time period (Gyárfášová et al. Reference Gyárfášová2015; Baboš, Világi, and Oravcová Reference Baboš, Világi and Oravcová2016; Kluknavská and Smolík Reference Kluknavská and Smolík2016; Guasti and Mansfeldová Reference Guasti and Mansfeldová2018).

Figure 2. Contentious Frames Analyzer Tool. Zoom on the period from the middle of July 2013 to the middle of March 2014. Below the main plot is a secondary plot that shows the entire time period from 2004 to 2014. Users can simply slide the window to the time period of interest and the main plot will zoom in on this period and identify ignored and escalated spikes for a given topic, which can be selected on the right (Minorities, Nation, Language, Interstate (Relations), Economics, EU/Enlargement), and for a given actor in the dyad (ethnic or radical right party), which can be selected below the main plot: Ethnic eliciting Radical reaction prediction (left) or Radical eliciting Ethnic reaction prediction (right).

Figure 2 displays spikes of attention over time to one topic (in this case, “Language”), by both camps (radical spikes are red and ethnic spikes are blue), and shows that the adversaries mobilize in bursts.Footnote ¹⁶ It also shows whether a spike from one camp is ignored or reacted to by the other camp in the form of a new spike on the same topic. The bottom panel in Figure 2 shows (in gray) the overall volume of contentious frames (2008–2011) during a period of intense debate over a very restrictive language law, adopted in September 2009 with the help of the SNS, and the ensuing efforts of ethnic parties to soften its negative impact on Hungarians. Finally, Figure 2 illustrates the volume of documents generated by radical (red line) and ethnic (blue line) political party outlets in Slovakia over the entire 10-year period (between 2004 and 2014) that we analyze.

Not all topics resonate within the dyad. The “predictions” panel on the right of Figure 2 shows a timeline (using alphabetic annotations) that corresponds to ethnic party spikes and predictions about whether they will lead to “escalation” as a result of the radical right parties responding. Green labels indicate a “hit” (correct prediction) and red labels indicate a “miss” (incorrect prediction) by the classifier. In the screenshot displayed, which covers from July 2013 to March 2014, the classifier correctly hits 9 of out 10 spikes.

Subject matter experts selected the key issues shown in Table 2 and mapped them onto the group (nationalism) and grid (state authority) dimensions. Focusing on these six topics, the framework categorizes spikes as either escalated or ignored for both radical right parties and for ethnic parties and then uses this information to predict partisan responsiveness. To determine whether a pair of consecutive spikes on one of these six topics is related, the mean similarity for each grid/group issue was used as the threshold. If the similarity between a spike (from one camp) and the following spike (from the other camp) exceeds the mean similarity for an issue, then the first spike is labeled escalated and otherwise it is labeled ignored.

Table 1. Number of Analyzed Documents.

Ethnic parties are: SMK—Strana mad’arskej koalície, Party of the Hungarian Coalition (now Strana mad’arskej komunity) and Most-Híd—Bridge).

Radical right parties are: SNS—Slovenská národná strana, Slovak National Party and Slovenská Pospolitost´, Slovak Brotherhood.

Table 2. Group (Nationalism) and Grid (State Authority) Issues.

To illustrate, Figure 3 shows the similarity measure ( $y$ -axis) for ethnic party spikes, for each of the six grid/group issues ( $x$ -axis). Dots represent spikes and boxes show the means (which vary between 0.35 and 0.45), along with the first and third quartiles, with whiskers for the 95% confidence intervals on the similarity measure. Once spikes from each camp are detected and categorized, the terms that comprise the topics are used as features for the SLEP classifier (Liu, Reference Liu, Ji and Ye2009b) to identify discriminative frames. Using 10-fold cross-validation (McLachlan, Reference Mclachlan, Do and Ambroise2004), we calculated the precision, recall and F-measure (Perry, Reference Perry, Kent and Berry1955).

3.2 Model Performance and Frame Detections Based on the LDA Model

Table 3 displays the performance of the vector space model, the word embedding model and LDA. In this table, the LDA model outperformed the other two models (F-measure).Footnote ¹⁷ Using the same set of LDA-based features, the SLEP classifier outperforms both the Naive Bayes and Random Forest classifiers in terms of the overall F-measure, which is consistently higher for SLEP than for either alternative classifier across all escalated and ignored topics.Footnote ¹⁸

Figure 3. Similarity Measures for Ethnic Spikes.

Table 3. F-measures of Three Models.

Figure 4 shows the “language” topics over which ethnic parties and radical parties fight (e.g., languag(e) Slovak, Hungarian school, minor(ity) nation, human right, educ(ational) minist(ry)). The Venn diagram shows two intersected circles, where the first circle belongs to the first spikes with top terms and the second circle represents the spikes from the other party. The intersecting area represents the common terms. Language education emerges as an intrinsic focal point spurring debate, in this case, prompted by ethnic parties and followed by a spike from radical right parties.

Figure 4. Intersecting Topics.

Table 4. Contentious Frames.

3.3 Contentious Frames and Polarization in Slovakia

To assess which topics are contentious and polarize public discourse and which topics are ignored, we turn to the intersecting topics and “contentious frames” identified in Figure 4 and Table 4, which lists issue-specific frames used by each camp that tended to elicit reactions from the other camp. We then compare these with the “ignored frames” in Table 5.

The left column of Table 4 depicts the Hungarian–Slovak political cleavage over language and Hungarian minority rights rather clearly. This is consistent with decade-old fights over the status of the Hungarian minority, particularly its language rights (Bútora Reference Bútora2007; Haughton and Ryba Reference Haughton and Ryba2008; Mesežnikov, Gyárfášová, and Smilov Reference Mesežnikov, Gyárfášová and Smilov2008; Deegan Krause and Haughton Reference Deegan Krause and Haughton2009). Radical parties tend to escalate on language policies: they respond strongly when ethnic parties talk about the “language law,” “minority language” and “mother tongue.”

Slovakia is home to two ethnic minorities: politically mobilized Hungarians and demobilized, impoverished Roma. The computational results show that radical parties and (Hungarian) ethnic parties react differently to Roma issues. While radicals respond to frames that advance the rights of the Hungarian speakers, the reverse is not true: if radicals challenge the right of Hungarians to be politically accommodated, ethnic parties do not escalate. Instead, ethnic parties escalate if radical right parties launch attacks on Roma: a different ethnic group and when they invoke the legacy of interwar fascism associated with an independent Slovak state. Ethnic parties also respond when the radical right parties discuss “protection of the republic,” “white race” and the “Gypsy (Roma) problem.” The historical dimension associated with attempts to whitewash the fascist legacy of Jozef Tiso, who collaborated with the Nazis, is most evident in the frames “Nation,” “Language” and “Minority.”

Scholars of Slovak politics know that radical right parties attack both Hungarians and Roma, but the computational results also reveal that ethnic Hungarian parties are more likely to respond to radical frames that are not related to the rights of Hungarians but rather to Roma, and to historical frames. If ethnic parties escalate on issues of Roma and interwar legacies, they frame radicals as fascists and xenophobes, and thereby diminish radicals as credible adversaries that can be engaged to debate policy. This may explain why ethnic parties stay quiet when radical right parties question policies that expand their (language) rights.

Table 5. Ignored Frames.

Turning to the frames that were largely ignored by the other side of the political spectrum in Table 5, we see that radical right parties did not respond when topics were discussed in cultural terms but rather did so when these topics were discussed in policy terms. Under the issue “Minority,” for example, radicals did not respond to the frames “cultural minorities,” “theater” and “cultural activities.” Although these frames may suggest concessions to minorities, they fall short of recognition as a “national minority,” which implies language rights as well as political and economic power-sharing. Similarly, ethnic parties ignored radical frames under the issues of “Nation,” “Language” and “Minority” that focused on religion (“Pope Benedict,” “Jan Hus,” “church provinces” and “Slovak Church”).

Tables 4 and 5 underscore the fact that the key issues of contention between radical right parties and ethnic political parties are almost exclusively related to three main issues: (1) rights of Hungarians as a national minority, (2) hostility toward Roma, an ethnic underclass and (3) an interwar fascist legacy: a historical cleavage that concerns the Nazi collaboration of the first independent Slovak state during World War II. The computational and qualitative text analysis advances understanding of party politics with a new approach to highlighting the issue-specific causes of political mobilization and polarization. It reveals that radical right parties mobilize when their adversaries discuss minority rights, whereas the ethnic parties respond to adversarial frames that evolve around racism and historical autocratic regime legacies. In sum, parties on the opposing poles of the political spectrum respond to each other and selectively react to each other’s polarizing frames.