1.1 Effects of context on cross-dialectal speech perception

While context has been argued to impact speech perception because listeners shift into a dialect-specific listening mode to facilitate processing (by activating exemplars, changing priors or mappings, etc.), it is not clear if a dialect-specific listening mode is available to listeners with limited familiarity with the contextually-cued dialect. If we look to the literature, we can build three predictions for how context may impact perception in these cases. On the one hand, context may have no impact on speech perception when experience is limited; a listener’s representation of the unfamiliar dialect is too weak to be activated by context, and listeners perform poorly with the less familiar accent regardless of whether there is facilitative context or not (cf. Evans & Iverson, 2004; Van der Feest & Johnson, 2016; Walker et al., 2020; Weatherholtz et al., 2014). On the other hand, context may impact the processing of familiar and less familiar accents alike by prompting listeners to activate specific mappings, distributions, or exemplars in both cases. The robustness of the dialect representation does not matter, and even a small amount of lived experience or knowledge of stereotypes can be invoked with context (McGowan, 2015; see Wade et al., 2023, and Walker, 2019, for similar arguments about speech production).

A third hypothesis is that listeners do not shift into a dialect-specific listening mode for less familiar dialects, but instead shift into a more uncertain listening mode, allowing for more flexibility in signal-to-word mappings. Work coming from the perceptual adaptation literature suggests that, in response to exposure to atypical pronunciations, listeners may relax their criteria for phonemic categorization, demonstrated by a general increase in the endorsement of pseudowords (rather than a specific increase based on exposure to the variant of interest; e.g., Babel et al., 2021; Bissell & Clopper, 2025; Weatherholtz, 2015; Zheng & Samuel, 2020).¹ Relatedly, there is evidence that listeners are more flexible in their mappings when the acoustic signal is overall less reliable (Brouwer et al., 2012; McMurray et al., 2019; Van Ooijen, 1996) or if they have reason to expect dialectal variation (Clopper & Walker, 2017). Based on these findings, listeners may respond to a less familiar dialect by shifting into an uncertain listening mode, characterized by relaxed categorization criteria or overall category broadening. In other words, the context may make listeners aware that different acoustic-phonetic mappings are likely to occur but not increase their confidence about what these mappings should be.²

1.2 Talker identity as a contextual cue to talker dialect

In the present study, we play listeners speech samples from actresses performing two different dialect guises: Mainstream U.S. English (MUSE), a term we use here to capture something akin to a pan-regional variety associated with the middle class and regionally most similar to Midland and West Coast dialect regions (Clopper & Bradlow, 2008; cf. Lippi-Green, 1997, p. 59) and Southern U.S. English (SUSE), a highly enregistered regional variety tied to the southeast U.S. (Labov et al., 2006; Preston, 2018). The listeners in our study are from western Pennsylvania (treated as a distinct dialect region between the Midland and Inland North in Labov et al., 2006), and MUSE is expected to be more familiar to them than SUSE.³ Other studies have found that similar participants perform worse in listening tasks with SUSE compared to MUSE accents (Clopper & Bradlow, 2008; Walker et al., 2018b). We are interested in how performance with these two dialects (one more familiar, one less familiar) changes with contextual cues.

The contextual manipulation in our study is talker identity. While most studies manipulate perceived talker identity by pairing a target speech sample with photos or videos of people of different genders, ethnicities, ages or social classes (e.g., Drager, 2011; McGowan, 2015; Strand & Johnson, 1996), we use the term “talker identity” to refer to a talker that listeners are already familiar with. They have heard this particular person speak before and therefore have had a chance to form expectations about this person’s broader linguistic patterns (i.e., their accent and style), as well as their idiosyncrasies and vocal tract characteristics. Kleinschmidt and Jaeger (2015, p. 171) frame talker identity as the most valuable type of information a listener can have in making predictions about speech. There is plenty of evidence that listeners are tracking fairly fine-grained patterns in the speech of known individuals (Podesva et al., 2015; Remez et al., 1997) and that processing speech is easier if produced by a previously encountered talker (Clapp et al., 2023; Nygaard et al., 1994; Nygaard & Pisoni, 1998; Palmeri et al., 1993; Sheffert & Olson, 2004). In short, manipulating talker identity should be a strong and realistic way to alter listeners’ expectations of a talker’s regional accent.

In the present study, we use video (audio-visual presentation) to familiarize our listeners with the talkers—and, critically, with their regional accents—to arm them with talker-specific knowledge for later speech processing. Bimodal presentation of speech has been shown to aid in speech perception (e.g., Summerfield, 1979; see also Walker et al., 2020) and the learning of specific voices (Sheffert & Olson, 2004). Of most relevance to our study, Molnar et al. (2015) showed that early bilingual listeners use audio-visual talker identity to form predictions about which language to expect from a talker, as evidenced by an increase in reaction time for incongruent talker-language pairs (see also Walker et al., 1995). Critically, they found that late bilinguals simply show a slow down for their less dominant language, regardless of talker-language (in)congruence. Taken together, these findings suggest that different levels of experience with a language variety impact the usefulness of talker identity.

In addition to manipulating talker identity, Molnar et al. (2015) also manipulated talker-language predictability. This study, and their related EEG study (Martin, Molnar et al., 2016), familiarized Spanish-Basque bilingual listeners with two types of talkers: monolingual talkers, who only used either Spanish or Basque during familiarization, and bilingual talkers, who switched between Spanish and Basque during familiarization. In other words, the language of monolingual talkers was predictable, and the language of bilingual talkers was unpredictable. Both studies found evidence that predictability impacted performance on the subsequent test phase: In terms of both accuracy and ERPs, processing the unpredictable bilingual talkers was more challenging than processing the predictable monolingual talkers. The authors argue that, in the case of the bilingual code-switching talkers, listeners could not use talker identity as a cue to predict the language of the upcoming signal. In fact, Martin, Molnar et al. (2016) even found differences between bilingual and monolingual talkers before the onset of audio, suggesting that listeners prepare to listen differently based on their linguistic expectations for a talker.

1.3 Regional dialects in online processing research

Like Molnar et al. (2015) and Martin, Molnar et al. (2016), we use talker identity as a cue to language variety, but with dialects, not languages. Specifically, we introduce participants to talkers who use MUSE (familiar), SUSE (less familiar), or who switch between both (unpredictable) during familiarization. We then see how listeners respond to the two dialects from these talkers in a lexical decision task. In addition to measuring lexical decision performance behaviorally,⁴ we also measure the impact of dialect familiarity and talker identity neurally with event-related potential (ERP) analysis of electroencephalographic (EEG) data. ERPs are measurements of electrical brain activity that are time-locked to an external event, such as hearing a word. ERPs provide a millisecond-by-millisecond measurement of the brain’s activity during mental processing as it unfolds over time, and behavioral measures (e.g., reaction times, accuracy) reflect the outcome of this process. ERPs can thus detect early stimulus-related changes in language processing that cannot be measured via behavioral responses, which typically capture the endpoint of processing. Combining neural (ERP) and behavioral (reaction time, accuracy) measures allows us to capture the different phases of language comprehension, beginning from the very onset of the stimulus and going through the participant’s manual response. In general, this combination of methods provides a full picture of the comprehension process. In our particular study, measuring both neural activity and behavior allows us to connect the sociolinguistic literature on cross-dialect communication to the neurocognitive literature on word recognition.

ERP research has found that familiarity with a particular dialect or accent impacts online processing of phonological (Brunellière & Soto-Faraco, 2013, 2015; Goslin et al., 2012), semantic (Martin, Garcia et al., 2016), and grammatical (Garcia et al., 2022; Weissler, 2021; cf. Zaharchuk et al., 2021) aspects of this variety. For example, Goslin et al. (2012) presented sentences in both the listeners’ own (D1) accent and several less familiar regional (D2) accents in a passive English listening task. They found that the D2 accents elicited larger phonological mapping negativities (PMN), indexing activation of phonological candidates, than the D1 accent. On the later N400 component, indexing lexico-semantic access, D1 and D2 processing did not differ. Together, these results suggest that mapping external acoustic signals onto internal phonetic categories requires more resources for less familiar regional accents than for the listener’s own accent; however, this effortful normalization process allows lexico-semantic access to proceed unimpeded. These results also demonstrate the benefit of investigating cross-dialectal communication with ERPs; observing differences on the PMN rather than the N400 component revealed that phonological rather than lexico-semantic processing was the source of comprehension difficulties in the D2. In other words, the latency of the ERP effects revealed the underlying mechanisms of cross-dialectal processing. By contrast, the behavioral measure of comprehension question accuracy only revealed that there were difficulties, but could not indicate at which stage of processing these difficulties arose.

In the present study, our participants are exposed to D1 and D2 accents through an auditory lexical decision task. In this task, listeners decide whether the token they heard was a real word or a pseudoword. ERP studies comparing real word and pseudoword processing typically observe a lexicality effect, where pseudowords elicit larger N400 amplitudes than real words (e.g., Chwilla et al., 1995; Holcomb et al., 2002; Liu & Van Hell, 2020; Martin, Molnar et al., 2016). A larger N400 is understood to reflect challenges in lexico-semantic access, with real words being easily found in the lexicon and pseudowords being searched for but never found (Federmeier, 2022; Kutas & Federmeier, 2011). The size of the N400 lexicality effect can also be influenced by talker identity; indeed, in their work with bilingual listeners, Martin, Molnar et al. (2016) found that bilingual talkers elicited smaller differences between real words and pseudowords on the N400 than monolingual talkers. In other words, lexical status was more easily established when the language was predictable. Differences in N400 amplitude have also been associated with neighborhood density, where real words and pseudowords with more lexical neighbors elicit larger N400 amplitudes than similar items with fewer neighbors (e.g., Carrasco-Ortiz et al., 2017; Meade et al., 2019). This N400 modulation is thought to be driven by an increase in global lexico-semantic processing due to the many coactivated neighbors. The effect of lexical competitors on the N400 is especially important to consider given the potential responses to dialect context we outlined in Section 1.1. If listeners respond to an unfamiliar dialect by broadening their expectations of possible mappings, this would introduce more lexical competition, reducing the N400 effect.

While lexicality effects are typically observed on the N400, Martin, Molnar et al. (2016), among others (e.g., Signoret et al., 2013), have also observed differences between real words and pseudowords on the N1, a much earlier component. The N1 is associated with processing the acoustic features of the signal (e.g., Helenius et al., 2002; Kapnoula & McMurray, 2021; for a review, see Getz & Toscano, 2021). A related component, the P2/P200, has been associated with processing phonetic category information (Brunellière et al., 2009). Both of these components are elicited by complex auditory stimuli and can be enhanced by attention (Čeponienė et al., 2005; Picton & Hillyard, 1974). In Martin, Molnar et al., the N1 lexicality effect—a larger response to pseudowords than to real words—was present only for the (predictable) monolingual talkers and not for the (unpredictable) bilingual talkers. This suggests that talker identity can facilitate lexical access at the earliest stages of auditory processing by cueing the upcoming acoustic-phonetic information in the speech signal. Additionally, as we mentioned earlier, Martin, Molnar et al. also observed differences in brain responses to talkers before the auditory stimulus was even played, with the visual presentation of the talker’s faces eliciting a larger P3b for bilingual talkers than for monolingual talkers. This pre-speech effect may reflect the activation of talker identity in anticipation of the upcoming auditory information. To summarize, the pre-lexical (i.e., earlier than the N400) effects of interest are the pre-speech P3b and the post-speech N1 and P2, which index different aspects of acoustic-phonetic processing. Together, these components may reflect the proactive use of talker identity for language processing.

1.4 Present study

Building on previous work, the present study investigates cross-dialectal (D1–D2) language processing both behaviorally and with EEG/ERPs. We manipulated talker identity—predictable monodialectal talkers and unpredictable bidialectal talkers—and accent familiarity—familiar Mainstream (MUSE) accents and less familiar Southern (SUSE) accents—to understand how listeners use their language experience for speech recognition. We audio-visually introduced participants to three types of talkers during an initial familiarization phase: Mainstream talkers, who used a MUSE accent throughout; Southern talkers, who used a SUSE accent throughout; and Unpredictable talkers, who switched between a MUSE and a SUSE accent. These videos established each talker’s identity, as well as the predictability of their accent. During the subsequent lexical decision task, listeners watched videos of the same talkers producing real word and pseudoword tokens. Critically, these tokens bore either a MUSE or a SUSE accent, which either aligned with or violated the listener’s expectations about the talker.

Following the work by Martin, Molnar et al. (2016), our hypotheses for the ERP data were focused on the N400 effect, where we had the strongest a priori predictions about the effects of lexicality. These predictions are laid out below in Table 1. We present three ways that talker and token accent could impact the N400, assuming that our western Pennsylvania listeners will be more familiar with MUSE compared to SUSE. First, if high familiarity with a dialect is necessary to form specific expectations about a talker’s pronunciation, we would expect the properties of the token to matter more than the properties of the talker. That is, we would expect pseudowords to elicit larger N400 effects than real words regardless of the talker’s expected accent, but this effect would be mediated by token accent (with the expectation of a weaker N400 lexicality effect for the less familiar SUSE accent). Alternatively, if listeners adopt a more uncertain listening strategy when faced with a talker of a less familiar dialect, we would expect to see strong lexicality effects for Mainstream talkers that would be attenuated for Southern and Unpredictable talkers. Finally, if listeners form robust expectations regardless of the degree of their previous experience with an accent, we would expect congruency between talker and token to matter more than the token itself. That is, we would expect SUSE real words and pseudowords to elicit weaker N400 effects than MUSE real words for the Mainstream talkers, and we would expect MUSE real words and pseudowords to elicit weaker N400 effects than SUSE real words for Southern talkers. For the Unpredictable talkers, we would expect a lexicality effect regardless of token accent, since neither variety has a stronger association with the talker than the other; however, this lexicality effect would probably be attenuated compared to the monodialectal talkers, who predictably use one variety or the other.

In addition, we were motivated to investigate three other ERP components: the pre-speech P3b and the pre-lexical N1 and P2. The pre-speech analysis was based on Martin, Molnar et al. (2016), who observed differential activation of talker identity for the bilingual versus the monolingual talkers. We expect a similar difference between our Unpredictable talkers and the two predictable talkers (Mainstream and Southern), which would reflect pre-activation of talker identity to facilitate perception of the unpredictable acoustics. We also investigated the pre-lexical N1 and P2, which index different levels of processing but tend to pattern together in response to salient stimuli. Martin, Molnar et al. observed a larger N1 response to pseudowords than to real words for monolingual but not for bilingual talkers. An analogous pattern in our study would be an N1 (and potentially P2) lexicality effect for Mainstream and Southern talkers but not for Unpredictable talkers. Together, these components should help us understand how the differences between a listener’s expectations, their underlying phonological representations, and the actual token accents are detected and resolved (or not) before lexico-semantic access takes place.

2.1 Participants

Thirty undergraduate students from Penn State’s subject pool received course credit for their participation. We recruited participants according to the following criteria: 18–35 years of age, right-handed, normal or corrected-to-normal vision, normal hearing, no history of head trauma, no history of language or neurological disorders, and monolingual English language background with limited second language experience. Two participants with bilingual language backgrounds were excluded prior to the EEG session. Five participants were unable to complete the EEG session due to technical difficulties. One additional participant was excluded after reporting abnormal hearing. Data from the remaining 22 participants were analyzed (Age: M = 18.36, SD = 0.58, Min = 18, Max = 20; Sex: 18 female, 4 male). All participants provided informed consent.

2.2 Materials

2.3 Design

Twelve experimental lists were created so that each word was said by all actresses in both accents across participants (6 actresses x 2 token accents) and each actress was presented with each talker identity in four lists (3 talker accents x 4). In each list, two of the actresses were presented as Southern talkers, two as Mainstream talkers, and two as Unpredictable talkers (Figure 1). An actress’ talker accent was reinforced throughout the lexical decision task, such that 58 of the 70 tokens from each SUSE talker (>80%; 38 real words and 20 pseudowords) were presented in a SUSE accent, 58/70 tokens from each MUSE talker were presented in a MUSE accent, and 35/70 tokens from each Unpredictable talker (50%; 25 real words and 10 pseudowords) were presented in each accent. In other words, only 12 of the 70 tokens for each monodialectal talker were produced with incongruent token accents (and these incongruencies were all real words). These distributions allowed us to maintain the Talker Accent established in the monologues (Southern, Mainstream, Unpredictable) while also investigating alignment with Token Accent (MUSE, SUSE).

Figure 1: Design for one of the 12 experimental lists. Numbers reflect how many trials were spoken by each individual talker at test, including both real words and pseudowords. For Predictable talkers, all of the incongruent trials were real words.

2.4 Procedure

Participants completed the language history questionnaire at the beginning of the experiment. The EEG session then began with the familiarization phase, in which each character’s Talker Accent was established. Recall that each actress was assigned two monologues. The monologues were presented to participants in two runs, such that the first monologue from each of the six talkers was presented in a row before the second set of monologues. Participants answered four short comprehension questions after each monologue (48 total) to encourage attention.

Immediately following the familiarization phase, participants completed the audio-visual lexical decision task while EEG was recorded. At the beginning of each trial, a fixation cross appeared for approximately 500ms. Next, the audio-visual stimulus began playing, in which one of the actresses was depicted producing either a real word or a pseudoword with either a MUSE or a SUSE accent. After the stimulus finished playing, participants saw “Real word?” on the screen and indicated whether the word was “real” or “fake” by pressing one of two buttons on the button box (response laterality was counterbalanced across participants). A response initiated the next trial. Talker accent, token accent, and token type were pseudo-randomized across trials (420 total), such that no more than three congruent trials from the same talker (e.g., a MUSE token produced by Sara as a Mainstream talker) appeared in a row and no more than two incongruent trials from the same talker (e.g., a MUSE token produced by Sara as a Southern talker) appeared in a row.

After the EEG session, participants were seated in a separate sound-attenuated booth at a desktop computer. The debriefing questionnaire was completed first, followed by the AX-CPT, O-Span, and verbal fluency tasks. Finally, participants completed the post-experiment questionnaire.

2.5 EEG acquisition and pre-processing

During the EEG session, participants sat in a comfortable chair approximately three feet from a computer monitor in a sound-attenuated booth. An elastic cap (Brain Products ActiCap, Germany) with 31 active Ag/AgCl electrodes located along the midline (Fz, FCz, Cz, Pz, Oz) and laterally (FP1/2, F7/8, F3/4, FC5/6, FC1/2, T7/8, C3/4, CP5/6, CP1/2, P7/8, P3/4, O1/2, PO9/10) recorded EEG. Bipolar recordings above and below the left eye (VEOG) and at the outer canthi of both eyes (HEOG) monitored for vertical and horizontal eye movements, respectively. Electrode impedances were kept below 10 kΩ. Online, the EEG signal used a vertex reference (FCz), was amplified by a NeuroScan SynampsRT amplifier with a 0.05–100 Hz bandpass filter (first-order Butterworth with 6 dB/octave roll-off), and was continuously sampled at a rate of 500 Hz. The EEG sessions were programmed with E-Prime 2.0 software. Auditory stimuli were presented over insert headphones (Etymotic Research Inc., Elk Grove Village, IL). Visual stimuli were presented on the computer monitor on a black screen. Button presses were recorded using a serial response box (Psychology Software Tools, Pittsburgh, PA).

EEG pre-processing and ERP measurement were conducted with the EEGLAB and ERPLAB MATLAB toolboxes (Brunner et al., 2013; Lopez-Calderon & Luck, 2014). We used a 30 Hz low-pass filter (24 dB/octave roll-off) and re-referenced the EEG signal to the average of the two mastoids. We conducted manual artifact rejection to remove any atypical eye or muscle activity or periods of line or channel noise. Head channels were removed if they exceeded a maximum flatline duration of five seconds, exhibited a channel correlation of lower than 0.6, or exceeded the line noise threshold of four standard deviations. The data were then submitted to independent component analysis (ICA). We removed ICA components associated with (1) eye activity over 70% and brain activity less than 25% or (2) channel noise over 90% and brain activity less than 10% (Number removed: M = 1.09, SD = 0.81, Min = 0, Max = 3). Any bad head channels that had been removed before ICA were then interpolated before we time-locked the EEG signal (Number removed: M = 1.23, SD = 1.51, Min = 0, Max = 6).

We conducted two time-locking procedures: one to the onset of the video (before the audio) to replicate Martin, Molnar et al.’s (2016) analysis of talker predictability; and one to the onset of the audio, to investigate processing of the speech signal itself. Time-locking to the onset of the word is a typical procedure for ERP research with auditory stimuli (see Kutas & Federmeier, 2011). Baseline correction (–200ms to 0ms) and artifact rejection (peak-to-peak activity exceeding ∓60 mV in eye channels or ∓100 mV in head channels) were conducted separately for the video and audio ERPs (from 0 to 600ms and from 0 to 1000ms, respectively). Overall, 3.32% of trials for the video analysis and 3.42% of trials for the audio analysis were removed during pre-processing. The data for each channel of interest (13) and trial (420 possible) were then extracted in 2ms increments. The channels of interest were frontal/fronto-central (F3, Fz, F4, FC1, FC2), central/centro-parietal (C3, Cz, C4, CP1, CP2), and parietal (P3, Pz, P4), as in Martin, Molnar et al.

2.6 Data preparation and analysis approach

Data preparation and analysis were conducted with R version 4.2.2 (R Core Team, 2022). Mixed-effects models were fitted to trial-level data with the lme4 package (Bates et al., 2015). Generalized linear mixed-effects models with a binomial family function were fitted to the behavioral accuracy data, while linear mixed-effects models were fitted to the behavioral RT and ERP data. The fixed and random effects structure for each model is described below. To test the main effects and interactions in each model, Type-III analysis-of-deviance tables were calculated and Wald chi-square tests were conducted with the car package (Fox & Weisberg, 2019). To investigate simple effects, estimated marginal means were calculated and pairwise comparisons were conducted with the emmeans package (Lenth, 2022). Pairwise p-values were adjusted with the Hommel method to control the family-wise error rate (Blakesley et al., 2009).

All models included Talker Accent (Mainstream, Southern, Unpredictable), Helmert contrast-coded, as a fixed effect. The fixed effect of Token Condition (MUSE real word, SUSE real word, Pseudoword), Helmert contrast-coded, was also included in all models except for the pre-audio ERP analysis, where it was not relevant (listeners had not yet heard the actual token). Token Condition combines Token Accent (MUSE, SUSE) and Token Type (real word, pseudoword) into one factor in order to model a full-rank interaction with Talker Accent. Recall that the monodialectal talkers only produced accent-congruent pseudowords (i.e., Mainstream talkers only produced MUSE pseudowords), while the bidialectal talkers produced pseudowords in both token accents. This approach was taken in order to balance (1) maintaining each actress’ Talker Accent throughout the study while (2) achieving an equal number of MUSE and SUSE tokens across conditions; however, this design resulted in an unbalanced three-way interaction among Talker Accent, Token Accent, and Token Type. Using Token Condition allowed us to model the interaction with Talker Accent without rank deficiency.

In addition to the fixed effects described above, models included mean-centered trial as a covariate to control for global changes in response patterns over the course of the experiment. Models also included random intercepts for Item, nested under Talker, and Participant. In addition, the ERP analyses included random intercepts for Channel. Model fitting began with random by-participant slopes for Talker Accent and Token Condition (except for the pre-audio ERP analysis). In the case of non-convergence, singularity, or correlations above 0.95, the random effects structure was simplified iteratively—first by removing the correlation between the slope and intercept estimates, then by removing random slopes but including the correlation parameter—such that the final model for each analysis reflected the maximally-supported structure (Matuschek et al., 2017).

3.1 Behavioral analysis

3.1.1 Accuracy

There was a significant main effect of Token Condition (χ²(2, N = 3) = 123.26, p < .001), qualified by a significant interaction with Talker Accent (χ²(4, N = 5) = 14.12, p = .007), on lexical decision accuracy. The main effect of Token Condition reflected a graded pattern of accuracy descending from MUSE real words (M = 0.96, 95% CI [0.95, 0.98]) to SUSE real words (M = 0.93, 95% CI [0.90, 0.95]) to pseudowords (M = 0.87, 95% CI [0.83, 0.90]). All three token conditions differed significantly from one another: MUSE real words versus SUSE real words (z = 6.33, p < .001), MUSE real words versus pseudowords (z = 11.07, p < .001), SUSE real words versus pseudowords (z = 5.91, p < .001).

To investigate the interaction between Token Condition and Talker Accent, we conducted pairwise comparisons between token conditions within each talker accent. These are shown in Figure 2. For Mainstream talkers, accuracy was higher on MUSE real words (M = 0.96, 95% CI [0.95, 0.98]) than on SUSE real words (M = 0.91, 95% CI [0.86, 0.94]; z = 4.92, p < .001) or pseudowords (M = 0.90, 95% CI [0.85, 0.93]; z = 6.11, p < .001), which did not differ from one another. For Southern talkers, accuracy was again higher on MUSE real words (M = 0.96, 95% CI [0.94, 0.98]) than on SUSE real words (M = 0.94, 95% CI [0.91, 0.96]; z = 1.99, p = .047) or pseudowords (M = 0.86, 95% CI [0.80, 0.90]; z = 5.82, p < .001). In addition, accuracy on SUSE real words was higher than on pseudowords (z = 5.95, p < .001). For Unpredictable talkers, accuracy was higher on MUSE real words (M = 0.97, 95% CI [0.95, 0.98]) than on SUSE real words (M = 0.93, 95% CI [0.90, 0.95]; z = 4.01, p < .001) or pseudowords (M = 0.85, 95% CI [0.79, 0.89]; z = 7.91, p < .001). Accuracy was also higher on SUSE real words than on pseudowords (z = 4.95, p < .001).

Figure 2: Test task results. Top panel: Boxplots and point plots show distributions of mean accuracy by participant for each combination of Talker Accent and Token Condition. Bottom panel: Boxplots and point plots show distributions of mean reaction times by participant for each combination of Talker Accent and Token Condition. In both panels, black dots represent estimated marginal means with 95% confidence intervals. Significant comparisons within Talker Accent are labeled with asterisks (*** < .001, ** < .01, * < .05).

To better understand the pattern of results within each talker accent, we also compared talker accents within each token condition. For pseudowords, accuracy was higher for Mainstream talkers (M = 0.90, 95% CI [0.85, 0.93]) than for Southern talkers (M = 0.86, 95% CI [0.80, 0.90]; z = 2.44, p = .029) or Unpredictable talkers (M = 0.85, 95% CI [0.79, 0.89]; z = 2.47, p = .027). In addition, for SUSE real words, accuracy was higher for Southern talkers (M = 0.94, 95% CI [0.91, 0.96] than for Mainstream talkers (M = 0.91, 95% CI [0.86, 0.94]; z = 2.42, p = .047). None of the other contrasts between talker accents within each token condition was significant (ps > .05).

3.2 ERP analysis

Figures 3 and 4 provide two ways of looking at ERP data. The left panel of Figure 3 and the top panel of Figure 4 show grand mean waveforms, or the overall trajectory of neural responses. These waveforms represent the mean voltage every 2ms across trials, electrodes, and participants. The gray boxes behind the waveforms show the time windows in which each analysis was conducted. The right panel of Figure 3 and the bottom panel of Figure 4 summarize the values used for each analysis. For example, consider a trial in which Participant A heard the real word dress spoken in a SUSE accent by a Mainstream talker. In the N400 time window, voltages were measured in 2ms increments, starting 500ms after the onset of this token and ending 300ms later, for a total of 150 data points per electrode. These data points were then averaged together to yield the N400 response for that particular stimulus. The analyses were performed on these individual values, but the figures show the values summarized by participant.

Figure 3: Test task ERPs time-locked to video onset (pre-speech analysis). Left panel: Grand mean waveforms for Talker Accent. Dotted black lines mark plot origin. Solid black line marks mean audio onset. Light grey box marks time window for analysis. Right panel: Boxplots and point plots show distributions of mean amplitude between 225 and 375ms post-video onset by participant for Talker Accent. Black dots represent estimated marginal means with 95% confidence intervals. Significant comparisons are labeled with asterisks (*** < .001, ** < .01, * < .05).

Figure 4: Test task ERPs time-locked to audio onset. Top panel: Grand mean waveforms for Token Condition within Talker Accent. Dotted black lines mark plot origin. Light grey boxes mark time windows for analysis. Bottom panel: Boxplots and point plots show distributions of mean amplitude by participant for each combination of Talker Accent and Token Condition within each time window. Black dots represent estimated marginal means with 95% confidence intervals. Significant comparisons within Talker Accent are labeled with asterisks (*** < .001, ** < .01, * < .05).

4.1 The effects of talker identity on lexical processing

Previous behavioral work shows that Northern listeners have a MUSE versus SUSE advantage (e.g., Clopper & Bradlow, 2008; Walker et al., 2018b). There is also a larger body of work showing that listeners process familiar regional accents more easily than less familiar regional accents (e.g., Floccia et al., 2006; Labov & Ash, 1997). In the present study, we observed evidence for this MUSE token accent advantage only in behavioral responses. Lexical decision performance was most sensitive to token accent, with significantly higher accuracy on MUSE real words compared to SUSE real words, even for Southern talkers. By contrast, RTs were only sensitive to lexical status, with faster responses to real words than to pseudowords, regardless of talker identity or token accent. The auditory ERP analyses were also relatively insensitive to token accent. During early auditory processing (N1), SUSE real words were more salient than (MUSE) pseudowords when produced by Mainstream talkers. For Unpredictable talkers, SUSE real words were less perceptually salient than MUSE real words or (mixed) pseudowords (N1) but caught listeners’ attention later during auditory processing (P2). Lexico-semantic processing was also conditioned on talker identity, with larger N400 responses to pseudowords compared to real words only for Mainstream talkers. Overall, our results only support a MUSE token advantage in the very latest stage of lexical processing (the lexical decision itself) and instead see evidence of a Mainstream talker advantage early on.

In the Introduction, we laid out three possible ways in which contextual cueing of talker identity could impact word recognition of less familiar dialects: (1) token accent may impact processing more than context; (2) context may facilitate processing by promoting dialect-specific listening strategies; (3) context may disrupt (or delay) processing by promoting an uncertain listening strategy. Our results align with the third option, with listeners adopting an uncertain listening strategy in the presence of Southern and Unpredictable talkers. We believe this was illustrated in two places. First, the absence of the N400 lexicality effect for these talkers suggests that listeners are entertaining more competitors. In Table 1, we said that if listeners simply maintained a MUSE advantage in the context of these talkers, we would expect to see a difference between SUSE and MUSE real words: We do not. We also said that if listeners were shifting their expectations to dialect-specific processing, we would see talker-accent congruency effects: We do not. Instead, what we observe in the N400 time window is that expectations about the talker matter, such that Southern and Unpredictable talkers broaden the possible mappings that listeners entertain.

Lexical decision accuracy further illustrates this uncertain listening strategy for unfamiliar dialects. We found that listeners were significantly more accurate on SUSE real words from Southern talkers than from Mainstream talkers (with SUSE words from Unpredictable talkers non-significantly falling in between). At first glance, we might take this to mean that our participants did activate Southern-accented mappings to some degree when seeing a Southern talker. However, we also found that a) listeners performed similarly well on MUSE real words across talkers and b) listeners made more mistakes on pseudowords from Southern and Unpredictable talkers compared to Mainstream talkers. Therefore, the higher accuracy for SUSE real words spoken by Southern talkers was not simply about shifting to a more SUSE-like perceptual system. If that had been the case, listeners should have had an easier time rejecting SUSE pseudowords and struggled more with MUSE real words. Rather, these results suggest that listeners broadened the signal-to-word mappings they considered from these talkers (Babel et al., 2021; Clopper & Walker, 2017; McMurray et al., 2019; Weatherholtz, 2015; Zheng & Samuel, 2020). This strategy was evident whether that challenge came from a talker who was overtly unpredictable (Unpredictable talker), or who simply had a predictably less familiar accent (Southern talker). While this uncertain listening strategy resulted in poorer lexical decision performance, this could be an optimal strategy in real-world situations where talkers are not likely to utter pseudowords. In the face of uncertainty, hearing tokens as real words is likely to result in successful comprehension (Ganong, 1980).

The transition from Mainstream talker advantage in the ERP data to MUSE accent advantage in the accuracy data highlights how contextual information shapes the time course of comprehension. During acoustic-phonetic processing, we observed sensitivity to SUSE-accented tokens on the N1 and P2 for Mainstream and Unpredictable talkers but not for Southern talkers. These early effects of acoustic-phonetic salience only transitioned to robust N400 effects for Mainstream talkers during lexico-semantic processing, with differences between real words and pseudowords that were insensitive to token accent. Southern and Unpredictable talkers did not elicit strong lexicality effects, suggesting that listeners were entertaining more competitors for these talkers. This uncertain listening strategy made listeners more likely to endorse SUSE real words and pseudowords as real words when they were produced by Southern talkers than when they were produced by Mainstream talkers. However, listeners were overall more accurate on MUSE real words than on SUSE real words regardless of the talker. Together, these differences in accuracy show how relying on talker identity during online processing led to less accent-specific mappings for SUSE tokens offline. We were able to observe this trade-off not only by disrupting the relation between talker identity and token accent, which have been perfectly aligned in previous work (e.g., Goslin et al., 2012), but also by measuring both online (EEG/ERPs) and offline (reaction time, accuracy) processing. Future work on cross-dialectal communication might also consider combining neurocognitive and behavioral methods, as well as pursuing other ways of disentangling bottom-up speech perception from top-down expectations.

So far we have framed the difference between the two accents in this study—MUSE and SUSE—as primarily being about the familiarity our western Pennsylvania listeners had with these accents. However, given that SUSE is also a more stigmatized variety (Hasty, 2018; Preston 2018), the two dialects also differ in overt prestige. Sumner and colleagues have argued that listeners encode prestigious dialects with higher social weight than less prestigious dialects, and that the benefit of familiarity cannot outweigh prestige (Clapp et al., 2023; Sumner & Kataoka, 2013; Sumner et al., 2014; see also Clopper & Bradlow, 2008; Evans & Iverson, 2007; Maher et al., under review; Zaharchuk et al., 2021). This means that the mental representations of SUSE and MUSE before the experiment and the responses to these accents during the experiment may have been shaped primarily by their relative prestige in the United States, rather than their relative familiarity to participants in the study. We cannot disentangle the issue of social status in this particular study; minimally, participants who frequently hear or use SUSE would be needed, which we are pursuing in other work. For now, however, we note that regardless of why the representations of SUSE and MUSE differed for our listeners, both prestige- and exposure-centric accounts would fundamentally predict the same thing—less robust SUSE representations—and what we ultimately argue here is that a weaker representation appears to lead to a less committed listening style.

4.2 The effects of talker identity on pre-speech processing

We also observed differences in talker identity in the pre-speech ERP analysis. The visual presentation of the Southern talkers elicited a negative-going deflection compared to the visual presentation of the Mainstream and Unpredictable talkers. Our enhanced negativity in the 225–375ms window to the visual presentation of unfamiliar Southern talkers is reminiscent of the enhanced negativity, peaking around 300ms after the onset of a visual stimulus, reported in the literature on processing complex visual stimuli (termed N300; e.g., Kumar et al., 2021). Taken to index predictive coding of complex visual objects and scenes, in particular the final stages of processing complex visual stimuli, this component has been shown to be sensitive to how well an exemplar fits into a category. Extending this reasoning to our data, the enhanced negativity for the presentation of the Southern talkers suggests that, to our listeners, the talkers who are designated as “Southern” are not as easily assigned to a talker identity category. Regardless of the specific interpretation, this effect suggests that talker identity can impact speech processing before speech even begins, and could be related to the well-established impacts of perceived talker ethnicity on speech perception performance (e.g., Kutlu et al., 2022; McGowan, 2015).

4.3 Comparison with Martin, Molnar, and Carreiras (2016)

Our study applied the experimental design from Martin, Molnar et al. (2016) to a bidialectal language context. Their study investigated the impact of talker identity on bilingual language processing by presenting bilingual Spanish-Baque listeners with monolingual Spanish talkers, monolingual Basque talkers, and bilingual Spanish-Basque talkers, whose bilingual identity was signaled by code-switching between Spanish and Basque. Our study investigated the impact of talker identity on cross-dialectal language processing by presenting our listeners with monodialectal MUSE talkers, monodialectal SUSE talkers, and bidialectal talkers. These two changes—from bilingual to bidialectal talkers and from bilingual to monodialectal listeners—resulted in different patterns of neural responses. In the pre-audio analysis, we both observed effects of talker identity; however, they were differentially impacted by predictability and familiarity. Specifically, Martin, Molnar et al. observed a positive-going deflection for the bilingual talker, which was interpreted as a marker of uncertainty in language prediction, while we observed a negative-going deflection for the Southern talkers. Together, these patterns of sensitivity to talker identity appear to be driven by a lack of predictability; in our case, from a lack of familiarity with Southern varieties.

The post-speech analyses diverged even further. On the N1, Martin, Molnar et al. (2016) observed an interaction between lexical status and talker identity, with pseudowords eliciting larger effects than real words for monolingual talkers but not for bilingual talkers. We did not observe such a lexicality effect on either the N1 or P2; rather, our early auditory effects reflected the perceptual salience of particular talker-token accent pairs. However, we did observe differences in real word and pseudoword processing on the N400 for the Mainstream talkers. Our N400 lexicality effect for Mainstream talkers is analogous to their N1 lexicality effect for monolingual talkers, in that lexical status was more easily detected when produced by predictable talkers; however, the difference in the timing of these effects shows that our listeners were unable to predict lexical status as early as their listeners.

Martin, Molnar et al. (2016) also observed typical N400 lexicality effects for both monolingual and bilingual talkers. While the lexicality effects were attenuated for the bilingual talkers, we failed to observe any lexicality effects at all for either our Southern or Unpredictable talkers. This comparison further highlights the uncertain listening strategy adopted by our participants. While their listeners were able to engage typical lexico-semantic processing mechanisms for both predictable (monolingual) and unpredictable (bilingual) talkers, our listeners engaged relatively shallow processing for unfamiliar or otherwise unpredictable talkers. These differences in lexical processing between their study and ours may reflect differences in processing languages versus dialects, the high level of proficiency and familiarity of their bilingual listeners with both varieties (versus the relatively low level of familiarity of our listeners with Southern accents), or the relative naturalness of switching between two different languages in a bilingual community versus categorical shifts between two different dialects (see below).

4.4 Limitations and future directions

We would like to highlight two aspects of our experimental design that could be the focus of future work. First, as mentioned in Section 2.2.2, our Unpredictable talkers exhibited extreme shifts between dialects in their monologues that are atypical for bidialectal speakers. We operationalized dialect switching this way as a direct parallel of the language switching condition in Martin, Molnar et al. (2016). The goal was to manipulate the predictability of the upcoming signal through talker identity; however, the social strangeness of the Unpredictable talkers’ patterns of speech may have exaggerated the effects of uncertainty in cross-dialectal communication. It is an open question whether listeners see more naturally occurring patterns of style-shifting as equally unpredictable; in other words, future studies may investigate to what degree, and on what dimensions, variation from a single talker is expected.

Second, our study implemented a mixed rather than blocked design, such that, trial-to-trial, participants did not know which talker or accent they would hear next. Apart from following Martin, Molnar et al. (2016), this decision allowed us to investigate the immediate and non-habitualized effect of talker type and token accent on speech perception. It possibly also pushed listeners into more categorical versus talker-specific approaches to processing (cf. Clapp et al., 2023, p. 13), which would mean our listeners were labelling the talkers as “Southern” and “non-Southern,” and using more general-Southern vs. talker-specific knowledge in processing (which is ultimately how we have interpreted our results). However, since previous work has shown that language processing can be different in experiments which are blocked by talker or by dialect and those that are not (Clapp et al., 2023), future work could investigate cross-dialectal processing when blocked by talker or by talker accent. We might expect overall accuracy to improve by facilitating talker-specific adaptation (see Luthra, 2024, for a review of talker variability); in addition, we may see evidence that western Pennsylvania listeners can form more dialect-specific expectations for Southern talkers when trial-to-trial uncertainty is reduced. Such a finding would suggest that the uncertain listening strategy we see in the current study is related to the high uncertainty of each trial rather than the predictability of the talker’s accent.

Appendix A. Acoustic analysis of actress’ vowels

In Figure A.1, we present average trajectories of six vowels (DRESS was not used in the present task, but we include it as an anchor for interpreting FACE) for the actors in their MUSE and SUSE performance (top row). The actresses make substantial changes across guises: in their SUSE guise, FACE and THOUGHT become more diphthongal, PRIZE becomes more monophthongal, and DRESS and KIT raise. In the second row, we show comparative vowel plots for five older women from southwest Virginia who demonstrate the Southern Vowel Shift (collected by Caleigh Hampton, taken from a mix of interview and citation speech), and four self-identified bidialectal younger southwest Virginia talkers, producing single words in a “Southern” guise (cf. Walker et al. 2018; Walker 2020). The actors have larger vowel spaces than the other two groups in both their guises. Apart from that, they look most similar to the older talkers, reflecting a more conservative Southern accent target.

Figure A.1. Vowel plots of actors across the two guises (top row), compared to women actually from SWVA (bottom row).

Appendix B. Analysis of word length

We compared the duration of each item in the lexical decision task by Token Accent with the following linear mixed-effects model (the default treatment contrast codes for Token Accent were used, with Southern as the reference level): Token duration ~ Token Accent + (Token Accent | Talker) + (Token Accent | Item). As Table B.1 shows, the difference in token duration between Token Accents was not significant.

Appendix C. Analysis of reaction time data