In-laboratory training has been used to demonstrate acquisition of very difficult sound contrasts in a learner’s second language (L2). For example, in a series of landmark papers, Logan and colleagues examined learning of English /ɹ/ and /l/ by native Japanese speakers (Bradlow et al., 1997, 1999 Lively et al., 1993; Logan et al., 1991). In these studies, they specifically examined how learners both perceive and produce the /ɹ/ /l/ contrast after significant amounts of training. This work was particularly groundbreaking because non-native contrasts had previously been thought to be extremely challenging to acquire in adulthood, and perhaps impervious to learning, even after significant exposure. Further, there was a concern that decontextualized learning in the laboratory may be inferior to classroom training and thus may not be amenable to investigating the processes underlying learning of novel speech sounds. In this set of studies, they not only demonstrated robust learning, but also elucidated some possible individual factors that impact learning. This work was critical for demonstrating that learning of novel speech sounds is possible in the laboratory, and that the laboratory is a valid testing ground for investigating myriad factors that may impact learning. Since these groundbreaking studies, researchers have investigated various aspects of training, including the amount of variability presented to participants during training (Iverson et al., 2005; Logan et al., 1991). In general, these aspects of training are a property of the stimuli and could remain stable in remote data collection, therefore we do not consider these studies further here.

Similarly, many linguistic factors have been shown to impact learning in these paradigms. Specifically, the relationship between a first and second language’s phonological system has been demonstrated to be critically important to the ease or difficulty of learning new speech sounds (Best & Tyler, 2007; Flege & Bohn, 2021). While these factors are also relatively easy to control in remote data collection, it is important to note that recruitment in remote protocols often requires participants to self-report about their language background. Many in-laboratory studies also use self-reporting for language background information, but it may be more difficult to verify this information in remote protocols that typically do not have a live experimenter observing the session. That is, it is possible a researcher in an in-person study could flag a participant as possibly not meeting the language background requirements, and this may not be possible in remote settings. Further, in remote protocols, participants have been shown to give incorrect information about themselves to access more studies (e.g., Chandler & Paolacci, 2017; Aguinis et al., 2021).

Other aspects that have been shown to influence learning of novel speech sounds include a variety of cognitive factors, which are typically described under the umbrella of “individual differences.” These properties are typically thought of as being relatively static. However, it is also known that some of these properties are impacted by external factors. For example, working memory has been shown to impact learning of L2 sounds (Darcy et al., 2015). However, working memory is not a static property within an individual, as it is impacted by distraction (e.g., West, 1999). Further, working memory and stress states have a reciprocal relationship (e.g., Matthews & Campbell, 2009), suggesting that working memory may vary substantially across testing environments.

Similarly, it is relatively easy to control for issues of attention and cognitive load during in-laboratory studies. In general, the circumstances for all participants in a given in-laboratory study are similar. That is, all participants in a particular study are usually tested in the same (or very similar) environments with limited distractions and are presumed to be primarily attending to the target task. However, this is not something that can be controlled in remote environments. While one participant may complete the experiment in a quiet, private room, others may complete the experiment in a noisy (both auditorily and visually) public space where there are competing demands on their attention. Because attentional control has been demonstrated to impact L2 speech sound learning (e.g., Mora & Darcy, 2023), it is possible these differences in attention and distractions during testing could impact participants differently in ways that are difficult to assess or predict in remote testing environments. While cognitive load has not been studied in as much detail in L2 speech sound learning (cf. Baese-Berk & Samuel, 2016), it is possible that distractions in the environment could impact a learners’ ability to acquire novel speech sounds in remote testing environments by increasing their cognitive load during learning.

In-laboratory training and testing is often limited in terms of the participants available to a researcher. That is, in much of the research in this area, investigations are limited to populations which are geographically close to an experimenter’s laboratory. This has resulted in samples from “WEIRD” societies (Western, Educated, Industrialized, Rich and Democratic) (Henrich et al., 2010; see Ortega, 2005, and Andringa & Godfroid, 2020, for a review of this issue in the applied linguistics domain). Indeed, in many L2 speech sound learning experiments, the typical participant for an in-laboratory study is a college student from a WEIRD society, resulting in participants who are even less representative of the general population than if participants were sampled broadly from the same community. Further, it is likely that in-laboratory studies that are not part of a university course also select for a specific type of person who is, perhaps, especially interested in the research topic, organized enough to set up times to come into the lab, or may have other special personality features. However, these features may not be common among everyone who wants to (or needs to) learn additional languages.1 Therefore, in-laboratory work has resulted in findings that are perhaps not robustly generalizable to a broader population.

In the present study, we treat the in-laboratory participants as the baseline and compare the remote group to them. This is in part because, as a field, we feel as though we have a better sense of what in-laboratory participants are like and what they can and cannot do as compared to remote participants. This is likely because we have decades of work with in-lab participants and relatively less experience with remote participants. However, it is possible that the in-lab participants are not a sufficient baseline for comparison. That is, in-person methods are likely to create their own non-representative patterns or artifacts, especially because the laboratory context deviates significantly from a participants’ daily experience. This has been demonstrated in previous work that has shown that being in a laboratory is sufficient to prime performance on a task using altered auditory feedback (e.g., Houde & Jordan, 2002).

Because we are unaware of any work that directly compares remote data collection for learning novel speech sounds to in-laboratory learning, we address here the basic issues of remote data collection for laboratory phonology and psycholinguistic research more broadly, which underlie the type of training we conduct in the present study.

In this study, we compare in-laboratory and remotely collected data for training of discrimination of novel speech sounds. As described above, these types of training studies are often quite challenging to conduct in person because they typically require multiple days of training which have logistical and scheduling challenges for both participants and researchers. Therefore, if remote data collection yields similar results as in-laboratory data collection, one could feasibly replace or supplement data collection using logistically simpler procedures.

Below, we describe a two-day training study which was conducted both in-laboratory and using remote data collection methods. We present the results and discuss both methodological and theoretical implications of these results.

Participants for the in-laboratory condition of the experiment were recruited via the Linguistics and Psychology Human Subject Pool at the University of Oregon. We set a one-term window for recruitment, so all participants were recruited during a single 10-week quarter. Eighteen participants completed one day of the in-laboratory experiment and 16 of those participants completed both days of training.

Participants for the remote condition of the experiment were recruited via Prolific. We set a 30-participant cap for the experiment, and full completion of this condition of the study took two days. On day one, all participants were recruited and completed the first day of training, but not everyone took part in the training on day two. In total, 30 participants completed one day of training, and 22 participants completed both days of training.

All participants reported their ages as being between 18–35 years (mean = 21.6). They reported that their first language was English, and they were not proficient in any other languages. Further, no participants reported experience with any languages that use a dental-retroflex contrast (e.g., Hindi), which was used in this study (see Section 2.2, below). Participants in both conditions were paid for their participation.

Stimuli were drawn from a synthetic continuum originally created by Stevens and Blumstein (1975). They consist of a six-step continuum from / ɖa/ to / d̪a/, representing the Hindi dental-retroflex contrast. They vary in the onset of the second and third formant frequencies and in the frequency of the burst. Specific details about the creation of these stimuli can be found in Stevens and Blumstein. All stimuli were normed for duration and were amplitude normalized.

The procedures for the in-laboratory and remote experiments were designed to be identical, except for location of the experiment. On the first day of training, participants began the experiment by giving informed consent for participation and completing a brief language background questionnaire to ensure they were members of our target demographic age range and language experience. Following this portion of the experiment, participants in the remote condition completed a headphone check (adapted from Woods et al., 2017) to ensure they were completing the experiment using headphones.2 No participants were excluded based on the results of the headphone check.

Each of two days of training then followed the same pattern: Participants first completed a pre-test, then training, and then the post-test. During all portions of the experiment, participants completed an ABX discrimination task. They heard two different sounds from the continuum (“A” and “B”) and then heard a third sound (“X”), which was identical to either “A” or “B”. They were asked to identify which of the two sounds they heard by pressing either “f” or “j” on their keyboard.3 The pre-and post-tests consisted of 72 trials each (a total of 288 trials across the pre- and post-tests on the two days). The training portion consisted of 504 trials per day (a total of 1008 trials across two days). This training and testing protocol is identical to those used in previous studies (e.g., Baese-Berk & Samuel, 2016, 2022).

Data were analyzed following procedures in Baese-Berk and Samuel (2016, 2022) using logistic mixed effects regressions to estimate the proportion correct for discrimination of each pair of stimuli. The dependent variable is whether the participants responded correctly (coded as “1”) or incorrectly (coded as “0”) to the discrimination task.

The full model included a comparison of all the peripheral pairs to the center pair (i.e., coded so that the average of the other pairs was compared to Pair 3–4, with the non-central pairs as reference level), training modality (In-laboratory reference level vs. Remote), training phase (Day 1 Pre-Test reference level vs. Day 2 Post-Test), and interactions between these factors as well as a random intercept for participants.

Given previous work in categorical perception, the expectation was that participants should perform at chance for discrimination among all adjacent pairs at pre-test. At post-test, if participants have learned the novel speech sound contrast, they should perform at chance for pairs away from the category boundary (i.e., pairs 1–2, 2–3, 4–5, and 5–6) and above chance for the pair that crosses the category boundary (i.e., pair 3–4).4

At pre-test, in-laboratory participants performed at chance for all pairs, as expected. Note the flat performance across all pairs in the grey line in Figure 1. This demonstrates that before training, participants were unable to discriminate between these unfamiliar speech sounds.

After training, we observe a “peak” for stimulus pair 3–4 in the black line in Figure 1, suggesting that in-laboratory participants have learned to differentiate between sounds that cross the trained category boundary, but not sounds within a single category. These results replicate decades of findings that after in-laboratory training, participants can learn to differentiate between two unfamiliar speech sounds.

When examining performance at pre-test for the remote participants, they perform at chance for all pairs, again as expected (see Figure 2). Again, this demonstrates that participants were unable to discriminate between these speech sounds before training.

At post-test, we again observe a “peak” at stimulus pair 3–4, showing that the remote participants learned to differentiate between sounds that cross the trained category boundary but not sounds within a single category (see Figure 2).

These results show that participants can learn novel speech sounds in remote experimental set ups. Below, we statistically compare learning across the two modalities.

Examining Figures 1 and 2, it is clear that participants in the in-laboratory condition show a higher discrimination peak than participants in the remote condition. Therefore, we used the logistic mixed effects model described above to investigate the role of training type on performance by predicting proportion correct as a function of test phase, pair, training type, and their interactions. The model is summarized in Table 1 below.

First, we see that pair is significant, demonstrating that participants discriminate between tokens 3 and 4 more accurately than other pairs along the continuum, which suggests categorical perception. Next, we see that the comparison between in-laboratory training and remote training is also significant, showing that participants in the in-laboratory training do, indeed, perform better than participants in the remote training group. Further, we see that test phase (i.e., Day 1 Pre-Test vs. Day 2 Post-Test) is significant, demonstrating a change in participant performance from Day 1 Pre-Test to Day 2 Post-Test. The interactions for pair by training type and pair by test phase are not significant; however, the interaction between training type and test phase is. This captures the observation above that participants in the in-laboratory training group are more accurate than participants in the remote training group at post-test, though they do not differ at pre-test. The three-way interaction is not significant.5 It is possible that the lack of significant interactions is driven by individual differences among participants, and that investigating that variability will be informative. These statistical results suggest that participants do learn during training, and performance for in-laboratory participants is superior to remote participants. Below we explore in more detail why we may see these results.

When evaluating the two conditions for training, it is important to evaluate not only learning performance but also the logistics of conducting such experiments. One key consideration is the time it takes to recruit participants. In the case of the in-laboratory study, we recruited 18 participants in a single 10-week quarter. Sixteen of these participants completed the study. While myriad factors impact our ability to recruit participants, this recruitment pattern was quite typical for our lab for a two-day training study. It is often quite easy to recruit participants for shorter studies (i.e., single day studies), but for multiple day studies recruitment is typically much slower.

In contrast, participants in the remote condition were recruited on a single day. In fact, it only took three hours to recruit 30 participants, 22 of whom completed the study. Because the experiment took two days to complete, the data collection also took two days, but this time period was much shorter than the 10-week period described above. Indeed, if we had aimed for a much larger sample size, we could have recruited even more participants in the aforementioned three-hour time period.

It is also important to compare attrition rates across the two studies. In the in-laboratory condition, 2 of 18 participants did not return for the second day (around 11% of participants). In the remote condition, 8 of 30 participants did not complete the second day of training (27% of participants). While the attrition rate is higher for the remote condition, it is not remarkably higher, as attrition rates in our lab are often between 15 and 25% for multi-day training studies. Therefore, we do not believe that this is a significant disadvantage for the logistics of running such a study in a remote setting.

There were not significant differences between the groups in how long it took them to complete the task; all participants in both groups took around one hour each day to complete the task. Therefore, we do not believe that overall group differences in performance can be attributable to time-on-task.

One final question is one of demographics. In our data, our in-laboratory participants all identified as White (n = 16), and the majority of participants identified as female (n = 10; male n = 5; non-binary n = 1). This is not surprising given the demographics of the university and specifically of the Linguistics and Psychology Human Subject Pool we used to recruit participants.6 The self-reported demographics of our remote participants were more ethnically diverse (White n = 18; Black n = 2; Asian n = 1; multiple racial ethnicities n = 1). The gender demographics were skewed more heavily toward male participants than our in-laboratory participants (male n = 13; female n = 7; prefer not to say = 2). Though we did not ask in our demographic questionnaire, it is also possible that our participant pools differed in other demographic characteristics including socioeconomic and educational status. The differences reported here between our in-lab and remote participant demographics are not substantial. However, it is important to note that it is possible to actively recruit for more diverse populations in remote studies, which is more challenging for in-laboratory studies, especially in locations where the general population and, specifically, the university population of question, are more homogeneous.

Taken together, the results of this study demonstrate the positive and negative aspects of conducting research remotely. While we are not the first to consider the trade-offs of these two means of data collection (see, e.g., Eerola et al., 2021), few studies have directly examined the pros and cons of conducting multi-day studies in these modalities and none has focused specifically on learning novel speech sounds. Below, we put forth a few recommendations for researchers deciding whether to conduct multi-day training studies remotely vs. in person.

First, researchers should consider logistical issues. If it is difficult to recruit participants to multi-day in-laboratory studies, or if they are hoping to target a specific population of participants (or a more diverse population than is available in their local area), remote data collection can provide a promising alternative to in-laboratory data collection.

Next, researchers should consider a number of methodological and analytical choices before beginning to conduct their research. In the present study, attrition was higher in the remote condition than the in-laboratory condition. How will the researcher handle attrition in the sample? Will they exclude participants who do not complete the entire experiment? Will they replace those who choose not to complete the study to ensure sufficient power? On the issue of power, the experimenter should consider whether a larger sample size is necessary to generate sufficient statistical power, as effect sizes may be smaller for remote experiments than for in-laboratory experiments.

The issue of potential distraction is also key to consider. How will a researcher determine whether a participant was too distracted? Or, more basically, how much does attention to task matter for the question being posed in a specific study? One could imagine a variety of controls for attention, including attention checks throughout the study, timed responses requiring participants to engage carefully on a given trial, or even use of remote eye-tracking tasks to note whether participants are looking at the screen during a given trial. The closer each task moves toward surveillance (e.g., a live experimenter being present via videoconference, for example), the more taxing this becomes for a laboratory and the more invasive the task becomes for participants (Castelli & Sarvary, 2021). This suggests a trade-off between controlled experimental settings and ease of remote data collection.

Further, the researchers should think carefully about how they collect data about their participants and whether verification of this data is mandatory in order to properly interpret their results. For example, it may be more difficult to verify some aspects of a participant’s language background online than it is in person. If having a clear understanding of language background is mandatory for a specific experiment, researchers should consider whether additional measures may be required to ensure participants have a specific language background. However, it is also important to consider whether such information is truly necessary for a particular study, given that “nativeness,” for example, is not a simple construct and may not impact study results as we expect (Cheng et al., 2021; Strand et al., 2024).

Given all of this, researchers should consider whether conducting data collection in a remote setting might be expected to impact their results in a meaningful way. That is, does shifting to remote data collection so drastically change the conditions for experimentation that it becomes a research question in and of itself?

Finally, and most crucially, researchers should be clear about their methodological and analytical decisions when reporting their results to ensure that results across studies are comparable and that results in any given paper are replicable.