1. Introduction

The prosodic realization of utterances is characterized by a large degree of variability, and there is generally no one-to-one relationship between prosody and meaning (Cruttenden, 1986; Grice et al., 2017; Nolan, 2020; Wells, 2006; Westera et al., 2020; Zimmermann & Onea, 2011). Nevertheless, there are certain constraints on prosodic realizations, i.e., certain tonal events or combinations thereof, which only occur in some contexts and not in others. For instance, assertions in German are typically produced with an H* pitch accent and an L-% edge tone, but other tonal realizations exist in this context. In perception, listeners make use of the prosodic realization of an utterance for its interpretation (Baumann, 2006; Bishop, 2012; Petrone et al., 2017; Petrone & D’Imperio, 2011; Petrone & Niebuhr, 2014; Snedeker & Trueswell, 2002; Weber et al., 2006). In German, for example, a sentence with declarative word order and a final fall (i.e., H* L-%) is likely interpreted as a (neutral) assertion, while a final rise may signal uncertainty or turn the utterance into a question (Cruttenden, 1994; Geluykens, 1987, 1988; G. Ward & Hirschberg, 1985).

This paper targets the role of tonal and nontonal prosodic cues to rhetorical (RQ) and information-seeking questions (ISQ) in German from a listener’s perspective. Linguistically, ISQs request information from an addressee (e.g., (1a)). RQs, on the other hand, often serve to make a point (Biezma & Rawlins, 2017). They commit the interlocutor to the answer presupposed in the interrogative (e.g., to the fact that nobody likes paying taxes in (1b)). RQs can be signaled by different kinds of linguistic means or by combinations of these, e.g., the use of particles, verb mood, negative-polarity items (Dehé et al., 2022; Dehé, Braun et al., 2024), but there are questions without specific morpho-syntactic or lexical marking that may signal either an ISQ or RQ, depending on their prosodic realization (Kharaman et al. 2019); see examples in (2). In prosodic research, the use of such string-identical questions is useful from a methodological point of view, because effects on interpretation resulting from nonprosodic cues are avoided (ignoring the influence of a potential linguistic context outside the question). Henceforth, we will refer to the contrast between wh-questions and polar questions (left-hand side vs. right-hand side in example (2)) as question type, and the one between ISQs and RQs ((a) vs. (b) in examples (1) and (2)) as illocution type.

German ISQs differ from RQs in several prosodic characteristics (see Section 2.1), specifically intonational realization, which is different for wh-questions and polar questions (see stylized f0 contours in (2), stressed syllables are marked by capitals), but ISQs and RQs also also exhibit nontonal prosodic differences, such as constituent duration and voice quality (not shown in (2)). We use the term prosodic realization/characteristics to refer to suprasegmental properties of utterances, including duration, voice quality and fundamental frequency. The term intonation refers to tonal properties of the entire utterance, i.e., prenuclear accents, nuclear accents, and edge tones (these individual parts are called tonal events).

The analysis of production data allows us to identify frequent prosodic realizations for each illocution type (and question type), but may not be able to reveal which cues are important for the listener to identify the intended illocution. This paper investigates the relative impact of individual cues and cue combinations on interpretation and whether they differ across question types. The results will help us gain a better understanding of the prosody of RQs and ISQs by further specifying the prosody-pragmatics interface, but will have implications well beyond the study of these two illocution types. In fact, these issues are pertinent to some core assumptions and questions in the field of prosody research.

First, intonational meaning can be attributed to individual tonal events or to tunes (combinations of tonal events, e.g., nuclear tunes). Regarding individual tonal events, there are reports that low edge tones signal assertions (Han, 2002), while high edge tones signal inquisitiveness (Altmann, 1984; Isačenko & Schädlich, 1966; Kohler, 2004). Also, nuclear L+H* has been associated with contrastive information (Braun et al., 2018; Watson et al., 2008), see Pierrehumbert & Hirschberg (1990) for a complete compositional semantics of pitch accents, phrase accents and boundary tones. Others report intonational tunes for specific purposes, e.g., the “uncertainty contour” in English (G. Ward & Hirschberg, 1985) or the “hat pattern” to signal contrastive topic-focus structures in German (Büring, 1997). Note that the British School of Intonation (A. Fox, 1984; O’Connor & Arnold, 1973; von Essen, 1964) associates intonational meaning with nuclear tunes. In autosegmental-metrical phonology, nuclear tunes are combinations of a unclear pitch accent and a following edge tone (Baumann et al., 2001). Roessig (2024) recently documented an inverse relationship between prenuclear and nuclear accents in German, suggesting a nonlocal (tune-based) dependency. For the current object of investigation, it is conceivable, for instance, that either certain tonal aspects of the stylized contours in (2b) trigger the rhetorical meaning (e.g., the steep rising-falling accent in rhetorical wh-questions (Zahner-Ritter et al., 2022) or the high plateau in rhetorical polar questions) or that the whole tune matters. Research question Q1 asks whether individual tonal realizations are relevant cues to illocution type or whether their combination strengthens the interpretation (tone vs. tune).

Second, prosody research has recently looked more beyond tonal characteristics and included voice quality and/or duration in signaling meaning, e.g., breathy voice to signal irony (Fünfgeld et al., 2024; Leykum, 2021; Niebuhr, 2014; Schmiedel, 2017) or longer durations, i.e., slower speaking rates, to signal indignation (Mozzionacci, 1995). It is an open question how listeners weigh tonal cues (fundamental frequency) compared to nontonal cues (e.g., Gobl et al., 2002). Research question Q2 asks how strong these nontonal prosodic cues are weighted (tonal vs. nontonal cues). The answer to this question will provide insights into the modelling of prosodic meaning: If nontonal prosodic cues are weighted equally strongly as tonal cues, one may have to consider their inclusion in descriptions of question prosody, alongside tonal cues.

Third, some authors suggested to incorporate nontonal prosodic cues with tonal cues to form prosodic constructions/configurations (Gras & Elvira-García, 2021; Neitsch & Niebuhr, 2019; Ogden, 2010), which forms a direct link from prosody to meaning. One example is the “bookended narrow pitch construction” (N. G. Ward, 2019). Such prosodic constructions could be independent of the syntactic surface form of an utterance. On the other hand, previous work on a number of languages has shown that neutral polar questions and neutral wh-questions occur with different nuclear contours (Grice et al., 2005), perhaps due to the differences in syntactic structure and corresponding ordering and positioning of the constituents making up the meaning of the utterance (see different tonal contours in (2)). Research question Q3 hence asks whether the interpretation of tonal and nontonal cues is affected by question type to answer the broader question of a direct link between prosody and meaning, or whether interpretation is mediated by question type.

Finally, the paper seeks to explore how well cue weights can be predicted from the prosodic analysis of production data. It is a useful assumption in prosody research that information on perception can be derived from behavior in production, and this practice fits well with the assumed links between production and perception (Beddor, 2015; Diehl et al., 2004; R. A. Fox, 1982; Newman, 2003). Research question Q4 asks to what extent cue weights in perception can be predicted on the basis of their frequency of occurrence in production.

To answer these questions, we interpret the results of two perception experiments. In Experiment 1, participants indicated whether they thought an auditory stimulus was intended as an ISQ or not. In Experiment 2, a different set of participants indicated whether they thought they heard an RQ or not. In short questions like those in (2), the initial word and the final noun are likely accent positions (Braun et al. 2019). The tonal options are limited by the intonational phonology of German (Grice et al., 2005), in particular the presence and type of prenuclear accent (4 different possibilities), the type of nuclear accent (6 different possibilities), and the type of edge tone (4 different possibilities), amounting to 96 tonal combinations in total. Nontonal differences may stem from voice quality or speaking rate. Such a large number of cues cannot be studied using classical psycholinguistic paradigms.

A novel contribution to the field of laboratory phonology is the application of Active Learning (AL), a machine learning algorithm that learns the weights of the cues from a limited number of items (see Section 2.2). The presentation of subsequent stimuli is based on an optimization of the classifier’s performance given the participants’ responses (Settles, 2009), which makes it highly efficient for complex designs such as the ones tested here. Given the large number of conditions, we started by modeling binary responses (‘ISQ’ vs. ‘no ISQ’, ‘RQ’ vs. ‘no RQ’).

The paper is organized as follows: Section 2 provides background information on what we already know about the prosodic realization of ISQs and RQs in production and perception (Sections 2.1 and 2.2), desiderata and hypotheses (Section 2.3), and information on how AL systems work in general (Section 2.4). Sections 3 and 4 present the methods and the results of the two experiments, respectively. Section 5 discusses the results of both experiments and Section 6 summarizes the conclusions.

2. Background

3. Experiments

Experiment 1 tested the interpretation of questions as information-seeking or not, Experiment 2 tested the same for the interpretation of questions as rhetorical or not. The experiments were approved by the Institutional Review Board of the University of Konstanz (IRB 05/2021).

3.1. Methods

3.1.1. Materials

We created twelve target utterances (lexicalizations), each consisting of a verb, the discourse particle (PRT) denn and a noun (e.g., spielen + denn + Badminton, ‘to play + PRT + Badminton’). All nouns were trisyllabic with stress on the first syllable. The finite verb form (3rd person singular) of the respective verb was monosyllabic. The PRT denn was included for the sake of eurhythmy.³ Each target utterance was used as polar question with the subject pronoun jemand ‘anyone’ and verb-first interrogative syntax (e.g., Spielt denn jemand Badminton?, ‘Plays PRT anyone badminton?’) and as a wh-question with wer ‘who’ as interrogative pronoun (e.g., Wer spielt denn Badminton?, ‘Who plays PRT badminton?’). The interrogatives were constructed from four different verbs (play, like, keep, eat) and twelve nouns, see Table 4 in Appendix 1. The interrogatives were pretested⁴ to make sure that they were suitable as both ISQ or RQ.

The stimuli were recorded by a phonetically trained, female native speaker of German. To keep prosodic realizations of the prenuclear regions consistent across items, the target sentences were split into a Part 1, consisting of the linguistic material before the noun (e.g., Spielt denn jemand, ‘Plays PRT anyone’) and a Part 2, consisting of the noun itself (e.g., Badminton), see Table 4 in Appendix 1. Each Part 1 was recorded in eight different ways: four prenuclear-accent conditions (none, L*, H*, L*+H on the first word) and two types of voice quality (breathy and modal on the first word). To avoid disruptions in the f0-contour between prenuclear and nuclear accents, each Part 1 was furthermore recorded in three different versions: (a) appropriate for the early-peak nuclear accent H+!H* (i.e., PRT ends in high pitch), (b) appropriate for the medial-peak accent H* (PRT ends in rising pitch), and (c) with a low-pitched PRT for the remaining nuclear accents starting with a low tonal target (L*, L+H*, L*+H and (LH)*). The nouns (Part 2) were produced with 24 different nuclear tunes: with six different nuclear accents (L*, H*, L*+H, L+H*, H*+!H*, and (LH)*) and four different edge tones (L-%, L-H%, H-%, and H-^{^}H%). Since the combinations (LH)* H-% and (LH)* H-^{^}H% were difficult to produce naturally for our speaker without also changing some nontonal prosodic cues, we manipulated the natural recordings for these contours in regard to the exact alignment of the starting point and maximum of the rise for the pitch accent (LH)* according to the values in Braun et al. (2019). Parts 1 and 2 were then spliced together (at positive zero crossings in the waveform).

After splicing, the duration of each sentence was PSOLA-resynthesized such that there was a long and a short version of each item (cue ‘duration’), by lengthening or shortening the total duration of each sentence by 10% of its original duration. The versions that were not manipulated for duration were not included in the experiment. In total, the whole procedure resulted in 9216 tokens (768 conditions × 12 items).

As practice items, we used three prosodic realizations of another item that was structurally identical to the experimental items (Wer mag denn Statuen?, ‘Who likes PRT statues?’) Furthermore, we used two catch items that were judged to be unambiguously information-seeking in the pretest described in Footnote 4 (Who wants spaghetti?, Who plays the trumpet?) and two that were interpreted as RQ (Who likes insects?, Who likes parades?); these were recorded naturally by the speaker with the intended illocution types.

3.1.2. Participants

For Experiment 1 (ISQ Experiment), data from 81 monolingual native speakers of German were used to train the AL system (average age = 24.3 years, SD = 3.5 years, 47 female, 34 male). In addition, 18 participants started the experiment but did not train the AL because the speakers were either bilingual (N = 5) or failed to correctly label 3 of the 4 catch items (N = 13), see Section 3.1.3 for details. For Experiment 2 (RQ Experiment), a total of 80 participants trained the AL system (54 female, 25 male, 1 not indicated, average age = 26.2 years, SD = 7.9). An additional 37 participants started the experiment but were not admitted to the AL phase, since they were either bilingual speakers (N = 32) or failed to label at least three catch items correctly (N = 5). Consequently, only attentive native speakers of German who had not learned any other language before the age of six trained the model.

3.1.3. Procedure

The experiments were conducted online, using a custom-made JavaScript program running on an in-house server at the University of Konstanz. At the beginning of each experiment, the participants gave informed consent and filled in a questionnaire on their language and demographic background. Next, they were given a definition of ISQs in Experiment 1 (‘the speaker is seeking an answer’) as well as an unambiguous example (see (1)). For Experiment 2, the definition of RQs was that an RQ is a question that does not demand an answer, because the answer is obvious or the speaker thinks the answer is obvious. The example given was Ist denn heut’ schon Weihnachten? ‘Is PRT today already Christmas?’, which suggests “no” as the obvious answer (this RQ was used in a frequently broadcasted German commercial).

The instructions for the experiment were provided in audio-visual form (i.e., participants could read the instructions and listen to them). The audio presentation was used to allow participants to adjust their computer’s audio setting to a comfortable level of loudness. The experiment had three phases: a practice phase with three items (to familiarize participants with the task), a grouping phase with 28 trials (see below for details), and the actual experimental AL phase with 64 trials. Only the data of the AL phase was used to update the AL system. Figure 1 shows a schematic overview.

Figure 1

Flow chart of the procedure.

Each question was played only once. Participants pressed “J” for ‘information-seeking question’ and “F” for ‘non-information-seeking question’ in Experiment 1. In Experiment 2, they pressed “J” for ‘rhetorical question’ and “F” for ‘non-rhetorical question’. Each trial started with the visual presentation of a centered fixation cross and the simultaneous start of the audio. Participants had 10 seconds to respond; after that, they were reminded to respond. After a response was entered, the next trial started. There was an inter-trial interval of 500 ms after each response. Overall, the experiment took about 10 minutes. Participants could enter a raffle: One Amazon voucher worth 25€ was allocated for every ten participants who entered a drawing.

The grouping phase consisted of 24 trials, one for each nuclear tune (with fixed values for all the other factors) and the four catch items (two ISQs and two RQs). The order of all items was pseudo-randomized so that there was no repetition of lexicalization in consecutive trials. If a participant failed to classify at least three of the four catch items correctly, the experiment ended after the grouping phase for this participant.

Otherwise, the responses in the grouping phase determined whether a participant was allocated to one of two groups that trained separate AL systems. Group I included participants who were sensitive to the tonal combination in the nuclear tune. Group II included participants who were not sensitive to it. The threshold for being assigned to Group II was that the proportion of one response key (“J” or “F”) exceeded 75% (i.e., little sensitivity to the tonal manipulation), otherwise participants were assigned to Group I. The threshold of 75% was arbitrary. In the AL phase, items were assigned to participants by the AL System (see Section 2.2); all combinations of cues could appear on any of the 12 lexicalizations.

3.2. Analyses

In Experiment 1 (ISQ Experiment), 62 of the 81 participants that entered the main AL phase (76.54%) were placed in Group I, and the remaining 19 (23.46%) in Group II. In Experiment 2 (RQ Experiment), again the majority of participants (N = 67, 83.75%) relied on the nuclear tune and was placed in Group I, while the minority (N = 13, 16.25%) formed Group II (see Table 5 in Appendix 2 for details). In our analyses, we focus on the larger Group I, which relied on the nuclear tune. There were too few participants to allow generalization of the patterns in Group II, but the analyses were still conducted and are provided in Table 7 in Appendix 6 for comparison.

The analyses were carried out in three stages (see Figure 2 for an outline): We first analyzed the responses using an extreme gradient boosting algorithm, XGBoost (Friedman, 2002) to test which cues and cue combinations would improve the model’s accuracy, and then calculated a linear regression model to test whether the remaining cues and cue combinations would significantly decrease or increase the likelihood of a given response. This reduced the initial set of 768 conditions to those that participants were sensitive to (see results for number of remaining conditions). In stage 3 we tested the remaining hypotheses using F tests and t tests.

Figure 2

Flow chart of the analyses.

3.2.1. Stage 1. Boosting algorithm

The probability of a question to be labelled as ISQ or not in Experiment 1 (or as RQ or not in Experiment 2) was estimated with XGBoost, a machine learning technique, which has proven good for high dimensional classification problems, like the current ones.⁵ In general, a machine learning tool, rather than a logistic regression, was needed to predict the probabilities due to the high dimensionality of the data. For prediction we used exclusively binary factors. For instance, it was coded whether an accent occurred (e.g., H*) or not (i.e., all other accents except H*). This resulted in 17 binary factors (see Appendix 4 for a full list). We further included three-way interactions between prenuclear accent, nuclear accent and edge tones and all possible two-way interactions between the tonal cues (to address Q1), and 3-way interactions between question type, edge tone and nuclear accent (to address Q3). This procedure resulted in 246 possible explanatory variables⁶ in predicting ISQ or RQ probability.

The boosting algorithm determines which cues and cue combinations improve the model’s accuracy. For a labelled cue we can compare the label coming from the user (y) and the label predicted by the model $\widehat{y}=1\left\{\widehat{p}>0.5\right\}$ , where the model predicts the question to be an ISQ if the estimated ISQ probability, $\widehat{p},$ is larger than .5. Accuracy measures how many mistakes the model made: labelled ISQ as non-ISQ and likewise for RQs. Every cue contributes to the machine learning model in a nonlinear way, and it can be calculated how much a particular cue or a cue combination improved the model’s accuracy.

The boosting algorithm determines those cues that improve the prediction accuracy by more than 1% (see Figure 2 for outline of procedure).

3.2.2. Stage 2. Linear regression models

The linear regression model determines which of the cues and cue combinations determined by the boosting algorithm significantly improve the predictions of a question as ISQ or RQ, relative to a baseline (α = .01). The baseline (intercept) included those cues and cue combinations that occurred most often when the predicted probability was between 49% and 51% in both experiments, i.e., where participants were around chance level when labeling the respective stimulus. This concerned 23 uncertain questions (items) in Experiment 1 (ISQ Experiment) and 17 in Experiment 2 (RQ Experiment). In this set of 40 uncertain questions, the most frequent cues were wh-questions, breathy voice quality, long durations, no prenuclear accent, an (LH)* nuclear accent, and an H-% edge tone. They are chosen as baseline settings for the linear regression models. Note that these baseline settings are settings that we predicted to signal an RQ (Table 1). The regression models allow us to see whether there are interactions between tonal cues, to estimate the contribution of individual tonal cues relative to the baseline (Q1), to test whether there are effects of nontonal cues (Q2), and whether prosodic cues interact with question type (Q3).

3.2.3. Stage 3. Further hypothesis testing

To evaluate Q1, beyond statistical comparisons with the baseline, we tested effects of individual cues. To this end we formulated appropriate null hypotheses and these linear restrictions were tested by means of F tests. The F test compared the sum of squared residuals of a model with and without the restriction imposed by the null hypothesis. For instance, to test whether each nuclear accent has the same contribution as the others (Q1), we formulated the null hypothesis in (4):

1. (4)

1. β(H*) = β(L*+H) = β(L+H*) = β(H+!H*) = β(L*)

If this null hypothesis is rejected (suggesting that one of these nuclear accents has a stronger contribution than the others, α = .05), these inequalities were tested with Student’s t tests. The Student’s t tests were one-sided and check whether inequalities hold (e.g., whether the contribution of H* is smaller than that of L*+H using the following null hypothesis: β(H*) ≤ β(L*+H)).

Regarding the relative weights of tonal and nontonal prosodic cues, the null hypothesis tested is shown in (5), where T stands for tonal cues and N for nontonal cues.

1. (5)

1. ${\displaystyle {\sum }_{j\in T,{\beta }_j>0}{\beta }_j\le }{\displaystyle {\sum }_{i\in N,{\beta }_i>0}{\beta }_i}$

3.3. Results and discussion for ISQs (Experiment 1)

Figure 3 shows the effects of those cues and cue combinations that were significant (see Table 6 in Appendix 5 for the full regression table). Positive values indicate a significant increase in ISQ probability relative to the baseline (24%); negative values, a significant decrease. The height of the bars indicates the percent of increase/decrease in responses provided by each significant cue and cue combination. For instance, nuclear H* leads to an increase in ISQ probability of 23% relative to the baseline (first positive bar). In Figures 3 and 4, cue combinations (interactions) are marked by an “&” on the x axis. To distinguish prenuclear from nuclear accents, prenuclear accents are typed in lowercase characters, and nuclear accents in standard uppercase characters. The bars are ordered and colored for research question: Q1: interactions between tonal cues first, followed by main effects of tonal cues (white), Q2: nontonal cues (light blue), Q3: interactions with question type (grey). Other effects and interactions, which were not predicted, are ordered last (dark blue).

Figure 3

Results of AL regression estimates of Experiment 1 (ISQ Experiment) in percent. Prenuclear accents are indicated by lowercase letters (e.g., h* for prenuclear H*) to distinguish them from nuclear accents (which are in standard formatting). The numbers indicate the percent increase or decrease with respect to the baseline. Colors code the three research questions (Q1: white, Q2: light blue, Q3: grey). Further results are shown in dark blue.

Figure 4

Results of AL regression estimates of Experiment 2 (RQ Experiment) in percent. Prenuclear accents in lower-case letters, nuclear accents in upper case letters (Q1: white, Q2: light blue, Q3: grey).

We first discuss the results with respect to the hypotheses and then indicate the cues and cue combinations that result in the highest ISQ probability.

Regarding Q1 (tune vs. tones, white bars), there was one significant 3-way-interaction: an overall low contour with prenuclear and nuclear L* followed by L-% (first bar). However, it decreased the ISQ probability and thus cannot serve as a tune for ISQs. In fact, there was no combination of tonal events that increased ISQ probability, allowing us to reject the idea of an intonational tune for ISQs. Instead, individual tonal events matter more for interpretation.

What is the contribution of these individual tonal events? With regards to prenuclear accents, the regression model showed that prenuclear H* and prenuclear L*+H (2^nd and 3^rd bars) decreased the ISQ probability relative to “no prenuclear accent” in the baseline, but the negative effect of prenuclear H* was reversed when combined with modal voice quality, as shown by the last bar. Neither prenuclear L+H* nor prenuclear L* changed the ISQ probability relative to the baseline at α = .01. Regarding nuclear accents, the linear regression model showed that all accents increased the ISQ probability relative to (LH)* in the baseline, rendering (LH)* a truly non-information-seeking accent (Zahner-Ritter et al., 2022). Regarding differences within the nuclear accents, we found no statistically significant difference between H* and H+!H* and no differences between H+!H* and L+H* (both p > .1) for wh-questions (in baseline), i.e., statistically, these nuclear accents have the same positive contribution on ISQ probability in wh-questions. Furthermore, the linear regression model showed that for polar questions, H+!H* significantly decreased ISQ probability (negative grey bar in Figure 3). Finally, the effect of L+H* was larger than that of L*+H and L* (both p < .0001). Regarding edge tones, the linear regression model showed that all edge tones increase the ISQ probability relative to H-% in the baseline. Beyond that, further hypothesis testing did not show a statistically significant difference between L-H% and H-^{^}H% (p > .2), so both have the same positive contribution to ISQ probability. The effect of L-H% was larger than that of L-% (p = .02), so L-% is clearly less appropriate to mark an ISQ. Summing up the data relevant for Q1, there is no relevant tune for ISQs, only individual tones matter: The ISQ probability is increased by the combination of prenuclear H* and modal voice quality, by nuclear H* or L+H* and by the edge tones H-^{^}H% or L-H%. For wh-ISQs, but not for polar ISQs, H+!H* is as good a nuclear accent as H* and L+H*.

Regarding Q2, the relative weighting between tonal versus nontonal cues, the results of the linear regression model showed a significant increase in ISQ probability for modal voice quality and short duration (light blue bars). Further hypothesis testing showed that the tonal cues increased the ISQ probability more compared to the nontonal cues (p < .0001). Furthermore, the positive tonal cues (nuclear accents and edge tones) increased the ISQ probability more than the interaction between tonal and nontonal prosodic cues (i.e., between modal voice quality and prenuclear H*, p < .0001). These data suggest that nontonal cues are truly secondary in signalling ISQs.

Regarding Q3, there were a number of interactions with question type (grey bars). The linear regression model showed that L-H% increased the ISQ probability more for polar questions than for wh-questions (in the baseline), and nuclear H+!H* decreased the ISQ probability for polar questions. These interactions speak against a strong version of prosodic constructions for ISQs, because these would have to hold independent of question type.

In summary, the highest ISQ probability that can be achieved for wh-questions (in the baseline⁷) is 87%. The best cues and cue combinations are:

• baseline                                                                                                                                 24%

  (wh-question, breathy voice quality, long duration, no prenuclear accent, nuclear (LH)*, H-%)

• nuclear H*                                                                                                                            +23%

• H-^{^}H%                                                                                                                                    +15%

• combination of prenuclear H* (h*) and modal voice quality                                        + 5% (–11%+16%)

• modal voice quality                                                                                                             + 8%

• short duration                                                                                                                       +12%

The contribution of the nontonal prosodic cues is smaller than that of the tonal cues (20% for nontonal cues on their own, 25% when factoring in the interaction with prenuclear H*). Without the nontonal prosodic cues, the highest ISQ probability would hence be 25% lower, i.e., at 62%.

For polar questions, the cues are largely identical, but L-H% emerges as a stronger cue to ISQs, leading to the highest ISQ probability of 95%:

• baseline                                                                                                                                 24%

  (wh-question, breathy voice quality, long duration, no prenuclear accent, nuclear (LH)*, H-%)

• nuclear H*                                                                                                                            +23%

• L-H%                                                                                                                                      +13%

• combination of polar question and L-H%                                                                        +10%

• combination of prenuclear H* (h*) and modal voice quality                                        + 5% (–11%+16%)

• modal voice quality                                                                                                             + 8%

• short duration                                                                                                                       +12%

Note that there are some other options to signal an ISQ with a high probability: with no significant differences, nuclear H* can be changed to L+H* or to H+!H* (for wh-questions only). Also, H-^{^}H% can be changed to L-H%. This optionality in signalling ISQs is another argument against assuming prosodic constructions for ISQs.

Regarding Q4, the possibilities and limitations of predicting perception from production, we see that many of the tonal predictions derived from production data were indeed confirmed by the perception data (modal voice quality, short duration, L-H% and H-^{^}H%, in addition H+!H* for wh-questions). However, this inference from production to perception has its limitations. Our multi-cue study showed that in polar questions prenuclear H* is relevant for perception (though it was not contrastive in production, as shown by the lack of grey shading in Table 1). Nuclear H* was a relevant cue in both question types, but was neither frequent nor contrastive in production. The same is true for L-H% for polar questions. We will revisit this issue in the General Discussion (Section 4).

4. General Discussion

This paper experimentally crossed tonal and nontonal prosodic cues and tested their roles for the interpretation of polar and wh-questions as information-seeking or rhetorical. The overarching research questions related to whether illocution type was signaled by tunes (combinations of tonal events) or individual tones (Q1), to the relative weighting between tonal cues versus nontonal cues (Q2) and to possible interactions between prosodic cues and question type (Q3). This allows us to further evaluate whether realizations that were frequent and contrastive in production are particularly relevant in perception (Q4).

The response to Q1 is at first glance different for ISQs and RQs: For ISQs, there is no evidence for tunes. Statistically, there was a three-way interaction, but the combination of tonal events decreased the ISQ probability. There were several nuclear accents (H*, L+H* for both question types, H+!H* for wh-questions only) that lead to high ISQ probabilities with no significant differences. Likewise, there was no statistical difference between L-H% and H-^{^}H% for wh-questions. The particular contribution of accents and edge tones may signal further pragmatic nuances, which participants in our experiments were not alerted to (e.g., incredulity or bias). It would be interesting to see whether an alternative instruction (e.g., which of these questions sound incredulous or interested?) might lead to a slightly different pattern that would allow us to distinguish further pragmatic nuances. For RQs, on the other hand, prenuclear L*+H, combined with nuclear (LH)* in the baseline, and L-% may form a tune, but only for wh-RQs. Polar RQs need an H-% edge tone to reach high probability. This dependence on question type (Q3) reduces the generalizability of such a tune and we do therefore not gain much explanatory power by postulating a rhetorical question tune, which is limited to wh-questions.

Regarding Q2, both ISQ and RQ probabilities were significantly affected by the nontonal cues duration and voice quality (in opposite ways, as expected: modal voice quality and short constituent durations increased ISQ probability, while breathy voice quality and long constituent durations increased RQ probability). Statistically and numerically, however, the contribution of nontonal cues was smaller than that of tonal cues (in line with Geib & Braun, 2022; Kharaman et al., 2019), see Table 2 for the numerically highest ISQ and RQ probabilities overall, and separate contributions for tonal cues (3^rd row) and nontonal prosodic cues (5^th row). The contribution of nontonal prosodic cues was numerically larger for ISQs than for RQs (25% vs. 17%) but a direct comparison is difficult because of the between-subject design.

Table 2 also indicates that RQs demand different prenuclear accents and edge tones for the two question types: prenuclear H* and H-% led to highest RQ interpretations for polar questions, prenuclear L*+H and L-% for wh-questions. For ISQs, we see an effect of question type in the alternative edge tones and nuclear accents (those that did not differ significantly from the optimal cue): L-H% was an alternative to H-^{^}H%, but only for polar questions, nuclear H+!H* was an alternative to nuclear H* or L+H*, but only for wh-questions. These results suggest that the interface between prosody and semantics is mediated to some extent by question type (Q3). Interestingly, the asymmetry observed in more spontaneous production tasks, which showed that speakers preferably produced polar questions in information-seeking contexts and wh-questions in rhetorical question contexts (Dehé, Wochner et al., 2024), is replicated here. The highest ISQ probability is achieved with polar questions (95% vs. 87% for wh-ISQs), while the highest RQ probability is achieved with wh-questions (76% vs. 70% for polar RQs).

Regarding the production-perception link (Q4), we need to compare the cues that were relevant in perception with those that were frequent and contrastive in production (Table 1). To this end, we reproduced Table 1 as Table 3, but crossed out those realizations that were not relevant in perception (N = 13: six times prenuclear realizations: four times no prenuclear accent, two times L*+H; four times nuclear accents: three times L*, once L+H*; three times edge tones: two times H-^{^}H%, once L-%, see Table 3). These mismatches would be picked out in a perception procedure that only tested the frequent and contrastive realizations. It is striking that nuclear L* and the high rising edge tone (H-^{^}H%) were very frequent and contrastive in production, but often not relevant in perception. It is possible that the L* nuclear accent is easy to produce and therefore frequent, but not particularly salient as a perceptual cue. The rising edge tones in production may be a remnant from primary school, where pupils are often told to produce a rise when they see a “?”. Clearly, the perception experiments give less weight to the high-rising contours. In boldface we printed realizations that were relevant in perception, but which were too infrequent in production (< 15%). These would likely be missed in a perception procedure that only includes frequent and contrastive cues. In our study, these unexpected perceptual cues were nuclear H*, nuclear L+H*, and L-H% in polar ISQs, nuclear H* in wh-ISQs and prenuclear L*+H in wh-RQs. Unlike in production, two further cues were contrastive in perception: nuclear L+H* for wh-ISQ and L-% for wh-RQ. More naturalistic and implicit tasks may be necessary to resolve the production-perception mismatches. In any case, relevant cues may be overlooked by only testing cues that are frequent and contrastive in production.

The results for ISQs show that a sequence of high tonal events (high prenuclear H* accent, high nuclear H* accent, rising H-^{^}H% or L-H% edge tones) already leads to an ISQ probability of 62% (not adding nontonal prosodic cues, cf. Table 2). It may seem surprising, however, that this tonal combination yields the highest ISQ probability for both polar and wh-questions: After all, wh-questions are often assumed to have a low edge tone (Grice et al., 2005). Note, however, that there is considerable variation in edge tones in natural discourse (e.g., Kohler, 2004; Oppenrieder, 1988), which may explain why the high rising edge tone (H-^{^}H%) received high ratings for wh-questions, too. More generally, our findings support the claim that questions need to have a high tonal event anywhere in the utterance (Herman, 1942; Ohala, 1983, 1984), but add that the combination of such high tonal events increases the ISQ probability for German. Analyses of Group II participants, not presented in detail in this paper, suggest that this generalization also holds for the participants that were not sensitive to nuclear tune in the grouping phase (see Table 7 in Appendix 6): nuclear H* and H-^{^}H% increased ISQ probability (and decreased RQ probability).

The highest probabilities for RQs (70% for polar questions and 76% for wh-questions) were generally a bit lower than for ISQ (95% and 87%, respectively). The lower probability of RQs in comparison to ISQs suggests that the prosodic realization alone may not be a sufficient cue to RQ interpretation. Given the pragmatic felicity conditions for RQs (to make a point, to try to commit listeners to the proposition in the question) and the fact that RQs still function as questions (in that an answer is possible), participants may be more reluctant to interpret an utterance as rhetorical, especially when lacking discourse context.

For our specific research question, the cues to RQs and ISQs, there are a number of prosodic cues that had opposite effects in the two experiments, despite the fact that the two illocution types were not presented as alternative response options (i.e., the participants in Experiment 1 were not told about RQs and the participants in Experiment 2 were not told about ISQs). These cues, which we conclude are particularly relevant for signaling the contrast between ISQ and RQ, are:

• Short duration: increased ISQ probability and decreased RQ probability.

• Modal voice quality: increased ISQ probability and decreased RQ probability.

• All accents except (LH)*: increased ISQ probability and decreased RQ probability.

• Edge tone H-^{^}H%: increased ISQ probability and decreased RQ probability.

The AL system proved useful to handle a large set of cue combinations and items. The AL system updated the XGBoost model (a separate model for Group I and Group II in every experiment) after each participant completed the experimental phase. This iterative process allowed the system to efficiently learn the contributions of individual cues and their interactions to the perception of illocution type. Importantly, the AL system ensured a balanced exploration of the experimental condition space, while minimizing redundancy in stimulus presentation. The implementation of this AL framework enabled the collection of sufficient data to evaluate the perceptual effects of a large number of prosodic and syntactic conditions while maintaining high predictive accuracy and minimizing participant burden. This methodology highlights the utility of AL for large-scale perceptual studies in linguistic research. In terms of further validating the predictions of the AL system, it would be interesting to run more tests to learn how responses of (temporarily) inattentive participants (simulated through random labeling) may disrupt the general weights of the AL system.

There are several directions for future research, most of which are concerned with testing the generalizability of the results. First, the speaker we recorded had a comparatively high pitch register and large pitch range. In future research it will be interesting to test whether our findings generalize to other speakers with different vocal characteristics, different phonetic implementations of pitch accents (with respect to scaling and pitch range) and with tokens from multiple speakers. This will also allow us to further test the effect of voice quality, which may differ across speakers. The challenge is to find trained speakers who can produce the different combinations of cues naturally; resynthesis of duration and f0 work well with PSOLA, but resynthesis of voice quality is difficult (cf. Mehta & Quatieri, 2005 for isolated vowels). Second, in the current experiments, two levels of constituent duration (lengthening and shortening of the entire utterance by 10%) and voice quality (breathy voice and modal voice on the initial constituent) were included. In future research it may also be useful to include more categories (or continuous acoustic measures of speaking rate or voice quality, cf. Ben Barsties v. Latoszek et al., 2020; Hillenbrand et al., 1994) for these cues to get a more fine-grained picture. Third, one may want to include syntactically longer and more varied utterances, such as the inclusion of adverbials (e.g., Wer kauft denn in der Innenstadt Klamotten? ‘Who buys PRT in the city center clothes?’), which may help to shed more light on the role of prenuclear accents and potential interactions with information structure. Finally, to test the specificity of the RQ interpretation it will be important to test questions with other connotations, such as biased questions or surprise questions in a similar paradigm. One final desideratum is to use more implicit tasks that do not require explicit semantic judgements.

5. Conclusions

In two experiments, we tested participants’ weighting of tonal and nontonal prosodic cues for the interpretation of information-seeking questions (ISQs, Experiment 1) and rhetorical questions (RQs, Experiment 2). The stimuli varied in question type (polar question vs. wh-question), presence and/or type of the prenuclear accent on the first word, type of nuclear accent on the sentence-final noun, type of edge tone, voice quality on the first word (modal vs. breathy voice) and global constituent duration (short vs. long). This large number of cues was handled by an Active Learning system, which controlled stimulus order and updated cue weights. Our results showed that the interpretation of both ISQs and RQs is predicted better by individual tonal events rather than tunes: For polar questions, ISQs were interpreted with highest accuracy with an H* prenuclear accent, high nuclear accent H*, and a low-rising edge tone (L-H%), while RQs were best interpreted with an H* prenuclear accent, a steep-rising nuclear accent (LH)*, and a high plateau (H-%). For wh-questions, ISQ interpretation was highest with a prenuclear H*, nuclear H*, L+H* (or H+!H* for wh-ISQs), and final rise (L-H% or H^{^}H%), while RQ interpretation was highest with a prenuclear L*+H, a steep-rising nuclear accent (LH)*, and a low edge tone (L-%). Overall, ISQs reached higher probabilities (95% probability for polar ISQs, 87% for wh-ISQs) than RQs (70% for polar RQs, 76% for wh-RQs) and ISQs had more tonal options than RQs (i.e., RQs were tonally more specific than ISQs). The nuclear realizations generally differed as a function of question type. The only exception is nuclear (LH)*, which was generally interpreted as RQ, independent of question type. Regarding nontonal prosodic cues, modal voice quality and short durations increased ISQ probability and decreased RQ probability for both question types. These nontonal prosodic cues had on average lower cue weights than the tonal cues. The data further showed that realizations that were frequent in production are not inevitably relevant in perception and, vice versa, that listeners may find cues relevant in perception even if they were not frequent in production. Given this discrepancy between frequency of realizations in production and relevance of cues in perception we stress the importance of methodological diversity for studying the prosody-pragmatics interface. Apart from this methodological contribution, our research further shows that the interpretation of prosodic cues is subject to question type and that nontonal prosodic cues are relevant, too. This raises further questions relating to the architecture of grammar, pertaining to additional (levels of) phonetic parameters, syntactic length and complexity, and more illocution types.

Appendix 1. Materials

Appendix 2. Metadata on participants

Appendix 3. Details on Active Learning

The Active Learning (AL) approach implemented in this study consists of three stages: the experimental data collection, the application of a machine learning model for prediction, and the uncertainty-based selection of stimuli for subsequent labelling. These stages are described in detail below.

The initial step involved a grouping phase to assess participants’ attentiveness and sensitivity to prosodic manipulations. Participants were presented with a fixed set of 28 stimuli, including unambiguous examples, ambiguous examples, and 4 catch items. Based on their responses, participants were assigned to one of two groups: Group I included those who showed sensitivity to nuclear tunes, while Group II included participants whose responses indicated low sensitivity, as determined by their performance on catch items and consistency in labelling ambiguous items. The threshold for being assigned to Group II was that the proportion of one response key (J or F) exceeded 75% (i.e., low sensitivity to the prosodic manipulation and therefore little attentiveness), otherwise participants were assigned to Group I.

In the experimental phase, the AL system was used to model the probability of each stimulus being classified as ISQ or RQ and to manage the presentation of the 768 experimental conditions. For estimating the probability XGBoost was chosen due to its ability to handle high-dimensional data and its robustness in accounting for nonlinear interactions between predictors. In this study, the predictors included binary features representing the experimental cues, such as prenuclear accent, nuclear accent, edge tone, voice quality, duration, and question type. To capture interactions among these factors, the model also incorporated second-order interactions between all cues and specific three-way interactions (https://xgboost.readthedocs.io/en/stable/).

XGBoost is a computationally effective algorithm to classify an item with a given set of cues to be ISQ/non-ISQ or RQ/non-RQ. The classification model is a tree, which is split into leaves based on the cues. An example of a tree is presented in Figure 5. In this example, a polar question with an L-H% edge tone (Tree 0) with a (LH)* nuclear accent (lower box in second “column”) and modal voice quality (lower box in “third column” ends up in the bottom leaf; its probability of being an ISQ question is determined as in (6)).

Figure 5

An example of a classification tree (see text for explanation) as exported from XGBoost.

Leaf cover is related to the variable importance, where a large value indicates an “important” leaf in terms of the classifying accuracy. In extreme gradient boosted tree algorithms (XGBoost) we build a lot of such trees and aggregate the results across those trees. The split into the leaves is decided based on the logistic loss function, which is also used in classical logistic regressions. The advantage of the XGBoost is not only its computational efficiency, but most importantly the ability to classify many experimental conditions (here: 768) based on a small number of labelled data (24 when the experiment started), which is not possible for the standard regression techniques. Therefore, XGBoost is implemented in real-time in the AL system to decide which experimental item should be labelled next. During Experiment 1 we estimate the ISQ probability for all 768 conditions and the next item which is displayed for a participant is the item where the ISQ probability is closest to .5 or a random choice if there are more cue combinations that are most uncertain for the model.

For selecting the next stimulus, for each participant, the AL system dynamically selected a stimulus with the cue combination that was most uncertain. This was implemented in JS (https://github.com/AL-perception-experiments/isq-rq), where the uncertainty of all of the 768 experimental conditions was estimated with XGBoost (https://xgboost.readthedocs.io/en/stable/).

Appendix 4. Binary factors

Lower case letters are used for prenuclear accents to avoid confusion with nuclear accents (printed in regular upper case letters), ¬ is the abbreviation for negation.

1. Voice quality: (modal vs. breathy)

2. Duration: (long vs. short)

3. No prenuclear accent (vs. any)

4. Prenuclear H* (vs. ¬H*)

5. Prenuclear L* (vs. ¬L*)

6. Prenuclear L*+H (vs. ¬L*+H)

7. Nuclear H* (vs ¬H*)

8. Nuclear L*+H (vs. ¬L*+H)

9. Nuclear L+H* (vs. ¬L+H*)

10. Nuclear H+!H* (vs. ¬H+!H*)

11. Nuclear (LH)* (vs. ¬(LH)*)

12. Nuclear L* (vs. ¬L*)

13. Edge tone L-% (vs. ¬L-%)

14. Edge tone H-% (vs. ¬H-%)

15. Edge tone L-H% (vs. ¬L-H%)

16. Edge tone H-^{^}H% (vs. ¬H-H%)

17. Question type: (polar vs. wh-question)

Appendix 5. Results for Group I participants

Appendix 6. Results for Group II participants

Acknowledgements

This research was supported by the German research foundation (grants BR 3428/4–2 to Bettina Braun, DE 876/3–2 to Nicole Dehé, and KE 740/17-2 to Daniel Keim) as part of the DFG Research Unit 2111 “Questions at the Interfaces”. We particularly thank our speaker for recording the stimuli as well as Justin Hofenbitzer, Friederike Hohl, Meike Rommel, Jasmin Pöhnlein, Johanna Schnell, Elena Schweizer, Daniela Wochner and Tianyi Zhao for help with annotation and preparation of the files. Furthermore, we thank the audience of the presentation at PaPE in Nijmegen 2023, the audience of the final workshop of the Research Unit (Konstanz, 2023), Ida Toivonen, Christoph Schwarze, María Biezma, Maribel Romero and the Bielefeld and Frankfurt Phonetics groups for feedback.

Competing interests

The authors have no competing interests to declare.

Author contributions

BB and ND: Initial conception; ME, AJ, KZR: preparation of stimuli; EK: statistical analyses; RS: Programming of Active Learning system; All: writing of the paper.