Virtual Pitch and Pitch Shifts in Church Bells

It is well established that musical sounds comprising multiple partials with frequencies approximately in the ratio of small integers give rise to a strong sensation of pitch even if the lowest or fundamental partial is missing—the so-called virtual pitch effect. Experiments on thirty test subjects demonstrate that this virtual pitch is shifted significantly by changes in the spacing of the constituent partials. The experiments measured pitch by comparison of sounds of similar timbre and were automated so that they could be performed remotely across the Internet. Analysis of the test sounds used shows that the pitch shifts are not predicted by Terhardt’s classic model of virtual pitch. The test sounds used were modelled on the sounds of church bells, but a further experiment on seventeen test subjects showed that changes in partial amplitude only had a minor effect on the pitch shifts observed, and that a pitch shift was still observed when two of the lowest frequency partials were removed, so that the effects reported are of general interest.


Introduction
Pitch is the characteristic that allows a set of sounds to be ordered in a musical scale. Sometimes, the pitch of a sound corresponds to a single frequency component present in the sound (so-called spectral pitch). However, a sound comprising a harmonic series with partial frequencies 2f, 3f, 4f etc. gives a strong perception of pitch at frequency f even if no energy at that frequency is present in the sound. This effect is variously known as the missing fundamental, residue pitch or virtual pitch. Terhardt [1] [2] proposed an algorithm for the determination of pitch. A software implementation of this model due to Terhardt and quencies and intensities.
The overview by Plack and Oxenham [4] of the work of various researchers shows that virtual pitch effects are most apparent in a restricted frequency range. Experiments documented in [5] show that, in church bells at least, virtual pitch dominates pitch perception when 2f (the lowest partial physically present) lies in the approximate range of 500 Hz to 1500 Hz.
Shifts in the frequencies of the partials of a sound from exact harmonic ratios give rise to a shift in the virtual pitch. Experiments reported by Patterson and Wightman in [6] show that the size of the virtual pitch shift due to changes in the partial frequencies depends on the absolute position in the audible spectrum, being larger at lower frequencies. The existence of the virtual pitch effect and pitch shifts in piano tones is investigated in work by Järveläinen et al. [7] and Anderson and Strong [8]. Experimental evidence of virtual pitch shifts in bells at a single point in the audible spectrum due to changes in various partials is presented in [9] and [10].
The term pitch shift is used for more than one effect in the literature. In particular in Terhardt's work on virtual pitch [2], pitch shift is used to describe shifts in pitch due to partial amplitude and the close proximity of pairs of partials. However, the experiments discussed by Plomp [11] show no significant change in pitch shifts between two cases, one with all partials of equal amplitude, and the other with amplitude falling off as 1/n where n is the partial number.
Meanwhile, work by Lin and Hartmann [12] shows that the virtual pitch effect survives when some of the partial frequencies are missing. In this paper, the term is used exclusively for shifts in virtual pitch due to changes in partial spacing.
The textbook definition of pitch given above mandates comparison by listeners as the only valid method of pitch estimation. Experiments show a variation in results between listeners-there is no absolute value for the pitch of a sound independent of a particular listener's ability to judge it. Various methods for pitch estimation are documented in the literature: • comparison of complex tones against sine tones, either from tuning forks or electronically generated; • the method of post vocalisation described by Terhardt and Seewann [13] in which test subjects are asked to sing or hum the pitch they perceive; • comparison of complex tones against test tones with similar timbre. This last method, used effectively by Järveläinen et al. [7] for work on piano tones, and also extensively validated by the author in [5], was used for the experiments described here.

Virtual Pitch in Church Bells
Church bells make an interesting case study for investigations of virtual pitch because perception of their pitch is dominated by virtual pitch effects. Terhardt documents investigations into virtual pitch in bells in [13]. An English translation of this paper appears in [5]. Figure 1 shows a typical spectrum of a bell sound (the old no. 9 bell from St Mary-le-Tower, Ipswich), comprising a mixture of inharmonic and near-harmonic partials. The identification of the significant partials follows the scheme described by Lehr in [14], except for the set of three partials labelled p1, p2 and p3.
These three partials, not usually tuned or recorded by bell founders, have an effect on the timbre and therefore potentially on the pitch of the bell. Table 1 shows the frequencies of the significant partials in the sound of the Ipswich bell together with the ratios between the partial frequencies. There are two series of partials with an approximately harmonic relationship (frequencies in the ratio 2:3:4), one based on the nominal and the other based on I-7, highlighted in Table 1. Experiments documented in [5] show that, for most bells, the series based on the nominal gives a dominant pitch sensation about an octave below the nominal. For low-pitched bells (nominals below 500 Hz), however, the higher series based on I-7 gives the dominant pitch sensation, and for high pitched bells (nominals above 1500 Hz), individual low partials dominate the pitch sensation as spectral pitches. For convenience in description, the nominal, superquint, octave nominal, I-7, I-8 etc. are commonly referred to as the "upper partials".
Due to the considerable importance of the nominal to the perceived pitch of bells, this partial is taken as the reference when considering the relationship where f p is the partial frequency and f n is the nominal frequency.
In bell sounds, the frequencies of the upper partials are either compressed together or stretched apart relative to the nominal, according to the shape or thickness of the bell. Informal observations suggest that these changes alter the perceived pitch of the bell by one quarter or more of a semitone. A simple relationship which allows the intervals of all the upper partials to the nominal to be represented by the interval between the octave nominal and the nominal is documented in [5]. The relationship can be visualised by imagining the spectrum of the bell to be drawn on a sheet of rubber. In thinner bells the rubber is stretched out more, moving the upper partials proportionally further apart.
The intervals of the other named partials to the nominal do not display this simple relationship.

Pitch Shift Experiment
An experiment was designed to quantify the shifts in perceived pitch due to changes in various partials. The independent variables chosen for the experiment, following some initial trials, were the intervals to the nominal of prime, tierce, quint, p1 to p3, and oc- them. This gave five independent variables in total to be explored.
The amplitudes and intervals to the nominal (expressed as a frequency ratio) for the partials used in the tests are shown in Table 2.
A simple amplitude envelope for the partials was used, based again on measurements on partials from a number of bell recordings. The envelope used was A/(1 + dt) where A is the initial amplitude (column labelled Amplitude in Table  2), t is time in seconds, and d is a decay constant set to 5 for all partials except the hum, for which it was set to 1. As will be demonstrated below, the virtual pitch shifts observed were relatively insensitive to amplitude. To allow the effect of five independent variables to be explored, in different test sets either the quint or p1, p2 and p3 was varied, with the other held at the average value. These two variables were treated in this way because, although experience suggested that they did not affect virtual pitch, for completeness it was desired to explore their contribution.
The scheme used to explore the effect on virtual pitch of the five independent variables across the nine test sets is shown in Table 3.
The sixteen test sounds, using the case of nominal = 1000 Hz as an example, involved the combinations shown in Table 4. As will be explained below, the tests were actually presented in random order to the test subject.

Reference Sounds
To allow delivery of the tests across the internet, pre-created reference sounds were used in the tests. For each individual test, the subject was asked to select which of 16 reference sounds was closest in pitch to the test sound. The reference sounds for all 16 tests in a test set were the same. The reference sounds were built from the same partials as the test sounds, with the same amplitudes, but with their intervals to the nominal set to the typical values as shown in Table   2. The nominals of the reference sounds were spaced geometrically at equal intervals apart. The spread of nominals in the reference sounds for each test was   Table 5.

Generation of the Test and Reference Sounds
The parameters for the test and reference sounds of all nine test sets were calculated in a spreadsheet and exported as csv files. These files of data were imported into bespoke software which generated each batch of test and reference sounds via additive synthesis of cosine waves.
The generated sounds were 16-bit with a sampling rate of 22,050 samples per second. The duration of all the sounds was set at 0.25 s. This short duration was intended to encourage pitch perception via virtual pitches rather than identification of individual partials (and is also typical of the spacing between bell sounds in English change ringing). During preliminary extensive trials of the tests, some tests with sounds 1 s long were conducted; the results were not significantly different from those at 0.25 s. In [13] Terhardt and Seewann report no significant difference in the pitch estimation ability of test subjects between sounds of 0.1 s and 3 s duration.

Execution of the Tests
Conduct of the tests and reporting of the results was automated in software so  This was converted into a pitch shift in cents using Equation (1).
Extensive trials of the approach employed were carried out to validate the test procedure and documentation, analysis of the results, the effect on test results of quality of sound reproduction equipment and musical experience of the subject, and ranges of the various test parameters. The experiment design was proven to be robust against all these criteria, and in particular the quality of sound reproduction equipment was eliminated as a concern.
The test results were analysed using the standard analysis of variance (ANOVA) statistical technique [15]. This allowed the effect of each of the independent variables on pitch shift to be isolated from that of the others, and also allowed the statistical validity of the test results to be established. The results were analysed using the standard ANOVA technique. Because the experiment design was orthogonal (i.e. all combinations of the independent variables were present; see Table 4) the ANOVA analysis allows two key sets of information to be obtained from the experiment results; 1) the effect of each independent variable individually on the pitch shift, averaged across all subjects and tests at a particular nominal frequency, and 2) the statistical significance of each of these effects.

Experimental Results
As an example of the results, Figure 2 shows the pitch shift due to upper partials experienced by 26 test subjects, for test tones at 1000 Hz. The quantity plotted is the interval in cents between the pitch observed with the upper partials at their low values, and pitch observed with the upper partials at their high values, averaged across all the tests at 1000 Hz. The order of the subjects is that in which they performed the test. The average shift for all subjects was 36 cents. Whether the spread in pitch shifts experienced was due to experimental variation or some characteristic of the subjects was not determined.
The statistical significance of each effect derived from the experiment results is expressed as a P-value. The P-value is the probability that a pitch shift of this size could have been observed (through chance variation) if in fact the independent variable did not give rise to a shift. A P-value of less than 0.05 indicates that the pitch shift is statistically significant at the 95% level. Table 6 shows the pitch shifts in cents due to each of the independent variables in the nine test sets, together with the P-value derived from the statistical analysis. The interpretation of these results is as follows: • the pitch shifts due to changes in the upper partials are both highly statistically significant, and musically significant. Pitch shifts range from 1/8 of a semitone at the highest frequency, through half a semitone in the middle range of frequencies, to well over a semitone at the lowest frequency;  • pitch shifts due to the quint and the middle partials are neither statistically nor musically significant at any frequency; • shifts due to prime and tierce are statistically significant at most frequencies, but musically quite small, generally a small fraction of the shifts due to the upper partials.
28 of the 30 test subjects gave an account of their musical experience, which ranged from near-professional experience to no experience at all. There was no correspondence between musical experience (or lack of it) and the size of the pitch shifts experienced by the test subjects.
As explained in Section 1.1, in bell sounds the spacing of the upper partials is uniquely parameterised by the interval of the octave nominal relative to the nominal. It is therefore convenient to show the upper partial pitch shift (in cents) for a one-cent shift in the octave nominal. The high and low frequencies of the octave nominal given in Table 2 are 1280 and 1140 cents above the nominal respectively, so the unit pitch shift can be calculated by dividing the upper partial effects in Table 6 by 1280 − 1140 = 140. The results, together with 95% confidence intervals, are as shown in Figure 3.
The dotted line is a regression assuming a simple log relationship (specifically ln(pitch shift) = −1.10012 ln(f n ) + 6.12396). No theoretical justification is proposed for this relationship; it just provides convenient shorthand allowing pitch shifts at an arbitrary frequency to be estimated. It is hypothesised that the very large pitch shifts at low nominal frequencies arise because, at this end of the spectrum, more subjects are hearing the pitch based on partial I-7 rather than the pitch based on the nominal.
A subsidiary experiment was conducted to explore the linearity or otherwise of the relationship between upper partial spacing and pitch shift. Test sounds were generated with the upper partial spacing in 16 steps from the low to the high values in Table 2, and all other partials set to their typical values.    cents. There is nothing in these results to suggest that the pitch shift effect due to upper partial spacing is not linear over the range of spacings used in the experiments.
Additional work also documented in [5] used a model derived from the results of the main experiment above to investigate the tuning of peals of bells.
Thick, heavy bells have compressed upper partials, and this experiment predicts that this will flatten their pitch. Comparison of the model against the partial frequencies of peals of bells known to have been tuned using virtual pitches rather than nominal frequencies, showed that the model successfully predicted the changes in bell pitch due to upper partial effects. The shifts found in practice can be up to 40 cents or 4/10 of a semitone, easily heard by a musical ear.

Effect of Partial Amplitude on Pitch Shift
A second experiment was constructed to investigate the effect of partial amplitude on the pitch shifts caused by changes in the upper partials. In this experiment, the amplitudes of superquint and octave nominal were varied from zero to 100% of the amplitudes in the previous experiment, while at the same time the frequencies of all the upper partials took on either the low or high values as defined in Table 2. All other partials took on their typical values. As before, the test subject was asked to match the pitch of the test tone against reference tones. The experiment was carried out at a single nominal frequency, 1000 Hz, chosen to be the middle of the range of nominals in the previous experiment.

Experimental Design and Procedure
To take advantage of the experimental procedure developed and proved in the previous experiment, this experiment followed a very similar approach and used the same software for delivery and execution.
The superquint and octave nominal amplitudes each took on four values; 0%, 33%, 67% and 100% of the amplitudes listed in Table 2. These values together with the low and high values for the upper partial frequencies give 32 combinations. For convenience, the combinations were divided into two sets of 16 tests to allow the established test procedure to be used. The two sets were constructed as shown in Table 7. The arrangement of independent variables in each test set allowed statistically valid results to be derived if the subject only completed one of the two sets of tests. The reference sounds used were identical to those used in the previous experiment. All other aspects of the design and conduct of this experiment were identical to those previously described.

Experiment Results
Subjects were again recruited through bell-ringing newsgroups. In total, 17 test subjects undertook the tests, one of whom was one of the authors of this paper.
One subject only completed one of the two sets of tests, so the average results of the remaining subjects were used to provide the missing results. In total, 528 individual tests were completed.

Comparison against Terhardt Virtual Pitch Model
Terhardt in [2] gives an algorithm for establishing perceived pitches, both spectral and virtual, from a set of partial frequencies and amplitudes. This algorithm was used in Terhardt's research on bell pitches described in [13]. In this work, the experimenters compared the predictions of the algorithm with pitch determination experiments on a number of bells. The predictions of the algorithm were a reasonable match for the experimental results as regards prediction of the note names of the pitches (i.e. to within half a semitone in either direction), but investigation of the detailed correspondence between measured and predicted pitches showed average deviations of between 15 and 40 cents. No attempt was made to explain the deviations as pitch shifts.
The Terhardt algorithm has been implemented in software (C program ptp2svp) by Terhardt and Parncutt [3]. This software implementation of the algorithm was used to investigate its predictions in respect of the first two experiments de- For the test sounds with nominal frequencies of 500 Hz and above, the algorithm predicted a single dominant pitch related to the nominal frequency. The effect on pitch of changes in individual partials was established by calculating contrasts and effects in the same manner as used in analysing the first experiment in this paper. The results obtained (quoted as pitch shifts in cents) are presented in Table 8.
Comparison with the experimental results given in Table 6 shows that the algorithm predicts no pitch shifts as a result of upper partial changes, whereas the experiments show that this is the dominant and significant effect. The prime and tierce effects predicted by the algorithm are at least an order of magnitude less than those seen in the experiments. The Terhardt algorithm therefore does not predict the pitch shifts seen in practice at these nominal frequencies.
The algorithmic results for test tones at nominal frequencies of 315 Hz and 397 Hz show multiple pitches with roughly equal perceived weight. This is in line with experience; due to the effect of the dominance region for virtual pitch, at lower nominal frequencies the higher harmonic series shown in Table 1 vies with the lower harmonic series as a generator of virtual pitch. The pitches predicted by the algorithm lie approximately two octaves below the nominal, one octave below the nominal, two octaves below partial I-7 (but not in every case at 397 Hz), and at the nominal frequency. Shifts in these pitches predicted by the algorithm due to changes in partials are shown in Table 9.
The pitch shifts seen in the experiment in Section 2 are an amalgamation of the various virtual pitches as predicted by the Terhardt algorithm, across the set  upper partial effect seen in the experiment results is not predicted by the algorithm; nor are the significant prime and tierce effects.
The Terhardt algorithm was also applied to the test sounds used in the subsidiary experiment, the results of which are shown in Figure 4. The test tones used in this experiment had a nominal frequency of 1000 Hz, and as seen in On the basis of these trials one can confidently say that the pitch shifts seen in practice are not predicted by the Terhardt algorithm.

Conclusions
The two experiments reported in this paper have shown that the perceived pitch of a complex tone due to the virtual pitch effect is shifted from the expected value (of half the lowest frequency in the harmonic series) if the frequencies of the partials in the harmonic series are stretched or compressed. The second experiment also shows that the size of this shift is only moderately affected by the amplitude of the next two higher frequency partials in the series. A pitch shift is still observed if these two partials are removed completely from the sound. The pitch shifts are large enough (at considerable fractions of a semitone) to be musically significant.
These findings have a number of consequences: • although the complex tones used in the experiments were modelled on the sound of church bells, the relative insensitivity of pitch shifts to partial amplitude means that the principles, if not the exact numerical results, are of general applicability; • many partials in the harmonic series, not just the lowest in frequency, contribute to the formation of the virtual pitch in the ear; • the failure of Terhardt's well-established model of virtual pitch to predict the pitch shifts observed suggests that further investigation into pitch formation mechanisms in the ear is desirable.