Consonance of Complex Tones with Harmonics of Different Intensity

The starting point of this study is the theory of tonal consonance by Plomp and Levelt. The major reference point is the theory of Critical Band and its use for evaluating consonance of simple tones. On the basis of such empirical criterion, our aim consists in providing a method of estimating the value of consonance of complex tones, by taking account not only of the number of harmonics, but also of the intensity of the partials which contribute to forming the tone. By introducing a specific algorithm, the level of consonance of a chord of complex tones is expressed in terms both of frequency differences and sound intensity of the partials. The results give rise to a series of considerations and applications, ranging from displaying the order of consonance of all intervals within the octave up to evaluating the hierarchy of chords in the frame of a musical scale.


Introduction
It is a common experience to perceive an agreeable acoustic sensation when tones with simple frequency ratios are sounded simultaneously. The musical theory denotes such intervals as unison (ratio 1:1), fifth (2:3), fourth (3:4) as perfect consonances and major third (4:5), major sixth (3:5), minor third (5:6) and minor sixth (5:8) as imperfect consonances. The association of consonance with simple ratios goes back to Pythagoras who, according to tradition, discovered the euphonic concordance of a vibrating string with a part of it, whose length was in a simple ratio with respect to the whole string (half, one third, ...). The same concept is formulated in terms of frequency in the 17-th century, when the dependence of pitch on the number of vibration in a unit time was clear. The experimental proof about existence of the harmonic partials (by J. Sauveur in 1701) forming a complex sound provided, in some sense, a scientific confirmation of the simple ratio theory of consonance: since the first partials (i.e. the first terms of the series of harmonics) correspond to the division of the string into small integer ratios, we may say that the pleasant sensation of two tones matches a physical explanation in the fact that one tone is somehow included into the other, by means of the overtones series. The relation between consonance and partial has been deeper investigated during the XIX century. The theory developed by H. von Helmholtz [1] aimed to explain consonance in terms of beats: the leading idea consists in realizing that a very small frequency difference produces few beats which are individually heard and not unpleasant. On the other hand, a large frequency difference gives rise to a considerable number of beats, so that they cannot be heard. Between the two opposite situations, von Helmholtz identified the frequency difference of 30 -40 Hertz (independently of frequency) as the maximum of roughness produced by two simple tones.
The hypothesis of von Helmholtz was essentially confirmed by Plomp and Levelt [2], with the important additional modification that the frequency at which maximum dissonance occurs actually varies over the frequency range. In order to evaluate the dependence of the maximum roughness of intervals on their mean frequency, Plomp and Levelt refer to the critical bandwidth experimentally measured by Zwicker, Flottorp and Stevens [3]. In a few words, we may define the critical band as a range (changing over the frequency range) of frequencies within which two pure tones (say 1 ν and 2 ν ) are perceived as one rough sound. The graphic in Figure 1 shows the width of the critical band [3], as a function of the mean frequency ( ) It can be remarked that for 0 650 Hz ν > the bandwidth keeps a bit under the minor third interval, while it ranges over the whole octave interval for values of the mean frequency 0 ν below 650 Hz. The minimum and maximum consonance, according to the hypothesis made by Plomp and Levelt [2] occur respectively at 25% and 100% of the critical bandwidth. As a matter of facts, the data summarized by Plomp and Levelt and achieved from laboratory experiments, led to the consonance curve plotted in Figure 2, which essentially confirms their conjecture.
It can be observed that the minimum consonance occurs approximately at one fourth of the critical bandwidth. On the other hand, we remark that the maximum consonance corresponds both to the unison and to the value * h h = , which is not exactly 1, but * 1.2 h  .

Evaluation of Consonance
The two curves of Figure 1 and Figure 2 can be used in order to give an evaluation of consonance of two simple tones of frequencies 1 ν and 2 ν , according to the following scheme  , derived from [2]: • calculate the mean frequency ( ) In order to make the algorithm effective, we need an analytical expression for the critical bandwidth ( Figure  1). For this purpose, we fit the data marked with a circle in Figure 1 in a least-squares sense. Fixing the degree of the approximating polynomial as 4, the following expression was found: ( ) 5 The same procedure is applied to the curve of Figure 2 and the polynomial of degree 5 fitting the data marked with a cross is the following: 48.808 29.693 6.8823 , 0 The scheme  can be easily extended to complex tones. Referring to Ohm's acoustical law, which states that the human ear carries out an analysis of the harmonic components of a complex tone [1], one may argue that the degree of consonance of two complex tones is simply obtained by considering the simoultaneous single occurrences of beats between the partials, taken two by two, and summing up all the contributes. In [2] such an argument is used in order to evaluate the consonance of complex tones. The procedure we deduce from their paper is the following. Let 1 f and 2 f be two complex tones each one with partials ( ) tones. In the spirit of Ohm's law, according to which each partial is analyzed as it were a single sinusoidal sound, we find it coherent to take rigorously account also of the possible presence in the critical band of two partials belonging to the same complex tone. However, in actual cases (that is 8 n ≤ and range of fundamental frequencies between 250 Hz and 6000 Hz) such an occurence is practically non-existent.
By considering that the match of a single partial with itself produces no contribution to the total dissonance, we find the number of matches that the algorithm  checks being 2 2 . In a more general case of chords formed by N complex tones with n partials each, the number of checks is 2 . It is evident that the total value of dissonance d is hardly affected by the number of partials n and the number of tones N . Anyway, we do not find it essential to normalize the value d .
One of the most relevant applications of the procedure consists in plotting the "consonance" curve of an octave interval, as illustrated in Figure 3.
A lower complex tone 1 f is fixed, while the upper tone 2 f varies from the unison The curve we obtain ( ) 6 n = is the same as the one in [2], page 556. As it is evident, the local minimum values of the curve correspond to the tonal interval with simple frequency ratios. The hierarchy of tonal intervals we deduce from the procedure of Plomp and Levelt is, in decreasing order of consonance, 1:1 and 1:2 (unison and octave), 2:3 (fifth), 3:5 (major sixth), 3:4 (fourth), 5:6 (minor third), 4:5 (major third). It is worth stressing that the choice of a larger value n , that is considering a wider range of harmonics which form the complex tones, entails the following facts: • the maximum value of dissonance increases if n increases, since a larger number of matches in the critical bandwidth are expected, • extra minimum values in the consonance curve will appear, as well as the superimposition of new simple tones in the series of harmonics determines the presence of new intervals (e.g., the seventh partial gives rise, among others, to the interval of minor seven). The presence of minima and, more generally, the properties of the consonance curves are studied in a extensive way by Sethares [4].
A further application of the procedure  consists in evaluating the consonance of tonal chords, i.e. the simultaneous emission of N complex tones, 2 N ≥ . Formally, let k f , 1, , k N =  be N complex tones each one with partials ( ) . The algorithm  is then applied to each of the pairs of simple tones ( ) , corresponding to the matching of the i-th harmonic of the complex tone h with the j-th harmonic of the complex tone k . The total dissonance is given by we find again the procedure relative to two complex tones explained above). An application of such a procedure will be presented and commented in Section 4.

Complex Tones with Different Loudness of Partials
As it is known, the timbre of a complex tone is mostly determined by the proportions in which the partials enter.
In the ideal situation we described in Section 2, it is assumed for simplicity that all the partials forming the complex tone are heard at the same intensity. However, it is quite natural to think that the sensation of dissonance produced by two tones depends somehow on the relative intensity of them: the roughness of an unpleasant interval is expected to vanish if one of the two tones is deadened and to expand if the loudness increases. The experimental setup made by Plomp and Levelt [2] planned a constant sound pressure of about 65 dB for each tone produced. In [5] the aspect of different sound intensities has been introduced, by considering a sound-pressure level for each single tone. On the other hand, Sethares [4] considers the amplitude of the simple sound-waves as a parameter which takes account of the intensity. The formula calculating the dissonance contains directly the amplitude.
We are going to suggest an alternative method to evaluate consonance which involves directly the scale of loudness, instead of sound pressure or amplitude, which refers to an objective measurement of intensity. The first and fundamental step consists in "correcting" the dissonance d produced by the algorithm  for two simple tones 1 ν and 2 ν . Formally, our aim is to find a suitable factor f , depending on the intensities, say x and y , of the two tones so that the new value of dissonance will be ( ) Once f is established, the total dissonance of two complex tones will be simply the sum of all the revised terms, in the same way we described in Section 2. At this point it is important to specify the scale of sound intensity we refer to. For our purpose it is quite natural to refer to the scale of loudness with phon as unit of measure, in the spirit of evaluating a subjective perception of consonance. Actually, a pair of tones with frequencies under the threshold of hearing (even if with non-zero decibel) will not produce any dissonance. Therefore, any possible data at our disposal expressed in decibel, corresponding to an objective scale of sound intensity, will be converted into phons by means of curves of equal loudness, which can be found, for istance, in [6]. The values 0 x = , 0 y = correspond to the threshold of hearing. We notice that the use of the Fletcher-Munson curves is properly suggested by Sethares [4].
We are going now to discuss the choice of the weighting function ( ) , f x y . First of all, it is natural to require that if one of the two tones 1 ν or 2 ν cannot be heard, then the resulting dissonance is null. Hence, we for- Besides that, it is quite natural to speculate that the sensation of roughness proceeds in the same direction as loudness. More specifically, we say that a second requirement consists in selecting f such that, whenever one of the two intensities is kept constant, then the dissonance increases (decreases) if the second intensity increases (decreases). In other words, f must increase along the straight line directions x constant, y constant, that is Let us consider now the special case of equal sound intensities x y = . Basing ourselves on the statements in [6], where the overall loudness of two simultaneous sounds is discussed, we deduce that when two simple tones are produced at the same sound intensity and their frequencies are within the critical band, then the total intensity remains the same. Along the direction x y = the function f is assumed to be By imposing the linear profile (5), we state simply that if the intensity of the tones x is doubled, tripled, reduced by half and so on, then the dissonance of the mixed sound (which is at the same intensity) increases or decreases by the same factor. We notice that in that case the maximum value of dissonance 1 (see Figure 2) is replaced by x on the dissonance scale, which is anyway arbitrary. A question arises about the possible simmetry of the function f . The experiments carried out by Kameoka and Kuriyagawa [5] on the consonance sensation show that two simple tones are more consonant when the higher frequency tone is louder with respect to the lower frequency tone than the opposite case (lower frequency stronger). In our scheme, x represents the intensity of the sound with lower frequency. Let us first consider the case x y ≤ . The selection for f can be further confined if we require that f depends on x and y by means of the arithmetic and geometric means in the following way: where α and β are non-negative real numbers. Any function of Type (6) verifies the properties (3) and (4).
Owing to (5) it must be Notice that for 0 α = we get the geometric mean, while for 1 α = we would get the arithmetic mean, which is however not consistent with (3). Furthermore, the set of functions depending on the parameter α ( ) ( ) ( ) is such that: , 2 x y xy f x y α + ≤ < , • for α tending to 0 (resp. 1)

( )
, f x y α converges uniformly to the geometric mean xy (not uniformly to the aritmetic mean ( ) 2 x y + ).
The opposite case x y > can be treated in the same way, by introducing a function of the same Type (8), with possibly a different exponent. We expect [5] an exponent 1 α α > , since a stronger lower frequency tone is claimed to determine a weaker consonance. In this way, we obtain the definition of the function x y x y f x y xy where sign is the sign-function ( In order to give a better idea of the function, we plotted in  It can be noticed that the slope of f , as x or y tends to zero, is "infinite". Moreover, the larger is α , the more vertical is the function, in proximity of the axis. In terms of the real meaning of the variables, this means that the larger is α the more abrupt is the deadening of the perception of the dissonance. If the conjecture of linearity (5) does not appear consistent with experience, then, still maintaining the structure (6) for f , gives the nonlinear dependence of dissonance on intensity x .

Some Applications of the Procedure
For the sake of clearness, we summarize the procedure we are going to follow: • given N complex tones k f , 1, 2, , • by algorithm  compute the value of dissonance ( ) • the total dissonance is given by . Nevertheless, any value in [ ) 0,1 for α and 1 α can be considered in order to fix f , whenever further speculations or experimental data lead to some specific choice.
The first instance we consider consists in drawing the curve of consonance relative to the interval of octave middle c (261.6 Hz) -c′ (523.2 Hz), considering 8 n = harmonics, with their specific sound intensities. We need at this point some data relative to the spectrum of a specific instrument. The values (in Decibel, then converted in phon by means of the Fletcher and Munson curves) deduced from [6] allowed us to fill Table 1, where the sound intensities of the i-th harmonic i ν , of c and c′ of piano are listed. In completing the values of harmonics intensities for the intermediate tones within the central octave c-c' we actually found the difficulty of no data available. We fill this gap by interpolating the existing data in a linear way: such an assumption was motivated by the fact that the energy is well distributed between the harmonics in the central octave of piano (as we deduced from [7]). As we expect, the timbre does affect the evaluation of consonance: for such reason, we avoid to maintain the same harmonical spectrum of each complex sound, independently of the frequency of the fundamental tone, as assumed by Sethares [4]. The evaluation of consonance with the second tone, ranging in frequency from 261.6 Hz to 523.2 Hz (upper octave), is checked at intervals of approximately 0.26 Hz (1000 steps), via a program in MATLAB. The curve of consonance we obtained is plotted in Figure 5.
Once again, the simple ratios intervals correspond to the minimal values of dissonance. As it was already observed in Section 2, the increase in number of partials n gives rise to the appearance of further minimum  values of dissonance (compare with Figure 2, where 6 n = ). Actually, the local minimum at 457.8 Hz corresponds to the seventh harmonic of c (a tone between a and # a in the Equal Temperament). The combination of such partial with some other harmonics gives rise to the two minimum values appearing between the vertical lines d and # d . As it is known, the simple ratios give rise to the Just Intonation tuning. For practical reasons (concerning mainly with keyboard instruments), many systems of tunings have been introduced over the past centuries. We will consider the probably most diffused (ancient and modern) systems of tunings for keyboards: Quarter (syntonic) Comma Meantone Tuning, Werckmeister Temperament III and Equal Temperament. An introduction to the various scales we consider is, for istance, in [8] and in [9]. In Figure 5 the discrepancies of a single interval tuned according to each specific scale (Meantone, Werck III and Equal Temperament) with respect to the Just Intonation simple ratio (corresponding to the minimum values on the curve) have been marked.
As a second instance of the procedure, we find it interesting to test the consonance of chords in different tuning. We consider the major chords starting from c-e-g, up to g-b-d', in the Just Intonation, Meantone, Werck III and Equal Tempered tunings. For the sake of clearness, we are going to write the values of tuning (expressed in cents) in each of the systems, starting from c . Let us assume to have a keyboard with twelve tones from c to b , namely c , # c , d , e♭, e , f , # f , g , a♭, a , b♭, b . We recall that the interval in cents between the frequencies 1 f and 2 f ( ) , c′ (1200). As it is known, fixing c as zero cents, the twelve chromatic tones of the equal tempered tuning #, , , c d c′  correspond to 100, 200,  , 1200 cents. The values in cents have been calculated directly by means of (10). The ratios of frequencies will be explained in more details throughout the following discussion.
We evaluate the consonance of the major chords (four semitones plus three semitones) taking as fundamental tone , #, , c c g  by means of the algorithm summarized above, with 3 N = , 8 n = . Having twelve tones in the octave at our disposal, the triad on # c for the non-enharmonic tunings (Just Intonation and Meantone) is # cf -a♭, and so on. The results are summarized in Table 2, where the values correspond to the corrected dissonance It is evident that in the Just Intonation system there is a remarkable difference in the value of dissonance among the triads. In particular, the triads which mantain the same proportions of major third (4:5, corresponding to 386 cents) and of fifth (2:3, corresponding to 702 cents) as the fundamental tone c show a value of dissonance very close to the triad over c : namely, a♭, f , g . The triads corresponding to non-usual chords (namely # c , e and # f ) show higher dissonances. Obviously, if we introduce the seventeen tones of the Just Intonation scale (sharp and flat tones), such dissonances would be appreciably milded: for istance, considering e -# gb ( # g corresponds to 772 cents), the value of dissonance of the triad would be 99.91. The triad over d is altered with respect to the natural proportion both in the third (364) and in the fifth (680): the high value of dissonance of such triad is not surprising. According to the Just Intonation tuning, only few tonalities would be comparable in terms of dissonance: this explains the necessity of different tunings (second, third and fourth columns in Table 2), at least for keyboard instruments equipped of not more than twelve keys in a octave. Actually, we see that the triad over d evidently shows a lower dissonance in the Meantone tuning: this is due to the preservation of the major third intervals in that system. As a matter of facts, the Meantone scale consists in reducing the four fifths cg , gd ′ , d -a , a -e′ by a quarter of syntonic comma, so that the tone e has the simple ratio 5/4 with respect to c . The resulting tuning is a chromatic scale where all the major thirds are natural (4:5) and all the fifths are tuned at 696.5 cents, except for # c -a♭, which corresponds to 738.5 cents. This chord corresponds indeed to the most dissonance triad. Once again, the triads with natural thirds ( c , d , e♭, f g ) give rise to the lower dissonant chords in the column. The tuning system Werck III uses the quarter Pythagorean comma to reduce the four fifths cg , gd ′ , d -a , b -# f ′ , so that the overlap in the circle of fifhts is closed. In this system no major third is natural not all the fifhts are tuned at 702 cents, so no major chord will play as well as a perfect major chord in the Just Intonation tuning. Werck III was introduced, as the Meantone tuning, in order to make easier the modulation from one tonality to another. The data of Table 2 show a further balancing of dissonance among the triads tuned in Werck III, even if some chords "play" better than others (namely c , d and g , the only ones with natural proportions of third and fifth). The levelling effect is finally much more evident in the Equal Temperament, where no interval is natural (the major third is always 400 cents, the fifth is always 700 cents). The values of dissonances are practically the same. The decreasing in dissonance with higher frequencies (last column) reflects in a consistent way the fact that the distance between the interval of third and the critical bandwidth tends to reduce, as the mean frequency increases (see Figure 1). We tested the algorithm also for minor triads. The values are plotted in Table 3. We remark that in this case triads are more dissonant than in major mode. The values read along both rows and columns of the table suggest the same considerations and comments we made for the major mode.
A second category of multiple sounds we can consider are chords of sevenths of different species ( 8 n = , 4 N = ). Table 4 summarizes the values we obtained from the algorithm.
As we have already noticed, the values of dissonance are manifestly higher with respect to the triads, since N is increased and more matches in the critical band are expected. However, we find it more interesting a relative comparison of the data table by table, rather than referring to a concept of absolute dissonance. Even though the partials we are considering refer to the modern piano (whose tuning is, from its appearance, the Equal Temperament), the data of Table 2 may be interpreted in order to make some comments. First of all, the seventh of II species (second row in Table 4) shows the minor dissonance in any system: actually, it has been largely used by composers starting from the Renaissance and many examples could be exhibited (Palestrina, Monteverdi, ...). Moreover, the tones forming such chord appear in melodic sequences even since the Middle Ages, namely in the first mode (dorian mode: d -efg -ab -c′ ). A second interesting aspect is that the seventh of V species is far and away the most dissonant chord in the Meantone and Werck III tunings, while it is comparable with the other species in the Equal Temperament. This could explain, besides any harmonic and stilistic reason, its large use in the XVIII century, contemporaneously with the spreading of Equal Temperament.
Another interesting test is made for the diminished chord B-d-f, whose results are in Table 5.   As we expect, the dissonance is larger with respect to major and minor chords. However, we remark that the inversion (sixth chord) is less dissonant than the fundamental chord. This reflects the practical rule according to which the inversion is more used in the harmonic sequences and cadences of the musical repertory, especially in the Eighteen century. On the other hands, if we detect the dissonance values of inversion chords, we realize that the major chords are more dissonant in the inversion than in the fundamental chord, while the minor chords present the opposite situation. We found indeed the following values ( Table 6), which have to been compared with the corresponding fundamental chords (same tones) of Table 2 and Table 3.
As we see, if on the one hand the major inversions are just a bit more dissonant than the fundamental chords, on the other hand the inverted minor chords are remarkably less dissonant than the corresponding fundamental chords. This is in correspondence with the practical musical rule of using the inverted minor chords in cadences.

Conclusions and Remarks
The procedure we suggested in order to evaluate the consonance of two or more simple or complex tones aimed to generalize the method of Plomp and Levelt [2], by encompassing the important aspect of the sound intensity. The main point of reference we used is the curve of Figure 2, which determines the degree of dissonance in terms of the critical bandwidth. Our method is based on the conjecture that the most dissonant interval is proportional to the critical bandwidth, independently of the loudness of tones. This point requires a special comment. If, on the one hand, it is reasonable to assume that the critical bandwidth does not show sensible change with sound pressure level [3] [5], so that the upper boundary in Figure 1 fits for a large range of intensities, on the other hand one may conjecture that the most dissonant frequency does increase with sound pressure level, as it is clearly stated in [5]. In that case, the algorithm we presented needs a slight modification, consisting in setting the frequency difference of maximum dissonance b f as a function of the lower frequency and of the intensities. Actually, in [5] we find f the frequency of the lower tone. In our opinion, the choice of the scale of sound intensity in phon is appropriate for our purposes: frequencies below the threshold of hearing cannot give a contribution to dissonance. Undoubtedly, a major difficulty consists in determining (via experiments) an appropriate value of the parameters α , 1 α entering the correcting function (8), which we speculated to have such a shape. By the moment, in our simulations we observed very slight differences in changing the values of the parameters: for istance, the simmetric case 1 0 α α = = , corresponding to the pure geometric mean of the two values x and y , produces essentially the same curve. Nevertheless, a series of experiments in order to get information about the parameter will be carried out in the frame of a degree program in our Department. On the other hand, the method we explained requires the knowledge of the harmonic spectrum of any complex tone involved in the procedure: as a matter of facts, there is not an easy availability of such data. In any case, the method presents the non-negligible advantage of ascribing to a specific instrument (with a specific spectrum of harmonics) a proper hierarchy of consonances, as it is expected to be. Actually, different relative intensities of the harmonics forming a complex tone will surely affect the profile of the curve in Figure 5, being understood that the minimum values of dissonances should always appear in correspondence of simple ratios frequencies. In this sense, an application we are going to carry out consists in drawing the curves of consonance for different instruments, once the data concerning their partial intensities will be at our disposal. Moreover, it will be interesting to detect the inverse problem, investigated by Sethares [4]: once the spectrum of an instrument is given, find the most appropriate scale. In such a context it is important to consider the real (i.e. non-harmonic) series of partials; the presence of local minima in the dissonance curve will locate the intervals of the peculiar scale. The theorical and numerical apparatus we presented could be used with very slight modifications in order to solve the problem.
In our procedure we did not distinguish "sensory consonance" (as the graded absence of annoying factors) from "harmony" (as music-specific principles of tonal affinity), according to the concept explained in [10]. Undoubtedly, an important component of musical consonance does refer to tonal affinity, compatibility, cognition. However, in our mind some conflicts appearing in psychoacoustic evaluation and musical theory or musical