Advertising Effectiveness : An Approach Based on What Consumers Perceive and What Advertisers Need

This paper examines how advertising effectiveness can be defined and measured. We propose a 3D-effectiveness key performance indicator involving the weighted prominence of three factors: recall, brand image and buying intention. In this way, the indicator proposed improves the metrics of this type known in the Spanish market. Using experimental data about 20 fixed brands and other 12 variable brands collected along the 52 weeks of a year, we find that this indicator reflects consumer’s perceptions and that it fits advertising company’s needs. We have developed a web tool to provide the market with a benchmark measure of advertising effectiveness. Finally, we discuss future research lines.


Introduction
One of the greatest challenges facing advertisers and agencies continues to be measuring the effectiveness of their advertising campaigns, typically one of the largest line items in the marketing budget.The situation is further complicated by the emergence of new advertising formats and platforms [1]- [4].So, what we now have is a different consumer [5], spreading his/her own messages [6].The result is that measurement becomes ever more problematic, making it impossible to measure "everything".
In parallel, advertisers are thirsty for metrics to help them with their day-to-day decision-making [7] [8], without having to wait a month pending the results of an investigation.Added to this is the lack of a precise de-finition of the concept of effectiveness [9], despite the multitude of articles that have been written on the subject and the studies that are available today in the market place.In this context, this paper aims, first, to define a new effectiveness key performance indicator (KPI), based on what consumers perceive, in such a way that the complexity of the new environment and the measurement difficulty would not matter so much.Second, this indicator should be simple in concept, but covering everything that may be important to advertisers, to whom ultimately must prove to be useful.And finally, it should be possible to measure it in a flexible way, and be available just one week after a communications campaign was aired.

Objectives: 3D-Effectiveness
Before we can decide whether a communication campaign has been effective or not, we need to clarify first what we actually mean by effectiveness, against what objectives should advertising effectiveness be evaluated [10].After many years of experience working closely with major companies around this concept, and given that the main purpose of this research was to be useful to advertisers, we understand that there are three key axes with respect to which a campaign can be considered effective or not (to some extent, this was already anticipated by White in 1998 when he wrote "All we need to know about the ads is whether people know they are there (Recall)... What matters is what they think about the brand, and whether they are going to buy it" [11].The three key axes that we suggest are: 1) the recall axis (cognitive factor [12]), where a recalled campaign is considered to be effective, 2) the image axis (emotional factors relating to a brand [13], where the aim is that the campaign should be able to improve the image that consumers have of the brand being advertised; and 3) the business axis (behavioral factor [14]), where the campaign is required to be able to increase the buying intention of the product being advertised.Regardless of the possible hierarchy which might exist between the three concepts-similar to the AIDA hierarchical model, linking "Attention" to "Interest", "Desire" and "Action" in terms of a cause-andeffect relationship [15], and their adaptation to marketing communications [16]-, our approach is not so much how the concepts relate and complement each other, but that all three can be easily measured and can be of interest to advertisers.Depending on each case and product life time, the importance of the axes may vary, so that the concept of effectiveness mutates over time and is not always the same.3D-Effectiveness is calculated as the weighted average of each axis, based on the a priori importance of each one of them.where α i are the weights that each advertiser determines based on the objectives which their communication had for each of the D i dimensions.This new way of defining effectiveness is better suited to the reality of advertisers, who sometimes run campaigns whose main objective is to achieve media coverage by getting many people to recall having seen the advertising, other times what they want is to improve brand image and at other times, however, all they want is to achieve rapid sales increases.
The objectives that we intend to address through this research are five: 1) Creating a concept of effectiveness that suits the needs of each advertiser (as seen in the previous section) and is valid for the different communication strategies that may be applicable; 2) That such effectiveness, that will be measured through fieldwork and therefore will be redefined as "perceived effectiveness", will be a type of effectiveness without the frequent biases that these kinds of methodologies have; 3) That we may set a benchmark in the market, so when comparing effectiveness between different brands, there will be a point of reference against which to measure performance; 4) That this information is open and free to everyone; 5) And finally, that the data may be available in a flexible way, so that we might speak of a "just-in-time" measurement of perceived effectiveness.

Methodology. Empirical Test of Hypothesis
Since January 2012, we have conducted a weekly tracking with CAWI methodology (Internet interviews), aimed at a representative sample of the Spanish population aged between 16 and 55.We have carried out 500 weekly interviews, with a total of about 26,000 annual interviews, based on quota sampling by age, sex and region.The sampling error is 4.4%.Each week the 20 fixed brands have always been tested and another 12 that have been changing each month.The 20 fixed brands were selected based on the Top of Mind advertising awareness 2011 ranking of IMOP (study done by the Public Opinion and Marketing Institute in Spain, which measures spontaneous advertising recall by media, without suggesting a brand or a product line), and the advertising investment ranking by INFOADEX company (source of investment data in Spain) also for 2011, with a maximum of 4 brands per sector.The performance indicators for these 20 brands have been tracked over the 52 weeks of the year.The other 12 brands that have rotated through each month represent two different sectors, so that in six months we have investigated the 12 sectors with the largest investment and enhanced awareness in the country.This has allowed us to have a full picture of each sector twice a year (see  The questionnaire covers the three axes of the perceived 3D-effectiveness: Recall, Image and Business, as well as questions about the media in which they recalled having seen the advertising, product consumption, overall assessment of the brand, and finally, and most importantly, a battery of questions on media consumption, which will help us to make a study of coincidence against the Media General Study (EGM) official source of audience measurement in Spain, which will give greater strength to our study ensuring that the media consumption is also representative of the general population (See an outline of the questionnaire used in Table 3).
When calculating the effectiveness index, we eliminated three common biases that characterize this type of research: 1) The brands engaged in a campaign in the week in which the fieldwork is carried out and have a large investment on that week, always have a higher recall than the ones which have a smaller investment.Therefore, a brand "A", out of the 20 analyzed each week, can have a high recall on a given week via a large investment versus another brand "B" having a smaller recall, because it had a very small advertising investment on that week.If we understand that having a high recall to be equated to having great effectiveness, we will then be penalizing brand "B".To avoid falling into this bias when estimating effectiveness, we performed some simple regression models between investments and recall levels, where R is the recall level and I the advertising investments.Then, we calculated the performance indicator in recall, D 1 , as the difference between the recall level R obtained from the questionnaire and R , the expected value of the recall based on the investment made.
[ ] Thus D 1 is the effectiveness dimension corresponding to recall and there will be campaigns with positive effectiveness and campaigns with negative effectiveness, what we will standardize at a later stage; 2) Also with simple regression models, we note that there is a high correlation between the value of a brand and how people perceive the advertising of that brand with a view to improving the image consumers had of the brand.If brand A is a highly valued brand, consumers will tend to claim that its advertising helps them improve the image they have of the brand, and vice versa when a brand is poorly valued.The way to overcome this bias in our effectiveness measurement will again be to compare the expected value according to the model, with the actual value obtained through the questionnaire; 3) Finally, in the business axis we can see the multivariate dependence of the brand image perception and its market share.A brand with a high value image and a high market share will always be favored in such questionnaires, falsifying the data unless this condition is isolated.With these three kinds of models, we obtained three effectiveness measurement indicators free from some of the main biases that characterize this type of research.

Limitations of the Method
In no circumstances this method could compete with research carried out by advertisers by means of questionnaires over long periods of time, most of the time face-to-face, exclusively designed to measure the effectiveness of communication campaigns one by one.Neither is designed nor intended to measure all the important attributes of the image of the brand, or going into the details of the variables such as buying intention or recommendation of a product.

Findings
The 3D-effectiveness key performance indicator proposed in this paper involves the weighted importance of three factors: recall, brand image and buying intention.In this way, this new metric reflects consumer's perceptions and it fits advertising company's needs.The indicator is built using experimental data about 20 fixed brands and other 12 variable brands collected along the 52 weeks of a year.The results obtained over the months have been enriching and in the following paragraphs we will discuss some of the main results.
Effectiveness KPI.First, we obtain a perceived effectiveness KPI, week by week, for the top 20 brands in Spain throughout the year 2012 (Figure 1 shows an example for the 26/11/2012 week, where we can see Renault, McDonalds and L'Oreal leading the ranking).Comparing this ranking with that of the previous week (Figure 2) we observe how it changes: in this case McDonalds, L'Oreal and Audi lead the ranking now.In both cases, the colors indicate whether or not the effectiveness has improved compared to the previous week.In the overall ranking, the perceived effectiveness KPI is built giving a third to each of the three effectiveness axes outlined above: recall, image and business.
Changing Concept of Effectiveness.Second, for each of the analyzed brands we can get the value of the 3 effectiveness axes, and, based on the principle that for each advertiser the concept of effectiveness may vary, each one of them can allocate the weights they may deemed valid.For example, if Nike considers that for their campaigns the weights should be 0.25 to recall and image, and 0.50 to business, they would move from the eleventh position, on the 26/11/212 week, to third position (Figure 3).
Top-20 Effectiveness.Third, we perform a weekly effectiveness tracking of the 20 top brands in Spain considering advertising awareness and investment.Such tracking will allow us to know how effectiveness changes  over time, besides being a benchmark for this indicator in the Spanish market, and which can be crossed with other more general variables.This will give rise to future research such as: what does effectiveness depend on, and whether it is affected, for example, by such variables as the consumer confidence index, reflecting a situation of crisis.
Effectiveness by Sector.Fourth, each month we have analyzed two sectors and every six months we repeated the study, so that each sector has been analyzed twice within a year.Based on this, we analyzed effectiveness by sectors, being able to compare each brand, no longer with the Top-20, but with their 6 or 7 main competitors.As future research lines in this case we want to emphasize the comparison of the evolution of the effectiveness by sectors and their drivers.
Effectiveness without Biases.Fifth, as explained above, we have eliminated some of the biases that frequently contaminate these researches.In Figure 4, we can see what the results would be in the beauty and health sectors without eliminating these biases.The bubble size is the recall, together with image on the horizontal axis and business on the vertical axis.A brand like Nivea is clear leader in image and business, with a recall slightly above average.However, in Figure 5, where we have already eliminated the biases, we see the different situation of Nivea, well positioned in image, but with a business axis below the average and an expected  recall close to the average.Our experience gathered throughout the 12-month study is that in general the reading that is obtained of effectiveness with or without biases comes out as significantly different.We believe that, as far as possible, it is important to eliminate the biases, and to this end we propose to use the methodology described above.Targeting Based on Effectiveness.Sixth, considering that each week the sample size is large enough, each month we can turn to the study of the effectiveness based on such variables as sex, age, educational level, region, media consumption or the fact of being a consumer or not of the brand.This will allow advertisers to address that segment of the target audience with low effectiveness and analyze the drivers of those sectors for which advertising has been highly effective.
Effectiveness for Each Media.Finally, from knowing which media the interviewees recall having seen the advertising, we can estimate the perceived 3D-effectiveness for each media.Figure 6 shows an example of this type of ranking.While this was not one of the initial objectives of our study, we believe it is one of our main findings, taking into account the importance that is being given today to the measurement of media effectiveness and the attainment of the optimum mix.

Conclusions and General Discussion
This project has largely achieved its initial objectives.We have been able to create an effectiveness indicator to suit the needs of each advertiser and which is valid for the different communication strategies that may exist (achieving coverage (recall), improving brand image and increasing buying intention).This indicator improves the metrics of this type known in the Spanish market and this improvement occurs in two ways: Firstly it addresses not only the traditional advertising recall, but also concepts as important for the advertiser as brand image or buying intention, and, secondly, it eliminates some of the biases that commonly crop up in these measurements.
Another handicap that these types of studies usually have, is the amount of time one has to wait until data are available, 2 or 3 weeks, at the earliest, after the end of the campaign.In our case, we have pursued speed and reliability of results, making them available on the week following the end of the campaign being tested.
Furthermore, in an effort to make this KPI known to the market and thus being capable to set an effectiveness benchmark against which to compare performance, we have developed a tool that has web access and which is open to everyone [17].
All this makes this research and its results especially relevant for the Spanish market advertisers; they can now know a little better what the effectiveness of their campaigns has been, and also what the effectiveness of the different media has been.This does not mean that this study and its methodology may be valid for any other country that would want to implement it, but it could be adapted to each particular case.
As future lines of research, several options are open, highlighting that of analyzing the evolution of effectiveness and to discover what are its main drivers, as well as comparing effectiveness by sectors, looking for possible explanations.In addition, as from January 2014, we will proceed to analyze the Top-100 instead of the Top-20.

Figure 6 .
Figure 6.Effectiveness for each media.

Table 1
(summary of the 20 fixed brands) and Table2(summary of the 12 sectors)).

Table 3 .
Outline of the questionnaire.