Assessment of Counselling for Acute Diarrhoea in North-Eastern German Pharmacies—A Follow-Up Study Using the Simulated Patient Methodology ()
1. Introduction
In the self-medication market with over-the-counter medications from pharmacies (OTC market), patients generally request medications without previously consulting a doctor. For this reason, a specialist advisory service offered by pharmacies is particularly important to be able to prevent or at least considerably minimise various risks [1] such as incorrect self-diagnosis, incorrect choice of therapy and infrequent but severe adverse effects. Providing good advice is not just an important criterion for patients. It also plays an important role in the competition between pharmacies. Although the density of pharmacies in Germany is below that of the overall European average [2] , there is intense competition with other local pharmacies due to the freedom of establishment, particularly in areas with high population densities [3]. The competitive situation in Germany has intensified since 2004 due to the abolition of price maintenance in the OTC sector [4] as well as the approval of mail-order sales of medicinal products [5]. A good advisory service can therefore provide an important competitive edge.
Although studies carried out in Germany to date on the quality of advice provided by pharmacies show significant shortcomings [6] [7] [8] [9] [10] , they do refer to other indications rather than “acute diarrhoea”. Acute diarrhoea is one of the most common illnesses in Germany [11]. If it is treated using self-medication, the pharmacy should provide information [12] because acute diarrhoea can be a symptom of a range of different diseases [13].
For Germany, the quality of advice provided by pharmacies for acute diarrhoea in adults was examined for the first time in 2014 by Langer et al. and significant shortcomings were identified [14]. The simulated patient methodology used is a well-tested approach for revealing any weak points [15]. Test persons simulate participation in an apparently real service provision process and subsequently evaluate the service provided and in this way identify any defects. One advantage of this method is that a true consultation situation can be replicated. The drawbacks include specifically the relatively high data collection costs as well as any margin in the evaluations (both between different test buyers and between different test purchases made by the same test buyer) [16]. Moreover, the test purchases remain ineffective if the deficits that are identified are not communicated to the pharmacies and if the pharmacies do not subsequently implement appropriate improvement measures. For this reason, the study conducted by Langer et al.―analogous to many other international studies [17] ―included a written individual performance feedback for each pharmacy so that the pharmacies evaluated can ideally initiate appropriate optimisation processes based on this feedback with the aim of increasing the quality of advice provided over the long term. Whether the pharmacies have actually taken steps to improve their performance based on this performance feedback could not be checked at the time because this would have required carrying out another test purchase study (possibly using the same criteria).
It was also not examined whether the quality of advice differed between the professional groups providing the advice (pharmacist with a 4-year pharmacy degree and subsequent 12 months’ practical training in the pharmacy; pharmacy technician with 3 years’ direct study or 4.5 years’ distance learning study in the former German Democratic Republic, which is no longer possible since German reunification; pharmaceutical technical assistant with a 2-year school-based education and subsequent 6 months’ practical training in the pharmacy) especially as there are inconsistent results in this regard from published international studies [7] [18].
To close this gap in the research, it was the primary aim of this study to analyse using a pre-post study whether from 2014 to 2017 the quality of advice dispensed by the pharmacies included in the study for acute diarrhoea in adults could actually be increased. The secondary aim as part of the follow-up examination was to analyse whether the quality of advice differed depending on the professional group providing the advice. A comparable study has not been carried out in Germany to date as far as the authors are aware.
2. Methodology
2.1. Design
It is particularly important for the validity of the study results to ensure that there is minimal distortion in the comparison between the baseline study [14] and the follow-up study. Great importance was therefore placed on not changing the study design as far as possible compared to 2014. Consequently, determining the quality of advice provided by the pharmacies investigated was again based on the simulated patient method as a form of participatory observation [19]. This method has been employed extensively in pharmacy practice-based research [17] [20].
2.2. Setting and Participation
The test purchases in the follow-up study were made between the start of May and the end of July 2017 in the city of Neubrandenburg (about 63,000 residents; the state of Mecklenburg-Vorpommern) [21]. The 21 pharmacies that were included in the 2014 study are still located in the metropolitan area of Neubrandenburg [22]. Each pharmacy was again visited once using four test scenarios, giving a total of 84 test purchases (4 scenarios × 1 visit × 21 pharmacies). To carry out the test purchases, a total of €353.01 was required (2014: €375.87) that was provided from the primary author’s own resources.
2.3. Outcomes and Outcome Measurements
To ensure comparability, all four test scenarios from the baseline study were carried out in an absolutely identical way in the follow-up study. The assessment form, which was also identical to that used in 2014, was completed immediately after the test purchases were made by the simulated patients and included 9 objective criteria, fulfilment of which was measured exclusively using dichotomous scales [14]. The test purchases were again covert, that is, they were conducted without informing the particular pharmacy beforehand, analogous to the baseline study and other international studies [17]. The reason for this is that the pharmacies being tested must not be influenced in the quality of the advisory service they provide because this would otherwise create a situation that no longer reflected a true consultation situation (Hawthorne effect).
2.4. Data Collection
Five Masters students from the Faculty of Health, Nursing, Management from the Neubrandenburg University of Applied Sciences acted as test buyers. The selection was again made on the basis of their participation in a 3 semester student research project, the results of which are reflected in this publication. Unlike 2014, instead of 5 female Masters students, 3 female and 2 male Masters students were available as test buyers.
Both the pre-tests and the test purchases were conducted in the same way as for the baseline study [14]. The distribution of the pharmacies being tested to the particular test buyer was also identical to 2014 and once again it was ensured that no pharmacy was visited more than once by a test buyer to minimise the risk of exposure.
The test buyers described their situation to the first pharmacy personnel who spoke to them. In addition to the approach used in 2014, the simulated patients this time attempted to determine during the test purchases the professional group of the pharmacy personnel who advised them by means of the name tag worn, details on the sales receipt or a telephone survey conducted once the study was completed (so as to not endanger the covert study design).
During the telephone calls it was also asked whether the ownership of the particular pharmacy had changed since the baseline study in 2014. This information is important for interpreting the study results because a change of ownership could be associated with a change in the advisory service provided.
Audio recordings are recommended in the literature for quality assurance of the test purchases [23]. For reasons of data privacy, however, this was omitted because otherwise the pharmacies would have to have been informed beforehand about the audio recording and the study design would no longer have been covert. However, the particular assessment forms were again completed by the test buyers immediately after visiting the pharmacies so that distortions in the study results due to faulty recall by the test buyers could be ruled out.
After evaluation of the collected data, each pharmacy again received written customised performance feedback, including graphically presented benchmarking which shows for each pharmacy its improvement or deterioration in regards to the individual variables compared to 2014 and to the average of the Neubrandenburg pharmacies. The pharmacies were thus provided with information about any change in their competitive position. This appears to be particularly important for the pharmacies that made no improvement or had even declined in the quality provided.
2.5. Data Management and Analysis
The statistical software program SPSS 23 was used to record and analyse the data. Continuous variables were expressed as mean ± SD. Because using the Shapiro-Wilk test indicated that the available data are not normally distributed, non-parametric tests (Wilcoxon signed rank test, Kruskal-Wallis test) were used. Results were assumed to be significant when the p < 0.05 threshold was reached by all statistical analyses. The effect strengths were measured using the Pearson correlation coefficient r, whereby according to Cohen, from 0.1 onwards there is a small effect, from 0.3 onwards there is a moderate effect and from 0.5 onwards there is a large effect [24].
2.6. Ethical Statement
This study involves observing pharmacists’ behaviour and does not interfere with patient care. The baseline study in 2014 was approved retrospectively by the local University ethics committee, which was established after the study had commenced. Ethics approval was not requested for the follow-up study because study methods and participating pharmacies are absolute identical to the baseline study. Following the “Guideline for the use of mystery research in market and social research” [25] , the information obtained was recorded so that the pharmacists involved could not be identified and the results were reported anonymously. This ensures that participating pharmacists are not at any risk of criminal or civil liability nor does their participation harm their employability or reputation. Recruited students provided informed consent to act as simulated patients.
3. Results
All planned 84 test purchases were again carried out in the same manner as in the baseline study in 2014. There were significant differences in the mean point number achieved between the first study in 2014 and the follow-up study in 2017 (Wilcoxon signed rank test: z = −2.065, p = 0.039, r = 0.225). In the follow-up study, 2.7 points of 9 possible points (30%) were achieved on average overall whereas 3.3 points (37%) were achieved in the first study in 2014. In 2017 the number of points achieved varied between 1.0 ± 1.2 points (11%) and 4.3 ± 2.4 points on average (48%) while in 2014 the spread varied between 1.0 ± 0.0 points (11%) and 5.0 ± 2.3 points (56%). Compared to the baseline study from 2014 in which 0 points (10%) were achieved in a total of 8 consultation situations across all the scenarios, in the follow-up study the 0-point results had reduced to 5 (6%). Three pharmacies achieved a point value of 0 in both 2014 and 2017 for a test purchase. These included one pharmacy that achieved 0 points in 2 scenarios in 2017. In the comparison between the particular scenarios 1 - 4, although there were differences between 2014 and 2017, they were not significant. Thus, in the first examination for test scenario 1 an average of 2.0 ± 1.3 points (22%) was scored while in the follow-up study an average of 1.7 ± 1.2 points (19%) was achieved for this test scenario (Wilcoxon signed rank test: z = −0.749, p = 0.454, r = 0.163). For test scenario 2 in the baseline study an average of 4.5 ± 1.9 points (50%) was scored whereas in the follow-up study an average of 3.6 ± 1.4 points (40%) was achieved for this test scenario (Wilcoxon signed rank test: z = −1.449, p = 0.147, r = 0.316). While in the baseline study for test scenario 3 an average score of 2.6 ± 2.2 points (29%) was achieved, in the follow-up study an average of 2.1 ± 1.4 points (23%) was achieved for this test scenario (Wilcoxon signed rank test: z = −0.595, p = 0.552, r = 0.129). For test scenario 4 a score of 4.0 ± 2.1 points (44%) was achieved on average in the baseline study while a score of 3.4 ± 1.6 points (38%) was achieved in the follow-up study (Wilcoxon signed rank test: z = −1.071, p = 0.284, r = 0.233) (see Table 1).
At the criteria level, an average of 6.3 points (30%) of a possible 21 points was achieved with very high variation with an average of 0.8 ± 0.5 points (4%) for the question regarding clarification by a doctor ranging to an average of 18.8 ± 1.0 points (90%) for advice about the dosage. In the baseline study from 2014, an average of 7.7 points (37%) was achieved with similar variation, whereby the mean values varied from 0.8 ± 0.8 points (4%) for advice about side effects to 18.3 ± 1.6 points (87%) for advice about the dosage (see Table 2).
Furthermore, it was determined that only 2 pharmacies (pharmacies 1 and 17) were managed by a new owner. For 75 of the 84 test purchases (89%) the professional status of the pharmacy personnel who provided the advice could also be identified (22 pharmacists, 32 pharmaceutical technical assistants and 21 pharmacy technicians). There were no significant differences identified for the advice provided by each of the professional groups (Kruskal-Wallis test: χ2(2) = 1.946; p = 0.378, r = 0.027). While the pharmacists achieved an average of 2.9 ± 1.9 points, the pharmaceutical technical assistants achieved 2.8 ± 1.4 on average and the pharmacy technicians achieved 2.2 ± 1.7 points on average (no figure).
4. Discussion
In this pre-post study carried out for the first time in Germany, it was not possible to achieve an improvement in the pharmacies investigated using feedback, in fact, the results showed that overall there was actually a significant deterioration. In a pre-post evaluation of a quality management project (including intensified training) in Finnish pharmacies, there were also no significant quality improvements observed depending on the scenarios investigated [26]. On the other hand, the international literature reports considerable improvements in some cases using various interventions. It is noteworthy, however, that these studies
![]()
Table 1. Assessment at pharmacy level.
used considerably more intense interventions in some cases than did the current study, including training [27] [28] [29] and supportive supervision [28] , repeated sequential verbal feedback loops [30] , distribution of educational pamphlets to pharmacies [31] and implementation and monitoring of guidelines [32] [33]. Therefore it is possible that very minor interventions―such as those used in this study―do not produce any significant improvements whereas (fundamentally) more comprehensive interventions do. However, for successful interventions, future studies should ask how sustainable the improvements achieved by these means are, that is, what changes are observed in the quality of the advice provided once the interventions cease.
All the pre-post studies presented above have not explicitly investigated the quality of advice provided for acute diarrhoea in adults over time and are therefore only comparable to this study to a limited degree because the quality of advice depends in part on the particular scenario or the particular indication [30].
![]()
Table 2. Assessment at variable level.
A recent Australian pre-post study that explicitly investigated the quality of advice provided for diarrhoea in adults (as 1 of 10 different scenarios) did not report any significant improvement in quality despite repeated feedback loops over time―similar to the present study [30].
Both in the baseline study and the follow-up study, there were considerable differences in the quality of the advice provided depending on the particular criteria investigated. Analogous to studies published in the national [6] [7] and international literature [34] [35] , for example, very little information was provided in both 2014 and 2017 for the criterion “side effects” while in both years advice about the dosage of the medication was frequently given.
Because pharmacy-internal test purchases such as those conducted by the Lower Saxony Board of Pharmacists―analogous to this study―also used feedback as an instrument for quality improvement, the question must be asked in light of the results of this study whether such measures actually have any effect. It should be clear to the pharmacy personnel that providing good quality advice is an important requirement for receiving the pharmacist-only restriction in the OTC sector [36]. This restriction could increasingly be lifted due to poor quality of advice, meaning that in future the government could consider having such preparations dispensed outside pharmacies―as is the case in many other countries [37].
An important prerequisite for providing adequate advice is again an appropriate training. This could, for example, include more rigorous teaching and evaluating of patient and customer consultations using examples to improve qualifications for consultations [38] , especially as to date teaching such “soft skills” in German universities is currently below average compared to Europe as a whole [39].
Regarding the education of the pharmacy personnel, no significant differences could be identified in the advice dispensed. Several other studies came to a similar conclusion but not for Germany and not for the indication acute diarrhoea in adults [18] [34] [40]. In contrast, the only German study to date that also investigated differences in advice dispensed based on professional group revealed significantly better advice from pharmacists compared to other professional groups. However, the advice was not sought for acute diarrhoea in adults but rather headaches [7]. A Turkish study, which specifically analysed the advice provided for acute diarrhoea in adults among others, also determined that pharmacists gave their patients significantly more recommendations than pharmaceutical technical assistants [41]. Due to the inconsistent international and national body of data, future studies should examine this influencing factor on the advice provided in more depth.
5. Limitations
The present study has limitations that must be considered when drawing any conclusions. This includes the fact that the study was conducted in a medium-sized city in Germany only and was restricted to the quality of advice dispensed for acute diarrhoea in adults.
Because the original study (2014) and the follow-up study (2017) were made by different test buyers, who also in part were different sexes, this may have an effect on the results [16]. On the other hand, the identical qualification of the test buyers (Masters students) and exclusive use of objective evaluation criteria should mean that this effect is minimised. For additional quality assurance, future studies could conduct and evaluate test purchases in parallel by two people (1 test buyer and 1 observer) [42].
It must also be pointed out that no control group was used and therefore the comparison results may be distorted by factors that are not related to the performance feedback from 2014 or the possible introduction of corresponding improvement measures by the pharmacies.
It is also plausible that the change in ownership of the pharmacies investigated would lead to distorted results because management of the pharmacies or the corresponding consultation services may no longer necessarily be comparable. However, only 2 pharmacies are managed by new owners, meaning that this aspect is likely to be of minimal relevance for the results of this study.
We were not able to test or determine whether the owner of the pharmacies as the recipients of the written feedback in 2014 forwarded the feedback information to all their staff. This is a key prerequisite for achieving any success with the feedback. There is also no guarantee that the simulated patients in the follow-up study encountered the same pharmacy personnel as in the baseline study, which could also produce distortions in the study results [30].
6. Conclusion
For the first time in Germany it could be shown in a follow-up study that the quality of advice dispensed for acute diarrhoea in adults declined over time despite that performance feedback intended to improve the quality. Interventions with a far greater impact are required to achieve an improvement in the quality of advice provided. The poor quality of advice provided in pharmacies could lead to further discussions about removing certain preparations from the pharmacist-only medications list because the (supposed) consultation in the pharmacy is the central justification for them being pharmacist-only medications. In the end, policy in Germany could follow the lead of other countries and shift the supply of many medications to health and beauty retailers or supermarkets. For example, if pharmacists and their professional organisations want to oppose these discussions, they are encouraged to make considerably greater efforts to improve the quality of advice provided than has been the case to date.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Funding
This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Acknowledgements
The authors thank the anonymous reviewers for their very helpful comments.
Author’s Contributions
Conceived and designed the experiments: BL, MK, SL, JS, SW, AW; Performed the experiments: MK, SL, JS, SW, AW; Analysed the data: BL, MK, SL, JS, SW, AW; Wrote the paper: BL, MK, SL, JS, SW, AW; Responsibility for responding to reviewer comments: BL. All authors state that they had complete access to the study data that support the publication.