The staircase of evidence – a new metaphor displaying the core principles of Evidence-based Veterinary Medicine
1Clinic for Animal Reproduction, Faculty of Veterinary Medicine, Freie Universität Berlin, Königsweg 65, 14163 Berlin, Germany.
*Corresponding Author (firstname.lastname@example.org)
Vol 1, Issue 1 (2016)
Published: 20 Jan 2016
The authors declare that they have no competing interests
Both authors developed the staircase metaphor, conducted the survey, analysed the results and drafted and approved the manuscript.
EBM Evidence-based Medicine
EBVM Evidence-based Veterinary Medicine
RCT Randomized Controlled Trial
The authors gratefully thank the students who took part in the survey.
Background: Veterinary practitioners and clinicians should base their clinical decisions on the best evidence obtainable to provide best patient care. The evidence of information concerning diagnostic or therapeutic procedures and disease prevention can be ranked from weak to strong based on the methodology the information is generated by. Teaching the concepts and strategies of Evidence-based Veterinary Medicine, however, can be challenging. Students and practitioners should become competent in defining a clinical problem, retrieving of information from the literature and independent critical thinking.
Several authors illustrated the levels of evidence ranked from strongest to weakest using a pyramid metaphor. The basis reflects information of the lowest evidence and towards the tip the evidence increases. The metaphorical concept of the pyramid, i.e. solid foundation and a narrow tip, however, does not adequately represent the hierarchy of evidence as the high evidence levels are represented by narrow fields and vice versa
Results: We propose an alternative schematic which may be used in order to display the respective evidence levels as a staircase. A reading pictogram character illustrates that the search for information should start at the highest step. To assess which metaphor can be more easily interpreted and is intuitively accessible a survey among 5th semester veterinary students was performed which was completed by 100 respondents. According to the results both metaphors adequately display the hierarchy of evidence. The staircase better represents the fact that high evidence information is more likely to be true and that one should start to look for high evidence when searching information. In contrast more students found that the pyramid represents distracting information i. e. that opinions and information from the internet provide a solid knowledge base.
Conclusions: The staircase of evidence displays better the evidentiary value of information and how to perform a search for information of high evidence compared to the pyramid of evidence. It therefore can help to teach information search and interpretation strategies in order to answer specific clinical questions.
Evidence based Medicine (EBM) has become broadly accepted as an appropriate basis for decisions around patient care (Ilic et al., 2011; Sargeant et al., 2014b). Hence, courses teaching EBM to students and postgraduates are held at several universities all over the world to various degrees (Miles et al., 2010; Petrisor et al., 2006). Veterinary medicine has also adopted the principles of EBM leading to the field of Evidence-based Veterinary Medicine (EBVM) (Vandeweerd et al., 2012b).
The aim of EBM and EBVM is to base decisions concerning treatments in practice on valid, clinically relevant research data (Evans et al., 2007). Sackett et al. (1996) defined EBM as the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. Hence, practicing EBM requires clinical expertise, and expertise in retrieving, critically appraising, interpreting, and applying the results of scientific studies. In addition, competence in communicating the risks and benefits of different treatments to patients or patient’s owners is necessary. (Greenhalgh et al. 2014) stated that usable evidence must be combined with context and professional expertise to achieve an optimal treatment for individual patients.
The hierarchy of evidence
Information concerning diagnostic or therapeutic procedures and disease prevention can be generated using different methods. These methods vary in regard of the certainty that the information is free from bias (Gugiu et al., 2010). Scientific evidence is the product of appropriately designed and carefully controlled research studies (Sargeant et al., 2014a). Hence, levels of evidence were defined and ranked from strongest to weakest. The levels of evidence that apply to interventions are presented in Table 1.
Slight modifications exist of the description of the level of evidence depending on the organisation or author (Harris et al., 2001); OCEBM Levels of Evidence Working Group, 2011; Petrisor et al., 2006b; Rubin, 2008; Salmond, 2007; Sargeant et al., 2014a).
The hierarchy of evidence is a categorisation of sources with the aim to highlight those that are most likely to avoid a biased outcomes (Cockcroft et al., 2003). Level 1 evidence is obtained from systematic reviews and metaanalyses of randomized controlled clinical trials (RCT) or single RCT with narrow confidence intervals (Gugiu et al., 2010; Holmes et al., 2007). Ideally, evidence should be derived from multiple studies investigating the same research question (Sargeant et al., 2014a). Metaanalyses summarize and combine information, weight and analyse the results of independent studies relating to a specific topic. Hence by increasing the sample size and statistical power the precision of an estimated outcome can be increased (Lean et al., 2009). Level 2 evidence is provided by controlled trials without randomisation and cohort studies. Level 3 evidence results from case reports and case series. Opinions and consensus reports are categorized as level 4 evidence, the weakest category (Bassler et al., 2000; Harris et al., 2001). Within each level of evidence, however, individual information may be evaluated as stronger or weaker after a thorough appraisal (Gugiu et al., 2010). This appraisal should address aspects like enrolment criteria, the allocation in experimental designs, blinding, the sample size, statistical methodologies and the objectiveness in the discussion of the results.
The pyramid of evidence
The hierarchy of evidence has been illustrated by several authors using a pyramid (Figure 1) with varying details (Arlt et al., 2010; Cardwell, 2008; Daly et al., 2007; Holmes, 2007; Kochevar et al., 2006; Sargeant et al., 2014a). The common component of all variations of the pyramid is that the basis reflects information of the lowest evidence. Results of low evidence are generally prone to bias (Petrisor et al., 2006b). Towards the tip of the pyramid the evidence increases. Hence, on the top randomised, controlled trials and metaanalyses (i.e. level 1 evidence) are located (Cockcroft et al., 2003). These study types have lower potential for bias and are more reliable in reproducibly demonstrating causes and effects.
Clinical decision making should be based on the best evidence obtainable (Sackett et al., 1996). Hence, the search for information should start at the top of the pyramid. If no sufficient information of the highest levels is available, we must rely on sources of more doubtful evidence and move towards the basis of the pyramid (Balshem et al., 2011).
The pyramid of evidence may result in disorientation of the reader what was articulated by several students in our courses on EBVM in the last years. One reason is that other pyramid metaphors usually present the best or most important categories at the broad basis. Furthermore, several students had the impression, that expert opinions and consensus reports represent a basis of knowledge supporting a superstructure of RCT and metaanalyses. Considering the historical evolvement of medical knowledge this context might be true for several areas. Nevertheless, decision strategies should avoid unduly considering low evidence levels.
The staircase of evidence
An alternative schematic is the staircase of evidence (Figure 2). The lowest step reflects the lowest level of evidence. With every step upwards, higher evidence is reached and represented at a higher level. The highest step implies the maximum overlap with the truth, although the metaphor indicates, that the truth is never reached to one hundred percent as no step fully overlaps with the truth.
Every step up on the staircase potentially leads to less potential bias and thus to a higher certainty of correctness of the information presented. The fact, that information of different steps can be combined to achieve even higher evidence (Donner-Banzhoff, 2009; Straus et al., 1998) is illustrated as a combination of the overlapping areas of different evidence levels covers a bigger proportion of the truth.
Every evidence level in the staircase has the same width. Hence, the width of the steps does not represent the certainty with which the information is correct. A limitation of the metaphor may be that a reader not familiar with EBM might conclude that he or she has to start at the first step during information search or clinical reasoning. Similar to the pyramid, it could also be misinterpreted that the first steps of the staircase are a basis of the higher levels.
Clinical decisions should be based on information of high evidence. Hence, the search for information should start at the highest step. This is illustrated by the reading pictogram character. Only if usable high level evidence is not available the reader should go down step by step and search for usable evidence at the respective level. This should be done in consciousness that with every step down the overlap with the truth declines.
Aim of this project was to develop the staircase and to assess if it can be more easily interpreted and is more intuitively accessible to students compared to the pyramid of evidence.
A total of 144 students attended the lectures whereas 100 students (69.4%) completed the survey (48 on Wednesday, 52 on Friday). Median (and interquartile range) age of the participating students was 23.0 (21.3 to 25.0) years, 4 respondents were males. In total 9 students stated that they already have heard about EBVM. Interestingly 11 participants stated that they would be able to explain EBVM and its aims. Further results of the survey are presented in Table 2.
According to the students both metaphors represent that quality of information is better when displayed at the top. To the statement referring to the staircase “that the metaphor adequately represents that information is more likely to be true if it is of high evidence” significantly more students agreed compared to the same statement referring to the pyramid metaphor (p<0.01, df = 2). In contrary, more students agreed that the pyramid adequately represents that opinions and information from the internet provide a solid knowledge base (p<0.01, df = 2). The statement when searching information one should start to look for high evidence information was more often supported when referring to the staircase metaphor (p<0.01, df = 2). Finally, students agreed more often to the statement more information of low evidence is available* referring to the pyramid (p<0.01, df = 2).
Gender and age of the respondents or the statements that they had heard already about EBVM or could explain the aims of EBVM had no influences on the outcomes.
The presentation order of the metaphors had no influence on the outcomes except on the statement that the staircase represents that opinions and internet information provide a solid knowledge base (p<0.01, df = 2). In the Wednesday group 25 students (52.1%) disagreed with this statement whereas in the Friday group 42 (80.0%) disagreed.
Some students (7.0%) provided comments within the free text comment section. Three students commented that the pyramid provides little or no information about the validity of the respective levels. Two students commented that the staircase better represents that opinions and information from the internet are less likely to be true. Two other students suggested that the pictogram character should walk upstairs.
Veterinary practitioners have to use optimal diagnostics, interventions and medications to examine and to treat their patients (Arlt et al., 2014). Additionally, an inherent responsibility of veterinary medicine is the protection of human health via caring for livestock and its reproduction. Major aspects are to ensure sufficient food production, to prevent infectious animal diseases including zoonoses and food borne diseases, and high quality and safety of foods and other products derived from animals.
In that regard, EBVM prompts the search for and utilization of information that are valid. Evidence levels are indicators of the strength of study designs to be not prone to bias (Cockcroft et al., 2003). The hierarchy of evidence aims not to completely disregard data from low evidence sources but alerts the clinician to the critical appraisal of the available information (Lohr, 2004).
Veterinary professional education has historically relied more on establishing solid foundations in physiology and pathophysiology and providing venues for experiential learning than it has on rigorous analytic training (Arlt et al., 2014). To be able to practice EBVM students and practitioners should become competent in defining a clinical problem, retrieving of information from the literature and independent critical thinking (Steele et al., 2013).
The body of published evidence on how to teach EBVM is small. Some papers of good quality are available but further research is required to determine the effects of timing, content and length of EBM and EBVM courses and teaching methods (Ilic et al., 2014).
Our survey reveals a dearth of EBVM knowledge of 5th semester veterinary students because only 9% stated that they had heard about EBVM. In addition, 2 participants seem to have engaged in Evidence based Medicine resulting in 11 people that stated to be able to explain EBVM and its aims. Due to the fact that only 5th semester veterinary students of the Freie Universität Berlin were included, one might question the representativeness of our survey. By choosing only 5th semester students we tried to ensure that most students did not know EBVM and the pyramid metaphor in advance to avoid bias based on previous knowledge.
As stated above, the graphic representation of a pyramid is well known and supposed to be intuitively to comprehend for the reader. Its concept consists of a solid foundation and a sharp tip.
One of the probably most popular pyramid schematics is the food guide pyramid (Achterberg et al., 1994). It was introduced in 1992 and displays how a diet for humans should be designed (Davis et al., 2001). The basis of nutrition should be bread, cereals, rice and pasta, illustrated by the foundation of the pyramid. Above this, vegetables and fruits are pictured, followed by milk-products, meat, eggs and nuts, meaning that these components are important but should be less consumed than items of the bread group. At the top a group containing fat, oils and sweets is represented. These products should be consumed sparingly. According to Achterberg et al. (1994) the pyramid also illustrates that the consumer can choose foods from a variety within a group but should rely on the illustrated proportions.
Another example is the learning pyramid (Bale, 2006) presenting the success of learning obtained via different techniques and media. The lowest retention rate is obtained by visiting lectures, presented at the top of the pyramid. The most successful methods are presented at the basis. These examples show that the metaphorical concept of the pyramid is usually used presenting the best or most important categories at the broad basis.
According to our results both metaphors adequately display the hierarchy of evidence, i. e. that the quality of information is better when displayed at the top. The staircase better represents the fact that high evidence information is more likely to be true and that one should start to look for high evidence when searching information. These core principles are of high relevance in the context of teaching the principles of EBVM.
In contrary, we presented statements indicating a disorientation of the reader. More students found that the pyramid represents that opinions and information from the internet provide a solid knowledge base. Even if the internet and opinions of colleagues play important roles in the search for veterinary information (Haimerl et al., 2013) they provide weak evidence as stated above. The foundation of a building represents a sound and solid basis for the superstructure. In regard to evidence levels one may argue that high evidence is indeed placed on a basis of less good evidence since RCTs are often based on findings of uncontrolled studies or case reports. Nevertheless, the pyramid of evidence may lead to an overestimation of the scientific value of publications of low evidence. In fact, this kind of information should be integrated in systematic reviews and critically appraised topics (Vandeweerd et al., 2012a). However, as soon as high quality research results from randomised, controlled clinical trials are available, not too much importance should be attached to data of low evidence. Integration of low quality results, however, may help to enhance the number of probands, to judge practical applicability of interventions or to identify sources of heterogeneity between different results (Lean et al., 2009).
An option would be to display the pyramid upside down, so that level 1 evidence is illustrated as the broadest. However, this may misleadingly be interpreted that a high number of level 1 evidence resources is available.
Finally, more students found that the pyramid better represents that more information of low evidence is available. Many authors who evaluated literature in human (Lohr, 2004) and veterinary medicine (Arlt et al., 2010; Simoneit et al., 2011) demonstrated that in the majority of topics numerous case reports, several clinical trials with or without a control group and only few metaanalyses could be found. This means, that in progressing from the bottom towards the top of the pyramid, the number of information available tends to decrease (Sargeant et al., 2014a). This frequency distribution is to a certain extend represented by the pyramid. Nevertheless, it can be assumed that the percentage of high quality publications will increase during the next years. In addition, this characteristic might distract the reader from the actual aims of consideration of the hierarchy of evidence.
Based on our findings, we suggest using the staircase metaphor in teaching contexts. In addition, using animated slides in a presentation the reading pictogram character could go down the staircase step by step while the lecturer explains the characteristics of every evidence level. Two students suggested that the character should walk upstairs. However, when we discussed this issue with 4th year students attending the clinical rotation (n = 17) all were of the opinion that the literature search process is better represented by a character walking downstairs.
When teaching evidence levels additional limitations should be taken into account and be discussed with the students.
A special issue is the opinion, usually categorized as evidence level four. Opinions, results from consensus meetings or information based on pathophysiological principles that are not supported by information of higher evidence are in general prone to bias. If an opinion or a consensus report is based on valid research data it may be regarded belonging to a higher evidence level. This categorisation does not want to completely disregard the advice of experts with a high expertise in their field. Nevertheless, also recommendations of experts may be outdated or incomplete (Antman et al., 1992). Hence, it should always be questioned what the expert´s advice is based on and if he or she is aware of possible limitations of the underlying information.
Furthermore, important issues related to the rating of evidence have been discussed controversially (Balshem et al., 2011). There is overlap between the different levels in the hierarchy since well-designed cohort studies may provide better evidence than poorly designed RCT (Cockcroft et al., 2003). It is challenging for example to evaluate the strength of evidence based on flawed RCT compared to well-conducted non–RCT (Gugiu et al., 2010). Science-based information regarding the repeatability of assessments of evidence levels is scarce (Simoneit et al., 2012). Therefore, it has to be emphasized again that all quality aspects of every individual study or information have to be appraised (Arlt et al., 2005; Schmidt, 2007).
On Wednesday 10th and Friday 12th of December 2014, we conducted a survey among veterinary medical students in their 5th semester (3rd year) attending a lecture about clinical propaedeutics in animal reproduction. At the beginning of the lecture held by a lecturer of the Clinic for Animal Reproduction, students were asked to voluntarily and anonymously participate in a survey (convenience sample). Completion of the survey indicated informed consent and took approximately 10 minutes. The survey contained questions concerning age, semester, and gender and existing knowledge about EBVM. In a short text limited information was provided to enable understanding of the context of the survey but avoiding bias caused by specific information:
“Veterinary practitioners have to use optimal diagnostics, interventions and medications to examine and to treat their patients. Therefore, it is important to base clinical decisions on recent and valid information. High evidence in this context means that information e. g. on the outcomes of a certain therapy can be generalized to other cases. When searching for information sources of high evidence should always be preferred compared to low level evidence information.”
On the rear page, the pyramid and the staircase metaphors were presented. To avoid a bias based on the order of the metaphors the Wednesday group found the pyramid metaphor on the left and the Friday group on the right, respectively. Below the figures statements were presented as pairs referring to the respective metaphor above. Participants responded to the statements using a three-point Likert scale (see below). A free text comment section was provided at the end of the survey.
The descriptive statistical analysis was performed using the statistical program SPSS for Windows (Version 22.0; SPSS Inc., Munich, Germany). We hypothesized that the responses to the specific statements did not differ. The data were considered to be quantitative. The data were analyzed using the Qi square test, expected frequencies and standardized residuals. The significance level was set as α = 0.05. The free text comment section was considered qualitative data.
Several authors have pointed out the need to train students and physicians in Evidence-based Medicine (Arlt et al., 2014; Hardin et al., 2006; Nault et al., 2011; Weiner et al., 2011). This need was confirmed by our survey among 5th semester students. Especially for introducing the concepts and levels of evidence an easily interpreted and intuitively accessible schematic would be helpful. The staircase of evidence better represents the evidentiary value of information and displays better how to perform a search for information of high evidence compared to the pyramid of evidence.
All the supplementary files are available here.