The staircase of evidence – a new metaphor displaying the core principles of Evidence-based Veterinary Medicine

Sebastian Arlt^1*

Wolfgang Heuwieser¹

¹Clinic for Animal Reproduction, Faculty of Veterinary Medicine, Freie Universität Berlin, Königsweg 65, 14163 Berlin, Germany.
^*Corresponding Author (sebastian.arlt@fu-berlin.de)

Background: Veterinary practitioners and clinicians should base their clinical decisions on the best evidence obtainable to provide best patient care. The evidence of information concerning diagnostic or therapeutic procedures and disease prevention can be ranked from weak to strong based on the methodology the information is generated by. Teaching the concepts and strategies of Evidence-based Veterinary Medicine, however, can be challenging. Students and practitioners should become competent in defining a clinical problem, retrieving of information from the literature and independent critical thinking.

Several authors illustrated the levels of evidence ranked from strongest to weakest using a pyramid metaphor. The basis reflects information of the lowest evidence and towards the tip the evidence increases. The metaphorical concept of the pyramid, i.e. solid foundation and a narrow tip, however, does not adequately represent the hierarchy of evidence as the high evidence levels are represented by narrow fields and vice versa

Results: We propose an alternative schematic which may be used in order to display the respective evidence levels as a staircase. A reading pictogram character illustrates that the search for information should start at the highest step. To assess which metaphor can be more easily interpreted and is intuitively accessible a survey among 5^th semester veterinary students was performed which was completed by 100 respondents. According to the results both metaphors adequately display the hierarchy of evidence. The staircase better represents the fact that high evidence information is more likely to be true and that one should start to look for high evidence when searching information. In contrast more students found that the pyramid represents distracting information i. e. that opinions and information from the internet provide a solid knowledge base.

Conclusions: The staircase of evidence displays better the evidentiary value of information and how to perform a search for information of high evidence compared to the pyramid of evidence. It therefore can help to teach information search and interpretation strategies in order to answer specific clinical questions.

Background

Evidence based Medicine (EBM) has become broadly accepted as an appropriate basis for decisions around patient care (Ilic et al., 2011; Sargeant et al., 2014b). Hence, courses teaching EBM to students and postgraduates are held at several universities all over the world to various degrees (Miles et al., 2010; Petrisor et al., 2006). Veterinary medicine has also adopted the principles of EBM leading to the field of Evidence-based Veterinary Medicine (EBVM) (Vandeweerd et al., 2012b).

The aim of EBM and EBVM is to base decisions concerning treatments in practice on valid, clinically relevant research data (Evans et al., 2007). Sackett et al. (1996) defined EBM as the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. Hence, practicing EBM requires clinical expertise, and expertise in retrieving, critically appraising, interpreting, and applying the results of scientific studies. In addition, competence in communicating the risks and benefits of different treatments to patients or patient’s owners is necessary. (Greenhalgh et al. 2014) stated that usable evidence must be combined with context and professional expertise to achieve an optimal treatment for individual patients.

The hierarchy of evidence

Information concerning diagnostic or therapeutic procedures and disease prevention can be generated using different methods. These methods vary in regard of the certainty that the information is free from bias (Gugiu et al., 2010). Scientific evidence is the product of appropriately designed and carefully controlled research studies (Sargeant et al., 2014a). Hence, levels of evidence were defined and ranked from strongest to weakest. The levels of evidence that apply to interventions are presented in Table 1.

Slight modifications exist of the description of the level of evidence depending on the organisation or author (Harris et al., 2001); OCEBM Levels of Evidence Working Group, 2011; Petrisor et al., 2006b; Rubin, 2008; Salmond, 2007; Sargeant et al., 2014a).

The hierarchy of evidence is a categorisation of sources with the aim to highlight those that are most likely to avoid a biased outcomes (Cockcroft et al., 2003). Level 1 evidence is obtained from systematic reviews and metaanalyses of randomized controlled clinical trials (RCT) or single RCT with narrow confidence intervals (Gugiu et al., 2010; Holmes et al., 2007). Ideally, evidence should be derived from multiple studies investigating the same research question (Sargeant et al., 2014a). Metaanalyses summarize and combine information, weight and analyse the results of independent studies relating to a specific topic. Hence by increasing the sample size and statistical power the precision of an estimated outcome can be increased (Lean et al., 2009). Level 2 evidence is provided by controlled trials without randomisation and cohort studies. Level 3 evidence results from case reports and case series. Opinions and consensus reports are categorized as level 4 evidence, the weakest category (Bassler et al., 2000; Harris et al., 2001). Within each level of evidence, however, individual information may be evaluated as stronger or weaker after a thorough appraisal (Gugiu et al., 2010). This appraisal should address aspects like enrolment criteria, the allocation in experimental designs, blinding, the sample size, statistical methodologies and the objectiveness in the discussion of the results.

The pyramid of evidence

The hierarchy of evidence has been illustrated by several authors using a pyramid (Figure 1) with varying details (Arlt et al., 2010; Cardwell, 2008; Daly et al., 2007; Holmes, 2007; Kochevar et al., 2006; Sargeant et al., 2014a). The common component of all variations of the pyramid is that the basis reflects information of the lowest evidence. Results of low evidence are generally prone to bias (Petrisor et al., 2006b). Towards the tip of the pyramid the evidence increases. Hence, on the top randomised, controlled trials and metaanalyses (i.e. level 1 evidence) are located (Cockcroft et al., 2003). These study types have lower potential for bias and are more reliable in reproducibly demonstrating causes and effects.

Clinical decision making should be based on the best evidence obtainable (Sackett et al., 1996). Hence, the search for information should start at the top of the pyramid. If no sufficient information of the highest levels is available, we must rely on sources of more doubtful evidence and move towards the basis of the pyramid (Balshem et al., 2011).

The pyramid of evidence may result in disorientation of the reader what was articulated by several students in our courses on EBVM in the last years. One reason is that other pyramid metaphors usually present the best or most important categories at the broad basis. Furthermore, several students had the impression, that expert opinions and consensus reports represent a basis of knowledge supporting a superstructure of RCT and metaanalyses. Considering the historical evolvement of medical knowledge this context might be true for several areas. Nevertheless, decision strategies should avoid unduly considering low evidence levels.

The staircase of evidence

An alternative schematic is the staircase of evidence (Figure 2). The lowest step reflects the lowest level of evidence. With every step upwards, higher evidence is reached and represented at a higher level. The highest step implies the maximum overlap with the truth, although the metaphor indicates, that the truth is never reached to one hundred percent as no step fully overlaps with the truth.

Every step up on the staircase potentially leads to less potential bias and thus to a higher certainty of correctness of the information presented. The fact, that information of different steps can be combined to achieve even higher evidence (Donner-Banzhoff, 2009; Straus et al., 1998) is illustrated as a combination of the overlapping areas of different evidence levels covers a bigger proportion of the truth.

Every evidence level in the staircase has the same width. Hence, the width of the steps does not represent the certainty with which the information is correct. A limitation of the metaphor may be that a reader not familiar with EBM might conclude that he or she has to start at the first step during information search or clinical reasoning. Similar to the pyramid, it could also be misinterpreted that the first steps of the staircase are a basis of the higher levels.

Clinical decisions should be based on information of high evidence. Hence, the search for information should start at the highest step. This is illustrated by the reading pictogram character. Only if usable high level evidence is not available the reader should go down step by step and search for usable evidence at the respective level. This should be done in consciousness that with every step down the overlap with the truth declines.

Aim of this project was to develop the staircase and to assess if it can be more easily interpreted and is more intuitively accessible to students compared to the pyramid of evidence.

A total of 144 students attended the lectures whereas 100 students (69.4%) completed the survey (48 on Wednesday, 52 on Friday). Median (and interquartile range) age of the participating students was 23.0 (21.3 to 25.0) years, 4 respondents were males. In total 9 students stated that they already have heard about EBVM. Interestingly 11 participants stated that they would be able to explain EBVM and its aims. Further results of the survey are presented in Table 2.

According to the students both metaphors represent that quality of information is better when displayed at the top. To the statement referring to the staircase “that the metaphor adequately represents that information is more likely to be true if it is of high evidence” significantly more students agreed compared to the same statement referring to the pyramid metaphor (p<0.01, df = 2). In contrary, more students agreed that the pyramid adequately represents that opinions and information from the internet provide a solid knowledge base (p<0.01, df = 2). The statement when searching information one should start to look for high evidence information was more often supported when referring to the staircase metaphor (p<0.01, df = 2). Finally, students agreed more often to the statement more information of low evidence is available* referring to the pyramid (p<0.01, df = 2).

Gender and age of the respondents or the statements that they had heard already about EBVM or could explain the aims of EBVM had no influences on the outcomes.

The presentation order of the metaphors had no influence on the outcomes except on the statement that the staircase represents that opinions and internet information provide a solid knowledge base (p<0.01, df = 2). In the Wednesday group 25 students (52.1%) disagreed with this statement whereas in the Friday group 42 (80.0%) disagreed.

Some students (7.0%) provided comments within the free text comment section. Three students commented that the pyramid provides little or no information about the validity of the respective levels. Two students commented that the staircase better represents that opinions and information from the internet are less likely to be true. Two other students suggested that the pictogram character should walk upstairs.

Veterinary practitioners have to use optimal diagnostics, interventions and medications to examine and to treat their patients (Arlt et al., 2014). Additionally, an inherent responsibility of veterinary medicine is the protection of human health via caring for livestock and its reproduction. Major aspects are to ensure sufficient food production, to prevent infectious animal diseases including zoonoses and food borne diseases, and high quality and safety of foods and other products derived from animals.

In that regard, EBVM prompts the search for and utilization of information that are valid. Evidence levels are indicators of the strength of study designs to be not prone to bias (Cockcroft et al., 2003). The hierarchy of evidence aims not to completely disregard data from low evidence sources but alerts the clinician to the critical appraisal of the available information (Lohr, 2004).

Veterinary professional education has historically relied more on establishing solid foundations in physiology and pathophysiology and providing venues for experiential learning than it has on rigorous analytic training (Arlt et al., 2014). To be able to practice EBVM students and practitioners should become competent in defining a clinical problem, retrieving of information from the literature and independent critical thinking (Steele et al., 2013).

The body of published evidence on how to teach EBVM is small. Some papers of good quality are available but further research is required to determine the effects of timing, content and length of EBM and EBVM courses and teaching methods (Ilic et al., 2014).

Our survey reveals a dearth of EBVM knowledge of 5^th semester veterinary students because only 9% stated that they had heard about EBVM. In addition, 2 participants seem to have engaged in Evidence based Medicine resulting in 11 people that stated to be able to explain EBVM and its aims. Due to the fact that only 5^th semester veterinary students of the Freie Universität Berlin were included, one might question the representativeness of our survey. By choosing only 5^th semester students we tried to ensure that most students did not know EBVM and the pyramid metaphor in advance to avoid bias based on previous knowledge.

As stated above, the graphic representation of a pyramid is well known and supposed to be intuitively to comprehend for the reader. Its concept consists of a solid foundation and a sharp tip.

One of the probably most popular pyramid schematics is the food guide pyramid (Achterberg et al., 1994). It was introduced in 1992 and displays how a diet for humans should be designed (Davis et al., 2001). The basis of nutrition should be bread, cereals, rice and pasta, illustrated by the foundation of the pyramid. Above this, vegetables and fruits are pictured, followed by milk-products, meat, eggs and nuts, meaning that these components are important but should be less consumed than items of the bread group. At the top a group containing fat, oils and sweets is represented. These products should be consumed sparingly. According to Achterberg et al. (1994) the pyramid also illustrates that the consumer can choose foods from a variety within a group but should rely on the illustrated proportions.

Another example is the learning pyramid (Bale, 2006) presenting the success of learning obtained via different techniques and media. The lowest retention rate is obtained by visiting lectures, presented at the top of the pyramid. The most successful methods are presented at the basis. These examples show that the metaphorical concept of the pyramid is usually used presenting the best or most important categories at the broad basis.

According to our results both metaphors adequately display the hierarchy of evidence, i. e. that the quality of information is better when displayed at the top. The staircase better represents the fact that high evidence information is more likely to be true and that one should start to look for high evidence when searching information. These core principles are of high relevance in the context of teaching the principles of EBVM.

In contrary, we presented statements indicating a disorientation of the reader. More students found that the pyramid represents that opinions and information from the internet provide a solid knowledge base. Even if the internet and opinions of colleagues play important roles in the search for veterinary information (Haimerl et al., 2013) they provide weak evidence as stated above. The foundation of a building represents a sound and solid basis for the superstructure. In regard to evidence levels one may argue that high evidence is indeed placed on a basis of less good evidence since RCTs are often based on findings of uncontrolled studies or case reports. Nevertheless, the pyramid of evidence may lead to an overestimation of the scientific value of publications of low evidence. In fact, this kind of information should be integrated in systematic reviews and critically appraised topics (Vandeweerd et al., 2012a). However, as soon as high quality research results from randomised, controlled clinical trials are available, not too much importance should be attached to data of low evidence. Integration of low quality results, however, may help to enhance the number of probands, to judge practical applicability of interventions or to identify sources of heterogeneity between different results (Lean et al., 2009).

An option would be to display the pyramid upside down, so that level 1 evidence is illustrated as the broadest. However, this may misleadingly be interpreted that a high number of level 1 evidence resources is available.

Finally, more students found that the pyramid better represents that more information of low evidence is available. Many authors who evaluated literature in human (Lohr, 2004) and veterinary medicine (Arlt et al., 2010; Simoneit et al., 2011) demonstrated that in the majority of topics numerous case reports, several clinical trials with or without a control group and only few metaanalyses could be found. This means, that in progressing from the bottom towards the top of the pyramid, the number of information available tends to decrease (Sargeant et al., 2014a). This frequency distribution is to a certain extend represented by the pyramid. Nevertheless, it can be assumed that the percentage of high quality publications will increase during the next years. In addition, this characteristic might distract the reader from the actual aims of consideration of the hierarchy of evidence.

Based on our findings, we suggest using the staircase metaphor in teaching contexts. In addition, using animated slides in a presentation the reading pictogram character could go down the staircase step by step while the lecturer explains the characteristics of every evidence level. Two students suggested that the character should walk upstairs. However, when we discussed this issue with 4^th year students attending the clinical rotation (n = 17) all were of the opinion that the literature search process is better represented by a character walking downstairs.

When teaching evidence levels additional limitations should be taken into account and be discussed with the students.

A special issue is the opinion, usually categorized as evidence level four. Opinions, results from consensus meetings or information based on pathophysiological principles that are not supported by information of higher evidence are in general prone to bias. If an opinion or a consensus report is based on valid research data it may be regarded belonging to a higher evidence level. This categorisation does not want to completely disregard the advice of experts with a high expertise in their field. Nevertheless, also recommendations of experts may be outdated or incomplete (Antman et al., 1992). Hence, it should always be questioned what the expert´s advice is based on and if he or she is aware of possible limitations of the underlying information.

Furthermore, important issues related to the rating of evidence have been discussed controversially (Balshem et al., 2011). There is overlap between the different levels in the hierarchy since well-designed cohort studies may provide better evidence than poorly designed RCT (Cockcroft et al., 2003). It is challenging for example to evaluate the strength of evidence based on flawed RCT compared to well-conducted non–RCT (Gugiu et al., 2010). Science-based information regarding the repeatability of assessments of evidence levels is scarce (Simoneit et al., 2012). Therefore, it has to be emphasized again that all quality aspects of every individual study or information have to be appraised (Arlt et al., 2005; Schmidt, 2007).

On Wednesday 10^th and Friday 12^th of December 2014, we conducted a survey among veterinary medical students in their 5th semester (3^rd year) attending a lecture about clinical propaedeutics in animal reproduction. At the beginning of the lecture held by a lecturer of the Clinic for Animal Reproduction, students were asked to voluntarily and anonymously participate in a survey (convenience sample). Completion of the survey indicated informed consent and took approximately 10 minutes. The survey contained questions concerning age, semester, and gender and existing knowledge about EBVM. In a short text limited information was provided to enable understanding of the context of the survey but avoiding bias caused by specific information:

“Veterinary practitioners have to use optimal diagnostics, interventions and medications to examine and to treat their patients. Therefore, it is important to base clinical decisions on recent and valid information. High evidence in this context means that information e. g. on the outcomes of a certain therapy can be generalized to other cases. When searching for information sources of high evidence should always be preferred compared to low level evidence information.”

On the rear page, the pyramid and the staircase metaphors were presented. To avoid a bias based on the order of the metaphors the Wednesday group found the pyramid metaphor on the left and the Friday group on the right, respectively. Below the figures statements were presented as pairs referring to the respective metaphor above. Participants responded to the statements using a three-point Likert scale (see below). A free text comment section was provided at the end of the survey.

The descriptive statistical analysis was performed using the statistical program SPSS for Windows (Version 22.0; SPSS Inc., Munich, Germany). We hypothesized that the responses to the specific statements did not differ. The data were considered to be quantitative. The data were analyzed using the Qi square test, expected frequencies and standardized residuals. The significance level was set as α = 0.05. The free text comment section was considered qualitative data.

Several authors have pointed out the need to train students and physicians in Evidence-based Medicine (Arlt et al., 2014; Hardin et al., 2006; Nault et al., 2011; Weiner et al., 2011). This need was confirmed by our survey among 5^th semester students. Especially for introducing the concepts and levels of evidence an easily interpreted and intuitively accessible schematic would be helpful. The staircase of evidence better represents the evidentiary value of information and displays better how to perform a search for information of high evidence compared to the pyramid of evidence.

1. Achterberg, C.; McDonnell, E. And  Bagby, R. (1994): How to put the food guide pyramid into practice. Journal of the American Dietetic  Association  94(9) pp 1030–1035. http://dx.doi.org/10.1016/0002-8223(94)92198-9
2. Antman, E. M. et al. (1992): A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. Journal of the American  Medical Association 268(2). pp  240-248   http://dx.doi.org/10.1001/jama.1992.03490020088036
3. Arlt, S.; Dicty, V. and  Heuwieser, W. (2010): Evidence-Based Medicine in Canine Reproduction: Quality of Current Available Literature. Reproduction in Domestic Animals  45(6). pp 1052-1058 http://dx.doi.org/10.1111/j.1439-0531.2009.01492.x
4. Arlt, S. and  Heuwieser, W. (2005): Evidenz-Basierte Veterinärmedizin. Deutsche Tieraerztliche Wochenschrift 112(4) pp 146-148. 
5. Arlt, S. P. and Heuwieser, W. (2014): Evidence-based Medicine in Animal Reproduction. Reproduction in Domestic Animals  49(s3) pp 11-15. http://dx.doi.org/10.1111/rda.12323
6. Bale, V. (2006): How my practice of using manipulatives in teaching multiplying and dividing fractions influences the student’s conceptual understanding of these operations. B.S. University of Central Florida 
7. Balshem, H. et al. (2011): GRADE guidelines: 3. Rating the quality of evidence. Journal of Clinical Epidemiology 64(4) pp 401-406. http://dx.doi.org/10.1016/j.jclinepi.2010.07.015
8. Bassler, D.and  Antes, G. (2000): Wie erhalte ich Antworten auf meine Fragen. In: Kunz, R et al Evidenzbasierte Medizin in Klinik und Praxis. Köln: Dt. Ärzteverlag  pp 89-97. 
9. Cardwell, J. M. (2008): An overview of study design. Journal of  Small Animal Practice 49(5) pp 217-218 http://dx.doi.org/10.1111/j.1748-5827.2008.00594.x
10. Cockcroft, P.and  Holmes, M. (2003): Research Studies. In: Cockcroft, P.and  Holmes, M Handbook of Evidence-Based Veterinary Medicine. Oxford:  Blackwell Publishing  pp 67-83 http://dx.doi.org/10.1002/9780470690833.ch5
11. Daly, J. et al. (2007): A hierarchy of evidence for assessing qualitative health research. Journal of Clinical Epidemiology  60(1)  pp 43-49 http://dx.doi.org/10.1016/j.jclinepi.2006.03.014
12. Davis, C. A. et al (2001):Past, present, and future of the Food Guide Pyramid. Journal of the American Dietetic  Association  101(8) pp 881–885 http://dx.doi.org/10.1016/S0002-8223(01)00217-6
13. Donner-Banzhoff, N. (2009):Pragmatic trials in a routine care setting. Zeitschrift fur Evidenz, Fortbildung und  Qualitat im  Gesundheitswesen 103(6) pp 404-409  http://dx.doi.org/10.1016/j.zefq.2009.05.022
14. Evans, R.; and O’Connor, A. (2007): Statistics and Evidence-Based Veterinary Medicine: answers to 21 common statistical questions that arise from reading scientific manuscripts. Veterinary Clinics of  North America: Small Animal Practice 37(3).pp 477-486 http://dx.doi.org/10.1016/j.cvsm.2007.01.006
15. Greenhalgh, T.; Howick, J.and Maskrey, N. (2014):Evidence based medicine: a movement in crisis? British Medical Journal 348.g3725  http://dx.doi.org/10.1136/bmj.g3725
16. Gugiu, P. C.;and Gugiu, M. R. (2010) A Critical Appraisal of Standard Guidelines for Grading Levels of Evidence. Evaluation and the Health Profssions 33(3) pp  233-255 http://dx.doi.org/10.1177/0163278710373980
17. Haimerl, P.; Arlt, S. P. and  Heuwieser, W. (2013):Entscheidungsfindung in der tierärztlichen Praxis. Tierarztliche Praxis 4(4) pp 229-236. 
18. Hardin, L. E.; and Robertson, S. (2006): Learning Evidence-Based Veterinary Medicine through Development of a Critically Appraised Topic. Journal of Veterinary Medical Education 33(3).pp 474-478 http://dx.doi.org/10.3138/jvme.33.3.474
19. Harris, R. P.; et al. (2001): Current methods of the U.S. Preventive Services Task Force: A review of the process. American Journal of Preventive Medicine 20(3S1) pp 21–35 http://dx.doi.org/10.1016/S0749-3797(01)00261-6
20. Holmes, M. and Ramey, D. (2007):An Introduction to Evidence-Based Veterinary Medicine. Veterinary Clinics of North America: Equine Practice 23(2) pp 191-200. http://dx.doi.org/10.1016/j.cveq.2007.03.001
21. Holmes, M. A. (2007):Evaluation of the evidence. Veterinary Clinics of  North America: Small Animal Practice 37(3). pp 447-62   http://dx.doi.org/10.1016/j.cvsm.2007.01.004
22. Ilic, D, and Maloney, S. (2014):Methods of teaching medical trainees evidence-based medicine: a systematic review. Medical Education 48(2) pp 124-135. http://dx.doi.org/10.1111/medu.12288
23. Ilic, D.; Tepper, K.; and Misso, M. (2011):Teaching evidence based medicine literature searching skills to medical students during the clinical years - a protocol for a randomised controlled trial. BMC Medical Education 11:49 http://dx.doi.org/10.1186/1472-6920-11-49
24. Kochevar, D.and  Fajt, V. (2006): Evidence-based decision making in small animal therapeutics. Veterinary Clinics of  North America: Small Animal Practice. 36(5) pp 943-959. http://dx.doi.org/10.1016/j.cvsm.2006.06.001
25. Lean, I. J.; et al  (2009):Invited review: Use of meta-analysis in animal health and reproduction: methods and applications. Journal of  Dairy Science  92(8) pp 3545-3565 http://dx.doi.org/10.3168/jds.2009-2140
26. Lohr, K. N. (2004): Rating the strength of scientific evidence: relevance for quality improvement programs. International Journal for Quality in Health Care 16(1) pp 9-18 http://dx.doi.org/10.1093/intqhc/mzh005
27. Miles, S. et al. (2010): Statistics teaching in medical school: opinions of practising doctors. BMC Medical Education 10:75  http://dx.doi.org/10.1186/1472-6920-10-75
28. Nault, A. J.and Baker, H. J. (2011):The power of information. Journal of Veterinary Medical Education 38(1). pp 3-4. http://dx.doi.org/10.3138/jvme.38.1.3
29. Oxford Levels of Evidence 2 [Oxford Centre for Evidence-Based Medicine – OCEBM Levels of Evidence Working Group] [online] Available from: http://www.cebm.net/index.aspx?o=5653 [Accessed 05/01/2016].
30. Petrisor, B. A. and Bhandari, M. (2006a): Principles of teaching evidence-based medicine. Injury 37(4) pp 335-339 http://dx.doi.org/10.1016/j.injury.2006.01.027
31. Petrisor, B. A.; Keating, J. and Schemitsch, E. H. (2006b): Grading the evidence: Levels of evidence and grades of recommendation. Injury 37(4). pp  321-327 http://dx.doi.org/10.1016/j.injury.2006.02.001
32. Rubin, A. (2008):Practitioner's guide to using research for evidence-based practice. Hoboken, NJ: John Wiley & Sons.
33. Sackett, D. L.; et al. (1996): Evidence based medicine: what it is and what it isn't. British Medical Journal 312(71) pp  71-72 http://dx.doi.org/10.1136/bmj.312.7023.71
34. Salmond, S. W. (2007): Advancing evidence-based practice: A primer. Orthopaedic Nursing 26(2) pp114–123  http://dx.doi.org/10.1097/01.NOR.0000265869.72265.0a
35. Sargeant, J. M.; Kelton, D. F. and  O'Connor, A. M. (2014a): Study Designs and Systematic Reviews of Interventions: Building Evidence Across Study Designs. Zoonoses and Public Health 61(S1). pp 10-17 http://dx.doi.org/10.1111/zph.12127
36. Sargeant, J. M.and  O'Connor, A. M. (2014b):Introduction to Systematic Reviews in Animal Agriculture and Veterinary Medicine. Zoonoses and Public Health 61(S1). pp3-9 http://dx.doi.org/10.1111/zph.12128
37. Schmidt, P. (2007):Evidence-based veterinary medicine: evolution, revolution, or repackaging of veterinary practice? Veterinary Clinics of  North America: Small Animal Practice 37(3). pp 409-417 http://dx.doi.org/10.1016/j.cvsm.2007.01.001
38. Simoneit, C.; Heuwieser, W.and Arlt, S. P. (2011):Evidence-based medicine in bovine, equine and canine reproduction:Quality of current literature. Theriogenology 76(6). pp 1042-1050 http://dx.doi.org/10.1016/j.theriogenology.2011.05.007
39. Simoneit, C.; Heuwieser, W.; and Arlt, S. P. (2012): Inter-observer Agreement on a Checklist to Evaluate Scientific Publications in the Field of Animal Reproduction. Journal of Veterinary Medical Education 39(2) pp 119-127 http://dx.doi.org/10.3138/jvme.1111.113R
40. Steele, M. et al. (2013): Online tools for teaching evidence-based veterinary medicine. Journal of Veterinary Medical Education 40(3). pp 272-277 http://dx.doi.org/10.3138/jvme.0113-010R1
41. Straus, S. E.and  Sackett, D. L. (1998):Using research findings in clinical practice. British Medical Journal 317(339). pp 339-342 http://dx.doi.org/10.1136/bmj.317.7154.339
42. Vandeweerd, J. M. Et al. (2012a):Using Systematic Reviews to Critically Appraise the Scientific Information for the Bovine Veterinarian. Veterinary Clinics of  North America: Food Animal Practice 28(1).pp 13-21 http://dx.doi.org/10.1016/j.cvfa.2011.12.002
43. Vandeweerd, J. M. et al. (2012b): Is evidence-based medicine so evident in veterinary research and practice? History, obstacles and perspectives. Veterinary Journal 191(1) pp 28–34 http://dx.doi.org/10.1016/j.tvjl.2011.04.013
44. Weiner, S. A.; Stephens, G.and  Nour, A. Y. M. (2011): Information-Seeking Behaviours of First-Semester Veterinary Students: A Preliminary Report. Journal of Veterinary Medical Education 38(1) pp 21-32 http://dx.doi.org/10.3138/jvme.38.1.21

All the supplementary files are available here.