Methodology of evidence-based medicine (EBM) О.G.Puzanova National O.O. Bohomolets medical university Chair of computer sciences

Medical practitioners are overloaded with information… Every day they encounter questions that need to be answered in order to make the best decisions about patient care. This is where EBM comes in. P.Glasziou, C.Del Mar, 2003 By the year 2020, 90% of clinical decisions will be supported by accurate, timely, and up-to-date clinical information and will reflect the best available evidence. IOM Roundtable on Evidence-Based Medicine

What is evidence-based medicine (EBM)? The term EBM was first used by David Sackett and his colleagues at McMaster University in Ontario, Canada, in the early 1990s. The term EBM was first used by David Sackett and his colleagues at McMaster University in Ontario, Canada, in the early 1990s. Patient Concerns Clinical Expertise Best research evidence EBM Sackett D, Rosenberg W, Gray J, Haynes R, Richardson W. Evidence based medicine: what it is and what it isnt. BMJ 1996;312:71-2 D.Sackett: Evidence-based medicine is the integration of best research evidence with clinical expertise and patient values

Which doctor do you want? William Osler, 1900 Wise & experienced Smart young doctor

In its original formulation, EBM reduced an emphasis on unsystematic clinical experience and pathophysiological rationale, and promoted the examination of evidence from clinical research: EBM is the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients. about the care of individual patients. The latter term is evidence-based practice (EBP).

The purpose of EBM is The purpose of EBM is to alert clinicians to important advances in different fields of medicine by selecting from the biomedical literature by selecting from the biomedical literature those original and review articles whose results are most likely to be both true and useful.

By individual clinical expertise we mean the proficiency and judgment that individual clinicians acquire through clinical experience and clinical practice. By best available external clinical evidence we mean clinically relevant research, often from the basic sciences of medicine, but especially from patient centred clinical research into - the accuracy and precision of diagnostic tests (including the clinical examination), - the power of prognostic markers, - and the efficacy and safety of therapeutic, rehabilitative, and preventive regimens.

Without current best evidence, practice risks becoming rapidly out of date, to the detriment of patients. Without current best evidence, practice risks becoming rapidly out of date, to the detriment of patients. Without clinical expertise, practice risks becoming tyrannised by evidence, for even excellent external evidence may be inapplicable to or inappropriate for an individual patient. Without clinical expertise, practice risks becoming tyrannised by evidence, for even excellent external evidence may be inapplicable to or inappropriate for an individual patient.

External clinical evidence both invalidates previously accepted diagnostic tests and treatments and replaces them with new ones that are more powerful, more accurate, more efficacious, and safer. External clinical evidence can inform, but can never replace individual clinical expertise, and it is this expertise that decides whether the external evidence applies to the individual patient at all and, if so, how it should be integrated into a clinical decision.

So, good doctors use both individual clinical expertise and the best available external evidence, and neither alone is enough. So, good doctors use both individual clinical expertise and the best available external evidence, and neither alone is enough. The practice of EBM means integrating individual clinical expertise with the best available external clinical evidence from systematic research. The practice of EBM means integrating individual clinical expertise with the best available external clinical evidence from systematic research. Doctors practising EBM will identify and apply the most effective interventions to maximise the quality and quantity of life for individual patients. Doctors practising EBM will identify and apply the most effective interventions to maximise the quality and quantity of life for individual patients.

In 1972, British epidemiologist Archie Cochrane highlighted that most treatment-related decisions were based on an ad hoc selection of information from the vast and variable quality scientific literature, on expert opinion, or on trial and error. Cochrane proposed that researchers and practitioners should collaborate internationally to systematically review all the best clinical trials (especially randomised controlled – RCTs). In the early 1990s, funds were provided by the UK National Health Care to establish a Cochrane Center in Oxford. The approach was further outlined at an international meeting organized by the New York Academy of Sciences and at first Cochrane Colloquium in 1993, when The Cochrane Collaboration was founded.

The total number of RCTs published has increased exponentially since the 1940s. A total of trials are published each year (with over in total) and approximately 50 new trials - every day. So, to keep up-to-date with RCTs alone, we have to read 1 study report every half hour, day and night !!! P.Glasziou, C.Del Mar, 2003

Epidemiological evidence is the cornerstone of EBM The goal of any therapeutic intervention is to increase the probability of improvement or to reduce the probability of harm. These probabilities are often reported as ratios, rates, or risks, in which the probability of an event occurring in one group, usually an experimental or treatment group, is compared with the probability of the event occurring in another treatment or placebo control group. Lang T.A., Secic M. How to report statistics in medicine. - American College of physicians, – P.19.

Differences between Clinical and Statistical Significance (1) SS essentially reflects the influence of chance on the outcomes; CS reflects the biological value of the outcome. SS essentially reflects the influence of chance on the outcomes; CS reflects the biological value of the outcome. Statistics are derived from groups of individuals; medicine is practiced on specific individuals. Statistics are derived from groups of individuals; medicine is practiced on specific individuals. Statistical conclusions require adequate amount of data to be valid; medical decisions must often be made with insufficient data. Statistical conclusions require adequate amount of data to be valid; medical decisions must often be made with insufficient data.

Differences between Clinical and Statistical Significance (2) Statistical answers are probabilistic; medical treatment requires committed decision. Statistical answers are probabilistic; medical treatment requires committed decision. Statistical analysis always requires measurement; medicine sometimes requires intuition. Statistical analysis always requires measurement; medicine sometimes requires intuition. The statistical and clinical applications of the term normal are often confused and vague. The statistical and clinical applications of the term normal are often confused and vague. Lang T.A., Secic M. How to report statistics in medicine.- American College of physicians, – P.19.

The 4 steps of EBM / EBP 1.Formulate an answerable question 2.Search for appropriate epidemiological evidence to help answer question - track down the best evidence of outcomes available 3.Appraise (critically) the evidence - find out how good is it 4.Apply the evidence - integrate the results with clinical expertise and patient values 3.Assess/ evaluate practice – to improve next time (clinical audit).

Generalizable knowledge for better clinical practice and healthcare knowledge from research (research evidence) knowledge from research (research evidence) knowledge from the analysis of routinely collected and audit data (statistics) knowledge from the analysis of routinely collected and audit data (statistics) knowledge from the experience of providers and patients (know how) knowledge from the experience of providers and patients (know how)

What is the evidence of EBM / EBP? clinically relevant evidence, sometimes from basic science, but especially from patient-centred clinical research into the accuracy of diagnostic tests*, the power of prognostic markers*, and the efficacy & safety of interventions* Modified from Sackett et al. EBM 2nd Edition 2000 * from clinical epidemiological studies

GATE: Graphic Appraisal Tool for Epidemiology The GATE Approach: every epidemiological study hangs on the GATE frame There is only one study design: RCT - interventions Cohort studies - prognosis / interventions / aetiology Cross-sectional studies - diagnosis Case-control studies - interventions /aetiology

P EC O GATE: Graphic Appraisal Tool for Epidemiology T GATE Frame: PECOT Population Exposure Comparison Outcome Time

Population P Comparison EC Outcomes (O) O yes no Exposure ab cd © GATE: epi study design (O)

Population P Outcomes: CHD O Randomised Controlled Trial (RCT): HERSTime T Randomised allocation Randomised allocation EGEGEGEG CGCGCGCG Exposure ( intervention ): HRT Comparison ( control ): Placebo Hulley et al. JAMA1998;280:605-13

Types of studies (1) Question Best study design Description Interven- tion Randomised controlled trial Subjects are randomly allocated to treatment or control groups and outcomes assessed. Aetiology and risk factors Randomised controlled trial Cohort study Case-control study The ideal study, but its usually not ethical or practical to conduct such a trial to assess harmful outcome. Outcomes are compared for matched groups with and without exposure or risk-factor (prospective study). Subjects with and without outcome of interest are compared for previous exposure or risk-factor (retrospective study).

Question Best study design Description Frequency and rate Cohort study Cross-sectional study As above Measurement of condition in a representative (preferably random) sample of people Diagnosis Cross-sectional study with random or consecutive sample Preferably an independent, blind, comparison with gold standart test Prognosis and prediction Cohort / survival study Long-term follow-up of a representative cohort Pheno- mena Qualitative Narrative analysis, or focus group; designed to assess the range of issues (rather than their quantification) Types of studies (2)

Population P Exposure GroupComparison Group non- randomised allocation Outcomes O Life is a cohort study: a natural experiment Cohort (Follow-up) study: archetypal epidemiological studyTimeT Real life time EGEGEGEG CGCGCGCG cohort ill-health

Population P Cohort of women diagnosed with breast cancer childbirth <2yrs before diagnosis Non- randomised allocation to prognostic groups yes death. no O Cohort study (prognosis): Danish Breast Cancer Cooperative 10 years T EGEGEGEG CGCGCGCG No childbirth <2 yrs before diagnosis © Kroman et al. BMJ 1997;315:851-5

Population P Cohort of British Doctors established in 1951 Smokers Non-smokers Non-randomised allocation (self- reported smoking) yes Lung Cancer. no O Cohort study (aetiology): British Doctors Study T EGEGEGEG CGCGCGCG 50 years Doll et al. BMJ 2004;328:1519

Population P Cohort of US Nurses HRT No HRT Non- randomised allocation (self- reported use) yes CHD. no O Cohort study (intervention): Nurses Health Study T EGEGEGEG CGCGCGCG 10 years Stampfer et al. NEJM 1991;325:

Population P EGEGEGEGC No one allocated to Comparison Group OutcomesO Time Case series T Exposure Grp ©

Descriptions of a clinical case or a case series are brief reports about successful treatment or threatening side-effects of pharmacothrapy (-) – poor evidence of information (+) – energy and efficiency of information !

What are your clinical questions? A 35 year old man says his brother recently died of a ruptured cerebral aneurysm. He is worried about whether he might have one and what the chances are that it would rupture. A 35 year old man says his brother recently died of a ruptured cerebral aneurysm. He is worried about whether he might have one and what the chances are that it would rupture.

Risk Factors Cause(s) Symptoms Signs, Tests Prognosis Treatment Effect Past current future Types of question and relevant studies : stroke Frequency Cohort Study Survey Inception Cohort Study Treatments Randomised Trial CT Scan Cross Sectional Study

Classes of evidence Class І – evidences and/or common agreement that intervention is useful and effective. Class ІІ – disputable evidences and/or differences in experts opinion about usefulness / efficacy of the intervention: ІІa – evidences / opinion for its efficacy; ІІa – evidences / opinion for its efficacy; ІІb – usefulness / efficacy of the intervention is not convincingly supported by the evidences / opinion. ІІb – usefulness / efficacy of the intervention is not convincingly supported by the evidences / opinion. Class ІІІ – evidences and/or common agreement that intervention is neither useful nor effective or is even harmful. Class ІІІ – evidences and/or common agreement that intervention is neither useful nor effective or is even harmful.

Levels of evidence А: data obtained in several RCTs. В: data obtained in isolated RCTs or in non- randomised trials. С: experts opinion or data from case series.

End-points of the controlled clinical trials - 5D: Death Disease - feeling of danger Discomfort - pain, dispnoe, dizziness, sickness… Disability - patient is not capable to activity (professional, or everyday, or self-service) Dissatisfaction - negative еmotional reaction to disease and treatment

It is usually defined as a probability of an unfavorable outcome occuring during a given period of time. Dictionary of Epidemiology, 1988 Risk is a combination of the probability of an event – usually an adverse event – and the nature and severity of the event. National Academy of Sciences (In: How to report statistics in medicine. - American College of physicians, – P.21). What is Risk?

Framingham Study: X 16 X 4.5 X 9 X 6 X 3 X 4 X 1.6 Паління Артеріальна гіпертензія Гіперхолестеринемія Combination of the cardiovascular risk factors

What are tests used for? Diagnosis Screening Monitoring Prognosis Treatment planning

Diagnostic Test Accuracy measurements Sensitivity is the probability of a positive test in a diseased person Specificity is the probability of a negative test in a non-diseased person

Sensitivity and specificity of diagnostic and screening tests are revealed in cross-sectional studies Sensitivity = the proportion of people with disease who have a positive test: Se = a / (a + c) Specificity = the proportion of people free of disease who have a negative test: Specificity = the proportion of people free of disease who have a negative test: Sp = d / (b + d) Disease +- TestTestTestTest+ab -cd

Predictive value (PV) of the test – the probability of presence or absence of the disease in case that result of the test is known. For test result interpretation this index is considered as the best !!! PV of the test depends on its sensitivity and specificity, as well as on prevalence of the disease. PV of the test depends on its sensitivity and specificity, as well as on prevalence of the disease. Prevalence - the baseline risk of a disorder in the population of interest: P = (a+c) / (a+b+c+d) Incidence: the number of new cases of illness commencing, or of persons falling ill, during a specified time period in a given population.

Positive predictive value (+PV) = the proportion of people with a positive test who have disease. + PV = a / (a + b) Negative predictive value (-PV) = the proportion of people with a negative test who are free of disease. - PV = d / (с + d) Dise- ase +- TestTestTestTest+ab -cd

2 by 2 table Disease Test Sensitivity Positive predictive value a b c

Systems Summaries Synopses Syntheses Studies Examples Computerized decision support Evidence-based textbooks Evidence-based journal abstracts Systematic reviews Original journal articles The evolution of information resources for evidence-based decisions

Premier evidence resources Systems: EMR with decision support Summaries: Clinical Evidence, PIER, UpToDate, Dynamed Synopses: ACP Journal Club, EBM Syntheses: via BMJUpdates+ Studies: via BMJUpdates+, PubMed Clinical Queries

The best evidence for different types of question (1) LevelTreatmentPrognosisDiagnosis I II Randomised trial Inception Cohort Cross sectional III

The best evidence for different types of question (2) LevelTreatmentPrognosisDiagnosis I Systematic Review of … II Randomised trial Inception Cohort Cross sectional III

Systematic review - an article in which the authors have systematically searched for, appraised, and summarised all of the medical literature for a specific topic (for one structured clinical question). Meta-analysis - a systematic review which uses quantitative methods to summarise the results.

What is a meta-analysis? Optional part of a systematic review Systematic review Meta-analyses

What makes a Review Systematic? Based on a clearly formulated question Based on a clearly formulated question Identifies relevant studies Identifies relevant studies Appraises quality of studies Appraises quality of studies Summarizes evidence by use of explicit methodology Summarizes evidence by use of explicit methodology Comments based on evidence gathered Comments based on evidence gathered

STEP 1: Ask focussed 5-part questions: 1. P articipants (patient/population group) 2. E xposure ( I ntervention if about therapy) 3. C omparison (there is always an alternative!) 4. O utcome 5. T imeframe

STEP 2: Search for best evidence using 3/4-part search terms: 1. P articipants (patient/population group) 2. E xposure (intervention if about therapy) 3. C omparison (e.g placebo) 4. O utcome

S S earching for best evidence 2 main databases of primary information origins we can use to search the evidence are Pub Med and Cochrane library. 2 main databases of primary information origins we can use to search the evidence are Pub Med and Cochrane library. PubMed - t he National Library of Medicine – is electronic on-line free database in Medline. Cochrane collaboration - a worldwide association of groups who create and maintain systematic reviews of the literature for specific topic areas.

Medline Embase Cochrane Trials Registry Comparing Databases Medline = Pubmed, Webspirs, OVID, …

Search Cascade

Using the question to guide searching You are interested in checking the hearing of elderly patients, and have heard that the whispered voice test is good. Question PopulationPopulation Indicator (Intervention, test, etc)Indicator (Intervention, test, etc) ComparatorComparator OutcomeOutcome Underline the key terms Number the order of importance from 1-4 Think of alternate spellings, synonyms, & truncations

You are interested in checking the hearing of elderly patients, and have heard that the whispered voice test is good. Question Population – in elderly patients doesPopulation – in elderly patients does Indicator – a poor whispered voice testIndicator – a poor whispered voice test Comparator – a normal whispered voice testComparator – a normal whispered voice test Outcome – predict poor hearing (audiogram)Outcome – predict poor hearing (audiogram) 1 2 3

Start up Start Explorer and enter Start Explorer and enter Put on CAPS lock Put on CAPS lock So that AND and OR are in CAPITALS Start 2nd Explorer window, enter Start 2nd Explorer window, enterwww.tripdatabase.com

* Means any other letters AND means both terms required Check the question type Check the emphasis

Structured Question Do adults (aged > 18) using oral anticoagulation therapy have fewer episodes of thrombotic events if they are self-managed than those that are managed by doctor/health practitioner? Do adults (aged > 18) using oral anticoagulation therapy have fewer episodes of thrombotic events if they are self-managed than those that are managed by doctor/health practitioner? population outcome intervention control

EBM / EBP Step 4: Applying to the individual What do the results mean on average? What do the results mean on average? What do they mean for this individual? What do they mean for this individual?

In 2002 the Guidelines International Network (GIN) was founded

Basic principles of giving medical help Clinical practice guidelines - systematically developed statements designed to assist health care professionals and patients make decisions about appropriate health care for specific clinical circumstances. Medical standarts – normative documents defining a list of norms and requests to medical care and recognizing an accordance to indicators National (local) clinical reports (statements) normative documents, detailed instructions

So, today we see… intensive development of medical sciences, intensive development of medical sciences, increase of cost of researches, increase of cost of researches, growth of biomedical information (scientific and popular), growth of biomedical information (scientific and popular), ageing of population, ageing of population, changes in morbidity structure (growth of chronic non- infectious diseases, co-existing of them in individual pts), changes in morbidity structure (growth of chronic non- infectious diseases, co-existing of them in individual pts), increase of cost of treatment while the resources of health care systems are restricted, increase of cost of treatment while the resources of health care systems are restricted, humanisation of world community, attention to pts rights, humanisation of world community, attention to pts rights, development and implementation of clinical recommenda- tions and medical standarts - to increase quality of medical care. development and implementation of clinical recommenda- tions and medical standarts - to increase quality of medical care.

Knowledge is the enemy of disease The application of what we know will have a bigger impact on health and disease than any single drug or technology likely to be introduced in the next decade. Sir Muir Gray, Director,NHS National Knowledge Service