EBSCO Health Evidence-Based Methodology

Step 1   |   Step 2   |   Step 3   |   Step 4   |   Step 5   |   Step 6   |   Step 7

Evidence-based is a descriptor that is often used to describe medically-related reference sources. Unfortunately, it is often used indiscriminately and without merit. For a clinical reference resource to truly be called evidence-based, conclusions must be based on the best available evidence. Conclusions can be based on the best available evidence only if the evidence is consistently and systematically identified, evaluated and selected.

Step 1: Identifying the Evidence

Systematic Literature Surveillance

To ensure that EBSCO Health proprietary medical content provides the best available evidence, an extensive set of current literature is monitored daily. Systematic Literature Surveillance is conducted using many journals, journal review services, systematic review collections, guideline collections and other sources considered relevant to a point-of-care clinical reference. These sources are derived from systematic evaluation of which sources and search strategies provide the greatest yield for identifying the most valid, relevant evidence.

New Topic Creation

As new topics are created, MEDLINE searches are conducted to identify recent guidelines and reviews of broad clinical scope. From these searches, general references are selected to guide the basic construct of the topic and support general statements where evidence is lacking or unnecessary. Original evidence is analyzed as needed for clarity and critical appraisal. Supplemental MEDLINE searches are conducted as needed to address specific clinical concepts. 

Step 2: Selecting the Best Available Evidence

Each article is assessed for clinical relevance and each relevant article is further assessed for validity relative to existing EBSCO Health Point-of-Care content. The most valid articles are summarized, the summaries are integrated with EBSCO Health content, and overview statements and outline structure are updated based on the overall evidence synthesis. Article selection is completed by editors with clinical expertise and training in scientific analysis.

Determining Relevance

Determining clinical relevance is the first consideration in systematically selecting the best available evidence from that identified. The relevance of medical information is different for every user. EBSCO Health Point-of-Care products are used in clinical care by practitioners with a wide range of experience and interests, and is also used in medical education. When adding information to EBSCO Health Point-of-Care products, the following considerations are used for determining relevance:

1.  Does this information have a direct bearing on patient-oriented outcomes?

  • Patient-orientated outcomes are outcomes that affect quality of life without extrapolation. Examples include mortality, incidence of myocardial infarction, and presence and severity of pain. These are also called clinical outcomes. Disease-oriented outcomes are used as surrogate markers for monitoring the effects of interventions ultimately intended to affect patient-orientated outcomes. Examples include cholesterol concentration, blood pressure and bone mineral density. Patients are only interested in these outcomes as a means for affecting clinically significant outcomes such as mortality or fracture incidence.
  • Because EBSCO Health Point-of-Care products are used primarily as clinical tools for use during patient care, patient-oriented outcomes information is considered relevant and included. Patient-oriented evidence is given priority over disease-orientated evidence, with disease-oriented evidence entered only if it adds substantially new information.

2.  In the absence of patient-orientated evidence, might this information be useful in clinical decision-making?

  • Much of medical knowledge is still lacking in terms of patient-oriented outcomes research. Clinical decisions based on extrapolated disease-oriented evidence are not proven to be appropriate. However, clinicians still need to make decisions in situations where patient-oriented evidence is not yet available.
  • Disease-oriented evidence is considered relevant for EBSCO Health Point-of-Care products in situations where patient-oriented evidence is lacking. Individual clinicians will have to determine if this information is considered relevant within their practice. When disease-oriented evidence is not presented as such in the supporting reference, commentary will be added and may appear as "patient-oriented outcomes not assessed," "clinical outcomes not assessed," or as specific commentary pointing out problems with extrapolating the information to clinical care.

3.  Is this information part of a clinical controversy?

In situations where the evidence does not clearly support or refute a clinical fact, opposing views are presented. EBSCO Health Point-of-Care products are not designed to resolve clinical controversies and strive to present information with as little "inappropriate" bias as possible. The inherent bias towards patient-oriented outcomes is considered appropriate. Information that questions "standard" approaches and has a potential bearing on patient-oriented outcomes is considered relevant for inclusion in EBSCO Health Point-of-Care products.

4.  Is this information that is of unique interest due to popularity?  

Some medical information is not clinically relevant but widely publicized. Summarization of this type of information (often with commentary) is relevant to users if it is likely that clinicians will be asked about it during clinical encounters. It is important for physician and patient educators to point out when this type of information is not clinically applicable.

Determining Validity

Clinically relevant articles must be assessed to determine the scientific validity of conclusions and facts presented before consideration for use. Conducting critical appraisal for all articles would be wasteful if these articles would not make a change to the existing EBSCO Health Point-of-Care product knowledge base.

Easily identifiable study features (e.g., study method, sample size) are compared with existing studies in EBSCO Health Point-of-Care products to determine if new articles potentially represent the best available evidence. Articles that do not provide relevant information with validity that meets or exceeds the existing content are excluded at this stage.

The evidence hierarchy depends on the type of conclusions.

For conclusions about causation (i.e., the effect of interventions or exposures on outcomes), the evidence hierarchy from highest to lowest validity includes:

a.  Randomized trials
b.  Quasi-randomized trials
c.  Cohort studies (observational comparisons of exposed and unexposed persons)
d.  Case-control studies (observational comparisons of persons with and without the outcome of interest)
e.  Case series (uncontrolled trials)
f.  Expert opinion 

For conclusions about diagnostic accuracy, the evidence hierarchy from highest to lowest validity includes:

a.  Diagnostic cohort studies (with representative sample of patients with diagnostic uncertainty)
b.  Diagnostic case-control studies (with non-representative samples of patients with known diagnosis and controls)
c.  Diagnostic case series (including only patients with diagnosis to evaluate sensitivity only, or only patients without diagnosis to evaluate specificity only)
d.  Expert opinion 

For conclusions about prognosis, the evidence hierarchy includes:

a.  Inception cohort studies
b.  Retrospective cohort studies
c.  Case series
d.  Expert opinion

Within each stratum of the evidence hierarchy, systematic reviews are considered more valid than individual studies of similar quality.

Determining which Guidelines to Summarize

All guidelines are listed but only a minority of guidelines are summarized. EBSCO Health Point-of-Care editors select which guidelines to summarize with selections based on:

a.  Scope of guideline (whole-topic view preferred over focused concept-specific guidelines)
b.  Currency of guideline
c.  Authority behind the guideline (national or international > local, professional association > non-affiliated expert consensus, etc.)
d.  Quality of guideline (use of evidence-based process)
e.  Relevancy of guideline
f.  Ease of use (are the recommendations easy to interpret—clear and unambiguous?)
g.  “Need” for additional guideline summarization to cover “missing information” in the topic 

Step 3: Critical Appraisal

Abstracts in research publications often do not accurately reflect the methodologic quality and results found in full-text articles. Article summaries in other publications often do not accurately reflect the methodologic quality and results found in full-text articles. 

Reports used for updating EBSCO Health Point-of-Care products represent the best available evidence for the specific content under consideration.  Evidence may be lableled in one of three levels as outlined under Levels of Evidence. 

Articles that potentially warrant the highest evidence ratings undergo complete critical appraisal. 

If serious methodological shortcomings are discovered (sufficient to affect clinically relevant results), then the evidence is labeled as mid-level evidence and the shortcomings are described.

Critical appraisal is completed by writers and editors trained in critical appraisal. 

Full-text evaluation of articles is required for:

a.  Any article rated as Level 1 [likely reliable] evidence or Grade A recommendation [consistent high-quality evidence].
b.  Any article potentially ratable as Level 1 or Grade A based on abstract-only information; full-text evaluation is necessary to provide lower levels or grades
c.  Any article for which definition of absolute magnitude of effect and/or detailed description of interventions or exposures are necessary, regardless of level of evidence
d.  Any article which represents the most important guidance for a DynaMed topic, regardless of level of evidence 

Step 4: Objectively Reporting the Evidence

When reporting the evidence, EBSCO Health Point-of-Care Editors consider all of the following questions:

a.  Were all relevant outcomes reported in the original article?
b.  What are the most relevant outcomes to report in topics?
c.  For relevant outcomes, what is the magnitude of effect? This may be represented by absolute rates and number needed to treat (NNT) or harm (NNH) abbreviations, or by absolute differences in continuous variables (e.g., mean decrease in 1.3 points on 0-10 visual analog pain scale).
d.  Were the findings clinically significant?
e.  In the case of no statistically significant differences, were the findings robust enough to rule out clinically significant difference?
f.  Are there any methodologic limitations sufficient to alter reliability of clinical conclusions? 

EBSCO Health Point-of-Care editors check the data against original study reports, and clinical editors review all summaries for validity and relevance at the point-of-care. Level of evidence labeling is done by protocol with explicit reasons stated for downgrading levels of evidence.

Editors check phrasing for possible ambiguity. Phrases which can be interpreted in more than one way are revised. Shortest unambiguous phrasing is used.

For individual studies, if results are available in both relative risk and absolute risk formats, the absolute risk data is used and NNT/NNH are calculated and presented (rounded up for NNT, rounded down for NNH) in the result lines for dichotomous outcomes. For systematic reviews, an NNT range is reported in results lines based on the odds ratio or risk ratio presented, the 95% confidence interval, and the control event rate.

Step 5: Synthesizing Multiple Evidence Reports

Evidence-based summarization of articles is necessary, but insufficient for a point-of-care reference. Understanding the best current evidence requires synthesizing multiple evidence reports.

Addition, deletion, and organization of information is done with consideration of levels of evidence. When multiple articles are present on the same topic, preference for inclusion and organization is based on the quality of methodology (e.g., preference given to data derived from randomized controlled trials over data from prospective observational studies, which is given preference over retrospective studies, which is given preference over anecdotal reports—see the evidence hierarchy discussion above in Step 2 for more details). When data of lesser quality does not add any substantially new or different information, this data is then deleted from EBSCO Health Point-of-Care products.

Evidence is also synthesized with guidelines, and areas of inconsistency are presented.

The outline format of  EBSCO Health Point-of-Care products  allows for rapid navigation and interpretation without the need to skim though large blocks of text.

Clinical editors review all synthesis results for validity and relevance at the point of care.

Step 6: Basing Conclusions on the Evidence

Deriving overall conclusions and recommendations from the evidence synthesis is required for a comprehensive point-of-care reference. In EBSCO Health Point-of-Care products, multiple evidence reports of similar quality are organized such that the overall conclusions quickly provide a synthesis of the best available evidence.

Treatment overviews (the ultimate synthesis of evidence for a clinical topic) are based upon all of the available evidence in the treatment section, and selection of the most important concepts. As new topics are created and existing topics are critically revised, treatment overviews are explicitly linked directly to the supporting evidence synthesis.

Editors confirm that overviews are clinically useful and accurately match supporting data.

Step 7: Updating Daily

The final step in EBSCO Health’s Point-of-Care evidence-based methodology is changing conclusions when new evidence alters the best available evidence. This step is crucial because new evidence is published every day. Having new evidence summaries handled separately from reviewed content in a manner requiring the clinician to search in two locations to synthesize the entire story would make finding the best available evidence more difficult.

As soon as new evidence is evaluated using the 6 steps governing systematic processing, it is added to the appropriate topic(s) in context. This process allows immediate and comprehensive access to the best available evidence as it occurs.

This process occurs every day in most EBSCO Health point-of-care products.

7 Step Proces

Other EBSCO Sites +