October 13,2001
Journal of Manipulative & Physiological Therapeutics
200 E. Roosevelt Road
Lombard IL 60138

To the Editor:

Dana J. Lawrence, D.C., Editor

Gatterman et al.'s recent paper, "Rating specific chiropractic technique procedures for common low back conditions" (1), is an effort to combine expert opinion with available evidence in order to rank the usefulness of 10 methods of treatment for patients with 15 categories of low back or lower extremity disorder (see Table 1). Ratings were made by eight chiropractors considered broadly knowledgeable in chiropractic technique and who were not associated with any proprietary technique. The authors did not participate as raters.

Table 1: Methods and conditions rated for quality of literature and effectiveness

Ten Treatment Methods
High velocity, low amplitude, no drop table (side posture)
High velocity, low amplitude, prone with drop table assist
Pelvic blocking procedures
Instrument adjusting
Mobilization, segmentally specific
Distraction techniques
High velocity, low amplitude, prone without drop table assist
Upper cervical
Non-thrust/reflex/low force
Lower extremity adjusting

Seven Acute Low Back or Lower Extremity Disorders
Low back pain
Low back pain with buttock or leg pain
Low back pain with buttock or leg pain and neurologic deficits
Buttock or leg pain (no low back pain)
Herniated disk
Posterior joint subluxation/syndrome
Sacroiliac joint dysfunction/syndrome

Eight Chronic Low Back or Lower Extremity Disorders
Low back pain
Low back pain with buttock or leg pain
Low back pain with buttock or leg pain and neurologic deficits
Buttock or leg pain (no low back pain)
Herniated disk
Posterior joint subluxation/syndrome
Sacroiliac joint dysfunction/syndrome
Spondylolisthesis

Two types of ratings (on eleven-point scales: 0-10) were made for each of the 150 cells (10 treatments x 15 conditions = 150 cells). The first was a rating of the available literature/evidence for each cell; randomized, controlled clinical trials (RCTs), cohort studies and case series were included if they provided clear descriptions of the intervention procedures and diagnostic characteristics. The second set of ratings involved judgments (based on evidence and/or personal clinical experience) of the effectiveness of each treatment method for each clinical condition. The ratings in each cell were averaged and the standard deviations within each cell were calculated.

The authors urge caution in comparing ratings among cells. However, such comparisons are presumably one of the reasons for engaging in the consensus process, and so this warning is a recognition of the severe deficiencies in the study. The deficiencies result not from the design or conduct of the literature review or consensus process, but from the dearth of information for most of the 150 treatment/condition cells. Indeed, there were more cells (150) than available research papers (139) to fill them, and the authors indicate that "Most of the cells have no literature in them, and all cells have inadequate literature in them." As well, there were "5 abstentions in the ratings of quality of literature and 327 abstentions in the ratings of effectiveness." The available data base can be likened to a block of Swiss cheese with more holes than cheese.

There was a strong linear correlation (r = 0.74, p<.0001) between ratings of the quality of the literature and ratings of effectiveness. This suggests that cells receiving lower effectiveness judgments were those for which the least (or no) empirical evidence exists, while those receiving higher effectiveness ratings were the better and more frequently studied methods. The authors appropriately caution the reader that “Lack of evidence in the literature is not evidence of lack of effectiveness.”

Given the many caveats and limits to interpretation in this project, perhaps the most important finding is the severe "lack of quality evidence in the literature." Earlier consensus ratings for chiropractic methods (e.g., the Mercy Conference guidelines) skirted this dilemma in part by rating treatment methods without relating them, in most cases, to the specific conditions for which they may be applied. However, subsequent guidelines will need to adopt a more focused approach, something akin to that offered by Gatterman et al. Clearly, the absence of relevant clinical outcome data for most treatment/conditions will be a serious limiting factor. Clinical judgment without empirical evidence amounts to guesswork by experts.

Unfortunately, Gatterman et al.'s effort to set the stage for future consensus guidelines may be misinterpreted by many in the profession. Within days of the study's publication one wag (2), ignoring Gatterman et al.'s admonition that the absence of information does not amount to lack of effectiveness, suggested that the use of low-ranked procedures (those for which little or no research currently exists) constitutes malpractice. Such leaps in reasoning are the bane of evidence-based practice. It is just as wrong to dismiss a clinical method without evidence as it is to fully accept one without hard data.

The scientific evolution of the chiropractic profession is now about a quarter century old (3). The development of RCTs conducted by chiropractors has barely more than a 15 year history (e.g., 4). We have been inching toward greater empirical accountability, and I believe we have made some worthy progress (e.g., 5) despite serious obstacles. But as Gatterman et al.'s project makes clear, our ability to integrate research evidence and consensus is severely limited at this time. We must guard against throwing out the baby with the bathwater.

To be sure, a continuing and pressing need within the profession is more and better quality research, and our recent successes in acquiring public funding for chiropractic investigations is promising. But research alone will not serve its intended purposes unless and until chiropractors acquire more sophisticated attitudes and skills in interpreting and acting upon the chiropractic and related health sciences literature. This latter component of our evolution will have to originate in our schools, and eventually become part of the culture of the profession. We have a ways to go.

Sincerely,

Arlan W. Fuhr. D.C

References:

1. Gatternan MI, Cooperstein R, Lantz C, Perle SM, Schneider MJ. Rating specific chiropractic technique procedures for common low back conditions. Journal of Manipulative & Physiological Therapeutics 2001 (Sept); 24(7): 449-56

2. Smith JC. The chiropractic top ten . E-maiI distribution, II October 2001

3. Gitelman R. The history of chiropractic research and the challenge of today. Journal of the Australian Chiropractors' Association 1984 (Dec); 14(4): 142-6

4. Waagen ON, Haldeman S, Cook a, Lopez D, DeBoer KF .Short-term trial of chiropractic adjustments for the relief of chronic low back pain. Manual Medicine 1986; 2(3 ):63- 7

5. Keating JC, Caldwell S, Nguyen H, Saljooghi S, Smith B. A descriptive analysis of the Journal of Manipulative & Physiological Therapeutics, 1989-1996. Journal of Manipulative & Physiological Therapeutics 1998 (Oct); 21(8): 539-52