— Read on

by Langston Hughes

Methodological Errors in “Treatments for Acute Pain – A Systematic Review” (AHRQ)

By Richard A Lawhern PhD

January 2021


This paper expands on comments by the author, offered to a September 2020 circulated draft of Agency for Healthcare Research and Quality (AHRQ) Comparative Effectiveness Review 240, titled “Treatments for Acute Pain:  A Systematic Review” [Ref 1, 2].  Author’s original comments to the draft were transmitted online to AHRQ and by email to the Director and senior staff of the Agency. This updated communication presents an evidentiary basis for a mandatory and substantial revision of Comparative Effectiveness Review 240 by AHRQ, followed by a window for public comments, before its republication. There is no other recourse to mitigate stigmatizing depictions of patients in pain who require opioid analgesia, as well as misleading and false interpretations of available opioid trials incorporated into the recently published review.

Analysis of Medical Evidence

           From the AHRQ Report Structured Abstract:  “Meta-analyses were conducted on pharmacologic therapy for dental pain and kidney stone pain, and likelihood of repeat or rescue medication use and adverse events….”  

“Results:  One hundred eighty-three RCTs on the comparative effectiveness of therapies for acute pain were included. Opioid therapy was probably less effective than nonsteroidal anti-inflammatory drugs (NSAIDs) for surgical dental pain and kidney stones, and might be similarly effective as NSAIDs for low back pain.”

Author’s comments:  

Study Design [Ref 3, 4]

In meta-analyses, mean changes from baseline to post-treatment on pain measures are pooled for the opioid and comparator, from which pooled opioid–comparator group differences are derived. 

Randomized controlled trials (RCTs) and meta-analyses measure central tendency, with a core assumption and necessary pre-condition being normal distribution (e.g., the “bell-shaped curve”) in treatment/comparator/placebo response.  

Analgesic response in chronic pain patients shows bimodal distribution with pain reduction either substantial or near-absent/absent. Bimodal distribution in opioid response appears within 2-4 weeks of randomization and treatment initiation. [Ref 3]  A clinical population with bimodal distribution in treatment response violates the core assumption of normal distribution in RCTs and meta-analyses. 

Meta-analyses and systematic reviews, when appropriately used, are powerful tools for understanding efficacy and safety in many areas of medicine. However, few understand that they are not useful in analgesic response. With meta-analyses and systematic reviews, the authors decide what is “evidence.” Roger Chou MD was a lead author of the 2016 CDC Guideline and its derivative systematic review – a document considered by many patients and medical professionals to be the exemplar of conclusions based on evidence selected and defined with prejudicial intent. 

The conclusions of the December 2020 AHRQ review are not helpful to clinicians treating pain patients.  This Review should be disqualified from incorporation in ongoing efforts by the CDC to expand and revise its 2016 guidelines for prescription of opioids to adults with chronic non-cancer pain. 

Opioid Selection [Ref 6-8]

Excluding (1) trials in acute renal colic pain with parenteral delivery of opioid and nonopioid, and (2) trials comparing opioid agonists to multimodal (e.g., tapentadol) or partial agonist (e.g., buprenorphine) opioids, 47 trials reported in this review compared opioids to NSAIDs or acetaminophen, with codeine or tramadol as the selected opioid: 

All oral opioid vs NSAIDs or acetaminophen comparisons: 38 of 47 trials (80.8%) 

Acute post-operative pain: 5 of 11 trials (45.5%) 

Acute low back, musculoskeletal, and dental pain: 33 of 36 trials (91.6%)

Thus, in trials comparing opioids to NSAIDs or acetaminophen (excluding parenteral delivery), the selected opioid was codeine or tramadol in 80.8% of trials overall, and in 91.6% of acute low back, musculoskeletal, and dental pain trials. There are several implications in this selection:

  • Codeine and tramadol are both weak opioid analgesics. Both require hepatic enzymatic conversion to an active analgesic metabolite. A significant proportion of the patient population possesses a genetic variation of cytochrome 2D6 that causes polymorphism.  Consequently, these individuals poorly convert codeine or tramadol, and experience side effects rather than analgesia. For this reason, codeine has fallen into relative disuse in pain management practice. 
  • Genetic variation in CYP450 expression alters opioid pharmacokinetics and may contribute to bimodal distribution in opioid response, along with pharmacodynamic variation and other factors.
  • Codeine and tramadol account for most (80.8%) oral opioid comparisons to NSAIDs or acetaminophen, and nearly all (91.6%) in acute low back, musculoskeletal, and dental pain. This needs to be stated clearly in the report, perhaps in this form:

“Weak Prodrug Opioids Accounted for >80% of Oral Opioid Comparisons to NSAIDs or Acetaminophen overall, and 91.6% in Acute Low Back, MSK and Dental Pain.”

  • Obvious, yet repeatedly ignored in this and other AHRQ reviews authored by Chou et al., is that meta-analyses are unsuited for evaluating the efficacy of opioids, which require a tailored approach to balance response and side effects. 

Additional Discussion

From the AHRQ Report Background:

“The key decisional dilemma in acute pain management involves selection of interventions to provide adequate pain relief, in order to improve quality of life, improve function, and facilitate recovery, while minimizing adverse effects and avoiding overprescribing of opioids.8 Evidence also suggests that adequate acute pain treatment may mitigate factors that promote the transition to chronic pain.3,9,10 However, shortcomings in acute pain care have been documented.11,12 In addition to the underlying cause of pain, patient factors that impact acute pain management include age, sex, race/ethnicity, pain severity, comorbidities (including mental health and substance use), genetic factors, pregnancy, or breastfeeding status.13-16 Timing of presentation and clinical setting can also influence acute pain management. For example, postoperative pain occurs at a specific point in time and is often managed with multimodal strategies in a monitored setting prior to discharge, whereas in outpatient clinic settings, timing of presentation of acute pain is variable, and assessing treatment response is often not feasible. Additionally, access and care options may vary.2,8 Different acute pain conditions (e.g., musculoskeletal pain, neuropathic pain, or visceral pain) may respond differently to treatments. Therefore, a treatment that is effective for one acute pain condition and patient in a particular setting may not be effective in others.” [Reference numbers in the original report]

Author’s Comments:It is highly revealing that despite this acknowledgement of the complexities of evaluating pain therapies, the AHRQ authors proceed to merge data of doubtful quality and to generalize their conclusions using terms such as “probably” and “might be”.  A section on “Research Gaps” in both the draft and the final report briefly mentions a wide range of confounding factors – which are then ignored as the authors race to their pre-ordained conclusions, attempting to disqualify prescription opioid analgesics from use in acute pain.  A far more supportable conclusion from this review is that the present state of medical literature lacks sufficient rigor and repeatability to arrive at any conclusions at all concerning best practices. 

In this context, the following observations apply:

  1. While the investigators’ desire to limit the scope of their “comprehensive review” is understandable, exclusion of German language studies from the AHRQ review is a significant omission.  The German government has invested major resources in evaluating opioid and non-opioid therapies.
  2. Ketorolac has strong analgesic potency that may be predicted to surpass that of Codeine or Tramadol in four comparative trials reported. Such a match-up seems inappropriate.
  3. Due to risks of GI toxicity, use of Ketorolac for longer than 6 days is contra-indicated on the FDA information label for this medication. How these patients with persistent pain should be managed is unclear. This significant limitation in “real-world” clinical practice is obscured by meta-analyses and similar big-data methods, and should be highlighted as a basis for disqualifying the selected studies.
  4. Likewise, all NSAIDs, even with acute use, are associated with potentially serious GI toxicity. Such side effects are nowhere discussed in the published Review.

For acute post-operative pain, it is unclear from the AHRQ report how or in what sense the performance of opioids is “no better” than that of NSAIDs.  The Review notes an abundance of limitations which are then ignored in its conclusions:  

 — “Evidence on how comparative effectiveness and harms of opioid therapy for postoperative pain vary according to patient and prescribing factors was lacking. The number of trials was small for each comparison and most trials had small sample sizes… No study [was] conducted within-study or across-study evaluations of subgroup effects. Evidence was too limited to determine effects of different opioid doses (converted into morphine milligram equivalents) on comparative effectiveness and harms.  The trials did not evaluate how effectiveness varied in subgroups defined according to the amount of opioid used.”

Also of note, no trial permitted opioid refills despite study durations up 15 days. This omission increases the potential of poorly or uncontrolled pain at final assessment, improperly biasing conclusions toward opioids being ineffective. 

  1. Two large retrospective studies are interpreted in the Review to claim that compared to persons not prescribed opioids, opioids for acute post-op pain impose “risks” of long-term opioid use – in one study 7.7% of patients at 1-year follow-up. The other study described the proportion of patients with opioid fills 90 to 180 days post-surgery (7.1%); opioid use lasting 90+ days in the period from 180 days post-surgery (1.0%) and either 10 or more opioid fills or 120 or more days’ supply (0.46%). As noted in the Review, such studies cannot adjust for factors not available in administrative claims, such as pain severity, functional status, level of psychiatric distress, or other measures of clinical status following surgery.”  [emphasis by the author] These omissions are essentially disqualifying in the context of the AHRQ Comparative Outcomes Review
  2. That retrospective evidence was allowed for opioid harms, and disallowed for opioid benefits, is telling. Post-surgical pain is a prevalent origin of chronic, life-altering pain. The discourse in this review stigmatizes patients experiencing persistent pain who require opioid analgesia. The Review section using retrospective data needs to be deleted. 
  1. The author suggests that post-operative use of opioids at one year is not a “risk”, but rather an “incidence” related to emergence of chronic pain from failed procedures. Long-term pain is often a consequence of poor analgesic control in the peri-operative or acute post-operative setting. In most patients, pain that becomes chronic is perpetuated by CNS alteration, divorced from peripheral nociceptive input from the original tissue injury. This makes prevention of chronic pain through aggressive analgesic control — which may include opioids — an imperative of practice. 

Chronic pain resulting from inadequate control of acute/post-acute pain is no less iatrogenic than recklessly exaggerated claims of opioid use disorder following prescribed opioids. For individuals with OUD, efforts are made to ease access to opioid agonist maintenance.  By sharp contrast, patients with chronic pain are punished for needing opioid pain control in the hostile regulatory environment fostered by CDC’s 2016 guideline on opioid prescribing in adults with chronic non-cancer pain. 

Contrary to the insinuations of the AHRQ report authors, prescription opioid analgesics do not “cause” addiction in patients who are not already predisposed by other factors. [Ref 4]  When pain in the peri-operative period is sufficient to require opioid analgesia, it is to be expected that a diminishing proportion of patients will need opioid pain control over time, with a smaller subgroup in whom pain has become chronic being continued on opioids as the only therapeutic measure that provides sufficient pain management.  This reality of practice is ignored – or perhaps deliberately suppressed – in the AHRQ comparative review.

  1. Genetic polymorphism may be a sufficiently strong effect to account for the bimodal structure of patient responses to opioids.  It is at least plausible that poor metabolizers and hyper-metabolizers can explain the low effectiveness of low-dose opioids in millions of patients, while “normal” metabolizers experience significantly better outcomes.  Although the AHRQ authors reference two papers that address these effects, they exclude any serious discussion of findings in those papers.  Other papers they do not reference are also pertinent.  [Ref 5 – Ref 9]


The published version of AHRQ Comparative Effectiveness Review 240 incorporates multiple and disabling errors of analytic methodology.  Arguably, the authors of this review have also cherry-picked and misinterpreted data in a manner that reflects a profound and unjustified bias against treatment of either acute or chronic pain by means of prescription opioids.  For these reasons, the report must be withdrawn immediately for an independent review by professionals qualified in statistical methods and epidemiology.  Given the checkered history of this document, the representation of patients and their advocates should be considered mandatory.  The revised AHRQ Review should then be submitted to the Federal Register for a 60-day public review and comment period, followed by a published AHRQ summary of comments.


[Ref 1] Roger Chou, Jesse Wagner, Azrah Y Ahmed, et al, “Treatments for Acute Pain:  A Systematic Review”, Agency for Healthcare Research and Quality, AHRQ Publication No. 20(21)-EHC006, December 2020.   

[Ref 2] Richard A. Lawhern, PhD., AHRQ correspondence by email, subject “Courtesy Copy – Comments Submitted to AHRQ”, September 13, 2020, with attachment.

[Ref 3]  Haeuser W, Toelle TR, “Meta-analyses of pain studies: what we have learned,” Best Practice & Research Clinical Rheumatology (2015),


Meta-analysis is a statistical procedure that integrates the results of at least two independent studies. The biggest threats to meta-analysis are publication bias due to missing studies with negative results and low-quality evidence due to methodological limitations imposed by included studies. Tools to improve the quality of meta-analysis have been developed by the Cochrane Collaboration and by the Preferred Reporting Items for Systematic Re-views and Meta-Analyses (PRISMA). Meta-analyses of trials have demonstrated that pain responses in patients with chronic pain, following treatment, are not normally distributed but have a bimodal distribution with the majority of patients having either very little or very good pain relief. The benefit can be detected within 2-4 weeks following drug administration. Further, the efficacy of drug and physical treatments is hampered by high placebo response rates, with modest average benefits with active treatments over placebo in both parallel and crossover design trials.

[Ref 4] R. A. Moore, S. Derry and P. J. Wiffen “Challenges in design and interpretation of chronic pain trials” British Journal of Anaesthesia 111 (1): 38–45 (2013)


[Ref 5] Nora D Volkow, MD, and Thomas A McLellan, Ph.D.,  “Opioid Abuse in Chronic Pain — Misconceptions and Mitigation Strategies” .  NEMJ 2016; 374:1253-1263 March 31, 2016].

[Ref 6] “Post-Op Opioid Prescribing Often Ignores CYP2D6 Pharmacogenics”, December 25, 2020, Anesthesiology News, reported by Josh Bloom, “Recognition of Genetic Differences in Opioid Metabolism, Finally” American Council on Science and Health, January 3, 2021

[Ref 7] Tom Lynch and Amy Price, “The Effect of Cytochrome P450 Metabolism on Drug Response, Interactions, and Adverse Events,” American Family Physician,  August 1, 2007

[Ref 8] Howard S Smith, MD, “Opiod Metabolism” Mayo Clinic Proceedings, 2009 Jul; 84(7): 613–624.

[Ref 8] Andrea M. Trescot, MD, and Semyon Faynboym, MD “A Review of the Role of Genetic Testing in Pain Medicine”, Pain Physician 2014:17 ISSN 1533-3159

[Ref 9] Bhushan A Kapoor, Prateek Lala, Julie L.V. Shaw,  “Pharmacogenics and Chronic Pain Management” Clinical Biochemistry, 2014.

Author Note:  Richard A Lawhern PhD is a technically trained non-physician patient advocate, with 24 years experience as a medical literature analyst and forum moderator for social media and peer-to-peer support groups for chronic pain patients and care providers.  He has published over 100 papers, articles, and public addresses in a mixture of medical journals and mass media, some of them co-authored with medical professionals.  

This paper has benefitted from research assistance provided by Mark Edmond Rose, author of “Are Prescription Opioids Driving the Opioid Crisis? Assumptions vs Facts”, Pain Medicine, Volume 19, Issue 4, April 2018, Pages 793–807, . However, any remaining errors herein are those of the author. 





Leave a Reply