A critical evaluation of the methodology of the literature on medication compliance

OBJECTIVE: To develop a simple evaluation tool to assess methodologic rigor of the literature on patient compliance with medications, and to apply the tool to a sample of the literature. METHODS: A computerized search of the MEDLINE database (January 1980-December 1996) was performed. All English-language articles on compliance with medications were identified, using the MeSH terms patient-compliance and drug-therapy. A 10% sample was then randomly selected for review. Methodologic rigor was assessed on eight standards: study design, specification of patient sample, power analysis, specification of disease, specification of therapeutic regimen, duration of follow-up, definition of compliance, and compliance measurement. The raw scores of the eight standards were then combined in three summary scores, standardized from 0 to 100: study design, disease-related features, and compliance issues. RESULTS: Seventy-two articles from 719 identified were reviewed. The majority of the research articles were descriptive (63.9%), and patients in these studies were selected mainly from a convenience sample (41.7%). Just nine studies were multicenter studies, and three employed power analysis. The compliance definition was replicable in 41.7% of the studies. In 22 articles neither the compliance measure nor the criteria were stated. One-quarter of the studies (18) used a nonvalidated measure of compliance. Only two studies reached a score of 6 in the compliance measure, and eight studies used two different measures of compliance simultaneously. The median values in the summary scores were: study design 8.3, disease 42.9, compliance issues 50. CONCLUSIONS: The quality of the compliance research was generally poor. These low scores reflect very important shortcomings in the methodology. Such oversights make it difficult for the reader to critically assess the validity of the conclusions.