Introduction
Clinical case reports have a time-honored and rich tradition in medicine and science.[1 ] Their significance for research and educational purposes is still thriving[2 ] as it allows in-depth, multi-faceted exploration of complex issues in their real-life
settings. Clinical case reports can, for instance, be considered for review purposes
when an experimental design is either inappropriate to answer the research questions
posed or impossible to undertake. Research of case reports, if carefully conceptualized
and thoughtfully undertaken and reported, can yield powerful insights into many important
aspects of healthcare delivery.[3 ]
Case reports are often insufficiently rigorous to be combined for data analysis, inform
research design, or guide clinical practice. To improve the transparency and quality
of clinical case reports, Gagnier et al developed the CARE reporting guideline for
use by healthcare stakeholders around the world.[4 ] Case reports, when written as per reporting guidelines, have the potential to offer
evidence from the point of care that can be useful for clinical research, inform clinical
practice guidelines, and improve medical education. Though clinical case reports are
justifiably graded as weak evidence for assessing causal relationships, they are often
the first “signals” of potential adverse drug reactions, and sometimes even single
cases can contribute significantly to establishing a causal link.[5 ]
In modern medicine, three approaches are used to assess the causal relationship between
drug treatment and the occurrence of adverse events: expert judgement (global introspection),
probabilistic approaches, and algorithms.[6 ] The Naranjo Criteria algorithm is one that has been utilized to classify the probability
that an adverse event is related to drug therapy based on a list of weighted questions.[7 ] The Modified Naranjo Criteria for Homeopathy (Causal Attribution Inventory) tool,
originally adapted by Rutten and further developed over several years by the Clinical
Data Working Group of the Homoeopathic Pharmacopoeia Convention of the United States
(HPCUS), is proposed to assess the likelihood of causal attribution of the clinical
outcome to the medicine(s) prescribed in homeopathic cases and case reports.[8 ]
[9 ]
[10 ]
When homeopathy is tested in clinical trials, understanding and appraisal are likely
to be improved if published reports contain details of prescribing strategies and
treatment as indicated in the homeopathy-specific extension (RedHot) of the CONSORT
guidelines.[11 ] Likewise, a homeopathy extension of the CARE clinical case reporting guideline (HOM-CASE)
recommends use of the Modified Naranjo Criteria for Homeopathy, which enables assessment
of the likelihood of assigning causal relationship between a homeopathic intervention
and a clinical improvement.[10 ] Although randomized controlled trials (RCTs) are considered the gold standard for
establishing causality, a pool of good-quality case reports published using HOM-CASE
CARE guidelines would offer an important additional contribution to knowledge.
The present study was undertaken to assess the reliability and validity of the currently
formulated Modified Naranjo Criteria for Homeopathy,[10 ] and proposes improved domain-by-domain wording based on the findings.
Methods
A purposive sampling strategy, a non-probability sampling wherein the selection process
involves identifying themes, concepts, and indicators through observation and reflection,[12 ] was adopted to identify information-rich cases from accessible publications. Published
case reports were searched in the Central Council for Research in Homoeopathy (CCRH)
Library (Janakpuri, New Delhi, India) and on the Web. Case reports were identified
using the following criteria: single case report, published in Medline-listed or non-indexed
journal or as a dissertation.
Preliminary evaluation of identified case reports as per HOM-CASE CARE guidelines
indicated that most of the cases covered around 9 items out of 30 on the generic CARE
checklist (all domains and sub-domains numbered from 1 to 30) and a minimum of 3 out
of 6 as per the HOM-CASE extension items (main and the sub-domains numbered from 1
to 6, except domain 10h3, which is the possible causal attribution of changes explicitly
assessed/discussed). Therefore, these criteria were chosen as a minimum threshold
for inclusion of cases. Case reports with poorly described prescribing symptoms, and
homeopathic patent medicines or compound formulations or proprietary products or combinations
where more than one medicine was administered simultaneously, were excluded.
The eligible case reports were independently evaluated as per the Modified Naranjo
Criteria for Homeopathy ([Table 1 ]) by four raters (VKG, CDL, NM, and AMM) using a specifically designed electronic
Case Recording Format. Their evaluations were sent to LR and RvH for compilation and
blinding of the raters during analysis, which was done to prevent bias. During analysis,
the face validity was assessed with a view to determining if the items of each domain
were sensible, appropriate, and relevant.
Table 1
Modified Naranjo Criteria
Domains
Yes
No
Not sure or N/A
1. Was there an improvement in the main symptom or condition for which the homeopathic
medicine was prescribed?
+2
–1
0
2. Did the clinical improvement occur within a plausible timeframe relative to the
drug intake?
+1
–2
0
3. Was there an initial aggravation of symptoms?
+1
0
0
4. Did the effect encompass more than the main symptom or condition (i.e., were other
symptoms ultimately improved or changed)?
+1
0
0
5. Did overall well-being improve?
( suggest using validated scale)
+1
0
0
6A Direction of cure: did some symptoms improve in the opposite order of the development of symptoms of
the disease?
+1
0
0
6B Direction of cure: did at least two of the following aspects apply to the order of improvement of symptoms:
–from organs of more importance to those of less importance?
–from deeper to more superficial aspects of the individual?
–from the top downwards?
+1
0
0
7. Did “old symptoms” (defined as non-seasonal and non-cyclical symptoms that were
previously thought to have resolved) reappear temporarily during the course of improvement?
+1
0
0
8. Are there alternate causes (other than the medicine) that—with a high probability—could
have caused the improvement? (Consider known course of disease, other forms of treatment,
and other clinically relevant interventions)
–3
+1
0
9. Was the health improvement confirmed by any objective evidence?
(e.g., laboratory test, clinical observation, etc.)
+2
0
0
10. Did repeat dosing, if conducted, create similar clinical improvement?
+1
0
0
Note: Maximum score = 13, minimum score = −6.
The analysis was done by CDL and RS. Thereafter, LR and RvH helped in reaching agreement/consensus
among raters.
The study was conducted in three phases as reflected in the study flowchart ([Fig. 1 ]). Inter-rater agreement for each domain was assessed via calculation and analysis
of the kappa value (κ) for nominal (i.e., categorical) variables. A measure is said
to be reliable if it gives the same results under consistent conditions. Hence, the
Modified Naranjo Criteria for Homeopathy algorithm was tested for reliability, and
thus consistency, by assessing inter-rater agreement by evaluating the ratings for
each domain given by four raters. The κ value enables an assessment of the level of
agreement “over and above chance” between different raters.
Fig. 1 Study flowchart. Domain 10h3: CARE guidelines domain 10, HOM-CASE extension3.
Kappa values[13 ] and percentage for inter-rater agreement were calculated based on 3-by-3 contingency
tables for the responses received from the raters under three categories: namely,
“Yes”, “No”, and “Not sure or Not applicable (N/A)”. The analysis was done using Statistical
Package for Social Sciences (SPSS).
In the present study, as there were more than two raters, percentage agreement for
all possible combinations of raters was calculated, and subsequently the mean level
of agreement across all four raters was calculated. The κ values[14 ] were interpreted as follows: κ < 0, “less than chance agreement”; κ 0.01 to 0.20,
“slight agreement”; κ 0.21 to 0.40, “fair agreement”; κ 0.41 to 0.60, “moderate agreement”;
κ 0.61 to 0.80, “substantial agreement”; κ 0.81 to 0.99, “almost perfect agreement”.
A minimum of “fair agreement”, in the absence of concerns with regard to the face
validity of the item, was taken as an indicator of validity of a given domain.
Results
Sixty of 80 case reports fulfilled the eligibility criteria ([Fig. 1 ]). Details of the 60 reports are listed in [Supplementary Table 1 ] (online only).
Domain-wise mean percentage agreement of four raters and κ calculation for the Modified
Naranjo Criteria domains were evaluated in the case reports. Inter-rater agreement
for each domain was as follows ([Table 2 ]): “perfect” for domains 1 (100%, κ = 1.00) and 2 (100%, κ = 1.00); “almost perfect”
for domain 8 (97.5%, κ = 0.86); “substantial” for domains 3 (96.7%, κ = 0.80) and
5 (91.1%, κ = 0.70); “moderate” for domains 4 (83.3%, κ = 0.60), 7 (67.8%, κ = 0.46)
and 9 (99.2%, κ = 0.50); and “fair” for domain 10 (56.1%, κ = 0.38). For domains 6A
(46.7%, κ = 0.03) and 6B (50.3%, κ = 0.18), there was “slight agreement” only. Thus,
the algorithm was found to be valid for each domain except 6A and 6B ([Table 2 ]).
Table 2
Domain-wise mean percentage agreement and kappa calculation between Modified Naranjo
Criteria for Homeopathy domains and evaluated case reports
Domain
Agreement (%)
Kappa (κ)
Inference
Valid (κ > 0.40)
1
100
1.00
Perfect agreement
Yes
2
100
1.00
Perfect agreement
Yes
3
96.7
0.80
Substantial agreement
Yes
4
83.3
0.60
Moderate agreement
Yes
5
91.1
0.70
Substantial agreement
Yes
6A
46.7
0.03
Slight agreement
No
6B
50.3
0.18
Slight agreement
No
7
67.8
0.46
Moderate agreement
Yes
8
97.5
0.86
Almost perfect agreement
Yes
9
99.2
0.50
Moderate agreement
Yes
10
56.1
0.38
Fair agreement
Yes
Table 3
MONARCH Inventory (improved version of the Mo dified Nar anjo C riteria for H omeopathy)
Domains
1
Was there an improvement in the main symptom or condition for which the homeopathic
medicine was prescribed?
2
Did the clinical improvement occur within a plausible timeframe relative to the medicine intake?
3
Was there a homeopathic aggravation of symptoms?
4
Did the effect encompass more than the main symptom or condition (i.e., were other
symptoms, not related to the main presenting complaint , improved or changed)?
5
Did overall well-being improve?
(Suggest using a validated scale or mention about changes in physical, emotional, and behavioral elements )
6A
Direction of cure: did some symptoms improve in the opposite order of the development of symptoms of
the disease?
6B
Direction of cure: did at least one
of the following aspects apply to the order of improvement in symptoms:
–from organs of more importance to those of less importance?
–from deeper to more superficial aspects of the individual?
–from the top downwards?
7
Did “old symptoms” (defined as non-seasonal and non-cyclical symptoms that were previously
thought to have resolved) reappear temporarily during the course of improvement?
8
Are there alternative causes (i.e. , other than the medicine) that—with a high probability—could have produced the improvement? (Consider known course of disease, other forms of treatment, and
other clinically relevant interventions)
9
Was the health improvement confirmed by any objective evidence? (e.g., investigations, clinical examination , etc.)
10
Did repeat dosing, if conducted, create similar clinical improvement?
Notes: Updated wording shown in bold (see domains 3, 4, 5, 6B and 9). Improvements
have also been made to the wording of domains 2 and 8. The scores per domain are the
same as for [Table 1 ].
Discussion
A clinical case report is considered a weak level of evidence for establishing causal
relationship. But keeping in view the highly individualistic approach of homeopathy,
conceptual difficulties with RCTs in the context of homeopathy and the resources involved,
we need to strengthen the reporting of case records. Tools to attribute the likelihood
of causal relationship that are aligned with the principles of homeopathy are important
in this regard.
The results of this study indicate that the Modified Naranjo algorithm is generally
feasible as a tool for assessing causality of homeopathic treatment, as there is,
after some instruction, good consensus about the qualifications of the domains. There
is, however, an exception for items 6A and 6B.
Findings and Recommendations Regarding the Modified Naranjo Criteria for Homeopathy
Domains
As successfully treated cases were selected, there was no ambiguity and a perfect
agreement among raters for domains 1 and 2 of the Modified Naranjo Criteria: therefore,
these items do not need modification or elaboration. In domain 3, initial aggravation
should clearly be attributable to homeopathy, because an aggravation may also be due
to a conventional drug or a disease. Also, it was reported in a few cases only, where
the raters largely agreed. The authors of case reports are encouraged to provide information
on the incidence of homeopathic aggravations ([Table 3 ]).
In assessing domain 4, some of the evaluators thought the description included minor
symptoms of the main condition, whereas some considered different symptoms not related
to the main condition. Therefore, the question should be re-phrased to “Did the effect
encompass more than the main symptom or condition (i.e., were other symptoms, not
related to the main presenting complaint, improved or changed)?” ([Table 3 ]). Any symptoms not related to the main presenting symptom or condition are to be
considered for assessing this domain.
Domain 5 was interpreted in different ways by the raters as most of the cases did
not use any validated scales for the assessment of general well-being. Improvement
in general condition or in associated complaints or other symptoms was considered
as overall improvement by the raters. Hence, multiple assessment options led to similar
conclusions. It is therefore recommended, if possible, to use a validated Quality
of Life scale that is either generic or specific for a given disease condition. A
note about improvement in physical condition, emotions, and behavior should be included
to help judge the overall well-being ([Table 3 ]).
Domains 6A and 6B, pertaining to direction of cure, cannot be considered as validated
due to insufficient reporting in the assessed case reports. The raters assessed domains
6A and 6B in most of the cases as “Not sure” or “Not applicable”. Therefore, only
a few cases remained, with considerable disagreement. Direction of cure is an important
aspect but either it was not observed, or it was inadequately reported in the analyzed
case reports. Also, there was no consensus among raters whether it was reported or
not and the raters principally agreed on the “Not sure” assessment. Domain 6A, which
specifically assesses improvement in the opposite order to the development of symptoms
of the disease, created the greatest difficulty in interpretation. In one case report,
it was actually stated that the symptoms improved in the opposite order, and here
there was consensus among evaluators. Otherwise, there was no consensus. The chronology
of development and improvement in the symptoms should be mentioned explicitly by case-report
authors so that the reader can better assess their sequence of onset.
Pertaining to domain 6B, in three case reports, observance of Hering's Law (which
determines the order of symptom improvement) was mentioned by the original authors,
on which basis the evaluators rated as “Yes” in this domain for its observance, but
on discussion it was found that all three cases had mentioned only one of the three
aspects, whereas at least two are required. Therefore, evaluators agreed that they
were not sure. This finding, together with the greater difficulty of basing a positive
score on varying combinations of two out of three items, leads us to recommend attributing
a positive score to this item if at least one of the three aspects is applicable ([Table 3 ]). Lowering the threshold from “at least two” to “at least one” of these aspects
gives these relatively uncommon but important items a greater “weight”.
A more general conclusion regarding domain 6 is that referring to the direction of
cure is generally neglected in homeopathy case reports. This is a surprising finding
given that the direction of cure is deemed to be important in homeopathy. It would
be useful to look further into the reasons for our findings, and to communicate to
the homeopathy community that this aspect should receive further attention when writing
up and assessing clinical cases. Also, one of the reasons for non-reporting could
be a lack of clear definitions: the terms “from organs of more importance to those
of less importance”, “deeper to more superficial” and “from the top downwards” could
usefully be further defined within domain 6B.
Regarding domain 7, there was no difference of opinion among raters in cases where
an author had clearly mentioned that the old symptoms reappeared, but in other cases
there was difference of opinion, subject to interpretation. Also, the raters agreed
that old symptoms were not reported in most cases. Therefore, the chronological sequence
of old symptoms in which these reappear should be summarized in the case reports to
enable uniformity in assessment.
In domain 8, there should be consideration of known course of disease, other forms
of treatment, other clinically relevant interventions, lifestyle changes, etc. Any
concurrent treatment should be mentioned in the case reports. This is to help further
substantiate the causal relationship between the homeopathic intervention and outcome.
In the cases evaluated, the high percentage agreement in the Naranjo algorithm was
for the response “No” (and an associated score of +1), which is consistent with its
validity in determining that alternative causes were explicitly considered and excluded.
In domain 9, clinical observation is mentioned as an example for any objective evidence,
which may be replaced by investigations or relevant clinical examination or photographs
(for dermatological conditions) or a validated questionnaire (especially for subjective
conditions) for better assessment ([Table 3 ]). It was adequately covered in the cases studied.
Repeat dosing, as referred to in domain 10, was infrequently observed in evaluated
cases and should be further defined. What is understood from this question is that
it should not be assessed based on the repetition of medicine during routine follow-up.
Instead, this item is only applicable when the disease has been absent or under remission
for quite a long time and the disease, with similar symptoms, reappears and is improved
with the previously selected medicine. Only then was the repeat dosing considered
to establish reproducibility. Diseases with relapsing remitting course, for example
multiple sclerosis and rheumatoid arthritis, can be assessed during a new episode
of relapse if the symptoms corroborate with the previous episode by repeat dosing,
rather than by assessing repetition of the medicine at subsequent follow-up visits
during the same disease episode.
There was thus good overall inter-rater agreement in assessing clinical case reports
using the Modified Naranjo Criteria as a tool for attributing the likelihood of a
causal relationship between homeopathic intervention and clinical outcome in quality
case reports. Therefore, except for the questions about direction of cure (domain
6), the reliability and thus the content validity of this Causal Attribution Inventory
were largely established. As found in this study, information to inform domains 3,
7, and 10 is not observed frequently in case reports but, when reported, these domains
are of value in establishing causal attribution. We may not find them in all successfully
treated cases but even their absence should be specified by the original author to
ensure unambiguous case reporting.
Although a thorough review of the international literature took place to identify
the case reports, the sample is likely to be biased toward a sub-set of case reports
predominantly from India. This, in turn, might be attributed to greater public acceptability
and case reporting in that country. The case reports were assessed as per contemporary
HOM-CASE CARE guidelines[10 ] for inclusion in the study. As per evaluation, most domains of these guidelines
were covered to a variable degree across the cases; however, none of the 60 cases
presented information related to intervention adherence, tolerability, or adverse
and unanticipated events. Therefore, it is important to follow these guidelines for
complete reporting and for making each case count in subsequent analysis.
It is a limitation of our study that, being an assessment of case reports that were
published before 2018, most of the cases were written before publication of the CARE
and HOM-CASE guidelines, reducing the likelihood of the reports' compliance with them.
Therefore, the domains assessed as per HOM-CASE CARE were not well represented in
these cases. However, it is expected that in future, when more cases are published
as per the HOM-CASE guidelines, firmer conclusions regarding the likelihood of a causal
relationship between intervention and outcome can be drawn.
Further work on domains 6A and 6B of the Modified Naranjo Criteria is needed with
a view to improving their validity. Describing the pattern of symptom unfolding or
improvement is imprecise, especially for long-standing illness. It relies on the vagaries
of memory, which may be subject to all sorts of contamination. Also, domain 5 brings
up the important point that much of the material here is based not on measurable evidence
or validated scales (such as Quality of Life) but personal recollection and medical
history taking. The concept of aggravation (pertaining to domain 3) is also somewhat
uncertain. What homeopaths attribute to the actions of the remedy is often explained
by their allopathic colleagues as either early and transient side effects of treatment
or a peculiar susceptibility of the patient to certain symptoms/adverse events. As
these domains are related to core areas of homeopathy, further work on defining these
basic principles is imperative to include them optimally for assessing causal relationship.
Conclusion
The Modified Naranjo Criteria for Homeopathy—Causal Attribution Inventory has been
identified as a useful tool for assessing the likelihood of a causal relationship
between homeopathic intervention and clinical outcome. Except for items relating to
“direction of cure” (domains 6A and 6B), the reliability and validity of all other
domains was largely established.
Some improvements to the wording of several domains of the Mo dified Nar anjo C riteria for H omeopathy are proposed: these are presented under the newly introduced acronym, “MONARCH”
([Table 3 ]). Further elaboration of the MONARCH domains 6A and 6B is needed to validate them
as these are related to some basic principles of homeopathy. Also, further validation
of the MONARCH Inventory via a formal assessment of content and construct validity
based on a broad set of clinical case reports is recommended.
The overarching CARE/HOM-CASE guidelines should be followed for standardized and therefore
more thorough case reporting.
