INTRODUCTION

Delirium is a common and a potentially life-threatening neuropsychiatric disorder. Delirium is a disturbance in consciousness with the cardinal and characterized disturbance of reduced ability to direct, focus, sustain, and shift attention. Inattention is accompanied by other cognitive deficits including of orientation, executive ability, language, visuo-spatial ability, learning and memory, as well as abnormalities of sleep-wake cycle, motor activity, affective control, higher level thought and perception. The onset of delirium may be acute or subacute. These symptoms are usually reversible and their severity fluctuates during the course of the day (1). Delirious patients, in particular the elderly, have worse longer-term outcomes following an episode of delirium (2). Recent research has delineated and validated three core domains for delirium symptoms reflecting cognitive, circadian and higher level thinking symptom clusters (3).

Advanced age is the most important factor for delirium risk, attributed to structural and neurodegenerative changes, reduced neurochemical flexibility, and oxidative stress associated with normal and/or abnormal aging. Preexisting cognitive impairment including dementia, comorbid medical conditions (e.g., cardiovascular, infectious, metabolic, major surgery) also increase the risk of delirium occurrence (4). The incidence and prevalence of delirium depends on the population examined and affects 10-30% of hospitalized patients (2,5), and at even higher rates in intensive care settings.

There are several tools used in research and clinical practice to assess delirium severity. The most widely used, well-validated scales are the Mini Mental State Examination (MMSE) (6), Delirium Rating Scale (DRS) (7), Memorial Delirium Assessment Scale (MDAS) (8) and Delirium Rating Scale–Revised 98 (DRS-R-98) (9). The MMSE is a 30-item bedside clinician-administered cognitive test that includes measurement of orientation, memory, comprehension, visuo-construction, and concentration (6). However, it measures only cognition and is not specific to a particular diagnosis (4). The DRS is a 10-item clinician-rated scale developed by Trzepacz et al. (7) in 1988 and was the first rating scale specifically designed to assess delirium phenomenology at a time when cognitive-only tools were used. It was intended for use in conjunction with a cognitive scale and therefore has only one overall cognitive disturbance item. Its items are anchored by descriptions of characteristics intended to differentiate delirium from other neuropsychiatric disorders as well as to quantitate delirium severity. It significantly distinguished delirium from dementia and other neuropsychiatric disorders. The MDAS is a 10-item clinician-rated scale to assess delirium severity with each item rated on a Likert scale but without phenomenological descriptions to anchor ratings (8). It is designed to allow for repeated administration within 24-hour period but not for delirium diagnosis.

The DRS-R-98 is a major overhaul of the DRS in structure and content and has become an essential tool in delirium assessment and research. Timmers et al. (10) claimed that DRS-R-98 is the best overall of delirium rating tool largely because of its range of symptoms and suitability for use by physicians and research assistants after training as do Kean and Ryan (11). The DRS-R-98 is available in a number of languages including Spanish, Japanese, Korean, Portuguese, Greek, Danish, Dutch, German, French, Lithuanian, Norwegian, Italian, Turkish, Hebrew, and both traditional and modern Chinese (4,10,11). The DRS-R-98 has been revalidated in several countries (12-18).

The purpose of this study is to evaluate the reliability and validity of the Turkish version of the DRS-R-98.

METHODS

Sample

This study was conducted in inpatient settings of Gulhane Military Faculty of Medicine, Ankara, during the calendar year 2010. Delirium patients were recruited consecutively from the Psychiatry Consultation-Liaison department whose research psychiatrist diagnosed delirium using DSM-IV diagnostic criteria. Patients who constituted the non-delirium group were recruited from the cardiovascular surgery department where patients are routinely screened and evaluated for delirium symptoms by the research psychiatrists after surgery. Research psychiatrists diagnosed delirium using DSM-IV diagnostic criteria and rated the DRS-R-98 for both groups. Delirium and non-delirium groups each had 30 patients, and included comorbid dementia patients determined by both previous diagnosis and DSM-IV criteria.

Ankara Ethics Committee Number 7 approved this study, and no financial support was taken from any source.

Measures

Mini Mental State Examination (MMSE): MMSE is a widely used screening test for cognitive impairment, which assesses orientation, attention, visuo-spatial ability, comprehension and memory including short-term recall. It was reported that serial assessment with the MMSE predicts delirium with 93% sensitivity and 90% specificity though it also detects dementia and other etiologies of cognitive impairment (19). The MMSE was validated in a Turkish population (20). The Turkish translation of the MMSE was administered independently from (by the other research psychiatrist prior to the DRS-R-98 rating) but on the same day as the DRS-R-98. Raters were alternated for MMSE and DRS-R-98. Its scores were not used to rate the DRS-R-98-T. MMSE scores were used to assess concurrent validity of DRS-R-98-T. MMSE was administered by one of the research psychiatrist before rating the DRS-R-98, performing psychiatrist was changed for the other patent evaluation.

Delirium Rating Scale Revised-98 (DRS-R-98): The DRS-R-98 is a 16-item clinician-rated scale usually rated over a 24-hour period for diagnosis. The Severity scale has 13 items to rate delirium severity and can be repeated in a chosen time frame. Items cover a broad range of cognitive and non-cognitive symptoms characteristic of delirium, each anchored by specific phenomenological descriptions for each level (0-3 points). Three diagnostic items that assess temporal course and etiological attribution were designed to differentially diagnose delirium from other disturbances such that the total scale can be used as a diagnostic instrument. The original DRS-R-98 had high validity, reliability, internal consistency, specificity, sensitivity, and differential diagnostic ability (9). A Cognitive subscale (items 9-13) and Neurobehavioral subscale (items 1-8) have been reported elsewhere (21) but not validated. Core domain items were analyzed in this report as a possible subscale (items 1, 5-13). After permission was granted by the copyright owner (PTT), the DRS-R-98 was translated into Turkish by M.A.C. and back-translated by an independent translator. The final version of the Turkish version of the DRS-R-98 was reviewed and determined by A.O.

Patients were evaluated between 08:00-10:00 AM and ratings of the DRS-R-98 scale items covered the preceding 24 hour period. The raters could choose their own standardized examination methods to rate memory, attention, and visuospatial ability items. Two researchers (M.A.C. and K.N.O.) visited all the patients together and rated the DRS-R-98-T independently to assess interrater reliability. One rater examined the patient while the other listened. After one rater completed his examination and left the ward, the other one performed any additional examination needed to complete his own scoring.

Statistical Analysis

Statistical analyses were performed using a commercially available statistical software packages (StatsDirect Ltd. StatsDirect statistical software. http://www.statsdirect.com. England: StatsDirect Ltd. 2008.: MedCalc Version 11.6.1© 1993-2011 MedCalc Software bvba MedCalc Software, Broekstraat 52, 9030 Mariakerke, Belgium). Descriptive features were described as mean±standard deviation (SD) values. Student’s t test, chi-Square test, and Fisher’s Exact test were used to analyze descriptive data. Data comparisons between the groups were made using Mann-Whitney-U test. The DRS-R-98-T was compared to MMSE and DSM diagnosis. Concurrent validity to the MMSE was assessed using the Pearson correlation test. MMSE and DRS-R-98T scores recorded by researcher #1 were used for the analyses. To assess interrater reliability for DRS-R-98 total and severity scores, intraclass correlation coefficients were calculated. Singular value decomposition (SVD), explained in detailed elsewhere, values were calculated for inter-battery factor analysis (22). Internal consistency was evaluated using Cronbach’s alpha. DRS-R-98-T Total and Severity scales were compared to DSM-IV delirium categorization using receiver operation characteristic (ROC) analyses to produce sensitivity and specificity at various scale cut-off values.

RESULTS

A total of 60 patients were recruited in the study, 30 in each group. The mean age was 70.2±15.38 for delirium and 62.8±12.63 for non-delirium groups with ranges encompassing young adults to the “old old” (Table 1). There were 16.6% (n=5) dementia patients in the delirium group and 10% (n=3) in the non-delirium group. MMSE scores were significantly different between groups (p<0.001, 95% CI=-13 to -9).

Median and minimum-maximum values for DRS-R-98 severity items and scale and subscale scores are listed in Table 2. All DRS-R-98 item, scale and subscale scores were significantly different between groups. The highest mean item scores on the DRS-R-98 in the delirium group were for sleep-wake cycle, lability of affect, thought process and all cognitive items except for short-term memory.

Figure 1 shows boxplot distributions of scores comparing groups (median and quartiles) for DRS-R-98 total and severity scales and MMSE scores. There was a significant difference between the groups for DRS-R-98 total and severity, and MMSE scores (significance is at p<0.001 level for all). Internal consistency was high - Cronbach’s alpha coefficient for the DRS-R-98 total was 0.96 and for the DRS-R-98 severity was 0.86 (Table 2). Individual items’ alpha coefficients ranged from 0.95-0.96 for the DRS-R-98 total and 0.81-0.86 for the DRS-R-98 severity.

The ICC for interrater reliability was high for both DRS-R-98 total and severity with 0.99 (SVD values were 203.74 and 282.98, respectively). Such reliability was also observed for subscales: 0.99 for the core domain and cognitive subscales (SVD values were 42.77 and 14.53, respectively), and 0.98 for neurobehavioral subscale (SVD value was 32.40).

Concurrent validity of the DRS-R-98 was assessed by comparison with MMSE to the extent that cognitive impairment is one component of delirium. Pearson correlation was high between DRS-R-98 total and severity scores and MMSE scores in the delirium group (r=-0.86 p<0.001 for each, n=30).

ROC analysis for DRS-R-98 scores comparing delirium and non-delirium groups revealed sensitivity and specificity at various cut-off values for DRS-R-98 total and severity scores are shown in Figure 2 and Table 4. The best cut-off score was 16 for the DRS-R-98 total score with 100% sensitivity (95% CI=86.2 to 100) and 88.5% specificity (95% CI=73.2 to 96.7), and 11 points for the DRS-R-98 severity score with 100% sensitivity (95% CI=86.2 to 100) and 85.7% specificity (95% CI=73.2 to 96.7).

DISCUSSION

Recognizing that delirium deserves much more attention in clinical practice, use of highly validated psychometric assessment tools will help detect and monitor delirium better. The DRS-R-98 is a highly validated and internationally recognized delirium assessment tool. We conducted a validation and reliability study of its Turkish version, which adds to the literature of its revalidations in different countries.

Interrater reliability, internal consistency, and construct validity of the DRS-R-98 total and severity scores were very high. The cut-off score from ROC analyses for the DRS-R-98 is compatible with the original DRS-R-98 score, which at 17.75 had 92% sensitivity and 95% specificity. Specificity and sensitivity values are also comparable with other translations of DRS-R-98. Cut-off scores have varied a little across various translations when revalidated in different countries (12-18), though they represent a tight range of scores taken together, and may be reflect the specific cohort evaluated in those studies. We acknowledge that selecting a cut-off score of 11 for the DRS-R-98 severity score improves sensitivity but realize that using 13 as a cut-off score may provide better balance between sensitivity and specificity and where the LR is the highest.

We also report subscale analyses for cognitive and neurobehavioral items and a subscale reflected by items on DRS-R-98 analyses that represent core domain items (4,23-25).

This information about the psychometrics of the Turkish DRS-R-98 will enable choice of tool selection for Turkish speaking populations. There is already a validated Turkish delirium severity assessment scale, the Delirium Rating Scale, developed by Aydemir et al. (26,27). This scale consists 10 items focusing on symptom severity. Consolidation of different psychomotor activities in one item limits its usefulness in differentiating motor subtypes of delirium, as in the DRS developed by Trzepacz et al. (7,26). Specificity and sensitivity values are higher than the scale developed by Aydemir et al. (26). We did not include any patient controls in the study that may contribute positively on specificity and sensitivity values. But, effect of this limitation on specificity and sensitivity values are not significant when compared other translations of DRS-R-98.

Limitations of this study include use of only the MMSE for construct validity without another delirium rating scale, though its correlation value was higher than in other DRS-R-98 validation reports. Also, allowing comorbid dementia patients was a limitation, though the number was small and about the same between groups, and may reflect cases in routine clinical care. While we did not evaluate longitudinal ratings, the ability of the DRS-R-98 to be sensitive to change has been shown by others (28,29). Small number of the patients included in the study may be considered as another limitation of the study. However, finding delirium cases is not easy to find and the time limit of the study prevent us to include more patients in the study.

While adequate training and clinical experience are key factors for producing reliable symptom ratings in delirium, we have discovered that nurses and clinical psychologists can easily administer the DRS-R-98 after a brief educational and clinical training experience. The DRS-R-98 Administration Guide is a resource that can assist in training for symptom assessment (obtained from PTT@lilly.com). In summary, we demonstrated that the Turkish version of DRS-R-98 is a highly reliable and valid instrument to measure delirium severity and diagnose delirium in Turkish medical settings.

REFERENCES

1. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. Fourth ed. Washington DC: American Psychiatric Publishing, Washington DC, 1994.

2. Francis J, Martin D, Kapoor WN. A prospective study of delirium in hospitalized elderly. JAMA 1990; 263:1097-1101.

3. Franco JG, Trzepacz PT, Meagher DJ, Kean J, Lee Y, Kim JL, Kishi Y, Furlanetto LM, Negreiros D, Huang MC, Chen CH, Leonard M, de Pablo J. Three core domains of delirium validated using exploratory and confirmatory factor analyses. Psychosomatics 2013; 54:227-238.

4. Trzepacz PT, Meagher DJ, Leonard M. Delirium: In Levenson J (editor). American Psychiatric Publishing Textbook of Psychosomatic Medicine. Second Ed. Washington DC: American Psychiatric Publishing Inc., 2011, 445-517.

5. Van der Mast RC, Roest FH. Delirium after cardiac surgery: a critical review. J Psychosom Res 1996; 41:13-30.

6. Folstein MF, Folstein SE, McHugh PR. Mini-mental state. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975; 12:189-198.

7. Trzepacz PT, Baker RW, Greenhouse J. A symptom rating scale for delirium. Psychiatry Res 1988; 23:89-97.

8. Breitbart W, Rosenfeld B, Roth A, Smith MJ, Cohen K, Passik S. The Memorial Delirium Assessment Scale. J Pain Symptom Manage 1997; 13:128-137.

9. Trzepacz PT, Mittal D, Torres R, Kanary K, Norton J, Jimerson N. Validation of the Delirium Rating Scale-Revised-98: comparison with the delirium rating scale and the cognitive test for delirium. J Neuropsychiatry Clin Neurosci 2001; 13:229-242.

10. Timmers JFM, Kalisvaart KJ, Schuurmans M. A Review of Assessment Scales for Delirium: In Kalisvaart CJ (editor). Primary Prevention of Delirium in the Elderly. Amsterdam: Academic Proefschrift, University of Amsterdam. 2005, 21-39.

11. Kean J, Ryan K. Delirium detection in clinical practice and research: critique of current tools and suggestions for future development. J Psychosom Res 2008; 65:255-259.

12. Fonseca F, Bulbena A, Navarrete R, Aragay N, Capo M, Lobo A, Trzepacz PT. Spanish version of the Delirium Rating Scale-Revised 98: reliability and validity. J Psychosom Res 2005; 59:147-151.

13. Lim KO, Kim SY, Lee YH, Lee SW, Kim JL. A validation study of the Korean version of Delirium Rating Scale–Revised–98 (K-DRS-98). J Korean Neuropsychiatr Assoc 2006; 45:518-526.

14. Kato M, Kishi Y, Okuyama T, Trzepacz PT, Hosaka T. Japanese version of the Delirium Rating Scale-Revised 98:reliability and validity. Psychosomatics 2010; 51:425-431.

15. de Rooij SE, van Munster BC, Korevaar JC, Casteelen G, Schuurmans MJ, van der Mast RC, Levi M. Delirium subtype identification and the validation of the Delirium Rating Scale–Revised 98 (Dutch version) in hospitalized elderly patients. Int J Geriatr Psychiatry 2006; 21:876-862.

16. Franco JG, Mejía MA, Ochoa SB, Ramírez LF, Bulbena A, Trzepacz PT. Delirium rating scale-revised-98 (DRS-R-98): Colombian adaptation of the Spanish version. Actas Esp Psiquiatr 2007; 35:170-175.

17. De Negreiros DP, da Silva Meleiro AM, Furlanetto LM, Trzepacz PT. Portuguese version of the Delirium Rating Scale-Revised-98: reliability and validity. Int J Geriatr Psychiatry 2008; 23:472-477.

18. Kato M, Kishi Y, Okuyama T, Trzepacz PT, Hosaka T. Japanese version of the Delirium Rating Scale, Revised-98 (DRS-R-98-J): reliability and validity. Psychosomatics 2010; 51:425-431.

19. O’Keeffe ST, Mulkerrin EC, Nayeem K, Varughese M, Pillay I. Use of serial Mini-Mental State Examinations to diagnose and monitor delirium in elderly hospital patients. J Am Geriatr Soc 2005; 53:867-870.

20. Gungen C, Ertan T, Eker E, Yasar R, Engin F. Reliability and validity of the standardized Mini Mental State Examination in the diagnosis of mild dementia in Turkish population. Turk Psikiyatri Derg 2002; 13:273-281. (Turkish)

21. Meagher DJ, Moran M, Raju B, Gibbons D, Donnelly S, Saunders J, Trzepacz PT. Phenomenology of delirium. Assessment of 100 adult cases using standardized measures. Br J Psychiatry 2007; 190:135-141.

22. Cherry S. Singular value decomposition analysis and canonical correlation analysis. J Clim 1996; 9:2003-2009.

23. Franco JG, Trzepacz PT, Mejía MA, Ochoa SB. Factor analysis of the Columbian translation of the Delirium Rating Scale Revised-98. Psychosomatics 2009; 50:255-262.

24. Meagher DJ, Leonard M, Donnelly S, Conroy M, Saunders J, Trzepacz PT. A comparison of neuropsychiatric and cognitive profiles in delirium, dementia, comorbid delirium-dementia and cognitively intact controls. J Neurol Neurosurg Psychiatry 2010; 81:876-881.

25. Kean J, Trzepacz PT, Murray LL, Abell M, Trexler LE. The accuracy and validity of a new provisional diagnostic tool for delirium in acquired brain injury. Brain Inj 2010; 24:1222-1230.

26. Aydemir O, Noyan A, Gulseren S, Kayahan B, Bodur Z, Elbi E, Kultur S. Development of delirium rating scale, reliability and validity. 3P 1998; 6:21-27. (Turkish)

27. Ok G, Aydemir O, Tok D, Erbuyun K, Turan E. New Delirium Rating Scale for ICU. Archives of Neuropsychiatry 2010; 47:36-39.

28. Meagher D, Adamis D, Trzepacz PT. Leonard M. Features of subsyndromal and persistent delirium. Br J Psychiatry 2012; 200:37-44.

29. Meagher DJ, Leonard M, Donnelly S, Conroy M, Adamis D, Trzepacz PT. A longitudinal study of motor subtypes in delirium: relationship with other phenomenology, etiology, medication exposure and prognosis. J Psychosom Res 2011; 71:395-403.