RADAR: A rapid detection tool for signs of delirium (6th vital sign) in emergency departments

Introduction Delirium is a neuropsychiatric syndrome whose core features are acute onset and fluctuating course, plus disturbances in cognition, consciousness and attention (American Psychiatric Association [APA], 2013). It is common among hospitalized older adults and is a sign of serious underlying health problems, such as infections, acute cardiovascular problems, or metabolic disorders. Its prevalence rate in older emergency department (ED) patients ranges between 10% and 14% (Elie, Rousseau, Cole, Primeau, McCusker, & Bellavance, 2000; Han, Wilson, Vasilevskis, et al., 2013; Singler, Thiem, Christ et al., 2014; Hustey, Meldon, Smith & Lex, 2003; Grossmann, Hasemann, Graber, Bingisser, Kressig, & Nickel, 2014), hence the importance of mental health assessment in emergency departments (Terrell, Hustey, Hwang et al., 2009).

To overcome these barriers, researchers developed RADAR (Recognizing Acute Delirium As part of your Routine). The assessment is based on the face-to-face interaction between nurse and patient during the administration of medication. RADAR contains three simple observation-based items (yes-no) that can be scored based on the interaction with the patient. The patient is never questioned directly. Based on more than 500 RADAR administrations by bedside nurses, completion of RADAR takes an average of seven seconds (Voyer, Champoux, Desrosiers et al., 2015;Voyer, Champoux, Desrosiers et al., 2016). When compared with DSM-5 criterion-defined delirium in validation studies, RADAR had a sensitivity of 73% and a specificity of 67% (Voyer, Champoux, Desrosiers et al., 2015), and a sensitivity and specificity of 100% and 77% in acute care settings and long-term care facilities respectively (Bilodeau, & Voyer, 2016). Among all participants with a positive RADAR, 89% to 100% had at least one symptom of delirium and it must be stressed that even one sign of delirium is detrimental to the health of patients (Tieges, McGrath, Hall & MacLullich, 2013;Cole, McCusker, Voyer, Monette, Champoux, Ciampi, Vu, Dyachenko et al., 2013;Cole, Ciampi, Belzile, & Dubuc-Sarrasin, 2013;Cole, 2013;Cole, Bailey, Bonnycastle, et al., 2016;McAvay, Van Ness, Bogardus et al., 2006).
Although RADAR on its own is a valid delirium-screening tool, we agree with the various expert groups that recommended the adoption of a two-step approach to the detection of delirium (Young, Murthy, Westby, Akunne, & O'Mahony, 2010;O'Regan, Ryan, Boland, et al., 2014) and, specifically, in the ED (Improved awareness, 2014). The first step is to screen for the 6th vital sign, which is a measure of the two core signs of delirium: attention and consciousness (Flaherty, Rudolph, Shay et al., 2007;Bellelli, Trabucchi, 2008). The second step is to administer a more comprehensive tool, such as the CAM or 4AT (Bellelli, Morandi, Davis, et al., 2014). By so doing, the process will be quicker and only positive cases from the first step comprehensively assessed. The objective of our study is to validate RADAR in the ED as a measure of the 6th vital sign. These reliability and validation tests were examined: sensitivity, specificity, positive and negative predictive value, and inter-rater agreement.

Study design
This validation study was conducted in the ED of the Hôpital de l'Enfant-Jésus -CHU de Québec, a university-affiliated acute care hospital. Data collection took place from March 2015 to May 2015. The Research Ethics Boards of the CHU de Québec approved the study.

Sample and setting
Patient enrolment -Patients were included in the study if they were aged 65 years or over and consulting at the ED for any medical or surgical health issue. Patients had a minimum eighthour ED stay, and were independent or semi-independent (5/7 Activities of Daily Living without any help). We excluded anyone living in a long-term care facility, with a history of psychiatric illness (specifically psychotic disorders, bipolar disorders and schizophrenia), those with moderate to severe dementia, or with intellectual disabilities, and those with delirium upon arrival or at the end of the first eight hours of ED stay. Also excluded were patients with unstable conditions that could lead to intensive care, and those who were unable to give verbal consent, to attend follow-up evaluations or to communicate in French or English.
A research assistant (RA) identified eligible patients using the Emergency Department Information System patient-tracking software. After appropriate consent, the RA first assessed patients for inclusion and exclusion criteria and then conducted an in-person evaluation of social-clinical, co-morbid, functional and psychological status. The RA used the Confusion Assessment Method (CAM) to confirm absence of delirium at this first encounter. Charts were reviewed and all confounder variables were collected.

measures
RAs were divided into Groups A and B to ensure that the RAs who complete the RADAR are not aware of the result of the CAM. To evaluate the presence of signs of delirium with CAM the RAs from Group A followed up patients with face-to-face interviews twice a day during their entire ED stay and over a 24-hour period on hospital ward. RAs from Group B were student nurses who scored RADAR by observing the patients during the distribution of medication (8 am, 12 noon, 5 pm, 8 pm) by ED nurses. If a patient did not take a medication at a specific hour, then RADAR was administered around the same time and was based on the interaction during other clinical activities (e.g., clinical assessment, measure of vital signs, blood sample, any treatments).
Primary measures -A positive 6th vital sign is defined as the presence of both an altered level of consciousness and inattention over the course of the day (Voyer, Champoux, Desrosiers et al., 2016;Bellelli & Trabucchi, 2008;Flaherty, Shay, Weir, et al., 2009;Flaherty, Rudolph, Shay et al., 2007). The presence of these delirium symptoms was measured with the CAM (Wei, Fearing, Sternberg & Inouye, 2008;Inouye, van Dyck, Alessi, Balkin, Siegal, & Horwitz, 1990). RADAR was validated in acute (medicine, cardiology, coronary intensive care unit) and long-term settings (Voyer, Champoux, Desrosiers et al., 2015;Voyer, Champoux, Desrosiers et al., 2016;Bilodeau & Voyer, 2016). It consists of three items: "When you gave the patient his/her medication… 1) Was the patient drowsy?; 2) Did the patient have trouble following your instructions?; 3) Were the patient's movements slowed down? A RADAR screening is considered positive when at least one item is checked "Yes". During the course of the study, the RAs assessed a sample of patients simultaneously and independently to test RADAR's inter-rater reliability. Training on how to use the RADAR was based on a 25-minute video.
Instruments for descriptive measures -Data extracted from the participants' medical charts included: demographic information (e.g., age, sex, education, and living arrangement), diagnosis of dementia, and other medical diagnoses. Information on medical problems was used to compute the Charlson Comorbidity Index (Charlson, Pompei, Ales, & MacKenzie, 1987), validated as a predictive index for survival among older adults (Bravo, Dubois, Hebert, De Wals, & Messier 2002;Buntinx, et al., 2002). Patients' functional status was measured using the Older American Adult Resources and Services (OARS) (Fillenbaum, & Smyer, 1981;McCusker, Bellavance, Cardin, Trepanier, Verdon, & Ardman, 1999 Statistical analysis -A RADAR screening was deemed positive if at least one of the three items was checked as present at one point in time during the study. We evaluated inter-rater reliability between the two RAs for each RADAR item; simultaneous assessments were conducted in 26.8% of all RADAR administrations. We then quantified the inter-rater agreement using the kappa statistic in conjunction with raw agreement percentages. Second, we computed sensitivities (SE), specificities (SP), positive and negative predictive values (PPV and NPV respectively) and their confidence intervals for RADAR, as a measure of the 6th vital sign. All analyses were carried out using SAS for Windows, version 9.3. [Émond et al., 2017]), funded by the Fonds de Recherche du Québec-Santé (FRQ-S). The INDEED study included 111 (68.5%) patients. Of these, 54 (48.6%) were not evaluated because RAs from Group B were not available, which left a sample of 57 patients for analysis. Table 1 shows the characteristics of these patients. The mean age of participants was 74.3 years, and they had a mean Charlson Comorbidity Index score of 1.8.

Description of the study population -This project was part of a larger multicentre project, the INDEED study (INcidence and impact measurement of DElirium induced by ED stay
A total of 256 RADAR were administered to these 57 patients during their stay in the ED, and each patient was assessed many times using the RADAR (4.5 ± 2.7 times). RADAR was positive for seven patients in our cohort (12.3%). Due to the fluctuating nature of delirium, patients with ≥ 4 RADAR assessments had more positive RADAR (6 of 7 patients, 85.7%).
Inter-rater agreement -To test RADAR's inter-rater reliability, the RAs assessed a sample of patients (n = 19/57; 33.3% of total sample) simultaneously and independently during the study. Percentage of agreement between the RAs was 89% and kappa value was 0.46 (CI 0.14-1.00), which indicates good inter-observer reliability.

Discussion
Delirium is often missed in the ED and this can cause severe consequences for the patients. Moreover, when delirium is undetected in the ED, it is often missed during the remainder of the patient's hospital stay. For this reason, researchers (Young, Murthy, Westby, Akunne, & O'Mahony, 2010;O'Regan, Ryan, Boland, et al., 2014) have suggested approaching the detection of delirium with a two-step process. The first step must be very fast and easy for nursing staff to implement. Therefore, we tested the validity of the RADAR for the detection of the 6th vital sign in the ED. This tool had previously been validated in other settings where it was well accepted by the nursing staff because of its fast administration time.  Given its 100% sensitivity, the RADAR screening tool appears to be a valid measure of the 6th vital sign. The negative impact of the presence in elderly patients of the mental health symptoms of the 6th vital sign is well known from earlier studies. Indeed, Cole, McCusker, Voyer, Monette, Champoux, Ciampi, Belzile, and Vu's study (2013) showed that, when compared to patients without delirium symptoms over a six-month period, the presence of only one symptom of delirium is associated with cognitive and functional impairments, as well as mood and behaviour problems. Similarly, Shim, DePalma, Sands and Leung (2015) found that, among elderly patients who underwent non-cardiac surgery, patients with at least one delirium symptom on the first day of surgery had a longer hospital stay and a functional decline one-month after surgery. Li, Chen, Chiu, Fu, Huang, and Chen (2015) showed similar results among patients who underwent cardiac surgery, where patients with sub-syndrome delirium [SSD] (one symptom of delirium) fell into an intermediate zone between non-delirious patients and delirious patients, for length of stay and cognitive problems. Finally, a study by Cole et al. (2016) showed that in a cohort of hospitalized elderly patients with SSD not developing into a full-blown delirium, 29% of patients still had SSD symptoms three months after onset. In view of these poor outcomes related to inattention or altered level of consciousness, a RADAR sensitivity of 100% for a positive 6th vital sign is clinically relevant (Shi, Warren, Saposnik, & Macdermid, 2013). These assertions also support the clinical importance of a RADAR+ even when not related to a positive 6th vital sign. In this study one patient was positive for the 6th vital sign, but RADAR was positive for six other cases and all of them showed signs of delirium.
In this study, we demonstrated that RAs (student nurses) with a 25-minute video training were able to use RADAR adequately (Kappa of 0.46). We know from previous studies that RADAR is a quick seven-second tool employing observations only. Given the fluctuating nature of delirium symptoms, this tool can, and must be used many times a day in order to better identify symptoms of delirium. However, these assessments should not be a burden for patients, and the patients should not improve their scores by learning the right answers (Voyer, Champoux, Desrosiers, 2015). Since RADAR is based solely on observation, these issues are unlikely to arise, and so multiple administrations are possible. This is clinically important since it was demonstrated that a number of participants who were not initially identified by RADAR as having signs of delirium, were found positive when tested again three hours later. This finding shows the obvious benefit of screening for signs of delirium at multiple points during a patient's ED visit to maximize delirium identification (LaMantia, Messina, Hobgood, & Miller, 2014), bearing in mind that the multiple screening process must be feasible for nursing staff.

Study limitations
The results of this study are based on a small sample size with only one case with a positive 6th vital sign, thus limiting their generalizability. Moreover, the number of research staff limited the capacity to include all eligible patients in this study, and may also have introduced a selection bias. Research staff (student nurses) administered RADAR and consequently another study should be conducted to test if sensitivity and specificity would be similar were bedside nurses to use the RADAR in their daily practice (as was the case in all other studies on RADAR). Nevertheless, this study has certain strengths. Attention and level of consciousness were measured using the CAM, an instrument well-recognized for its psychometric properties. Second, ratings of these signs of delirium were based on a seven-hour NPV: 100% (95% CI: 92.9 -100) CI = confidence interval observation period, a cognitive assessment and additional sources of information (staff, family members, medical chart review and a baseline cognitive assessment). Third, in order to minimize contamination bias, the RAs responsible for administering RADAR were blinded to the CAM results.

Conclusions
RADAR was developed for the detection of signs of delirium and is a brief screening tool that appears to be valid for the 6th vital sign. RADAR is certainly a tool worth considering as a first step in the detection of delirium process among ED patients. Finally, only a 25-minute training session is needed to be able to properly use the RADAR (video freely accessible at http:// radar.fsi.ulaval.ca/).