Assessing Anxiety and Depression in General Populations

¶ … Test Development

This research is a mixed methods study designed to explore the perceptions of self-identifying individuals with anxiety and depression regarding any relation between their conditions and their ability to access appropriate healthcare under the Patient Protection and Affordable Care Act. Five respondents completed the questionnaire constructed explicitly for this research study. A review of the literature serves as a canvas of instruments also developed for assessing Axis 1 disorders as defined by the Diagnostic and Statistical Manual (DSM-5). The research on instrumentation included the following: 1) The SCID, 2) the Composite International Diagnostic Interview (CIDI), 3) the Beck Depression Inventory (BDI), 4) the Center for Epidemiologic Studies of Depression Scale (CES-D), and 5) Severity Measure for Generalized Anxiety Disorder -- Adult (an emerging online measure provided in association with the DSM-5).

Their responses negate the theoretical construct, however, an insufficient number of respondents in this pilot study meant that the hypothesis could not be tested. Recommendations for future research include selecting only respondents with diagnoses obtained from certified psychotherapists or clinical psychologists, or board certified psychiatrists.

Rationale/Problem Statement:

The construct of interest is whether co-morbid anxiety and depression are exacerbated by concerns about the availability of healthcare and potential treatments for the condition. Uncertainty about life events or access to services can contribute to increased anxiety. Difficulty accomplishing daily tasks is common for people suffering from depression. Taken together, the two conditions can present barriers that make it difficult for clients or patients to stay on top of their disordered conditions. To one degree or another, changes are challenging to nearly everyone. The construct of interest is the relation between co-morbid anxiety and depression and the prospect of difficulty obtaining appropriate care. The construct assumes that the patients or clients will anticipate difficulties even when they do not have direct experience with any particular healthcare related barrier.

The instrumentation developed for this test development proposal essentially functions as a screening device which relies on self-report and, as such is used to gauge the perceptions of the patients or clients with regard to their actual difficulties or their perceptions regarding anticipated difficulties accessing and obtaining adequate healthcare for their anxiety and depression. It is important to recognize that the proposed instrument does not actually measure behaviors or actions. This factor makes it all the more important for the instrument to demonstrate solid reliability and validity, and -- optimally -- to show correlation with instrumentation that has undergone rigorous evidence-based trials or research recognized by the American Psychological Association or other certifying body.

Assumptions

Explicitly, the focus of the proposed research is to determine whether anxiety and/or depression is further exacerbated by the fact that the Patient Protection and Affordable Care Act (PPACA) has been enacted and by worries that treatment for anxiety and depression will be as readily accessible as it was before the Patient Protection and Affordable Care Act was enacted. The Patient Protection and Affordable Care Act passed in the Senate on December 24, 2009, passed in the House on March 21, 2010, and was signed into law by President Barack Obama on March 23, 2010. Moreover, the Patient Protection and Affordable Care Act was upheld in the Supreme Court on June 28, 2012. Publicity about the law has been excessive, partisan spin regarding the possible ramifications of the law have been exaggerated, and the every hiccup expected of a new federal program of such scope has been treated as an authoritative proof that the law is ill-conceived and poorly administered. However, as the Patient Protection and Affordable Care Act has been implemented over the past year, the media and a number of well-funded key political groups have been shown to have engaged in hyperbole.

Regardless of the progress made during implementation and the evidence running contrary to the proclamations of dire effects, any patient or client who pays attention to the media will have heard, read, or seen stories predicting enormous problems and difficulties for those who must rely on the new healthcare law for their care. For patients or clients suffering from conditions characterized by anxiety and depression, this scenario has the potential to exacerbate the symptoms resident in the conditions, as described by the DSM-5.

Limitations

It is important to note that current professional practice in the assessment of Axis 1 disorders of the DSM-5 distinguishes between patients who have received a diagnosis of depression and/or general anxiety from those clients who have not been so diagnosed.

In a remarkable break with conventional practice, in Section III of the DSM-5, an offering of a body of instruments that have been termed emerging measures. The purpose of providing these new instruments in this manner is to further the research and clinical evaluation of the measures ("APA," 2013). The intention is for these instruments to be administered by clinicians and researchers at initial interviews with clients, and then used to monitor client progress ("APA," 2013). It is important to recognize that the emphasis is on the initial symptomatic status -- as reported by the client -- and on the patient reported outcome (PRO) information ("APA," 2013). Moreover, the emerging measures are designed for use in conjunction with conventionally anchored severity assessment instruments ("APA," 2013). The DSM-5 includes the instructions, scoring information, and interpretation guidelines that clinicians or researchers need to access in order to use the emerging measures in their practice or research ("APA," 2013).

In future iterations of this research, it may be that the study participants will not self-identify, but will need to have received an official diagnosis of anxiety or depression. This would be consistent with earlier research in anxiety, depression, and -- for instance -- post traumatic stress disorder (PTSD). At present, the research instrument requests that respondents: 1) Self-identify as depressed and anxious patients or clients; and 2) self-assess the severity of their symptoms in relation to the perceived threats to adequate healthcare in the near and far future.

Research Question

The research question for this study is as follows: Is either anxiety or depression, (or in the case of co-morbid anxiety and depression) or anxiety and depression exacerbated in patients or clients when considering the near-term and far-term healthcare under the Patient Protection and Affordable Care Act (PPACA)?

The hypotheses derived from this research question include:

HO: Clients who self-identify as having clinically significant depression and anxiety experience exacerbated symptoms associated with their disorders when they think about the PPACA.

HA: Clients who self-identify as having clinically significant depression and anxiety experience no difference in the symptoms associated with their disorders when they think about the PPACA.

Literature Review

The purpose of the literature review is to provide a thorough explanation of the construct of co-morbid anxiety and depression and, further, to review a number of instruments used to measure the construct that have been validated by the psychometric, psychiatric, and psychotherapeutic industries. The validated instruments serve as a benchmark of sorts against which the proposed instrument can be evaluated. Specifically, the literature review enabled a review that extends to the validity, reliability, and utility of the proposed measure. To that end, the strengths and weaknesses of the proposed measure are discussed, and the difference between the proposed measure and the established instrumentation are considered, including any justification for the development of the new instrument.

The industry validated instruments for assessing general anxiety and depression are introduced in this paragraph and reviewed in detail in the sections for each instrument that follow. The SCID is a semi-structured interview for making the major Axis I DSM-5 diagnoses. It is administered by a clinician and includes an introductory overview followed by nine modules, seven of which represent the major axis I diagnostic classes. Because of its modular construction, it can by adapted for use in studies in which particular diagnoses (e.g., depression only) are of interest. The Composite International Diagnostic Interview (CIDI) is a structured diagnostic interview designed to be used by trained interviewers who are not clinicians (Kessler et al., 1994). The Beck Depression Inventory (BDI) is a list of 21 symptoms and attitudes that are each rated in intensity. This instrument has been used to measure severity of depression in depressed samples but has also been used to assess depression in general population samples. It is also associated with other self-report measures of depression. The Center for Epidemiologic Studies of Depression Scale (CES-D) was designed to measure current level of depressive symptomatology, and especially depressive affect. Any one of the above instruments could be used in conjunction with the instrument designed for the current study. Correlation between the scores on different tests would help to verify the presence and severity of depressive symptoms. The same type of instrument could be used for verifying the presence of anxiety disorder.

1. SCID has undergone a revision in order to correspond to the DSM-5. The new version, called SCID-5, is currently in review by American Psychiatric Publishing, Inc. (APPI), the sole distributor of the SCID-5. A research version is expected to be released in the spring of 2015. A substantial number of reliability studies have been conducted on the SCID-I, the earlier version of the SCID-5. Reliability is reported for categorical constructs that correspond with the DSM-IV diagnoses that are assessed by the SCID.

Table Summarizing Selected SCID-I Reliability Studies

Reference (See Below)

Skre et al., 1991

Zanarini et al., 2000

Zanarini et al., 2000

Segal et al., 1995

Williams et al., 1992

Zanarini et al., 2001

Zanarini et al., 2001

Lobbestael et al., 2010

Population Studied

N=54

N=27

N=52

N=40

N=592;

N=45

N=30

N=151

Version of SCID

DSM-IIIR

DSM-IIIR

DSM-IIIR

DSM-IIIR

DSM-IIIR

Design of Reliability Study

Joint;

Audio-Tape

Joint;

84 Rater-Pairs from 4 sites

7-10 Day Interval Test-Retest

Joint;

Audio-Tape

1-3-Week Interval Test-Retest

Joint;

Observed Live

7-10 Day Interval Test-Retest

Joint; Audio-Tape

Major Depressive Disorder

.93

.80

.61

.90

.64

.90

.73

0.66

Dysthymic Disorder

.88

.76

.35

.53

.40

.91

.60

.81

According to convention, reliability is reported by using the Kappa statistic, which corrects for chance agreement, with any Kappa value above .70 reflecting that the agreement is good. Accordingly, Kappa values in the range from .50 to .70 indicate fair agreement, and Kappa values below .50 show poor agreement. Many of the factors that can influence reliability in an interview instrument are beyond the scope of this paper, but for purposes of reference the primary factors are as follows: Interviewer training; type of administration (e.g., joint interviews vs. A test / retest design); specific population; and base rate. Note that, for statistical reasons, it is more difficult to obtain strong reliability for a rare diagnosis of a disorder than for a common diagnosis. What this means in practical terms is that the reliability of SCID for Major Depressive Disorder will be higher, for instance, in a clinic that treats mood disorders than in a community sample where the base rate of Major Depressive Disorder will be much lower.

The validity of a technique for diagnostic assessment is measured by comparing the technique to a hypothetical, elusive gold standard. In an unavoidable tautological situation, the SCID has been used as the gold standard in a number of studies. Another flawed standard is the best estimate diagnosis, which has been operationalized in a format termed the LEAD standard. To achieve a measure of validity, a longitudinal assessment (L) is conducted by expert (E) diagnosticians who use all data (AD) available about the subjects of the study. The LEAD standard is conceptually attractive, but it is difficult to implement. Other researchers have created and administered versions of the LEAD standard, which have proven to demonstrate validity superior to the SCID over standard intake clinical episodes.

2) World Health Organization's Composite International Diagnostic Interview (CIDI)

The CIDI uses a multi-step procedure that produces short form (CIDI-SF) screening scales for eight disorders from the DSM and the U.S. National Comorbidity Survey (NCS). Members of a respondent subsample selected the CIDI stem question for each disorder, and then a series of stepwise regression analyses was employed to select a subset of screening questions. The idea was to create a reproduction of the full CIDI diagnosis. Indeed, three to eight screening questions for each disorder were sufficient to account for the significant relationships between the CIDI diagnosis and the symptom ratings of the screening tool. What this means in practical terms is that the summary scales can be used to classify clients at a rate between 77 to 100% of the CIDI cases and between 94 to 99% of the CIDI non-cases in the NCS, depending on the diagnosis. For an overall classification, an accuracy rate can be expected of 93% (the lowest rank) for major depressive episodes to 99% 99 (the highest rank) for generalized anxiety disorder. Note that pilot testing via a representative telephone survey found that the full CICI-SF scales take an average of seven minutes to administer compared to over a hour for the full CIDI. Clinicians can expect that the CIDI-SF scales will reproduce the diagnostic classifications that are found in the full CIDI, although more independent tests with other data sets are needed for verification.

3. The Beck Depression Inventory (BDI)

The BDI is a 21-item, self-report rating inventory that measures characteristic attitudes and symptoms of depression (Beck, et al., 1988). A number of forms of the BDI have been developed, including online versions and the BDI-11 short form of 13 items. A modest 5th or 6th grade reading level is required of clients in order to adequately understand the question items, and administration takes roughly 10 minutes. The internal consistency (reliability) range for the BDI and the BDI-11 is from .73 to .92 with a mean of .86. Internal consistency is high across populations, with alpha coefficients of .86 and .81 for psychiatric and non-psychiatric populations respectively.

4. Center for Epidemiologic Studies of Depression Scale (CES-D)

The CES-D is a 20-item measure that asks caregivers to rate how often over the past week they experienced symptoms associated with depression. Response options range from 0 to 3 for each item (0 = Rarely or None of the Time, 1 = Some or Little of the Time, 2 = Moderately or Much of the time, 3 = Most or Almost All the Time). Scores range from 0 to 60, with high scores indicating greater depressive symptoms.

The CES-D performs well with good sensitivity and specificity for identifying individuals at risk for clinical depression; the instrument shows high internal consistency (Lewinsohn, Seeley, Roberts, & Allen, 1997). Application of the CES-D across a broad range of chronological ages has been successful (Lewinsohn et al., 1997). The CES-D has been shown to be sensitive to differences between caregivers and non-caregivers (Pinquart & Sorensen, 2003). Moreover, the CES-D is sensitive to changes in caregiver depressive symptoms after intervention, which can be a tremendous help to clinicians (Pinquart & Sorensen, 2006). The CES-D was developed with somewhat different factor structures across racial and ethnic groups, making it useful for diverse caregivers populations (Roth et al., 2008).

5. Severity Measure for Generalized Anxiety Disorder -- Adult

The GADSS is an interview rating scale designed for assessing the severity of symptoms of generalized anxiety disorder (GAD). The GADSS has shown positive psychometric data in a sample of adult primary care patients with GAD and panic disorder (Spitzer, et al., 2006). The validity and reliability of the GADSS with older adults has demonstrated adequate internal consistency, strong inter-rater reliability, adequate convergent validity, poor diagnostic accuracy, and mixed discriminant validity (Spitzer, et al., 2006). From this, clinicians can assume mixed preliminary support for use of the GADSS with older adults (Spitzer, et al., 2006).

Generalized anxiety disorder (GAD) is one of the most common mental disorders, but there are few brief clinical measures for assessing GAD. One criterion-standard evaluation of a 7- item (GAD-7-item scale had 2740 adult patients complete the questionnaire, followed by 965 patients with a telephone interview by a mental health professional (Spitzer, et al., 2006). For criterion and construct validity, GAD self-report scale diagnoses were compared with independent diagnoses made by mental health professionals; functional status measures; disability days; and health care use (Spitzer, et al., 2006). The brief anxiety scale (GAD-7) demonstrated good reliability and good criterion, construct, factorial, and procedural validity (Spitzer, et al., 2006). Optimized sensitivity and specificity were identified at a cut off point, at 89% and 82%, respectively (Spitzer, et al., 2006). On the GAD-7, as scores increased on the scale, there was a strong association with multiple domains of functional impairment (Spitzer, et al., 2006). A factor analysis of the scale demonstrated distinct dimensions for the co-occurring symptoms of GAD and depression, which is a particularly useful attribute to clinicians. Indeed, the symptoms of GAD and depression showed different and independent effects on functional impairment and disability (Spitzer, et al., 2006). Another strong point is that good agreement between self-report forms and interviewer-administered versions of the brief scale were observed (Spitzer, et al., 2006). The GAD-7 is viewed as a valid and efficient screening tool for GAD and lends itself well to assessing the symptom severity of GAD in clinical practice and in research (Spitzer, et al., 2006).

The construction of a valid and reliable instrument for screening and diagnostic purposes is a highly complex endeavor involving multiple steps and numerous tests across a broad spectrum of presenting and environmental variables. The review of the SCID, provided at the beginning of this section, offers a good window into this complexity, and the unpredictable nature of the outcomes. Recall that the reliability factors reported for the SCID varied considerably across the studies listed. Indeed, as with many evaluations of instrumentation, the studies were conducted across several years and under widely varying conditions. Crucial variables were considered to be the type of administration employed, the specific population makeup of the study participants, the type and extent of training provided to interviewers who administer the tests, and the base rate -- a variable that is altered with each new iteration of the Diagnostic and Statistical Manual (DSM) as new disorders are identified or reclassified, thereby altering the diagnostic base rate for affected disorders. This is in addition to the anomaly created by the statistical procedures utilized in analyzing the base rate across studies in, say, a meta-analysis.

The justification for the proposed instrument is primarily that the author wished to know if a particular change event contributed to or exacerbated the disorders of depression and anxiety in clients and patients. The change event is the authorization and implementation of the Patient Protection and Affordable Care Act. None of the instruments reviewed in this paper would adequately perform in this regard as the instruments are all designed as general screening tools or in-depth diagnostic tools.

Methodology

The participants. The respondents were not selected according to a sample plan or a sample frame. Rather, the respondents were selected from a convenience sample that utilized snowball referrals from friends and acquaintances. Participants all completed a consent form before beginning the questionnaire. The informed consent described the purpose of the study, explained how the data would be utilized, stored, and disseminated. Participants were informed that their data would be available to others only in the aggregate. Moreover, participants were told that they were to engage in the study only as an informed and willing participant, and that they could end the survey or any other participation in the study at any time and without providing a reason to anyone. Informed consents were provided via email and participants were required to return the informed consent with a signature and as an attachment to the email.

Measures. The results of this pilot test do not support the construct theory, which suggests that the instrument was not sufficiently discriminatory, that the sample was too small for this research approach. Consideration is being given to developing a case study research proposal and delving deeper into qualitative methods to better understand the lived experience of the study participants.

The instrument contains 15 items and used a 5-point Likert scale for the quantitative component and a set of open-ended questions for the qualitative component of the research. The question items were developed by reviewing a number of industry validated instruments, such as the pre-existing scales discussed in this paper.

Data collection. The questionnaire was administered online by using a free trial of a commercially designed and supported survey tool. The respondents were given precise instructions on how to complete the online questionnaire before they clicked to the first question. And each question item contained a brief explanation of how to respond to the question, explaining each time the meaning of each point on the 5-point Likert scale (as the language changes somewhat from item to item) and how to complete the open-ended questions.