Advanced Nursing Field Experience

Quality Improvement Activity (QIA) Form Instructions

When to Use the QIA Form

This document is a guide for completing NCQA's Quality Improvement Activity (QIA) form. This form can be used for the QIA required NCQA accreditation and certification programs, as applicable. It must be used to meet the Quality Improvement Projects required for Medicare Advantage Deeming.

You are not required to use the QIA form; however, you must provide the data it requests in order for NCQA to review your QIAs completely and accurately. Submit a QIA for each activity you present by attaching it to the applicable element in the Survey Tool using the Attach Document feature in the Survey Tool.

Detailed instructions on attaching documents to the Survey Tool are found in the Survey Tool Instructions under Help on the Main Menu bar.

The purpose of the QIA form is to summarize the clinical and service quality activities that you are using to demonstrate meaningful improvement in the applicable element.

You should not complete the QIA forms for service or clinical activities that you use to demonstrate compliance with other standards that require data collection and analysis such as member/enrollee satisfaction, availability and access and satisfaction with UM. Document compliance with these standards as you would document any other standard.

All data points must be final when your organization submits the Survey Tool.

NCQA does not recommend using this form to report on activities that have only one data point (e.g., baseline only).

Consult the appropriate Explanation for the meaningful improvement standard for the accreditation or certification program for which you apply.

Remember that you cannot achieve a score of 100% with only one data point.

The activity will not be considered.

Achieving Meaningful Improvement

Submit enough data

To receive "credit" for meaningful improvement, you must submit enough data to allow an evaluation of any seasonal variations that could affect the results. On the service side, open-enrollment seasons can affect such activities as ensuring access to primary care and reduction in referral time frames. In most cases you must present:

annual measurement occurring during the same season (e.g., comparing the first quarter of one year to the first quarter of the following years) for areas that show seasonal differences, such as provision of enrollment cards five quarters of data fifteen months of data.

Note: If you do not have adequate data to satisfy the above conditions or if you believe that the results are not biased by seasonal issues, provide an explanation as it relates to QI 12 and QI 13 under Other Pertinent Methodology Features, in Section I.

The improvement must meet the time period covered in the survey

To receive "credit" for meaningful improvement, the improvement must have occurred in the three-year period covered in the survey. For example, if you have annual data on member satisfaction since 1996, but the date of the survey for which this QIA is being prepared is January 2008, only data beginning in 2005 should be shown.

In other words, the improvement must have started at some point during the three years immediately prior to the survey and have been subsequently sustained.

For Renewal Surveys, you may need to present measurements for the year prior to the current survey period if these data were not available for your previous survey.

The QIA Form

The form's five sections

The QIA form is divided into five sections:

Section I Activity Selection and Methodology

Section II Data/Results Table

Section III Analysis Cycle

Section IV Interventions Table

Section V Chart or Graph

Activity name and activity examples

The form first asks you to supply an activity name. The activity name should succinctly encompass the purpose of the activity and begin with an action word that accurately states what the activity is designed to do (e.g., "improving," "increasing," "decreasing," "monitoring"). Examples are listed below.

decreasing the risk of congestive heart failure improving claims turn-around time to practitioners increasing the rate of diabetic foot exams improving access to behavioral health services decreasing practitioner complaints with the referral process.

QIA Instructions and Form 15

2 QIA Instructions and Form

QIA Instructions and Form 1

Effective July 1, 2007

Effective July 1, 2007

Section I:

Activity Selection and Methodology

This section asks you to provide the rationale for choosing this QI activity for your organization. Explain why the clinical or service activity affects your members or practitioners.

NCQA requires you to choose service improvements based on their impact on members. NCQA also accepts improvements in practitioner satisfaction that relate to utilization management (UM) processes or effects (e.g., issues identified in UM 11) for one service QIA.

Examples are listed below:

improvements in turnaround time for prior-authorization requests decrease the time that members wait to receive care requiring authorization and/or increase productivity for practitioners improvements in UM decision making turn-around-time ensure more satisfied members and/or practitioners improvements in referral to specialist turnaround time reduce the number of complaints and appeals regarding referrals.

Rationale

Define the rationale for selecting the activity

This section asks you to define your rationale for selecting this activity for improvement.

Why was it chosen over others?

Why is it important to your members or practitioners?

Why is it worth the resources your organization is spending on it?

Using objective information provide as much information that is specific to your organization as possible.

You do not have to provide generic defenses for most clinical or service issues. For example, do not include explanatory phrases such as "member services departments serve many important functions," or "neuropathy of the foot is a serious condition that affects thousands of diabetics nationwide."

Nor is it necessary to provide literature source cites on the importance of a clinical or service issue to members unless it is an unusual topic. Focus on the importance of the activity to your organization.

Importance of activity

Include pertinent organization data or community demographic data that reflect the importance of the activity to your organization's membership. Describe the magnitude of the issue related to the activity in quantifiable terms.

Activity examples

Examples are listed below.

Between 2004 and 2005, hospitalization due to diabetic foot neuropathy rose 9%. This was the largest increase in any diabetes related hospitalization. Research has shown that periodic foot screening of diabetics and self screening by diabetics can decrease rates of foot neuropathy.

Practitioner dissatisfaction turnaround time with UM decisions increased from 5 to 15% between 2004 and 2005. This was the largest increase in practitioner dissatisfaction the organization has received for four years. In addition, this 15% dissatisfaction rate was the highest dissatisfaction rate on the practitioner survey.

Quantifiable Measures

Quantifiable measures clearly and accurately measure the activity

This section asks you to list all quantifiable measures you use in this activity, including those added over time. Quantifiable measures should clearly and accurately measure the activity being evaluated. List your baseline benchmarks and goals and if you modify them over time, list the updated benchmark or goal in the table in Section II.

Multiple measures

You may use one or more measures for each activity. For some activities, multiple measures are useful. For example, practitioner complaints and actual turn-around-time for UM decisions would be two measures that are closely linked to the timeliness of UM decisions.

In other cases, multiple measures may not be useful. For example, you may display multiple measures associated with a CHF disease management (DM) program, only one of which shows improvement. Unless the intervention is clearly focused to address that measure, NCQA may not consider the improvement meaningful.

Denominator

Describe here the event being assessed or the members who are eligible for the service or care. Indicate whether all events or eligible members are included, or whether the denominator is a sample. Examples of responses are listed below:

all physician complaints

members 35 years of age and older during the measurement year who were hospitalized and discharged alive from January 1 -- December 24 of the measurement year with a diagnosis of congestive heart failure all survey respondents

14 QIA Instructions and Form

Numerator

Describe here the criteria being assessed for the service or care:

all physician complaints concerning UM decision turn-around-time members meeting the criteria for inclusion in the denominator who received an ambulatory prescription for ace inhibitors within 90 days of discharge survey respondents who do or do not like the event in the denominator

First measurement period

State here the time period covered by the initial assessment.

For clinical issues, this is typically an entire calendar year (e.g., January 1, 2008 -- December 31, 2008).

For service issues, the measurement period is often monthly or quarterly (e.g., January 2008 or 1Q 2008). Measurement periods may vary by measure. For example, the first measurement period for UM decision timeliness may be the first quarter of 2008, but the measure addressing timeliness may not have started until the third quarter of 2008.

Baseline benchmark

Include here information on how the benchmark was derived as well as the benchmark rate. NCQA defines "benchmark" as the industry measure of best performance against which the organization's performance is compared. It should be directly comparable to your QI measure.

You may describe the benchmark in numerical terms (e.g., the 90th percentile), or in terms of the comparison group (e.g., the best published rate in our state, 85%).

The benchmark may be a best practice in an industry based on published data or the best performance within a corporation with multiple organizations. NCQA requires a benchmark or a goal, but not both. Many service activities do not have benchmarks. If you are not using a benchmark, insert "NA" in response to this query.

Remember: Benchmarks are not averages; they are the best in class.

The average for a national organization or corporation with multiple organizations is not a benchmark.

The organization's best rate would be considered a benchmark.

Benchmark source

If you give a benchmark, list the organization or publication from which it was obtained and the time period to which it pertains.

Baseline goal

The performance goal is the desired level of achievement for the measure within a reasonable time. It does not have to be based on actual best practices, but it should reflect the level of achievement your organization has targeted.

The goal should be quantitative and stated in numerical terms (e.g., 90%, 0.3 appeals per thousand, 3 days).

Most organizations do not set performance goals until after they have collected baseline results. If that is the case, enter NA here.

Words such as "improve," "decrease" or "increase" are not acceptable in stating goals unless they are accompanied by a numerical quantifier (e.g., "improve one standard deviation from baseline" or "decrease by 5 percentage points from the last remeasure").

Remember to use the words "percent" and "percentage" precisely.

An increase in practitioner satisfaction with the UM referral system from 35% to

40% is a 5 percentage point increase, not a 5% increase.

State the first goal you set (which, generally, is set after baseline results have been analyzed). NCQA expects that as you achieve your goals, you set new ones. Section II has a space to list updated goals. Examples are listed below.

Goal example

Measure: Pre-service UM decisions.

Numerator: Number of preservice decisions less than 4 days.

Denominator: Number of preservice decisions.

Benchmark: NA

Baseline Goal: 80% of preservice decisions are made within 3 days of the request.

Note: NCQA does not consider achievement of a prespecified goal or benchmark alone as a demonstration of meaningful improvement.

Baseline Methodology

This section uses tables, check boxes and narrative to enable you to describe your methodology. The more precisely you describe the data you used and how they were obtained; the sampling procedures, if any, that were applied; and any special factors that could have influenced the results, the more easily NCQA can assess the validity and reliability of the findings.

C.1 Data sources

Check all the data sources used. If you used other sources that are not listed, check "Other" and describe the sources completely. Indicate the number of the measure from Section B. next to the data source used.

C.2 Data collection methodology

This section is divided into:

medical/treatment record survey administrative.

Because you may use different data collection methodologies for different measures, check all that apply. Indicate the number of the measure from Section B. next to the data source used. If you collected survey data using more than one of these techniques, check all that apply. If you used different techniques, or if you used other methods to collect administrative data, mark "Other" and describe your data sources completely. You are not limited to the options provided.

Most of these methodologies are self-explanatory. The definitions for the survey data collection methodology are listed below.

Definitions

Personal interview

A face-to-face interview.

Mail

A survey mailed to and returned from the respondent and involving no personal contact.

Phone with CATI script

A telephone interview using a computer-assisted script containing prompts beyond the actual questions that can be used according to a set protocol.

Phone with IVR

A telephone interview involving an interactive voice recognition system rather than a live person.

Internet

A survey conducted using the Internet and involving no personal interaction.

Incentive provided

A survey in which the respondent was given an incentive (e.g., gift certificate, cash) for participating.

Note: Regardless of the survey methodology, mark this box if the respondent is given any incentive to complete the survey.

Other

Any other survey methodology different from those listed above.

C.3 Sampling

For each measure that involved sampling, state the sample size, the method used to determine the size and the sampling methodology. If the size is the same for all measures, state "All Measures" and give the information only once. Also provide the size of the full population from which you drew the sample.

Remember that the sampling methodology here relates to your baseline measurement only.

Any change to this sampling methodology is reported in Section I.D of this form.

Table elements

Measure. You may use the measure number from the measures listed in Section I.B and abbreviate the name.

Sample size. State the number of the full sample selected, including any oversampling. The denominator listed in Section II provides the number included in the measure.

Determining the sample size. To determine the size, explain the parameters used to determine the sample size, which typically include:

the assumptions or requirements of the statistical test to be used to verify the significance of observed differences the desired degree of confidence in the statistical test (alpha level)

statistical power (the sensitivity of the statistical test to detect differences; bigger samples yield greater power)

the margin of error to be allowed when assessing the hypothesis the oversample rate the oversample is the extra cases included in the sample to replace cases rejected because of contraindications, ineligibility, etc. (In survey measurement, the oversample should be large enough to replace expected nonresponses.) Examples of oversampling are shown below.

Oversampling example

You plan to improve the time required for members to obtain a referral. You conduct telephone surveys of different groups of members who obtained referrals at two points in time, asking them how many days it took for them to get the referral. You have these expectations about the survey:

the distribution of responses about the "number of days to referral" is normally distributed for both the pre- and post-survey groups the t-test is used to test the significance of the pre- and post-differences at alpha = 0.05 and 80% power a pilot survey showed that the standard deviation of "number of days to referral" responses is 5.25

the program reduces the average number of days from 8.5 days to 7 days the response rate is 85%.

Sample size calculations based on the above parameters indicate that you require a sample of 193 completed surveys. You expect that 15% of the sampled members will not respond, so you sample 227 members to account for the nonresponse (X *0.85 = 193; X = 193/85; X = 227). This calculation includes 193 members in the original sample plus an oversample of 34 patients to replace those who do not respond.

Sampling method

State the sampling methodology (simple random sample, stratified random sample, convenience sample). State the reasons for exclusions from the sample, if there were any (e.g., "Simple random sampling was used. During the claims pull, three claims were excluded because they were miscoded.").

Remember that if your sampling methodology involves a survey, it is not necessary to complete this table because you have included the Survey Tool and the survey protocol

(requested in Section I.C.2).

C.4 Data collection cycle and data analysis cycle

Check the box that applies or describe the frequency of data collection and analysis. Indicate the number of the measure from Section B. next to the data source used. For many service activities, the data collection cycle is more frequent than the analysis cycle.

For example, hospitalization data may be collected weekly, but analyzed monthly or quarterly. Survey data may be collected quarterly and analyzed at six-month intervals.

C.5 Other pertinent methodology features

Describe any other methodological decisions or issues that could affect the analysis of the data or influence the results, such as:

coding definitions claims-processing specifications unique to your organization claims-processing delays unique survey response coding or benefit design (e.g., pharmacy benefits).

If your QIA does not include sufficient data as specified by NCQA policy, or if you believe the results are not biased by seasonal issues because of the definition of the measure, provide your rationale for considering this for QI 12 and QI 13.

Mark this section "NA" if there are no other methodological features that need to be brought to NCQA's attention. You are not required to complete this section past this point.

Changes to Baseline Methodology

This section asks you to describe any methodology changes that were made after the baseline measurement was taken. To compare results accurately, it is best to use the same methodology over time. However, you may need to change methodology in order to strengthen the validity and reliability of the outcome, correct inadequacies in the initial process, or accommodate for lack of resources. Specifying changes that were made is important because those changes influence analysis of the results.

For each affected measure, you must describe:

the dates during which the changed methodology was used how the methodology was changed the rationale for the change the anticipated impact of the change on the analysis.

If you changed the sampling methodology in the same way for several measures you need to provide the information only once. If the sampling methodology is the same, but the sample size has changed, show only those changes.

Section II:

Data/Results Table

This section consists of a table of the results of the baseline measurement and all of the remeasurements that you are presenting for consideration for the QIA. You may substitute a table of your choice as long as it includes all of the required elements. If there are more than five remeasurement periods, add a row for each additional measure. If you measured a service issue more frequently than quarterly, combine the data by recalculating the numerator and denominator and enter the quarterly result in the table.

Table Description

Quantifiable measure

You may use the measure number from the list of measures completed in Section I and abbreviate the name.

Time period covered

State the time period the measurement covers. It could be quarterly (e.g., 1Q 2008), twice a year (e.g., January -- June and July -- December 2008), yearly (e.g., 2008), or every other year (e.g., January -- December 2006 and January -- December 2008).

Numerator/

denominator

List the numerator and denominator for each remeasurement period.

If the measure uses survey methodology, state the number of people who met the numerator criteria (numerator) and the number of people who responded to the question (denominator).

Rate or results

Convert the fraction (numerator/denominator) to a percentage.

Comparison benchmark / comparison goal

List the goal and/or benchmark period in effect during the remeasurement cycle. The comparison goal is blank for the baseline measurement unless you have established a goal prior to pulling the baseline data. A goal based on baseline data that is in effect for the first remeasurement cycle should appear in the comparison box on remeasurement line 1. If you met your goal but there is still opportunity for improvement, NCQA suggests you increase your goal.

If you changed your goal for any other reason, explain the basis for doing so in Section III: Analysis Cycle. You may also add benchmarks that you did not have at the baseline period.

Statistical test and significance

NCQA does not require you to test for statistical significance. Consult the appropriate Standards and Guidelines for the accreditation or certification program for which you are applying for additional information on the requirements for achieving meaningful improvement.

If you have performed such tests and choose to report them, however, state the time periods that you compared and the type of statistical test used for each measure. The table has been left open-ended to allow you to compare any time period you choose. Most organizations compare the latest remeasurement to the previous one and the latest remeasurement to the baseline measurement.

Statistical testing is generally not necessary when measures are based on the entire eligible population, and may not be appropriate if the denominator is not based on a random or probability sample or if the measure specifications substantially changed since the last remeasurement period.

For the most common test (comparing two independent rates), the chi-square test of proportions or the z-test of proportions can be used (e.g., a z-test to compare the baseline to remeasure #1, p value = 0.2992; and baseline to remeasure #5, p value = 0.001).

These tests are not appropriate when the same members are being measured at different time periods, in which case the McNemar test for correlated proportions might be appropriate.

If you measure nonrate data, such as average wait times, the t-test or z-test for comparing means would be appropriate, depending on the size of the sample. If you have several independent remeasurements, based on samples, you may want to do an ANOVA test of linear trend to show that the rate is increasing over time.