Survey Questionnaire Design: Critique and Recommendations
This paper offers a section-by-section methodological critique of a live survey questionnaire, drawing on established survey design literature. It addresses issues including survey length and dropout rates, inconsistent recall periods, inappropriate use of dichotomous measurement scales, double-barreled questions, confusing question wording, and the suboptimal placement of demographic items. For each identified problem, the paper recommends specific improvements — such as converting dichotomous attitude items to ordinal or Likert-scale formats, simplifying complex sections, rewording potentially alienating knowledge questions as opinion items, and relocating the demographic section to the beginning of the instrument to maximize data recovery from incomplete responses.
- Survey Length and Early Placement of Critical Questions: Dropout rates and prioritizing key questions early
- Measurement Consistency and Recall Period Issues: Recall periods and question consistency across sections
- Dichotomous Scales and Level of Measurement: Converting dichotomous items to ordinal or interval scales
- Double-Barreled Questions and Ambiguous Items: Double-barreled wording and confusing question structures
- Demographic Question Placement: Moving demographic questions to survey beginning
- References: Cited sources in APA format
✍️ How to write this paper — guide, tools & examples ▾
What makes this paper effective
- Each critique is grounded in a specific citation, giving the recommendations scholarly authority rather than appearing as personal preference.
- The paper moves systematically through the questionnaire section by section, making it easy for the reader to locate and act on each recommendation.
- Concrete examples (e.g., converting L4 to a Likert-scale format, rewording Section P knowledge questions as opinion items) translate abstract methodological principles into actionable edits.
Key academic technique demonstrated
The paper demonstrates applied methodological critique: it identifies a specific design flaw in each section, names the technical problem (e.g., inappropriate level of measurement, double-barreled question, inconsistent recall period), links it to a relevant research finding, and proposes a targeted fix. This pattern of flaw → principle → remedy is a strong model for any methodological evaluation paper.
Structure breakdown
The paper opens with a general concern about survey length and dropout, then proceeds through individual questionnaire sections (A, B, D, E, L, P, Q, R, X, Y) in order. Each paragraph addresses one or two related sections. The paper closes with a standalone recommendation on demographic placement before listing references in APA format. The structure mirrors the questionnaire itself, making the argument easy to follow.
Survey Length and Early Placement of Critical Questions
Because the modal time for dropouts in live surveys — whether short or long — is at the halfway point, the researcher needs to place the most crucial information early in the survey (Cape, 2010). Which information qualifies as crucial depends on the focus of the researcher (demographic information is addressed separately below). Additionally, the questionnaire is somewhat cumbersome and will require considerable training before interviewers can use it effectively, particularly in sections such as Section F. It would be better to add clearer guidelines, simplify some content, and make the instrument more user-friendly. The fewer errors an interviewer makes, the lower the bias and the greater the reliability — and hence validity — of the measure (Bradburn, Sudman, & Wansink, 2004; Kalton & Schuman, 1982). Specific issues with individual sections are discussed below.
Measurement Consistency and Recall Period Issues
Question A4 asks about reading within the past week, whereas other questions in the same section ask about more typical behavior patterns over time. For the sake of validity, it is best to keep questions measuring similar parameters consistent with respect to the time frame they reference (Bradburn et al., 2004). Furthermore, the optimal recall length that participants have for ingrained behaviors — such as reading the newspaper or watching the news — is longer and more reliable than it is for other behaviors, so questions of this nature can be framed more generally (Clarke, Fiebig, & Gerdtham, 2008).
Create your account
Always verify citation format against your institution’s current style guide requirements.