What makes a good reflection question?? What makes a good survey question? While studying reflection and journal writing for my current projects, I looked back at some of the principles I developed about what makes a good survey question for youth and family surveys. I think that the principles are the same.
Youth-friendly questions should be:
Comfortable: Questions that a person can comfortably more than one way, without feeling that they ought to give a particular answer.
Realistic: Questions that are realistically likely to provide a variety of responses.
Respectful: Respecting individual styles and cultures.
Reasonable length: In-depth enough to be meaningful but not overly time-consuming.
Focused: Questions focused on topics relevant to program goals and activities.
Useful: Questions that will provide participants with a valuable opportunity to reflect and that may also produce useful information for the program.
Integral to programming: Making journal writing, reflection and/or information-gathering a natural and integral part of programming.
Principle #1: Comfortable. Questions that a reasonable person can comfortably answer more than one way, without feeling that they ought to give a particular answer.
Discussion: It is well known that surveys/reflections should avoid explicitly leading questions (such as “Don’t you think that ____ is getting too old to be an effective mayor?”) but it is also possible that some questions have an “ought-to” tone even if they are phrased neutrally.
For example, questions about healthy habits or family activities can take on an “ought-to” tone because other media messages have said that families “ought to” eat dinner together regularly and that children should eat a good breakfast daily. Even asking a series of questions in a certain way can suggest that the survey-writer feels that you ought to do these things. Ideally, survey-takers should be able to choose from a variety of options on checklists, or use rating scales that allow a range of answers, so they aren’t forced to say “no” explicitly to something they know ought to be a yes.
Principle #2: Realistic: Questions that are likely to provide a variety of responses.
Discussion: Questions should realistically reflect the likely interests and experiences of the population. For example: a career interest survey asks a series of realistic questions like “do you enjoy performing music” or “do you enjoy working with children” but also asks questions like “do you enjoy filling out income tax forms.” This question is probably not realistic for a younger population because young people probably haven’t filled out income tax forms yet. (And for an older population, income tax forms might not be something people realistically think of as “enjoyable” even if they enjoy working with information and filling out forms.
Generally, questions should focus on activities that children/teens might have experience with, including some that may be a “stretch,” but not on activities that are unlikely to be relevant.
Principle #3: Respectful. Respecting individual styles and cultures.
Discussion: It is helpful to have several people look over proposed questions to look for potential cultural bias. For example, different cultures may use different wording when talking about volunteering, service learning, community service, or helping out in the community. Therefore, questions about community involvement have to be worded in a way that makes sense across cultures. Or, for example, different cultures (and individual families) may emphasize different aspects of family life, so questions about family life and parent involvement should recognize and respect a variety of approaches.
One important part of respectful question design is to avoid conveying biased or negative images of participants’ lives, families and neighborhoods. For examples, surveys with a heavy focus on risk behaviors suggest to youth that the survey-writers expect these behaviors. Or, for example, surveys that ask about possible deficits in the participants’ lives suggest that the survey-writers expect these deficits. If it is important to survey youth about these issues, it can be helpful to balance the survey by asking about positive behaviors and assets as well, and perhaps to use rating scales, checklists with multiple options or other survey methods to allow a range of positive or mid-range responses.
It is important to remember that for youth especially (but also for individuals of all ages) taking a survey or responding to a reflection question is a learning experience. Youth form impressions about the world, about themselves, and about your program based on the collection of questions posed to them.
Principle #4: Reasonable Length. In-depth enough to be meaningful but not overly time-consuming.
Discussion: It’s important to seek balance between “ease of collection” and “quality of information” in determining the length/complexity of questions. As a corollary, if answering a question provides people with a positive opportunity to reflect on experiences or interests, or provides an opportunity for staff to learn about something important, then it is often worthwhile to “invest” more time and use questions that are more in-depth.
Principle #5: Focused: Questions focused on topics relevant to program goals and activities.
Discussion: Questions should be focused on the program goals and activities. This can include questions about relevant background and interests, reflections about future goals, and questions about current program activities. This can also include questions that extend beyond the current program activities to explore applications of the concepts and skills that participants are learning.
Principle #6: Useful: Questions that will produce useful information that can be analyzed and used for reporting and/or improving programming.
Discussion: Whatever the format of the question, including open-ended questions, yes/no, multiple choice, rating scale, etc., it is valuable to gather information that can be summarized, analyzed and shared with staff and others for improving programming, describing activities and outcomes, evaluating the program, or seeking support for future programming.
Principle #7: Integral to programming: Making journal writing, reflection and information-gathering a natural and integral part of programming.
Discussion: Wherever possible, programs should use reflections, goal setting activities, career interest checklists, or other materials that are integral to the programming. Staff and participants should get accustomed to a few different types of information-gathering activities such as pre-program, mid-program and post-program questionnaires or tests, reflection questionnaires after an activity, or periodic goal-setting questionnaires, interest questionnaires or college/career planning checklists. Whatever the formats, data collection should be seen as an integral part of programming – and may be seen as a valuable opportunity to reflect and learn, or at least a reasonably comfortable and pleasant part of the program.
Skills Library on Facebook