1. Questionnaires, interview schedules, and scales, particularly as used in interview surveys, are an important way to develop measures of people’s characteristics, behaviors, opinions and attitudes. Opinions are personal preferences; attitudes are more general predispositions to act in particular ways. We are often more interested in studying the more firmly established attitudes such as personal self-esteem rather more transient opinions about say for instance current trends in popular culture.
2. In order to use questionnaires effectively to tap people’s opinions and attitudes, questions in such questionnaires need to be constructed carefully so that they can serve as reliable and valid measurement instruments and are therefore able to avoid measurement error as much as possible.
3. Many items in a questionnaires, interview schedules or even self-administered questionnaires are not questions but batteries of agree/disagree statements designed to elicit responses in order to determine what each person’s attitudes are about a variety of personal, social and other concerns. Often sets of these items are built to develop a composite scale summarizing the extent to which each person studied has a particular attitude, behavior pattern or personal characteristic.
4. Questions can be open-ended or closed-ended. Open-ended questions allow the respondent to state their response as they best see fit using an extended narrative usually of a few sentences. Close-ended items are designed to get respondents to choose from a fix set of possible responses. While open-ended responses often lead to more qualitative information,they too can be later reduced to categories that are given numbers. Close-ended questions can often be highly structured, for instance as in a five-point equal-interval scale ranging from strongly agree to strongly disagree.
5. Good questions/items are most often have all of the following characteristics— clarity, brevity, relevance to the respondents, stated in ways the respondents can readily understand, no negative characterizations, no "double-barreled" choices where respondents are asked to choose two things at once, the avoidance of built-in bias favoring one possible response over another.
6. The questionnaire format overall should meet certain criteria. For self-administered questionnaires should have a particularly clear layout and uncluttered appearance.. All types of questionnaires, it is important to have clear instructions , a logical order, filter questions to ensure that subsequent questions are only answered by those respondents for whom they are appropriate, thoughtful sequencing of questions and items to avoid acquiescent response-set or unreflective answering of sets of questions according a pattern determined by the questions rather than what the respondent thinks.
7. Measurement instruments need to be sensitive to language and cultural differences that could affect how people interpret questions as well as how inviting the find the questionnaire. These issues are effectively discussed in the Land and Hudson article.
8. Many key concepts concerning attitudes which we wish to measure such as self-esteem can not be reliably and validly measured simply with point-blank questions. Instead, some subtlety needs to be practiced to develop a reliable and valid measure. Multiple item scales or indices are often used.
9. One of the simplest techniques is a Likert-type scale where respondents are asked to choose whether they strongly agree, agree, are neutral, disagree, or strongly disagree with each item in a battery or series of items all related to the same conceptualized attitude, such as self-esteem or sense of personal well-being. All responses for all items are scored 1 to 5. Each respondent is given a total score indicating the extent to which they possess that attitude at what level of strength, such as a score of 25 for five items indicating the highest level of self-esteem.
10. Another simple approach, used frequently still today, is a semantic differential where respondents are asked to choose between two opposite positions such as whether they find one item related to say going to work for the first time "scary or friendly," while for the next related item about going to work for the first time even though it is at a job far below the status they had initially hoped for was in their mind "encouraging or depressing." To avoid response set, the placement of the possible responses could be place in some non-obvious pattern. All the responses would be coded 1 or 0 depending on what you were trying to measure. Again we would total the responses for each respondent and give them a composite index or scale score.
11. Eventually a sample population to be studied must be selected, qualified and appropriate interviews or people to administer the questionnaire must be chosen and other factors must be considered in order to get reliable and valid data on your variables.
------------------------------------
Group Work for 10/9/01
Operationalization in Questionnaire Construction
Measuring a Dependent Factor in a Survey
Break into groups of four, use the Land and Hudson reading on measuring client satisfaction as reproduced in our course booklet as illustration. Develop a short questionnaire for an interview survey or for a self-administered survey.
Choose a research topic, develop a research question, indicate your unit of analysis, specify your key concepts, think about how to operationalize them as measurable variables, and then develop an hypothesis that specifies the relationship of a independent, causal factor to a dependent, effect factor. For instance: Healthier clients at a local senior citizen center were more likely to be satisfied with the center's performance as a service agency.
For this exercise, as is common in this sort of research, the causal factor can be something relatively easy to measure, the effect factor should be an attitude that you think is affected by variation in the causal factor such as sense of personal well-being as a dependent factor that is affected by participation in a particular treatment program.
Remember you are trying to test your hypothesis with data collected from a self-administered survey. Think of a dependent factor that can be measured in a self-administered questionnaire.
Write up a short multiple item index to measure the dependent factor you are studying and also another questionnaire item or schedule to measure the independent factor.
Use the following checklist; spend most of your time on the last item:
Topic:
Question:
Unit of Analysis:
Key Concepts:
Measure of the Independent Factor:
Index to Measure the Dependent Factor: