This article is a follow-up to this one, and it refers to the questions presented as an example here:
Let’s take it to step by step. First of all, methodologically. The first question uses a scale with an even number, which requires the responder to mark only one answer. The second item presents an odd number of choices, of which one, several, or all can be picked by the responder. This creates confusion because it requires the responder to use a different rule for answering question 2 from the rule used to answer question 1. No matter how well you explain, it is subject to mistakes. You must eliminate these mistakes out of respect for your work, your personal branding as a competent, non-time waster professional, your responsibility towards providing actionable intelligence, and the responders’ intellectual effort.
The second question uses an unclear scale for choices, which, combined with the words used in the item, may create confusion.
If it’s a scale, it must be from white to black, from dark to light, and from alpha to omega. You cannot play around with the responders’ perception. If the first choice is “daily”, the other choices may be “2-3 times a week”, “weekly”, “2-3 times a month”, “about once a month”, “3-4 times a year,” “Yearly”.
“Pretty often “is a poor choice because what is “pretty often” for someone can be very different than what is “pretty often” for someone else. It could range in the individual perception from once a month to twice a day to all the time. So don’t use relative language. Use precise words when you measure frequency or intensity. And if you want both frequency and intensity, for example, measure them with different items. Yes, that means putting the same questions again, with similar scales.
“Sometimes” is a different item for measuring frequency than “pretty often”, but it may be similar. You may use these terms together in rather a six points scale like “never”, “rarely”, “sometimes”, “pretty often,” “very often”, or “always”. You can also cut the extremes and have it a four points scale, conditioned that the rest of the items besides “pretty often” and “sometimes” are “rarely” and “very often”. I wouldn’t use vague terms, but they are not wrong.
Let’s take a look at the first words of the item: “How often do you usually” continued with option d), “rarely”. So the question the responder has to ask himself to understand what the item means, in this case, is: “How often do I usually rarely…? Do I rarely often usually… ?”. It’s nonsense. Use logic. If you aren’t used to logical thinking, learn it or drop human resources NOW.
Another issue with the first question is the word “feel”. It’s a tricky word to use in a subjective description. In general, people are already very subjective when completing a questionnaire. Especially when you measure subjective experience of frequency, it is preferable to use moderate, neutral words. Besides, some people could relate more to seeing or hearing than feeling. In formulating an item, you must respect the reality that people have different perception styles, which you must respect to speak in their own language. For these two reasons, it is recommendable to use the word “find” instead of “feel”. It appears to describe a more objective experience, but in reality, the subjectivity comes from the term “tension”. When using subjective nouns in items, don’t enhance them by adding subjective verbs. “Find” is more objective than “feel”. Pay attention to the choice of words. “If you ask wrong questions, you will get wrong answers” (3).
“When starting work” describes an unclear event. It could mean the minute entering the building, the moment actually starting work (after coffee, talking to the boss, or a morning conference), or coming back to work after a meal. To which moment does the researcher refer?
Another thing to pay attention to is that the first question refers to “working with colleagues” when the second question also relates to other departments or families. When you want to evaluate the organizational environment in a specific office, you focus your questions in such a manner to discover which elements inside the system are not working. If the researcher wants to find out also the external factors, then item 1 must be restated to integrate the external causes. Not to mention there is a high difference in perception of whether the responder has an internal locus of control or an external locus of control in the context of working in that specific environment.
The first question uses the term “tension”, and the second uses the term “pressure”. Although they may be synonyms, if you used one term for the first item, then you must also use it in the second item, too. Some people might relate differently to “tension” than to “pressure”, not to mention that “tension” is often perceived as internal, and “pressure” is external. It’s not a rule, but there are subtilties to which you have to pay attention.
The second question is wrong from the beginning. If the purpose is to find the cause of distress, you must first consider all the reasons for the distress. Let’s say, for example, that an employee has a good chair at his own office, but when joining others to work on a joint project, the table around them has the wrong chairs for his back. The person feels physical tension, but none of the variants are responsible for that. It’s the chair. It’s not someone’s fault; it’s something. So define well what you want to measure and what words you use.
Please also consider this list of articles in Romanian about research.
Author’s experience in questionnaire design. The Author has worked as a research group member developing organizational evaluation tools. From November 2006 to June 2007, he contributed to five questionnaire designs in students’ groups. Also, he designed the first psychological questionnaire for application in political branding using NLP (Iaşi, 2004). For that project, he was awarded the first prize at the students’ contest EconomMix in 2005, the management-marketing section.
Besides that, he designed psychological questionnaires for his own research on parental education (2005), self-esteem (2007, 2009, 2011), insurance (2009), and memory (2009), and he enhanced an evaluation questionnaire for career consulting (2008).
Copyright © Marcus Victor Grant 2012-present, all rights reserved.
The materials on this blog are subject to this disclaimer.
Analytic Vision 