I love this scene from Annie Hall, where Annie and Alvin are telling their respective therapists how often they have sex.

Alvin: Hardly ever. Maybe three times a week.

Annie: Constantly. I’d say three times a week.

I always encourage clients to watch this clip when they want to ask questions about frequency.  Too often, people want to use vague quantifiers like often, sometimes, occasionally, and rarely in survey questions.  Here’s an example:

In the past month, how often did you get at least 30 minutes of exercise?

Regularly * Sometimes * Occasionally * Never

As a self-proclaimed “gym rat,” I consider 5-6 times a week “regular” exercise; anything less than that is “sometimes.” Other people consider 3 times a week “regularly.”  On the other hand, if you asked me this question, I would interpret it very differently:

How frequently do you clean your home?

Regularly * Sometimes * Occasionally * Never

I’m not sure if this question is asking me about the past week or the past year, but on average, I clean once a week—that’s regular enough for me.  Plenty of people would call that “occasionally,” and insist that “regular” housecleaning is done every day. In short: these terms can be interpreted wildly differently by different people– or even by the same person in different contexts.

Questions with vague quantifiers are easy to write and don’t require a lot of thought. But you rarely have any idea how your respondents are interpreting these words or what frequency they think they are conveying.

There are situations where this type of question can work, but if you really want to gauge frequency, I’d offer two things that you need to think about:

What is an appropriate reference period?  Are you interested in knowing about behavior in the last year? The last week? The last 24 hours?

What ranges would be reasonable answers? Do your response options start with “fewer than 3 times” or “fewer than 20 times?” Is the high range “more than four times” or “more than 100 times?” Or is this something people might do “more often than once a week” to “less than once a year?”

At the risk of sounding like a broken record, I’d recommend pre-testing to make sure you’re getting this right!