There are many possible reasons besides the obvious problems related to sampling error:
- It is possible that the studies were conducted at different times, even if they were published at the same time. If many people’s attitudes are flexible and prone to change with events, then it is possible that both surveys are good, and that people’s attitudes have changed between the earlier and later surveys.
- It is possible that the researches focused and therefore included different groups that were investigated (eg different ages, regions, ethnic groups, etc.)
- It is possible that the research was conducted using different methods. The results can be the so-called “mode effect”, which means that some people may, consciously or subconsciously, give different answers depending on whether they are asked in person by the interviewer, or whether they fill out a mailed or e-mail/internet questionnaire themselves. There is some evidence that anonymous self-administered surveys can provide greater openness on some sensitive topics than face-to-face or telephone surveys.
- It is possible that different questions were asked in the surveys. The wording of the question is important, especially when it comes to issues where many people do not have strong opinions. That’s why it’s worth checking how the question was asked correctly, if there are differences in the results.
- There may be an “order effect”. One survey may ask a certain question “cold”, at the beginning of the survey, and another survey may ask the same question “warm”, after a series of other questions related to the same topic. Differences sometimes appear between the two sets of results, again in the case that many people do not have strong opinions, and some people may give different answers depending on whether the question was asked “out of nowhere” or if they have considered some aspects of the topic first.