Do respondents to surveys tell the truth?

Daniel Gile

February 16, 2006

One way to obtain information about translational behavior is to ask translators to answer questions about it, essentially through interviews and questionnaires. Despite appearances, the efficient conduct of surveys is far from simple. One fundamental question is whether respondents tell the truth. The concern here is about the existence of subjective factors which unconsciously bias their responses. The risk of such bias is well known. The double-blind paradigm used in medical research, in particular when testing the therapeutic effect of drugs, is associated with the risk of experimenters tending to detect more improvement in patients who have received a drug as opposed to a placebo and with the risk of patients reacting not to the chemical effects of the drug, but to the idea that it is supposed to help them. The double-blind paradigm addresses this issue by masking to both the experimenter and to the patient (hence the “double blindness”) the nature of whatever is administered to each patient, drug or placebo.

What are the factors most likely to bias a respondent’s statements? One is the effect the answer may have in his/her own eyes on his/her self-image. Inter alia, it is reasonable to assume that it is psychologically difficult to make a statement showing that one’s knowledge and/or skills are limited, that one’s attitude as a professional is not as honest, conscientious or systematic as it should be according to conventional norms. One can therefore assume a risk of bias in a respondent’s answers to questions which might call for such statements. Another source of bias is the respondent’s idea of how his/her answer may affect his/her environment. For instance, if translators believe that information provided by them may lead to a deterioration of working conditions, they may, unconsciously (or not), provide a response different from what they would say if they felt the information would have no consequences.

Different steps can be taken to limit such risks of untruthfulness due to the respondents’ bias. One is to avoid questions associated with such risks and try to collect the relevant information by other methods. Another is to formulate the questions in a way aimed to reduce the likelihood of their association with negative (or positive) images of the respondents. Yet another is to introduce “control-questions” to detect possible inconsistencies. Still another is to train interviewers to interact with respondents in ways which will limit the risk of such interference with truthful responses.

Biased answers can still be used conservatively: for instance, if translators asked how often they make sure that their terminological choices are backed by reliable sources answer that they do most of the time, the truth may be that they do so less than most of the time, but their reports can reasonably be interpreted as acknowledging that sometimes, they do not. In other words, while the accuracy of the statements is difficult to assess, they can be used conservatively with some reliability.