The editorial process through the
Daniel Gile - Université
Lumière Lyon 2
Gyde Hansen - Copenhagen Business
Published in Gyde, Hansen, Kirsten Malkmjær
and Daniel Gile (eds).
2004. Claims, Changes and Challenges in Translation Studies.
Amsterdam/Philadelphia: John Benjamins. 297-306.
scholarly publications are gatekeepers who decide whether to publish
manuscripts or not, but their decisions are constrained by institutional
factors, sociological factors, and the availability of referees for
peer-reviewing. The quality of the input of such referees depends not only on
their thematic and methodological expertise, but also on their motivation, on
their personal bias and on sociological factors. Although this suggests that
editorial decisions are less objective and reliable than their idealized
representation, the referees’ comments are often convergent to an encouraging
degree. The main comments made by referees on 42 manuscripts submitted to
these proceedings are listed. They share a large common denominator, which
suggests that the fundamental norms they are based on are the same.
proceedings are a well-established tradition, and are often awaited
impatiently not only as a reflection of the meeting, but also as a
publication vector for participants, and frequently as a justification to
their university for their presence and participation in the event. However,
publishers do not necessarily allow sufficient space for all papers to be
published, and editors must choose. How they do this is not always clear to
authors who submit their manuscripts.
the process is very similar to the editorial process in other types of
collective volumes and in journals in that it is based on the peer review
system. This paper discusses the concept, as well as recurrent issues, in
particular as they crystallise in the field of
Translation Studies; it then discusses issues in refereeing, and briefly
mentions follow-up possibilities after comments are received from the
reviewers. This paper is based on its authors’ experience as editors in the
field, and also includes, by way of illustration, a case study with an
analysis of the most common types of weaknesses reported in papers submitted
for publication in proceedings in the field of TS.
editor's role as a gatekeeper
scientific community, where a scholar’s
publications are his/her visiting cards, passports and stepping stones
towards professional promotion, editors of journals and collective volumes
have an important three-pronged institutional role to play. Firstly, they
create and offer publication space to individual scholars. Secondly, they act
as gatekeepers and guarantors of the scientific quality of texts they publish
(see Judd et al. 1991:17f). Thirdly, through the peer-review system, they
offer guidance and text-improvement opportunities.
gatekeepers who decide to publish or not to publish, editors seem to be in a
position of power. And yet, their action is constrained by several factors,
including constraints from the publishers and pressure from the research
community, difficulties in finding adequate referees for proper
implementation of the peer review system, and “political” and diplomatic
considerations that inevitably influence their decision-making process.
Constraints and policies
system which is theoretically designed to provide reliable guidance and
assessment to editors in their role as gate-keepers in the publication
process has in-built limitations which can be difficult to overcome.
limitations, one should add the expectations of the publisher in some cases,
and of the researchers' communities in other cases (for instance, when
editing the proceedings of a conference), let alone the expectations and
reactions of individual researchers, who may need to publish for
institutional or personal reasons and who may find it difficult to accept the
editors' decision to reject their paper, or even the critical comments from
of policy are observed in the field. The easiest one to implement consists in
accepting virtually all papers, and merely performing some cosmetic surgery
on the wording, correcting errors in the lists of references, re-formatting
papers to applicable standards. While this may not be the most efficient
policy in terms of scholarly quality and promotion of solid scientific norms,
it can be considered legitimate when the purpose is only to reflect the state
of the art in a field, or to provide the written record of a scholarly
At the other
extreme, editors may decide to implement a strict quality policy, accepting
only manuscripts which pass severe scrutiny. Such a strategy is possible for
editors of prestigious journals in established disciplines where there is
much research production and much competition. In TS, due to the small size
of the community and the general state of research, it is doubtful whether
any editorial board can afford to act in such a strict manner.
the editorial policy with respect to quality is somewhere in-between, and
quality thresholds for acceptance vary as a function of supply and demand: if
many good papers are offered and fill the required number of pages, there is
no need to accept mediocre submissions. When there is more publication space
than material, acceptance standards fall.
In such a
situation, it is difficult to set standards in advance. For each issue of a
journal or collective volume, submissions may have to be refereed first, and
the decision to publish or not to publish is taken when the total picture is
Referees in the peer review system
As its name
indicates, the peer review system is based on the work of scholars with a
similar background to the authors', whose role is to provide useful input,
which takes the form of both assessment for the benefit of the editor, and
guidance to the authors. It is significant, but not often noticed, that in
its very name, the system refers to “peers” rather than to “experts” or some
other authority-carrying agent: the idea, which is corroborated by experience
very regularly, is that even without a substantial difference in training and
experience between referees and authors, the very fact that it is easier for
anyone to detect another person's weaknesses than one’s
own, makes it possible for the former to help the latter (“…well-trained
scientists can make mistakes and sometimes overlook important problems of
design” – McBurney 1990:16).
So much for
theory. In practice, the system is a bit more complicated. Firstly,
"peers" are not always peers. Sometimes, acknowledged authorities
are asked to do the reviewing, and sometimes, referees are much less
experienced and qualified than the authors themselves. In the 'guidance'
part, i.e. specific comments on errors or weaknesses and specific suggestions
for improvement, this is generally not too problematic. In fact, excellent
criticism is often received from conscientious beginners, who may react with
high sensitivity to all segments in a text which contain unclear, ambiguous
writing or faulty logic, whereas more experienced readers already familiar
with the field, who may also be busier, may focus on more specific points and
overlook such segments. With respect to overall assessment of manuscripts by
referees as opposed to specific comments, experienced, more highly qualified
scholars are clearly in a better position than beginners to provide useful
established disciplines with a large community of scholars, referees may not
be too difficult to enlist. In other cases, and in
particular in the rather small, but diversified TS community, finding such
reviewers may be a real problem. Firstly, in some sub-areas of a discipline,
there may simply not be more than a very small number of scholars competent
enough for refereeing purposes. The problem in international events (as
opposed to national ones) is compounded in papers written in languages other
than English, which has more or less become the international language of
science, in the field of TS as in other fields. More generally, refereeing
effectiveness can be discussed under several components:
The language issue
Though not a scientific or intellectual criterion, the language barrier is
formidable, even in a multilingual society such as the TS community (or
perhaps especially in this community, where authors may insist on their right
to write in their language and count on their fellow-researchers’
comprehension). Not only are language skills essential when assessing and/or
correcting the style of a text, but they are also important in terms of
comprehension, especially in theory-oriented papers. How many TS scholars can
review German texts on court interpreting, Spanish texts on research
methodology, French texts on contrastive linguistics and translation? The
problem was a very concrete one in this volume as well, when scholars well
versed in certain fields turned out not to have the language in which the
relevant papers were written. In several instances, the less than ideal
choice was between refereeing a paper written in a language one knew well,
but in a field not familiar enough, and refereeing a paper addressing a field
one knew well, but written in a language in which one did not trust one’s comprehension ability fully.
The requirement for sufficient relevant knowledge on the specific theme in
the field is obvious, but in practice, it is not always easy to meet, because
editors do not always have access to referees with the required
qualifications. When they lack ‘thematic’ knowledge, readers may miss errors
in the manuscript, especially in the literature review and references, but
also in the interpretation of quotations and in the use of concepts. On
epistemological and methodological issues, they may also fail to see the
relevance of certain points made by authors, or miss gross errors.
When the referee’s lack of familiarity with a field leads to
unjustified criticism, the refereeing process can become counter-productive.
The most conscientious and careful referees faced with this dilemma tend to focus
on the structure and rationale of the paper and to formulate questions rather
than assertive criticism when in doubt, thus alerting the authors and editors
to the existence of potential problems, without making any damaging claims.
In a field as heterogeneous as TS, research paradigms are diverse and quite
different from each other. Some scholars may adhere to the methods and norms
of cognitive psychology, others to those of literary studies, of
philosophical discourse, of sociology, of formal linguistics, etc. These are
not necessarily mutually compatible, and if referees adhere to one paradigm
and the author of a refereed text to another, this may lead to negative
assessments predicated on an opposition between schools of thought, not on
actual weaknesses in the paper within the paradigm chosen by the author.
Challenging a paradigm is legitimate, and a potentially powerful driver of
scientific progress, but criticising an author’s method using the criteria of a paradigm other
than the one s/he has chosen to follow can be counter-productive (see Kjørup 1996: 109, 113, 116 f.).
However ‘objective’ and impersonal the peer review system is supposed to be
within the idealistic view of science, personal factors
may come in strongly in the refereeing process. Critical reading is serious
work, as it involves reading a text carefully,
making sure one understands it, assessing it, reporting on the findings, and
making suggestions for improvement. If there are serious design flaws, and if
weaknesses are numerous, it may take much time and effort to list them
comprehensively and to suggest alternative formulations, inferences or
actions to help an author understand the problem and do the appropriate
remedial work. In such cases, the willingness of referees to devote
considerable work to the process can make much difference, and it is not rare
to see one reviewer only send back one or a few general paragraphs as input
to the editors, and another offer several pages of detailed comments on the
Personal bias and sociological factors
One final component, which is not in line with the ideal of objectivity in
science, but which is nevertheless endemic in human society, is personal
bias. Scientific society is a competitive one, both
in terms of theories and schools of thought and in terms of personal rivalry
(see Plastow and Igarashi 1989). Actually, as analysed by Kuhn and many others (see Kuhn 1970, Chalmers
1990), such sociological forces may reasonably be considered useful in that
they drive researchers into accumulating evidence to attack existing
paradigms (or defend them, depending on which side they are on), thus
contributing to scientific development. Nevertheless, in an activity having
such a strong assessment component as refereeing, positive or negative
attitudinal bias, whether conscious or subconscious, can result in unbalanced
judgement. Scholars may be eager to find fault with
their opponents' or rivals' work, or be reluctant to criticise
their allies' and friends' writings.
is compounded by status issues: when reading a well-known, experienced
scholar's work, referees may be reluctant to criticise,
not only out of respect for the author, but also out of fear of ill feelings
or 'retaliation' if recognised. Indeed, in a
community as small as that of Translation scholars, it is often easy to recognise an 'anonymous' author and an 'anonymous'
referee through their distinct style, ideas and references.
problems in submitted papers: a case study
After the EST
Congress in Copenhagen,
more than 50 papers were submitted for inclusion in the proceedings.
Following the tradition and conventions of scientific editing and publishing,
they were submitted to the peer-review process. All papers were reviewed by
at least 2 referees, and most of them by three. In a
few cases, only general assessments were received (see above), but specific
comments were made on 42 of them. Referees mentioned both positive aspects,
such as the presence of innovative components, interesting information,
interesting ideas, etc., and problematic areas. In line with its focus as
explained in the introduction, this paper analyses the latter, where
constructive work could be done, but should not be misinterpreted as a
pessimistic statement or diagnosis.
types of perceived problems were reported by referees. Since this is a single
case study, they are given not as a demonstration of any general proposition,
but by way of an illustration of what we consider a not atypical pattern.
that for reasons that readers will understand easily, no attempt will be made
to present the full set of data, with all the comments made by each referee
on each paper. This information will remain confidential, and only a synopsis
will be given here, in the hope that it will be sufficient to provide
interesting and useful input.
Frequently mentioned weaknesses in papers for this volume
The most numerous comments from referees report missing
data: in order to gain better understanding of the author’s
claims or conclusions, reviewers felt that they needed data which was not
provided in the relevant papers, on experimental conditions, on actual
results, on the nature of the texts involved, etc. (also see Strauss/Corbin
1998: 268ff, Judd et al. 460ff.) This problem was reported in 18 out of the
42 papers (43%) for which detailed comments were received from referees.
In a related but not identical category, in 11 papers
(26%), reviewers reported problems with the bibliographical references, which
were missing, incorrect, and/or used inappropriately from secondary sources
(also see Gibaldi 1999).
The second largest number of comments pertained to
weaknesses in the rationale of papers. Reviewers reported that authors made
unsubstantiated or excessive claims, speculated, mixed categories, over-generalised, and more
generally made inferencing errors in 17 out of the
42 papers (40%) (see Judd et al. 1991: 27ff, 258 f. on various types of
validity and bias).
This problem in the rationale is linked with
problems in the design of empirical studies. Out of 20 papers reporting such
studies, 12 (60% - the highest proportion) were reported as having a design
that did not correspond to the stated aim of the study, or as comparing
groups that were not comparable, or using samples that were too small, or not
representative, or as failing to control relevant variables (also see Judd et
al. 1991: 75ff, 215ff.).
The third largest set of comments considered that
papers (13 out of 42 - 31%) presented no innovation or added value.
Other frequent comments had to do with specialised concepts, which were not defined at all or
not defined clearly enough, or used in a way different from their established
use in other disciplines or in paradigms from which they were taken (9 papers
out of 42 - 21%).
Referees also commented on the excessive length of
papers or segments thereof, in particular the introduction (8 papers out of
42 - 19%).
Still other comments reported clumsy or pompous
style, repetitions in the structure of the paper, missing units in graphs,
abbreviations and acronyms which were not spelled out, lack of indications as
to the objectives of the papers or studies (see Judd et al. 1991: 472 about
procedure and style).
Assessing the truthfulness of the referees’ comments
Before analysing these data, a few points must be made: Firstly,
no strong claim is made as to the degree of representativeness
of the sample of papers, of referees and/or of comments. The numbers are
given here as a rough indication. Nevertheless, the following could be
The set of more than 42 papers reviewed for these
proceedings can be considered of reasonable size for a sample of this nature,
and we (the authors of this paper and editors of this volume) did not feel it
differed markedly in respect of problem areas from other sets of papers
received and reviewed for other volumes in the field in recent years.
An important point is that while there were a few
disagreements (less than a handful) between the referees and/or editors on
the general value of papers, no disagreement was expressed on any specific
comment. Thus, while it is not claimed that referees are holders of the truth
(they are as fallible as any scholar - see the discussion in Section 2.3), in
this case, there is good reason to consider that their judgement
on specific weaknesses has some reasonable inter-subjective value
Referees and editors come from different countries
and have different backgrounds. These factors suggest that the picture which
emerges is not likely to be a highly biased reflection of the general
situation in TS. The only potentially marked difference between this case and
other, similar cases, is a possibly stronger will on the part of the editors
(there was at least one editor among the referees of each paper) to stress
quality in the editorial process, in spite of the other factors mentioned in
3.4 Analysing the data
the data, it appears that referees have reacted on the basis of a number of
norms, which are made explicit in research textbooks (see for instance, McBurney 1983, Shipman 1988, Babbie
1992, Judd et al. 1991, Gibaldi 1999,
Strauss/Corbin 1998), in particular the requirements that science be
explicit, in providing all the information required for readers to be able to
understand and assess reports on scientific studies (“Great care is taken to
specify the exact conditions under which observations are made so that other
scientists can repeat the observations if they desire and can try to obtain
the same results as the first scientists”- McBurney
1990: 7; also see Strauss/Corbin 1991: 266f about replicability),
that it be logical in its inferences, and that scientific endeavours
contribute at least some innovation, rather than just review existing data
and information (in the words of the editor of the Journal of Bone and
Joint Surgery who gave a plenary at a medical conference in Paris in
the early nineties, “the reviewer will ask three questions: Is it true? Is it
new? Does it matter?” - also see Beaud
large proportion of papers received, more than one in three, did not observe
these principles. This suggests a lack of specific awareness of the
importance of these norms in research which, in turn, could be ascribed to
the lack of appropriate training in research principles and their concrete
interesting observation is the relatively high number of problems arising
from trans-disciplinary ventures. As they explore new avenues with theories
and concepts from related disciplines, scholars with a solid track record in
TS seem to become susceptible to misperceptions. Beyond the case study of
this volume of proceedings, problems have been observed when TS scholars
without the corresponding background ventured into statistics, philosophy of
science, linguistics, and cognitive psychology. Such weaknesses could
probably be avoided by co-operating with researchers from the relevant
disciplines, and, of course, by appropriate training in such disciplines.
Referees’ comments and beyond
The next and
obvious questions are, firstly, what authors can/should do after they receive
the referees’ comments. Most authors only view the peer review process as an
assessment, and focus on its short-term effects in terms of acceptance or
rejection of their manuscript and of requirements for rewriting. However, it
could also be an excellent opportunity to improve research and writing
skills. Comments can be classified into three levels of
Some comments are easy to deal with. Following minor
suggestions for grammatical correction and stylistic improvement, correcting
factual errors, correcting errors in the bibliographical references,
improving the graphical presentation of data are ‘mechanical’ and should
cause no problems to authors.
At a second level of difficulty, the referees’
suggestions challenge the authors’ ‘local’ assessments (on specific points):
does a previous finding really mean what the paper says it means? Does
another author really say what the paper reports it is saying? Is a quoted
piece of work good enough to be used as a reference? Should another study not
be cited to strengthen the case? Is the newly coined terminology appropriate,
or should it be replaced by better terms? Should a particular assertion based
on the findings not be toned down?
At the highest level of difficulty are suggestions
that challenge fundamental choices and inferences made by the author. In some
cases, if the design of an empirical study is seriously flawed, if a
theoretical discourse on a concept, model or theory is predicated on a
serious misunderstanding of its nature, the paper cannot be salvaged. In
other cases, it is possible to improve it without starting again from
scratch. For instance, the data may be re-processed with a different method,
and findings can be re-interpreted, which may lead to a new conclusion, which
may not be in line with the ideas that authors set out to defend. Again, this
may be associated with considerable psychological difficulty for them.
peer-review process and the subsequent decisions and actions discussed here
are not the only components of editorial work. Upstream, there are contacts
with a or several publisher(s), definition of
editorial policy, sometimes calls for papers with specific guidelines as to
the length and format of texts. Downstream, when revised papers have been
accepted, there is still much reading, checking, writing, copy-editing, etc.
However, the most stimulating and constructive part certainly lies in dealing
with the issues highlighted in this paper.
preparation of proceedings involves a chain of actions, decisions and
reactions from authors, referees and editors. However clear and
straightforward the principles may be, their
implementation is therefore lengthy, complex, and sometimes a bit painful.
However, if referees and editors have enough time and good environmental
conditions for serious work on the manuscripts, and if authors accept their
comments as potentially useful input, the editorial process offers excellent
skill honing opportunities for all parties in the process.
Babbie, Earl. 1992. The practice of social research.
(Sixth Edition). Belmont, California:
Beaud, Michel.1988. L’art
de la thèse. Paris: Editions la découverte.
1982. What is this thing called Science? (Second Edition). Milton Keynes: Open University Press.
Gibaldi, Joseph. 1999. MLA Handbook for
Writers of Research Papers. (Fifth Edition). New
York: The Modern Language Association of America.
Charles M. & Eliot R. Smith & Louise H. Kidder. 1991. Research
Methods in Social Relations. (Sixth Edition). Orlando:
Harcourt Brace Jovanovich
Kjørup, Søren. 1996. Menneskevidenskaberne. Roskilde: Roskilde Universitetsforlag.
1970. The structure of scientific revolutions. (Second Edition,
enlarged). Chicago: The University of Chicago
McBurney, Donald. 1990. Experimental
Psychology. Belmont, California:
Plastow, Michael & Yoshihide
Igarashi. 1989. The Mind of Science. Tokyo: Kyoritsushuppan.
1988. The limitations of social research. (Third Edition). London and New
Anselm & Juliet Corbin. 1998. Basics of Qualitative Research.
(Second Edition). California:
[back to Resources]