Accurate assessment of a text’s level of translation difficulty is critical for translator training and accreditation, translation research, and the language industry as well. Traditionally, people rely on their general impression to gauge a text’s translation difficulty level. If the evaluation process is to be more effective and the results more objective, an instrument needs to be developed.
In order to develop an instrument to measure translation difficulty, two basic research questions must be answered: what to measure and how to measure it. The potential sources of translation difficulty include translation factors (i.e., text difficulty and translation-specific difficulty [i.e., translation problems in a task]) and translator factors (i.e., translation subcompetences). Accordingly, to measure translation difficulty, we need to measure text difficulty, recognize translation-specific difficulty, and assess translation difficulty (i.e., mental workload) for the translator.
NASA Task Load Index (TLX), a multidimensional scale for measuring subjective workload, can be used to assess translation difficulty for the translator, and it was proved to be reliable in this study.
Performance measures are often used to derive an index of difficulty, and two commonly used indicators are speed (i.e., time-on-task) and accuracy. In this study, it was shown that time spent on a translation was significantly, but weakly, related to translation difficulty level as measured by NASA-TLX. The translation difficulty level self-assessed by high scorers was not consistently lower or higher than that by low scorers, and this indicates that accuracy (i.e., translation quality score in this case) may not be a reliable indicator of difficulty level.
Readability formulas are for measuring text difficulty, and are based on such measurable factors as average sentence length and percentage of difficult words. It was hypothesized that they might be able to measure translation difficulty. This study found that the readability score was weakly correlated with translation difficulty score. That is, a text’s readability only partially accounts for its level of translation difficulty.
A formula was developed using multiple regression to predict a text’s level of translation difficulty (as measured by NASA-TLX) for a translator by using the translator’s pre-translation rating. This will greatly facilitate future studies.