The recent global surge in audiovisual content has emphasised the importance of accessibility for wider audiences. The Shaping Multilingual Access through Respeaking Technology (SMART) project addressed this by exploring interlingual respeaking, a novel practice combining speech recognition technology with human interpreting and subtitling skills to produce real-time speech-to-text services across languages. This method evolved from intralingual respeaking, which is widely used in broadcasting to create live subtitles for the deaf and hard-of-hearing. Interlingual respeaking, which involves translating live content into another language and subtitling it, could revolutionise subtitle production for foreign-language content, overcoming sensory and language barriers. Interlingual respeaking involves two shifts: interlingual (from one language to another) and intermodal (from spoken to written). This practice combines the challenges of simultaneous interpreting with the requirements of subtitling. Respeakers must accurately convey messages in another language to a speech recognition system, adding punctuation and making real-time edits for clarity and readability. This method leverages speech recognition technology and human translation skills to ensure efficient and high-quality translated subtitles. Interlingual respeaking offers immense potential for making multilingual content accessible to international and hearing-impaired audiences. It is particularly relevant for television, conferences, and live events. However, research into this practice is still in its early stages. The SMART project's main goals were to study interlingual respeaking's complexity, focusing on the acquisition and implementation of relevant skills (not only procedural, but also cognitive and interpersonal ones), and the accuracy of the final subtitles. The research involved fifty-one language professionals with backgrounds in relevant language-related practices (namely interpreting, translation, subtitling, and intralingual respeaking). The research programme examined three areas: process, product, and upskilling. It sought to understand the variables contributing to language professionals' performance, challenges faced during performance, and how performance can be sustained. Regarding the product, it aimed to identify factors affecting the accuracy of interlingual respeaking and the impact of various individual and content characteristics on accuracy. For upskilling, the focus was on the challenges and strengths of the course delivered as part of the experiment. Key findings included the importance of working memory in predicting high performance and the enhancement of certain cognitive abilities through training. Interpersonal traits like conscientiousness and integrated regulation were also examined. In terms of product accuracy, the average achieved across 153 performances three per participant) after 25h of upskilling was 95.37%, with omissions being the strongest negative predictor of accuracy. High performers outperformed low performers across all scenarios. The upskilling course was innovative, focusing on modular training and combining intralingual and interlingual practices. It addressed real-world challenges and was tailored to different professional backgrounds. The approach proved effective, with 82% of participants finding the course met their expectations and 86% acknowledging its challenging nature. The study confirmed the benefits of a modular and personalised training approach, highlighting the need for flexibility and adaptability to different skill levels and backgrounds.Our day and age is characterised by a proliferation of live multimedia and multilingual content, such as news and TV programmes. Such content is not accessible to everyone. In many countries, live subtitles in the same language are required by law to enable deaf and hard-of-hearing audiences to enjoy access to information, culture and entertainment. In the UK and other countries (e.g., Canada, Spain, Switzerland), intralingual respeaking is the most well-established technique to produce these subtitles: it relies on human-machine interaction whereby a respeaker listens to the original sound of a live programme or event and simultaneously dictates it to a speech recognition software that turns each sentence into subtitles displayed on the screen. Respeaking so far has been used to produce intralingual subtitles, i.e., in the same language. Given the current multilingual content boom, the SMART project aims to investigate whether respeaking can be used to produce interlingual subtitles, i.e. in a different language.

Topics: