Expediting transcription through a new computer system

At Tufts, students of all ages and experience levels join labs and gain invaluable research experience through hands-on work and close mentorship from faculty, fellow students, and postdoctoral scholars. But not only did Muhammad Umair, E21, participate in research in computer science and psychology that laid the groundwork for further innovation in automated transcription (and for his own PhD studies, which he is now conducting at Tufts), he also published his research as a primary author in an academic journal – a rare feat as an undergraduate student.

The Human Interaction Laboratory (HI Lab) at Tufts combines psychology and computer science in the study of cognition and psycholinguistics. As a bridge professor with appointments in both the Department of Computer Science (School of Engineering) and the Department of Psychology (School of Arts and Sciences), Professor JP de Ruiter focuses on the cognitive foundations of human communication. He and a team of researchers including Umair developed GailBot, a computer system that also captures paralinguistic features in automated transcripts of human conversation.

Other automated systems cannot transcribe paralinguistic features such as variations in speech rate, overlapping speech, or laughter. Instead, researchers have historically relied on human transcribers working by hand to painstakingly capture these nuances of human conversation. GailBot expedites the transcription process, creating an accurate, faster, and more cost-effective way for researchers to generate a first draft transcription. Researchers can also add their own plug-ins to the system to further extend GailBot’s abilities, when new algorithms become available through advances in Natural Language Processing.

"This project is a great example of bridging computer science and psychology research, spearheaded by a computer science undergrad," says de Ruiter. Computer science major Umair took a leading role in the research. He was the first author on the paper - titled "GailBot: An automatic transcription service for Conversation Analysis" - published in the prestigious journal Dialogue and Discourse. After completing his BS in computer science, he is now continuing his work in the HI Lab as a PhD candidate studying human-robot interaction. 

Umair’s leading role in the research process as an undergraduate student demonstrates the School of Engineering’s continued commitment to developing the next generation of engineers, and his work has directly contributed to advancements in automated transcription.