DOES SIZE MATTER? FONT SIZE MANIPULATIONS IN PICTURE-WORD INTERFERENCE TASKS FOR HUMANS AND MACHINE LEARNING MODELS.

Abstract

Recent developments in Machine Learning have developed multimodal models capable of detecting concepts across text and images. These models are however highly susceptible to font size manipulations in adversarial text-based attacks, reducing their applications to real world tasks. Studies into how human cognitive functions handle such adversarial attacks would hence be a useful direction for developers to emulate in their models. However, there is a dearth in the psychological literature on the effects of font size manipulations of distractors. This study investigated the effects of font size and semantic relatedness manipulations on distractor words using the picture-word interference paradigm in an online study, hypothesising that larger font sizes and semantically related distractor words would impede performance in both cognitive models. The findings supported the hypothesised effects of larger font sizes on human reaction times and machine learning models’ accuracy but did not find an effect of semantic relatedness. These findings contribute to the present research gap in font size manipulations as well as an alternative approach in integrating the fields of cognitive psychology and machine learning. Challenges of this approach along with future research implications for psychology and machine learning development are discussed.