Please use this identifier to cite or link to this item: https://doi.org/10.18653/v1/2021.acl-long.321
Title: Reliability Testing for Natural Language Processing Systems
Authors: Tan, Samson
Joty, Shafiq
Baxter, Kathy
Taeihagh, Araz 
Bennett, Gregory A
Kan, Min-Yen 
Keywords: Science & Technology
Social Sciences
Technology
Computer Science, Artificial Intelligence
Computer Science, Interdisciplinary Applications
Linguistics
Computer Science
Issue Date: 1-Jan-2021
Publisher: ASSOC COMPUTATIONAL LINGUISTICS-ACL
Citation: Tan, Samson, Joty, Shafiq, Baxter, Kathy, Taeihagh, Araz, Bennett, Gregory A, Kan, Min-Yen (2021-01-01). Reliability Testing for Natural Language Processing Systems. Joint Conference of 59th Annual Meeting of the Association-for-Computational-Linguistics (ACL) / 11th International Joint Conference on Natural Language Processing (IJCNLP) / 6th Workshop on Representation Learning for NLP (RepL4NLP) abs/2105.02590 : 4153-4169. ScholarBank@NUS Repository. https://doi.org/10.18653/v1/2021.acl-long.321
Abstract: Questions of fairness, robustness, and transparency are paramount to address before deploying NLP systems. Central to these concerns is the question of reliability: Can NLP systems reliably treat different demographics fairly and function correctly in diverse and noisy environments? To address this, we argue for the need for reliability testing and contextualize it among existing work on improving accountability. We show how adversarial attacks can be reframed for this goal, via a framework for developing reliability tests. We argue that reliability testing - with an emphasis on interdisciplinary collaboration - will enable rigorous and targeted testing, and aid in the enactment and enforcement of industry standards.
Source Title: Joint Conference of 59th Annual Meeting of the Association-for-Computational-Linguistics (ACL) / 11th International Joint Conference on Natural Language Processing (IJCNLP) / 6th Workshop on Representation Learning for NLP (RepL4NLP)
URI: https://scholarbank.nus.edu.sg/handle/10635/229387
ISBN: 9781954085527
DOI: 10.18653/v1/2021.acl-long.321
Appears in Collections:Staff Publications
Elements

Show full item record
Files in This Item:
File Description SizeFormatAccess SettingsVersion 
2105.02590v3.pdfPublished version603.03 kBAdobe PDF

OPEN

Post-printView/Download

Page view(s)

39
checked on Sep 29, 2022

Download(s)

11
checked on Sep 29, 2022

Google ScholarTM

Check

Altmetric


Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.