1. Adam M, Wessel M, Benlian A (2020) AI-based chatbots in customer service and their effects on
user compliance. Electron Markets. https://doi.org/10.1007/s12525-020-00414-7
2. Adiwardana D, Luong MT, So DR, Hall J, Fiedel N, Thoppilan R, Le QV (2020) Towards a human-like
open-domain chatbot. arXiv preprint. arXiv:2001.09977
3. Amershi S, Weld D, Vorvoreanu M, Fourney A, Nushi B, Collisson P, Teevan J (2019) Guidelines for
human–AI interaction. In: Proceedings of the CHI 2019 (paper no. 3). ACM, New York
4. Araujo T (2018) Living up to the chatbot hype: the influence of anthropomorphic design cues and
communicative agency framing on conversational agent and company perceptions. Comput Hum
Behav 85:183–189
5. Araujo T (2020) Conversational agent research toolkit: an alternative for creating and managing
chatbots for experimental research. Comput Commun Res 2(1):35–51
6. Ashktorab Z, Jain M, Liao QV, Weisz JD (2019) Resilient chatbots: repair strategy preferences for
conversational breakdowns. In: Proceedings of CHI 2019 (paper no. 254). ACM, New York
7. Baez M, Daniel F, Casati F (2019) Conversational web interaction: proposal of a dialog-based natural
language interaction paradigm for the Web. In: Chatbot research and design. Third international
workshop, CONVERSATIONS. Springer, Cham, pp 94–11
8. Bavaresco R, Silveira D, Reis E, Barbosa J, Righi R, Costa C, Moreira C (2020) Conversational agents
in business: a systematic literature review and future research directions. Comput Sci Rev 36:100239.
https://doi.org/10.1016/j.cosrev.2020.100239
9. Bender EM, Gebru T, McMillan-Major A, Mitchell M (2021) On the dangers of stochastic parrots: can
language models be too big? In: Proceedings of the 2021 ACM conference on fairness, accountability,
and transparency. ACM, New York, pp 610–623
10. Bickmore T, Picard RW (2005) Establishing and maintaining long-term human–computer relationships. ACM Trans Comput Hum Interact 12(2):293–327. https://doi.org/10.1145/1067860.1067867
11. Bittner E, Shoury O (2019) Designing automated facilitation for design thinking: a chatbot for supporting teams in the empathy map method. In: Proceedings of the 52nd Hawaii international conference
on system sciences. Scholar Space, Honolulu, pp 227–236
12. Bobrow DG, Kaplan RM, Kay M, Norman DA, Thompson H, Winograd T (1977) GUS, a framedriven dialog system. Artif Intell 8(2):155–173
13. Bott N, Wexler S, Drury L, Pollak C, Wang V, Scher K, Narducci S (2019) A protocol-driven, bedside
digital conversational agent to support nurse teams and mitigate risks of hospitalization in older adults:
case control pre-post study. J Med Internet Res 21(10):e13440
14. Brandtzaeg PB, Følstad A (2017) Why people use chatbots. In: Proceeedings of the international
conference on internet science—INSCI 2017. Springer, Cham, pp 377–392
15. Breazeal C (2003) Toward sociable robots. Robot Auton Syst 42(3–4):167–175
16. Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, Amodei D (2020) Language models
are few-shot learners. arXiv preprint. arXiv:2005.14165
17. Candello H, Pinhanez C, Pichiliani M, Cavalin P, Figueiredo F, Vasconcelos M, Carmo DH (2019)
The effect of audiences on the user experience with conversational interfaces in physical spaces. In:
Proceedings of CHI 2019 (paper no. 90). ACM, New York
18. Cassell J, Bickmore T, Billinghurst M, Campbell L, Chang K, Vilhjálmsson H, Yan H (1999) Embodiment in conversational interfaces: Rea. In: Proceedings of CHI ‘99. ACM, New York, pp 520–527
19. Catania F, Di Nardo N, Garzotto F, Occhiuto D (2019) Emoty: an emotionally sensitive conversational
agent for people with neurodevelopmental disorders. In Proceedings of the 52nd Hawaii international
conference on system sciences. Scholar Space, Honolulu, pp 2014–2023
20. Chen H, Liu X, Yin D, Tang J (2017) A survey on dialogue systems: recent advances and new frontiers.
ACM SIGKDD Explor Newsl 19(2):25–35
21. Chung H, Iorga M, Voas J, Lee S (2017) Alexa, can I trust you? Computer 50(9):100–104
22. Ciechanowski L, Przegalinska A, Magnuski M, Gloor P (2019) In the shades of the uncanny valley:
an experimental study of human–chatbot interaction. Future Gener Comput Syst 92:539–548
23. Corkrey R, Parkinson L (2002) Interactive voice response: review of studies 1989–2000. Behav Res
Methods Instruments Comput 34(3):342–353
24. Cranor LF (2008) A framework for reasoning about the human in the loop. In :UPSec’08: Proceedings
of the 1st conference on usability, psychology, and security. USENIX Association, Berkeley
25. Dale R (2016) The return of the chatbots. Nat Lang Eng 22(5):811–817
123
Future directions for chatbot research…
26. Daniel G, Cabot J, Deruelle L, Derras M (2020) Xatkit: a multimodal low-code chatbot development
framework. IEEE Access 8:15332–15346
27. DeVault D, Artstein R, Benn G, Dey T, Fast E, Gainer A, Morency LP (2014) SimSensei Kiosk: a
virtual human interviewer for healthcare decision support. In Proceedings of the 2014 international
conference on autonomous agents and multi-agent systems, pp 1061–1068
28. Deriu J, Rodrigo A, Otegi A, Echegoyen G, Rosset S, Agirre E, Cieliebak M (2021) Survey on
evaluation methods for dialogue systems. Artif Intell Rev 54:755–810
29. Devlin J, Chang MW, Lee K, Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers
for language understanding. arXiv preprint, arXiv:1810.04805
30. European Commission (2019) Ethics guidelines for trustworthy AI. Retrieved from: https://digitalstrategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai
31. Farooq U, Grudin J (2016) Human–computer integration. Interactions 23(6):26–32
32. Feine J, Gnewuch U, Morana S, Maedche A (2019a) A taxonomy of social cues for conversational
agents. Int J Hum Comput Stud 132:138–161
33. Feine J, Gnewuch U, Morana S, Maedche A (2019b) Gender bias in chatbot design. In: Chatbot
research and design. Third international workshop CONVERSATIONS 2019. Springer, Cham, pp
79–93
34. Ferrara E (2020) What types of COVID-19 conspiracies are populated by Twitter bots? First Monday.
https://doi.org/10.5210/fm.v25i6.10633
35. Fitzpatrick KK, Darcy A, Vierhile M (2017) Delivering cognitive behavior therapy to young adults
with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a
randomized controlled trial. JMIR Mental Health 4(2):e19
36. Floridi L, Chiriatti M (2020) GPT-3: Its nature, scope, limits, and consequences. Mind Mach
30(4):681–694
37. Frey CB, Osborne MA (2017) The future of employment: How susceptible are jobs to computerisation? Technol Forecast Soc Chang 114:254–280
38. Følstad A, Brandtzæg PB (2017) Chatbots and the new world of HCI. Interactions 24(4):38–42
39. Følstad A, Brandtzaeg PB (2020) Users’ experiences with chatbots: findings from a questionnaire
study. Quality User Exp. https://doi.org/10.1007/s41233-020-00033-2
40. Følstad A, Brandtzaeg PB, Feltwell T, Law ELC, Tscheligi M, Luger E (2018) Chatbots for social
good. In: Extended abstracts of the 2018 CHI conference on human factors in computing systems
(paper no. SIG06). ACM, New York
41. Følstad A, Araujo T, Papadopoulos S, Law EL-C, Luger E, Goodwin M, Brandtzaeg PB (eds) (2020)
Chatbot research and design 4th international workshop, CONVERSATIONS 2020. Springer, Cham
42. Go E, Sundar SS (2019) Humanizing chatbots: the effects of visual, identity and conversational cues
on humanness perceptions. Comput Hum Behav 97:304–316
43. Goel AK, Polepeddi L (2016) Jill Watson: A virtual teaching assistant for online education. Georgia
Institute of Technology. Retrieved from https://smartech.gatech.edu/handle/1853/59104
44. Gorwa R, Guilbeault D (2020) Unpacking the social media bot: a typology to guide research and
policy. Policy Internet 12(2):225–248
45. Grudin J, Jacques R (2019) Chatbots, humbots, and the quest for artificial general intelligence. In:
Proceedings of CHI 2019 (paper no. 209). ACM, New York
46. Guzman AL, Lewis SC (2019) Artificial intelligence and communication: a human–machine communication research agenda. New Media Soc 22(1):70–86
47. Hagendorff T (2020) The ethics of AI ethics: An evaluation of guidelines. Mind Mach 30(1):99–120
48. Hall E (2018) Conversational Design. A Book Apart, New York
49. Hao K (2020) We read the paper that forced Timnit Gebru out of Google. Here’s what it says.
MIT Technology Review. Retrieved from https://www.technologyreview.com/2020/12/04/1013294/
google-ai-ethics-research-paper-forced-out-timnit-gebru/
50. Harms JG, Kucherbaev P, Bozzon A, Houben GJ (2018) Approaches for dialog management in
conversational agents. IEEE Internet Comput 23(2):13–22
51. Harwell D (2018) The accent gap. The Washington Post. Retrieved from https://www.washingtonpost.
com/graphics/2018/business/alexa-does-not-understand-your-accent/
52. Ho A, Hancock J, Miner AS (2018) Psychological, relational, and emotional effects of self-disclosure
after conversations with a chatbot. J Commun 68(4):712–733
123
A. Følstad et al.
53. Hobert S, Berens F (2019) Small talk conversations and the long-term use of chatbots in educational
settings–experiences from a field study. In: Chatbot research and design. Third international workshop
CONVERSATIONS 2019. Springer, Cham, pp 260–272
54. Hobert S, Meyer von Wolff, R (2019) Say hello to your new automated tutor—a structured literature
review on pedagogical conversational agents. In: Proceedings of the 14th international conference on
Wirtschaftsinformatik. Association for Information Systems, pp.301–313
55. Hoffman G, Breazeal C (2004) Collaboration in human–robot teams. In: AIAA 1st intelligent systems
technical conference. https://doi.org/10.2514/6.2004-6434
56. Hsieh T, Chaudhury B, Cross ES (2020) Human–robot cooperation in prisoner dilemma games: people
behave more reciprocally than prosocially toward robots. In: Companion of the 2020 ACM/IEEE
international conference on human–robot interaction. ACM, New York, pp 257–259
57. Jain M, Kumar P, Kota R, Patel SN (2018) Evaluating and informing the design of chatbots. In:
Proceedings of the 2018 designing interactive systems conference. ACM, New York, pp 895–906
58. McTear MF (2002) Spoken dialogue technology: enabling the conversational user interface. ACM
Comput Surv (CSUR) 34(1):90–169
59. Jokinen K, McTear M (2009) Spoken dialogue systems. Synthesis lectures on human language technologies. Morgan & Claypool Publishers, Williston
60. Jovanovic M, Baez M, Casati F (2020) Chatbots as conversational healthcare services. IEEE Internet
Comput. https://doi.org/10.1109/MIC.2020.3037151
61. Jurafsky D, Martin JH (2020) Dialogue systems and chatbots. In: Speech and language processing.
Draft of December 30, 2020. https://web.stanford.edu/~jurafsky/slp3/24.pdf
62. Kim S, Lee J, Gweon G (2019) Comparing data from chatbot and web surveys: effects of platform
and conversational style on survey response quality. In: Proceedings of CHI 2019 (paper no. 86).
ACM, New York
63. Kocaballi AB, Laranjo L, Coiera E (2019) Understanding and measuring user experience in conversational interfaces. Interact Comput 31(2):192–207
64. Kopp S, Gesellensetter L, Krämer NC, Wachsmuth I (2005) A conversational agent as museum
guide–design and evaluation of a real-world application. In: International workshop on intelligent
virtual agents. Springer, Heidelberg, pp 329–343
65. Kretzschmar K, Tyroll H, Pavarini G, Manzini A, Singh I (2019) Can your phone be your therapist?
Young people’s ethical perspectives on the use of fully automated conversational agents (chatbots) in
mental health support. Biomed Inform Insights. https://doi.org/10.1177/1178222619829083
66. Kvale K, Sell OA, Hodnebrog S, Følstad A (2019) Improving conversations: lessons learnt from
manual analysis of chatbot dialogues. In: Chatbot research and design. Third international workshop
CONVERSATIONS 2019. Springer, Cham, pp 187–200
67. Laban G, George J, Morrison V, Cross ES (2021) Tell me more! Assessing interactions with social
robots from speech. Paladyn J Behav Robot 12(1):136–159
68. Laban G, Araujo T (2019) Working together with conversational agents: the relationship of perceived
cooperation with service performance evaluations. In: Chatbot research and design. Third international
workshop CONVERSATIONS 2019. Springer, Cham, pp 215–228
69. Laban G, Araujo T (2020) The effect of personalization techniques in users’ perceptions of conversational recommender system. In: IVA ’20: Proceedings of the 20th ACM international conference
on intelligent virtual agents (paper no. 34). ACM, New York
70. Latour B (2005) Reassembling the Social - An introduction to actor-network-theory. Oxford University Press, Oxford, UK
71. Lee S, Choi J (2017) Enhancing user experience with conversational agent for movie recommendation:
effects of self-disclosure and reciprocity. Int J Hum Comput Stud 103:95–105
72. Lester J, Branting K, Mott B (2004) Conversational agents. In: Singh MP (ed) The Practical Handbook
of Internet Computing. Chapman & Hall/CRC, Boca Raton
73. Liao QV, Geyer W, Muller M, Khazaen Y (2020) Conversational interfaces for information search.
Understanding and improving information search. Springer, Cham, pp 267–287
74. Liao QV, Mas-ud Hussain M, Chandar P, Davis M, Khazaeni Y, Crasso MP, Geyer W (2018) All
work and no play? In Proceedings of CHI 2018 (paper no. 3). ACM,New York
75. Luger E, Sellen A (2016) “Like having a really bad PA”—the gulf between user expectation and
experience of conversational agents. In: Proceedings of CHI 2016. ACM, New York, pp 5286–5297
76. McAfee A, Brynjolfsson E (2017) Machine, platform, crowd: harnessing our digital future. W.W.
Norton & Company, New York
123
Future directions for chatbot research…
77. McTear M (2021) Conversational AI: Dialogue Systems, Conversational Agents, and Chatbots. Morgan & Claypool, Williston
78. Meyer von Wolff R, Hobert S, Masuch K, Schumann M (2020) Chatbots at digital workplaces - A
grounded-theory approach for surveying application areas and objectives. Pac Asia J Assoc Inf Syst
12(2):64–102
79. Moore RJ, Arar R (2019) Conversational UX design: a practitioner’s guide to the natural conversation
framework. Morgan & Claypool, Williston
80. Nordheim CB, Følstad A, Bjørkli CA (2019) An initial model of trust in chatbots for customer
service—findings from a questionnaire study. Interact Comput 31(3):317–335
81. Ntoutsi E, Fafalios P, Gadiraju U, Iosifidis V, Nejdl W, Vidal ME, Ruggieri S, Turini F, Papadopoulos S, Krasanakis E, Kompatsiaris I (2020) Bias in data-driven artificial intelligence systems—an
introductory survey. Wiley Interdiscipl Rev Data Min Knowl Discov 10(3):e1356
82. Oh CS, Bailenson JN, Welch GF (2018) A systematic review of social presence: definition,
antecedents, and implications. Front Robot AI 5:114
83. Paikens P, Znoti ¸nš A, B¯arzdi ¸nš G (2020) Human-in-the-loop conversation agent for customer service.
In: Proceedings of the 25th international conference on applications of natural language to information
systems, NLDB 2020. Springer, Cham, pp 277–284
84. Pérez JQ, Daradoumis T, Puig JMM (2020) Rediscovering the use of chatbots in education: a systematic literature review. Comput Appl Eng Educ 28(6):1549–1565
85. Perski O, Crane D, Beard E, Brown J (2019) Does the addition of a supportive chatbot promote user engagement with a smoking cessation app? An experimental study. Digital Health
5:2055207619880676
86. Piccolo LS, Mensio M, Alani H (2018) Chasing the chatbots. In Internet Science. INSCI 2018
international workshops. Springer, Cham, pp 157–169
87. Porcheron M, Fischer JE, Reeves S, Sharples S (2018) Voice interfaces in everyday life. In: Proceedings of CHI 2018 (paper no. 640). ACM,New York
88. Press G (2019) AI Stats News: 62 % of US Consumers like using chatbots to interact with businesses. Forbes. Retrieved from https://www.forbes.com/sites/gilpress/2019/10/25/ai-stats-news-usconsumers-interest-in-using-chatbots-to-interact-with-businesses-rise-to-62/
89. Rodríguez-Gil L, García-Zubia J, Orduña P, Villar-Martinez A, López-De-Ipiña D (2019) New
approach for conversational agent definition by non-programmers: a visual domain-specific language.
IEEE Access 7:5262–5276
90. Roller S, Dinan E, Goyal N, Ju D, Williamson M, Liu Y, Boureau YL (2020) Recipes for building an
open-domain chatbot. arXiv preprint. arXiv:2004.13637
91. Ruane E, Birhane A, Ventresque A (2019) Conversational AI: social and ethical considerations. In:
Proceedings for the 27th AIAI Irish conference on artificial intelligence and cognitive science—AICS
2019, pp 104–115. CEUR Workshop Proceedings, Vol-2563. http://ceur-ws.org/Vol-2563/
92. Rubin VL, Chen Y, Thorimbert LM (2010) Artificially intelligent conversational agents in libraries.
Library Hi Tech 28(4):496–522
93. Salem M, Kopp S, Wachsmuth I, Rohlfing K, Joublin F (2012) Generation and evaluation of communicative robot gesture. Int J Social Robot 4(2):201–217
94. Schaake M (2021). The European commission’s artificial intelligence act. Policy Brief, Stanford HAI.
Retrieved from https://hai.stanford.edu/sites/default/files/2021-06/HAI_Issue-Brief_The-EuropeanCommissions-Artificial-Intelligence-Act.pdf
95. Schuller BW (2018) Speech emotion recognition: two decades in a nutshell, benchmarks, and ongoing
trends. Commun ACM 61(5):90–99
96. Seering J, Luria M, Ye C, Kaufman G, Hammer J (2020) It takes a village: integrating an adaptive
chatbot into an online gaming community. In: Proceedings of CHI 2020 (paper no. 579). ACM,New
York
97. Shevat A (2017) Designing bots: creating conversational experiences. O’Reilly Media, Boston
98. Shum HY, He XD, Li D (2018) From Eliza to XiaoIce: challenges and opportunities with social
chatbots. Front Inf Technol Electron Eng 19(1):10–26
99. Strubell E, Ganesh A, McCallum A (2019) Energy and policy considerations for deep learning in
NLP. arXiv preprint, arXiv:1906.02243
100. Sundar SS (2020) Rise of machine agency: a framework for studying the psychology of Human–AI
Interaction (HAII). J Comput Mediat Commun 25(1):74–88
123
A. Følstad et al.
101. Suresh H, Guttag JV (2019) A framework for understanding unintended consequences of machine
learning. arXiv preprint, arXiv:1901.10002
102. Syvänen S, Valentini C (2020) Conversational agents in online organization–stakeholder interactions:
a state-of-the-art analysis and implications for further research. J Commun Manag 339–362
103. Ta V, Griffith C, Boatfield C, Wang X, Civitello M, Bader H, Loggarakis A (2020) User experiences
of social support from companion chatbots in everyday contexts: thematic analysis. J Med Internet
Res 22(3):e16235
104. Toxtli C, Monroy-Hernández A, Cranshaw J (2018) Understanding chatbot-mediated task management. In: Proceedings of CHI 2018 (paper no. 58). ACM,New York
105. Vaidyam AN, Wisniewski H, Halamka JD, Kashavan MS, Torous JB (2019) Chatbots and conversational agents in mental health: a review of the psychiatric landscape. Can J Psychiatry 64(7):456–464.
https://doi.org/10.1177/0706743719828977
106. Van der Goot MJ, Pilgrim T (2019) Exploring age differences in motivations for and acceptance
of chatbot communication in a customer service context. In: Chatbot research and design. Third
international workshop CONVERSATIONS 2019. Springer, Cham, pp 173–186
107. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017)
Attention is all you need. In: Conference on neural information processing systems—NIPS 2017, pp
5998–6008
108. Vaziri M, Mandel L, Shinnar A, Siméon J, Hirzel M (2017) Generating chat bots from web API
specifications. In: Proceedings of the 2017 ACM SIGPLAN international symposium on new ideas,
new Pparadigm, and reflections on programming and software. ACM, New York, pp 44–57
109. Vinyals O, Le Q (2015) A neural conversational model. arXiv preprint, arXiv:1506.05869
110. Väänänen K, Hiltunen A, Varsaluoma J, Pietilä I (2019) CivicBots–Chatbots for supporting youth in
societal participation. In: Chatbot research and design. Third international workshop CONVERSATIONS 2019. Springer, Cham, pp 143–157
111. Wallace R (2003) The elements of AIML style. Alice AI Foundation, 139
112. Weizenbaum J (1967) Contextual understanding by computers. Commun ACM 10(8):474–480
113. Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, Cistac P, Rault T, Louf R, Funtowicz
M, Brew J (2019) HuggingFace’s transformers: state-of-the-art natural language processing. arXiv
preprint, arxiv:1910.03771
114. Xu A, Liu Z, Guo Y, Sinha V, Akkiraju R (2017) A new chatbot for customer service on social media.
In: Proceedings of CHI 2017. ACM, New York, pp 3506–3510
115. Yaghoub-Zadeh-Fard MA, Zamanirad S, Benatallah B, Casati F (2020) REST2Bot: bridging the gap
between bot platforms and REST APIs. In: Companion proceedings of the web conference 2020.
ACM, New York, pp 245–248
116. Yang Q, Steinfeld A, Rosé C, Zimmerman J (2020) Re-examining whether, why, and how Human-AI
Interaction is uniquely difficult to design. In: Proceedings of CHI 2020 (paper no. 174). ACM,New
York
117. Zamora J (2017) I’m sorry, Dave, I’m afraid I can’t do that: Chatbot perception and expectations.
In: Proceedings of the 5th international conference on human agent Interaction. ACM,New York,pp
253–260
118. Zarouali B, Van den Broeck E, Walrave M, Poels K (2018) Predicting consumer responses to a chatbot
on Facebook. Cyberpsychol Behav Soc Netw 21(8):491–497
119. Zierau N, Engel C, Söllner M, Leimeister JM (2020) Trust in smart personal assistants: A systematic
literature review and development of a research agenda. In: Proceedings of the 15th international
conference on Wirtschaftsinformatik—WI2020. Association for Information Systems,pp 99–11
120. Zou J, Schiebinger L (2018) AI can be sexist and racist—it’s time to make it fair. Nature 559:324–326
![[img]](https://eprints.gla.ac.uk/style/images/fileicons/text.png)
References
References