Task-based evaluation of context-sensitive referring expressions in human–robot dialogue

References

• Argyle, M., & Graham, J. A. (1976). The Central Europe experiment: Looking at persons and looking at objects. Environmental Psychology and Nonverbal Behavior, 1(1), 6–16. doi:10.1007/BF01115461
   Web of Science ®Google Scholar
• Ariel, M. (1991). The function of accessibility in a theory of grammar. Journal of Pragmatics, 16, 443–463. doi:10.1016/0378-2166(91)90136-L
   Web of Science ®Google Scholar
• Bard, E. G., & Aylett, M. P. (2004). Referential form, word duration, and modeling the listener in spoken dialogue. In J. C. Trueswell & M. K. Tanenhaus (Eds.), Approaches to studying world-situated language use: Bridging the language-as-product and language-as-action traditions (pp. 173–192). Cambridge, MA: The MIT Press.
   Google Scholar
• Bard, E. G., Bekkering, H., Bicho, E., de Bruijn, E., Cuijpers, R., Erlhagen, W., … Sousa, M. (2009). Final report on the evaluation of the robot system (Deliverable No. 5.13). JAST Project.
   Google Scholar
• Bard E. G., Hill R., Foster M. E., & Arai M. (2013, in press). Tuning accessibility of referring expressions in situated dialogue. Language and Cognitive Processes.
   Google Scholar
• Bartneck, C., Kulić, D., Croft, E., & Zoghbi, S. (2009). Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots. International Journal of Social Robotics, 1, 71–81. doi:10.1007/s12369-008-0001-3
   Web of Science ®Google Scholar
• Belz, A., Kow, E., Viethen, J., & Gatt, A. (2010). Generating referring expressions in context: The GREC task evaluation challenges. In E. Krahmer & M. Theune (Eds.), Empirical methods in natural language generation (pp. 294–327). Berlin/Heidelberg: Springer.
   Google Scholar
• Belz, A., & Reiter, E. (2006). Comparing automatic and human evaluation of NLG systems. In Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2006) (pp. 313–320). Trento. Retrieved from http://aclweb.org/anthology/E06-1040.pdf
   Google Scholar
• Bicho, E., Erlhagen, W., Louro, L., & Costa e Silva, E. (2011). Neuro-cognitive mechanisms of decision making in joint action: A human-robot interaction study. Human Movement Science, 30, 846–868. doi:10.1016/j.humov.2010.08.012
   PubMed Web of Science ®Google Scholar
• Bicho, E., Louro, L., & Erlhagen, W. (2010). Integrating verbal and nonverbal communication in a dynamic neural field architecture for human-robot interaction. Frontiers in Neurorobotics, 4(5). doi:10.3389/fnbot.2010.00005
   Google Scholar
• Binsted, K., Pain, H., & Ritchie, G. (1997). Children's evaluation of computer-generated punning riddles. Pragmatics & Cognition, 5, 305–354. doi:10.1075/pc.5.2.06bin
   Google Scholar
• Campana, E., Tanenhaus, M. K., Allen, J. F., & Remington, R. (2011). Natural discourse reference generation reduces cognitive load in spoken systems. Natural Language Engineering, 17, 311–329. doi:10.1017/S1351324910000227
   Web of Science ®Google Scholar
• Carenini, G., & Moore, J. D. (2006). Generating and evaluating evaluative arguments. Artificial Intelligence, 170, 925–952. doi:10.1016/j.artint.2006.05.003
   Web of Science ®Google Scholar
• Carletta, J., Hill, R., Nicol, C., Taylor, T., de Ruiter, J., & Bard, E. (2010). Eyetracking for two-person tasks with manipulation of a virtual world. Behavior Research Methods, 42, 254–265. doi:10.3758/BRM.42.1.254
   PubMed Web of Science ®Google Scholar
• Clark, H. H., & Bangerter, A. (2004). Changing ideas about reference. In I. A. Novacek & D. Sperber (Eds.), Experimental pragmatics (pp. 25–49). Basingstoke: Palgrave Macmillan.
   Google Scholar
• Clark, H. H., & Wilkes-Gibbs, D. (1986). Referring as a collaborative process. Cognition, 22(1), 1–39. doi:10.1016/0010-0277(86)90010-7
   PubMed Web of Science ®Google Scholar
• Dale, R., & Reiter, E. (1995). Computational interpretations of the Gricean maxims in the generation of referring expressions. Cognitive Science, 19, 233–263. doi:10.1207/s15516709cog1902_3
   Web of Science ®Google Scholar
• Foster, M. E. (2008). Automated metrics that agree with human judgements on generated output for an embodied conversational agent. In Proceedings of the 5th International Conference on Natural Language Generation (INLG 2008) (pp. 95–103). Salt Fork, OH. Retrieved from http://www.aclweb.org/anthology/W08-1113
   Google Scholar
• Foster, M. E., Bard, E. G., Hill, R. L., Guhe, M., Oberlander, J., & Knoll, A. (2008). The roles of haptic-ostensive referring expressions in cooperative, task-based human-robot dialogue. In Proceedings of the 3rd ACM/IEEE International Conference on Human Robot Interaction (HRI 2008) (pp. 295–302). Amsterdam. doi:10.1145/1349822.1349861
   Google Scholar
• Foster, M. E., Giuliani, M., Isard, A., Matheson, C., Oberlander, J., & Knoll, A. (2009). Evaluating description and reference strategies in a cooperative human-robot dialogue system. In Proceedings of the 21st International Joint Conference on Artificial Intelligence (IJCAI-09) (pp. 1818–1823). Pasadena, CA. Retrieved from http://ijcai.org/papers09/Papers/IJCAI09-302.pdf
   Google Scholar
• Foster, M. E., Giuliani, M., & Knoll, A. (2009). Comparing objective and subjective measures of usability in a human-robot dialogue system. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP (ACL-IJCNLP 2009) (pp. 879–887). Singapore. Retrieved from http://www.aclweb.org/anthology/P09-1099
   Google Scholar
• Garrod, S., & Pickering, M. J. (2009). Joint action, interactive alignment, and dialog. Topics in Cognitive Science, 1, 292–304. doi:10.1111/j.1756-8765.2009.01020.x
   PubMed Web of Science ®Google Scholar
• Gatt, A., & Belz, A. (2010). Introducing shared tasks to NLG: The TUNA shared task evaluation challenges. In E. Krahmer & M. Theune (Eds.), Empirical methods in natural language generation (pp. 264–293). Berlin/Heidelberg: Springer.
   Google Scholar
• Giuliani, M., Foster, M. E., Isard, A., Matheson, C., Oberlander, J., & Knoll, A. (2010). Situated reference in a hybrid human-robot interaction system. In Proceedings of the 6th International Natural Language Generation Conference (INLG 2010) (pp. 67–76). Trim. Retrieved from http://aclweb.org/anthology/W10-4207
   Google Scholar
• Giuliani, M., & Knoll, A. (2008). MultiML: A general purpose representation language for multimodal human utterances. In Proceedings of the 10th International Conference on Multimodal Interfaces (ICMI 2008) (pp. 165–172). Chania, Crete. doi:10.1145/1452392.1452424
   Google Scholar
• Grice, H. P. (1975). Logic and conversation. In P. Cole & J. Morgan (Eds.), Syntax and Semantics: Vol 3, speech acts (pp. 43–58). New York: Academic.
   Google Scholar
• Grosz, B. J., Weinstein, S., & Joshi, A. K. (1995). Centering: A framework for modeling the local coherence of discourse. Computational Linguistics, 21, 203–225. Retrieved from http://aclweb.org/anthology/J95-2003
   Web of Science ®Google Scholar
• Guhe, M., & Bard, E. G. (2008). Adapting referring expressions to the task environment. In Proceedings of the 30th Annual Meeting of the Cognitive Science Society (CogSci 2008) (pp. 2404–2409). Washington, DC. Retrieved from http://csjarchive.cogsci.rpi.edu/proceedings/2008/pdfs/p2404.pdf
   Google Scholar
• Gundel, J. K., Hedberg, N., & Zacharski, R. (1993). Cognitive status and the form of referring expressions in discourse. Language, 69, 274–307. Retrieved from http://www.jstor.org/stable/416535doi:10.2307/416535
   Web of Science ®Google Scholar
• Horton, W. S., & Keysar, B. (1996). When do speakers take into account common ground? Cognition, 59(1), 91–117. doi:10.1016/0010-0277(96)81418-1
   PubMed Web of Science ®Google Scholar
• Karasimos, A., & Isard, A. (2004). Multi-lingual evaluation of a natural language generation system. In Proceedings of the 4th International Conference on Language Resources and Evaluation (LREC 2004) (pp. 829–832). Lisbon. Retrieved from http://www.lrec-conf.org/proceedings/lrec2004/pdf/134.pdf
   Google Scholar
• Kelleher, J. D., & Kruijff, G.-J. M. (2006). Incremental generation of spatial referring expressions in situated dialog. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics (COLING-ACL 2006) (pp. 1041–1048). Sydney.
   Google Scholar
• Koller, A., Staudte, M., Garoufi, K., & Crocker, M. (2012). Enhancing referential success by tracking hearer gaze. In Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGdial 2012) (pp. 30–39). Seoul. Retrieved from http://www.aclweb.org/anthology/W12-1604
   Google Scholar
• Koller, A., Striegnitz, K., Gargett, A., Byron, D., Cassell, J., Dale, R., … Oberlander, J. (2010). Report on the second NLG challenge on generating instructions in virtual environments (GIVE-2). In Proceedings of the 6th International Natural Language Generation Conference (INLG 2010) (pp. 243–250). Trim. Retrieved from http://aclweb.org/anthology/W10-4233
   Google Scholar
• Krahmer, E. (2010). What computational linguists can learn from psychologists (and vice versa). Computational Linguistics, 36, 285–294. doi:10.1162/coli.2010.36.2.36201
   Web of Science ®Google Scholar
• Krahmer, E., & Theune, M. (2002). Efficient context-sensitive generation of referring expressions. In K. van Deemter & R. Kibble (Eds.), Information sharing: Reference and presupposition in language generation and interpretation (pp. 223–264). Stanford: CSLI Publications.
   Google Scholar
• Krahmer, E., & van Deemter, K. (2012). Computational generation of referring expressions: A survey. Computational Linguistics, 38, 173–218. doi:10.1162/COLI_a_00088
   Web of Science ®Google Scholar
• Kranstedt, A., Lücking, A., Pfeiffer, T., Rieser, H., & Wachsmuth, I. (2006). Deictic object reference in task-oriented dialogue. In G. Rickheit & I. Wachsmuth (Eds.), Situated communication (pp. 155–207). Berlin: Mouton de Gruyter.
   Google Scholar
• Kranstedt, A., & Wachsmuth, I. (2005). Incremental generation of multimodal deixis referring to objects. In Proceedings of the 10th European Workshop on Natural Language Generation (ENLG-05) (pp. 75–82). Aberdeen. Retrieved from http://aclweb.org/anthology/W05-1608
   Google Scholar
• Larsson, S., & Traum, D. (2000). Information state and dialogue management in the TRINDI dialogue move engine toolkit. Natural Language Engineering, 6, 323–340. doi:10.1017/S1351324900002539
   Google Scholar
• Litman, D. J., & Pan, S. (2002). Designing and evaluating an adaptive spoken dialogue system. User Modeling and User-Adapted Interaction, 12(2–3), 111–137. doi:10.1023/A:1015036910358
   Web of Science ®Google Scholar
• Mellish, C., & Dale, R. (1998). Evaluation in the context of natural language generation. Computer Speech & Language, 12, 349–373. doi:10.1006/csla.1998.0106
   Web of Science ®Google Scholar
• Nielsen, J., & Levy, J. (1994). Measuring usability: Preference vs. performance. Communications of the ACM, 37(4), 66–75. doi:10.1145/175276.175282
   Web of Science ®Google Scholar
• Oviatt, S. (1999). Ten myths of multimodal interaction. Communications of the ACM, 42(11), 74–81. doi:10.1145/319382.319398
   Web of Science ®Google Scholar
• Reiter, E. (2011). Task-based evaluation of NLG systems: Control vs real-world context. In Proceedings of the UCNLG + Eval: Language Generation and Evaluation Workshop (pp. 28–32). Edinburgh. Retrieved from http://www.aclweb.org/anthology/W11-2704
   Google Scholar
• Spanger, P., Iida, R., Tokunaga, T., Terai, A., & Kuriyama, N. (2013). A task-performance evaluation of referring expressions in situated collaborative task dialogues. Language Resources and Evaluation. Advance online publication. doi:10.1007/s10579-013-9240-5
   Web of Science ®Google Scholar
• Staudte, M., & Crocker, M. W. (2009). Producing and resolving multi-modal referring expressions in human-robot interaction. In Proceedings of the PRE-CogSci Workshop at CogSci 2009. Amsterdam. Retrieved from http://pre2009.uvt.nl/pdf/staudte.pdf
   Google Scholar
• Steedman, M. (2000). The syntactic process. Cambridge, MA: The MIT Press.
   Google Scholar
• Tokunaga, T., Iida, R., Terai, A., & Kuriyama, N. (2012). The REX corpora: A collection of multimodal corpora of referring expressions in collaborative problem solving dialogues. In Proceedings of the 8th International Conference on Language Resources and Evaluation (LREC 2012). Istanbul. Retrieved from http://www.lrec-conf.org/proceedings/lrec2012/pdf/676_Paper.pdf
   Google Scholar
• van Breemen, A., Yan, X., & Meerbeek, B. (2005). iCat: An animated user-interface robot with personality. In Proceedings of the 4th International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS 2005) (pp. 143–144). Utrecht.
   Google Scholar
• van Deemter, K., Gatt, A., van der Sluis, I., & Power, R. (2012). Generation of referring expressions: Assessing the incremental algorithm. Cognitive Science, 36, 799–836. doi:10.1111/j.1551-6709.2011.01205.x
   PubMed Web of Science ®Google Scholar
• van Deemter, K., van der Sluis, I., & Gatt, A. (2006). Building a semantically transparent corpus for the generation of referring expressions. In Proceedings of the 4th International Conference on Natural Language Generation (INLG 2006) (pp. 130–132). Sydney. Retrieved from http://www.aclweb.org/anthology/W06-1420
   Google Scholar
• van der Sluis, I. (2005). Multimodal reference: Studies in automatic generation of multimodal referring expressions (Doctoral dissertation). Tilburg, Tilburg University.
   Google Scholar
• van der Sluis, I., & Luz, S. (2011). A cross-linguistic study on the production of multimodal referring expressions in dialogue. In Proceedings of the 13th European Workshop on Natural Language Generation (ENLG 2011) (pp. 53–62). Nancy. Retrieved from http://www.aclweb.org/anthology/W11-2807
   Google Scholar
• Viethen, H., Dale, R., & Guhe, M. (2011). The impact of visual context on the content of referring expressions. In Proceedings of the 13th European Workshop on Natural Language Generation (ENLG 2011) (pp. 44–52). Nancy. Retrieved from http://www.aclweb.org/anthology/W11-2806
   Google Scholar
• Walker, M. A., Litman, D. J., Kamm, C. A., & Abella, A. (1997). PARADISE: A framework for evaluating spoken dialogue agents. In Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics (ACL 1997) (pp. 271–280). Madrid. doi:10.3115/976909.979652
   Google Scholar
• White, M. (2004). Reining in CCG chart realization. In Proceedings of the 3rd International Conference on Natural Language Generation (INLG 2004) (pp. 182–191). Brockenhurst. doi:10.1007/978-3-540-27823-8_19
   Google Scholar
• White, M. (2006). Efficient realization of coordinate structures in Combinatory Categorial Grammar. Research on Language and Computation, 4(1), 39–75. doi:10.1007/s11168-006-9010-2
   Google Scholar
• White, M., Foster, M. E., Oberlander, J., & Brown, A. (2005). Using facial feedback to enhance turn-taking in a multimodal dialogue system. In Proceedings of HCI International 2005 (Vol. 8). Las Vegas, NV.
   Google Scholar