Audio-augmented arboreality: wildflowers and language

ABSTRACT

Before colonization, there were over 250 languages spoken in Australia. Today only thirteen Indigenous languages are still being taught to children). Language has an important part to play in cultural maintenance and ‘closing the gap’ in terms of First Peoples’ cultural heritage, identity, and sense of belonging. In this work, we aim to develop an engaging and easy way to teach and learn the local Indigenous names of wildflowers using a mobile device. This paper presents the development of a phone application that runs on a local machine, recognizes local wildflowers through its camera, and plays associated sounds and displays associated text in the Noongar language. The prototype mobile application has been developed with MobileNets model on the TensorFlow platform. The dataset is derived from Google searches, while the sound files are generated from label text by running an apple script. UI and interactivity have been developed by using Vuforia and the Unity game engine. Finally, the Android Studio is used to deploy the app. At this point in time, the prototype can only recognize ten local flowers, with 85%∼99% of accuracy. We are working with a larger dataset towards developing the full application.

KEYWORDS:

• Wildflower
• object detection
• machine learning
• TensorFlow
• Noongar language

Acknowledgements

The authors like to thank Matthew Hallberg for assistance.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes on contributors

Dr Hafizur Rahaman is a Research Fellow at School of Media, Creative Arts and Social Inquiry (MCASI) in the Humanities Faculty of Curtin University, Perth, Western Australia. He has been actively involved in heritage research, visualization and interpretation for the last 12 years and has a PhD degree in Digital Heritage Interpretation from The National University of Singapore. He is a founding member of ICOMOS Bangladesh and has received the best paper award from SIGGRAPH 2008 USA and CAADRIA 2010 Hong Kong.

Dr Michelle Johnston is a Screen Arts Senior Lecturer with extensive experience in Aboriginal community engagement and producer of the national television programme ‘Noongar Dandjoo’. Her role includes Aboriginal Community Engagement unit coordinator and lecturer, liaison and relationship building with the Curtin Centre for Aboriginal Studies and the Noongar community; facilitating screen studies students’ partnerships with local Aboriginal organizations and supervising and assessing their production and other assignments; and research and project results dissemination.

Professor Erik Champion is the former UNESCO Chair of Cultural Heritage and Visualisation, and Professor at MCASI, in the Humanities Faculty of Curtin University, Perth, Western Australia. He was previously Project Leader of DIGHUMLAB Denmark, and before that was Director of Research and Postgraduate Studies at the Auckland School of Design, Massey University, in New Zealand. He has taught game design, interaction design, and design history, and researches in the area of virtual heritage and serious games. His books includes Organic Design in Twentieth-Century Nordic Architecture, The Phenomenology of Real and Virtual Places, Cultural Heritage Infrastructures in Digital Humanities, Critical Gaming: Interactive History And Virtual Heritage, Game Mods: Design, Theory and Criticism, Playing With The Past.

Additional information

Funding

This study was supported by 2019 MCCA Small Grant, granted by the School of Media, Creative Arts and Social Inquiry (MCASI), Curtin University, Perth, Australia.