2,258
Views
24
CrossRef citations to date
0
Altmetric
Articles

Touching Products Virtually: Facilitating Consumer Mental Imagery with Gesture Control and Visual Presentation

References

  • Alibali, M.W. Gesture in spatial cognition: Expressing, communicating, and thinking about spatial information. Spatial Cognition and Computation, 5, 4 (2005), 307–331.
  • Babin, L.A.; and Burns, A.C. Effects of print ad pictures and copy containing instructions to imagine on mental imagery that mediates attitudes. Journal of Advertising, 26, 3 (1997), 33–44.
  • Balaji, M.; Raghavan, S.; and Jha, S. Role of tactile and visual inputs in product evaluation: A multisensory perspective. Asia Pacific Journal of Marketing and Logistics, 23, 4 (2011), 513–530.
  • Bang, Y.; Lee, D.-J.; Han, K.; Hwang, M.; and Ahn, J.-H. Channel capabilities, product characteristics, and the impacts of mobile channel introduction. Journal of Management Information Systems, 30, 2 (2013), 101–126.
  • Bargh, J.A. What have we been priming all these years? On the development, mechanisms, and ecology of nonconscious social behavior. European Journal of Social Psychology, 36, 2 (2006), 147–168.
  • Barsalou, L.W. Perceptions of perceptual symbols. Behavioral and Brain Sciences, 22, 04 (1999), 637–660.
  • Barsalou, L.W. Grounded cognition. Annual Review of Psychology, 59(2008), 617–645.
  • Barsalou, L.W. Grounded cognition: Past, present, and future. Topics in Cognitive Science, 2, 4 (2010), 716–724.
  • Barsalou, L.W.; Niedenthal, P.M.; Barbey, A.K.; and Ruppert, J.A. Social embodiment. Psychology of Learning and Motivation, 43(2003), 43–92.
  • Barsalou, L.W.; Simmons, W.K.; Barbey, A.K.; and Wilson, C.D. Grounding conceptual knowledge in modality-specific systems. Trends in Cognitive Sciences, 7, 2 (2003), 84–91.
  • Beilock, S.L.; and Goldin-Meadow, S. Gesture changes thought by grounding it in action. Psychological Science, 21, 11 (2010), 1605-1610.
  • Bhalla, M.; and Proffitt, D.R. Visual–motor recalibration in geographical slant perception. Journal of Experimental Psychology: Human Perception and Performance, 25, 4 (1999), 1076.
  • Blajenkova, O.; Kozhevnikov, M.; and Motes, M.A. Object‐spatial imagery: A new self‐report imagery questionnaire. Applied Cognitive Psychology, 20, 2 (2006), 239–263.
  • Brasel, S.A.; and Gips, J. Tablets, touchscreens, and touchpads: How varying touch interfaces trigger psychological ownership and endowment. Journal of Consumer Psychology, 24, 2 (2014), 226–233.
  • Brasel, S.A.; and Gips, J. Interface psychology: Touchscreens change attribute importance, decision criteria, and behavior in online choice. Cyberpsychology, Behavior, and Social Networking, 18, 9 (2015), 534–538.
  • Campos, A. Spatial imagery: A new measure of the visualization factor. Imagination, Cognition and Personality, 29, 1 (2009), 31–39.
  • Campos, A.; Lopez, A.; and Perez, M.J. Vividness of visual and haptic imagery of movement. Perceptual and Motor Skills, 87, 1 (1998), 271–274.
  • Cano, M.B.; Perry, P.; Ashman, R.; and Waite, K. The influence of image interactivity upon user engagement when using mobile touch screens. Computers in Human Behavior, 77(2017), 406–412.
  • Castaño, R.; Sujan, M.; Kacker, M.; and Sujan, H. Managing consumer uncertainty in the adoption of new products: Temporal distance and mental simulation. Journal of Marketing Research, 45, 3 (2008), 320–336.
  • Chin, W.W. The partial least squares approach for structural equation modeling. In G. Marcoulides (ed.), Modern Methods for Business Research. Mahwah, NJ: Lawrence Erlbaum Associates, 1998, pp. 295–333.
  • Choi, Y.K.; and Taylor, C.R. How do 3-dimensional images promote products on the Internet? Journal of Business Research, 67, 10 (2014), 2164–2170.
  • Chung, S. Do touch screen users feel more engaged? The impact of touch interfaces on online shopping. ACR Asia-Pacific Advances in NA - Advances in Consumer Research, eds. Kristin Diehl and Carolyn Yoon, Duluth, MN : Association for Consumer Research, 43, (2015), 488.
  • Cian, L.; Krishna, A.; and Elder, R.S. This logo moves me: Dynamic imagery from static images. Journal of Marketing Research, 51, 2 (2014), 184–197.
  • Cohen, J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Hillsdale, NJ: Erlbaum, 1988.
  • Cummings, J.J.; and Bailenson, J.N. How immersive is enough? A meta-analysis of the effect of immersive technology on user presence. Media Psychology, 19, 2 (2016), 272–309.
  • Deighton, K. M&S Enters the World of VR with Loft Homeware Tour. 2015. Accessed on 30 June2019: https://www.campaignlive.co.uk/article/m-s-enters-world-vr-loft-homeware-tour/1364085.
  • Dimoka, A.; Hong, Y.; and Pavlou, P.A. On product uncertainty in online markets: Theory and evidence. MIS Quarterly, 36, 2 (2012), 395–426.
  • Dourish, P. Where the Action Is: The Foundations of Embodied Interaction. Cambridge, MA: MIT Press, 2004.
  • Elder, R.S.; and Krishna, A. The “visual depiction effect” in advertising: Facilitating embodied mental simulation through product orientation. Journal of Consumer Research, 38, 6 (2012), 988–1003.
  • eMarketer. Retail Ecommerce Sales Worldwide, 2016-2021. 2018. Accessed on 30 June 2019: https://www.emarketer.com/Chart/Retail-Ecommerce-Sales-Worldwide-2016-2021-trillions-change-of-total-retail-sales/215138.
  • Farah, M.J.; Hammond, K.M.; Levine, D.N.; and Calvanio, R. Visual and spatial mental imagery: Dissociable systems of representation. Cognitive Psychology, 20, 4 (1988), 439–462.
  • Faul, F.; Erdfelder, E.; Lang, A.-G.; and Buchner, A. G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 2 (2007), 175–191.
  • Gu, Z.J.; and Tayi, G.K. Consumer pseudo-showrooming and omni-channel placement strategies. MIS Quarterly, 41, 2 (2017), 583–606.
  • Gurman, M. Apple Working on Touchless Control and Curved iPhone Screen. 2018. Accessed on 30 June 2019: https://www.bloomberg.com/news/articles/2018-04-04/apple-is-said-to-work-on-touchless-control-curved-iphone-screen.
  • Hadi, R.; and Block, L. I take therefore I choose? The impact of active vs. passive acquisition on food consumption. Appetite, 80(2014), 168–173.
  • Harvie, D.S.; Broecker, M.; Smith, R.T.; Meulders, A.; Madden, V.J.; and Moseley, G.L. Bogus visual feedback alters onset of movement-evoked pain in people with neck pain. Psychological Science, 26, 4 (2015), 385–392.
  • Hegarty, M.; and Kozhevnikov, M. Types of visual–spatial representations and mathematical problem solving. Journal of Educational Psychology, 91, 4 (1999), 684.
  • Hess, T.J.; Fuller, M.A.; and Mathew, J. Involvement and decision-making performance with a decision aid: The influence of social multimedia, gender, and playfulness. Journal of Management Information Systems, 22, 3 (2005), 15–54.
  • Hong, Y.; and Pavlou, P.A. Product fit uncertainty in online markets: Nature, effects, and antecedents. Information Systems Research, 25, 2 (2014), 328–344.
  • Hu, G.; Bin Hannan, N.; Tearo, K.; Bastos, A.; and Reilly, D. Doing while thinking: Physical and cognitive engagement and immersion in mixed reality games. Proceedings of the 2016 ACM Conference on Designing Interactive Systems: ACM, 2016, pp. 947–958.
  • Jetter, H.-C.; Leifert, S.; Gerken, J.; Schubert, S.; and Reiterer, H. Does (multi-) touch aid users’ spatial memory and navigation in “panning” and in “zooming & panning” UIs?, Proceedings of the International Working Conference on Advanced Visual Interfaces: ACM, 2012, pp. 83–90.
  • Jiang, Z.; and Benbasat, I. Virtual product experience: Effects of visual and functional control of products on perceived diagnosticity and flow in electronic shopping. Journal of Management Information Systems, 21, 3 (2004), 111–147.
  • Jiang, Z.; and Benbasat, I. The effects of presentation formats and task complexity on online consumers’ product understanding. MIS Quarterly, 31, 3 (2007), 475–500.
  • Jiang, Z.; and Benbasat, I. Investigating the influence of the functional mechanisms of online product presentations. Information Systems Research, 18, 4 (2007), 454–470.
  • Kim, J.-S.; Gračanin, D.; Yang, T.; and Quek, F. Action-transferred navigation technique design approach supporting human spatial learning. ACM Transactions on Computer-Human Interaction (TOCHI), 22, 6 (2015), 30.
  • Kim, Y.; and Krishnan, R. On product-level uncertainty and online purchase behavior: An empirical analysis. Management Science, 61, 10 (2015), 2449–2467.
  • Klatzky, R.L.; and Lederman, S.J. Haptic object perception: Spatial dimensionality and relation to vision. Philosophical Transactions of the Royal Society of London B: Biological Sciences, 366, 1581 (2011), 3097–3105.
  • Klatzky, R.L.; and Peck, J. Please touch: Object properties that invite touch. IEEE Transactions on Haptics, 5, 2 (2012), 139–147.
  • Kosslyn, S.M.; Thompson, W.L.; and Ganis, G. The Case for Mental Imagery. Oxford University Press, New York, 2006.
  • Krishna, A. An integrative review of sensory marketing: Engaging the senses to affect perception, judgment and behavior. Journal of Consumer Psychology, 22, 3 (2012), 332–351.
  • Krishna, A.; Elder, R.S.; and Caldara, C. Feminine to smell but masculine to touch? Multisensory congruence and its effect on the aesthetic experience. Journal of Consumer Psychology, 20, 4 (2010), 410–418.
  • Krishna, A.; Lee, S.W.; Li, X.; and Schwarz, N. Embodied cognition, sensory marketing, and the conceptualization of consumers’ judgment and decision processes: Introduction to the issue. Journal of the Association for Consumer Research, 2, 4 (2017), 377–381.
  • Krishna, A.; Morrin, M.; and Sayin, E. Smellizing cookies and salivating: A focus on olfactory imagery. Journal of Consumer Research, 41, 1 (2014), 18–34.
  • Krishna, A.; and Schwarz, N. Sensory marketing, embodiment, and grounded cognition: A review and introduction. Journal of Consumer Psychology, 24, 3 (2013), 159–168.
  • Lakoff, G.; and Johnson, M. Philosophy in the Flesh: The Embodied Mind and Its Challenge to Western Thought. Basic Books, New York, 1999.
  • Lakoff, G.; and Johnson, M. Metaphors We Live By. Chicago, IL: University of Chicago Press, 2008.
  • Landau, M.J.; Meier, B.P.; and Keefer, L.A. A metaphor-enriched social cognition. Psychological Bulletin, 136, 6 (2010), 1045.
  • Lederman, S.J.; and Klatzky, R.L. Hand movements: A window into haptic object recognition. Cognitive Psychology, 19, 3 (1987), 342–368.
  • Lederman, S.J.; and Klatzky, R.L. Extracting object properties through haptic exploration. Acta Psychologica, 84, 1 (1993), 29–40.
  • Lederman, S.J.; and Klatzky, R.L. Haptic perception: A tutorial. Attention, Perception, & Psychophysics, 71, 7 (2009), 1439–1459.
  • Lee, Y.; and Chen, A.N. Usability design and psychological ownership of a virtual world. Journal of Management Information Systems, 28, 3 (2011), 269–308.
  • Li, H.; Daugherty, T.; and Biocca, F. Impact of 3-D advertising on product knowledge, brand attitude, and purchase intention: The mediating role of presence. Journal of Advertising, 31, 3 (2002), 43–57.
  • Lien, N.-H.; and Chen, Y.-L. Narrative ads: The effect of argument strength and story format. Journal of Business Research, 66, 4 (2013), 516–522.
  • Lunden, I. Amazon Sumerian, a Platform for Building AR, VR and 3D apps, is Now Open to All. 2018. Accessed on 30 June 2019: https://techcrunch.com/2018/05/15/amazon-sumerian-a-platform-for-building-ar-vr-and-3d-apps-is-now-open-to-all/.
  • MacInnis, D.J.; and Price, L.L. The role of imagery in information processing: Review and extensions. Journal of Consumer research, 13, 4 (1987), 473–491.
  • Meißner, M.; Pfeiffer, J.; Pfeiffer, T.; and Oppewal, H. Combining virtual reality and mobile eye tracking to provide a naturalistic experimental environment for shopper research. Journal of Business Research, 100, (2017) 445–458.
  • Minas, R.K.; and Dennis, A.R. Visual background music: Creativity support systems with priming. Journal of Management Information Systems, 36, 1 (2019), 230–258.
  • Moreau, C.P.; Markman, A.B.; and Lehmann, D.R. “What is it?” Categorization flexibility and consumers’ responses to really new products. Journal of Consumer Research, 27, 4 (2001), 489–498.
  • Nunnally, J.C. Psychometric Theory. New York: McGraw-Hill, 1978.
  • Obrist, M.; Velasco, C.; Vi, C.; Ranasinghe, N.; Israr, A.; Cheok, A.; Spence, C.; and Gopalakrishnakone, P. Sensing the future of HCI: Touch, taste, and smell user interfaces. Interactions, 23, 5 (2016), 40–49.
  • Ostinelli, M.; and Bockenholt, U. On believing our imagination: The role of mental imagery in belief generation and resilience. ACR North American Advances, eds. Ann L. McGill and Sharon Shavitt, Duluth, MN : Association for Consumer Research, 36, (2009),722–723.
  • Overmars, S.; and Poels, K. Online product experiences: The effect of simulating stroking gestures on product understanding and the critical role of user control. Computers in Human Behavior, 51(2015), 272–284.
  • Papies, E.K.; Best, M.; Gelibter, E.; and Barsalou, L.W. The role of simulations in consumer experiences and behavior: Insights from the grounded cognition theory of desire. Journal of the Association for Consumer Research, 2, 4 (2017), 402–418.
  • Park, W.; and Han, S.H. An analytical approach to creating multitouch gesture vocabularies in mobile devices: A case study for mobile web browsing gestures. International Journal of Human-Computer Interaction, 30, 2 (2014), 126–141.
  • Parrish, K. Microsoft Patent Paves Way for a Touch- and Smudge-Free Tablet Future. 2018. Accessed on 30 June 2019: https://www.digitaltrends.com/computing/new-microsoft-patent-eliminates-smudges-with-touchless-input/.
  • Pasch, M.; Bianchi-Berthouze, N.; Van Dijk, B.; and Nijholt, A. Immersion in Movement-Based Interaction. International Conference on Intelligent Technologies for Interactive Entertainment. Springer,Amsterdam, The Netherlands, 2009, pp. 169–180.
  • Pasqualotto, A.; and Proulx, M.J. The role of visual experience for the neural basis of spatial cognition. Neuroscience & Biobehavioral Reviews, 36, 4 (2012), 1179–1187.
  • Peck, J.; Barger, V.A.; and Webb, A. In search of a surrogate for touch: The effect of haptic imagery on perceived ownership. Journal of Consumer Psychology, 23, 2 (2013), 189–196.
  • Peck, J.; and Childers, T.L. To have and to hold: The influence of haptic information on product judgments. Journal of Marketing, 67, 2 (2003), 35–48.
  • Peck, J.; and Shu, S.B. The effect of mere touch on perceived ownership. Journal of Consumer research, 36, 3 (2009), 434–447.
  • Petrova, P.K.; and Cialdini, R.B. Fluency of consumption imagery and the backfire effects of imagery appeals. Journal of Consumer research, 32, 3 (2005), 442–452.
  • Podsakoff, P.M.; MacKenzie, S.B.; Lee, J.-Y.; and Podsakoff, N.P. Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88, 5 (2003), 879.
  • Podsakoff, P.M.; and Organ, D.W. Self-reports in organizational research: Problems and prospects. Journal of Management, 12, 4 (1986), 531–544.
  • Radhakrishnan, S.; Lin, Y.; Zeid, I.; and Kamarthi, S. Finger-based multitouch interface for performing 3D CAD operations. International Journal of Human-Computer Studies, 71, 3 (2013), 261–275.
  • Rempel, D.; Camilleri, M.J.; and Lee, D.L. The design of hand gestures for human–computer interaction: Lessons from sign language interpreters. International Journal of Human-Computer Studies, 72, 10(2014), 728–735.
  • Roggeveen, A.L.; Grewal, D.; Townsend, C.; and Krishnan, R. The impact of dynamic presentation format on consumer preferences for hedonic products and services. Journal of Marketing, 79, 6 (2015), 34–49.
  • Saffer, D. Designing Gestural Interfaces: Touchscreens and Interactive Devices. Sebastopol, CA: O’Reilly Media, 2008.
  • Schlosser, A.E. Experiencing products in the virtual world: the role of goal and imagery in influencing attitudes versus purchase intentions. Journal of Consumer research, 30, 2 (2003), 184–198.
  • Schlosser, A.E. Learning through virtual product experience: The role of imagery on true versus false memories. Journal of Consumer Research, 33, 3 (2006), 377–383.
  • Sheehan, A. How These Retailers Use Augmented Reality to Enhance the Customer Experience. 2018. Accessed on 30 June 2019: https://www.shopify.com.sg/retail/how-these-retailers-are-using-augmented-reality-to-enhance-the-customer-experience.
  • Shen, H.; and Sengupta, J. If you can’t grab it, it won’t grab you: The effect of restricting the dominant hand on target evaluations. Journal of Experimental Social Psychology, 48, 2 (2012), 525–529.
  • Shen, H.; Zhang, M.; and Krishna, A. Computer interfaces and the “direct-touch” effect: Can iPads increase the choice of hedonic food. Journal of Marketing Research, 53, 5 (2016), 745–758.
  • Shu, S.B.; and Peck, J. Psychological ownership and affective reaction: Emotional attachment process variables and the endowment effect. Journal of Consumer Psychology, 21, 4 (2011), 439–452.
  • Simmons, W.K.; Martin, A.; and Barsalou, L.W. Pictures of appetizing foods activate gustatory cortices for taste and reward. Cerebral Cortex, 15, 10 (2005), 1602-1608.
  • Spence, C.; and Gallace, A. Multisensory design: Reaching out to touch the consumer. Psychology & Marketing, 28, 3 (2011), 267–308.
  • Steinmann, S.; Kilian, T.; and Brylla, D. Experiencing Products Virtually: The Role of Vividness and Interactivity in Influencing Mental Imagery and User Reactions. 35th International Conference on Information Systems, Auckland, New Zealand, 2014.
  • Streicher, M.C.; and Estes, Z. Multisensory interaction in product choice: Grasping a product affects choice of other seen products. Journal of Consumer Psychology, 26, 4 (2016), 558–565.
  • Suh, K.-S.; and Lee, Y.E. The effects of virtual reality on consumer learning: An empirical investigation. MIS Quarterly, 29, 4 (2005), 673–697.
  • Sujan, M.; and Dekleva, C. Product categorization and inference making: Some implications for comparative advertising. Journal of Consumer Research, 14, 3 (1987), 372–378.
  • Symes, E.; Ellis, R.; and Tucker, M. Visual object affordances: Object orientation. Acta Psychologica, 124, 2 (2007), 238–255.
  • Tan, D.S.; Pausch, R.; Stefanucci, J.K.; and Proffitt, D.R. Kinesthetic Cues Aid Spatial Memory. CHI’02 Extended Abstracts on Human Factors in Computing Systems: ACM, 2002, pp. 806–807.
  • Thompson, B. Exploratory and Confirmatory Factor Analysis: Understanding Concepts and Applications. American Psychological Association, Washington, DC, 2004.
  • Tucker, M.; and Ellis, R. On the relations between seen objects and components of potential actions. Journal of Experimental Psychology: Human Perception and Performance, 24, 3 (1998), 830–846.
  • Verhagen, T.; Vonkeman, C.; Feldberg, F.; and Verhagen, P. Present it like it is here: Creating local presence to improve online product experiences. Computers in Human Behavior, 39(2014), 270–280.
  • Wachs, J.P.; Kölsch, M.; Stern, H.; and Edan, Y. Vision-based hand-gesture applications. Communications of the ACM, 54, 2 (2011), 60–71.
  • Weibel, D.; Wissmath, B.; and Mast, F.W. Influence of mental imagery on spatial presence and enjoyment assessed in different types of media. Cyberpsychology, Behavior, and Social Networking, 14, 10 (2011), 607–612.
  • Wesp, R.; Hesse, J.; Keutmann, D.; and Wheaton, K. Gestures maintain spatial imagery. American Journal of Psychology, 114, 4 (2001), 591.
  • Wexler, M.; Kosslyn, S.M.; and Berthoz, A. Motor processes in mental rotation. Cognition, 68, 1 (1998), 77–94.
  • Wohlschläger, A.; and Wohlschläger, A. Mental and manual rotation. Journal of Experimental Psychology: Human Perception and Performance, 24, 2 (1998), 397.
  • Xu, D.J.; Benbasat, I.; and Cenfetelli, R.T. A Two-stage model of generating product advice: Proposing and testing the complementarity principle. Journal of Management Information Systems, 34, 3 (2017), 826–862.
  • Xu, J.; Benbasat, I.; and Cenfetelli, R.T. The nature and consequences of trade-off transparency in the context of recommendation agents. MIS Quarterly, 38, 2 (2014), 379–406.
  • Xu, K.; Chan, J.; Ghose, A.; and Han, S.P. Battle of the channels: The impact of tablets on digital commerce. Management Science, 63, 5 (2016), 1469–1492.
  • Yi, C.; Jiang, Z.; and Izak, B. Enticing and engaging consumers via online product presentations: The effects of restricted interaction design. Journal of Management Information Systems, 31, 4 (2015), 213–242.
  • Yoo, J.; and Kim, M. The effects of online product presentation on consumer responses: A mental imagery perspective. Journal of Business Research, 67, 11 (2014), 2464–2472.
  • Zagermann, J.; Pfeil, U.; Fink, D.; von Bauer, P.; and Reiterer, H. Memory in Motion: The Influence of Gesture-and Touch-Based Input Modalities on Spatial Memory. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems: ACM, 2017, pp. 1899–1910.
  • Zatorre, R.J.; and Halpern, A.R. Mental concerts: Musical imagery and auditory cortex. Neuron, 47, 1 (2005), 9–12.
  • Zhu, K.; Ma, X.; Chen, H.; and Liang, M. Tripartite effects: Exploring users’ mental model of mobile gestures under the influence of operation, handheld posture, and interaction space. International Journal of Human–Computer Interaction, 33, 6 (2017), 443–459.

Reprints and Corporate Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

To request a reprint or corporate permissions for this article, please click on the relevant link below:

Academic Permissions

Please note: Selecting permissions does not provide access to the full text of the article, please see our help page How do I view content?

Obtain permissions instantly via Rightslink by clicking on the button below:

If you are unable to obtain permissions via Rightslink, please complete and submit this Permissions form. For more information, please visit our Permissions help page.