Applications of augmented reality and virtual reality on computer-assisted teaching for analytical sketching of architectural scene and construction

ABSTRACT

Analytical sketching is a method to document an inspired understanding from direct observations and analysis of an architectural environment through freehand sketching. A course dedicated to teaching analytical sketching has encountered problems of how to integrate different teaching activities involving in-class lectures and on-site practices. A mobile learning application (Architectural Sketching) was previously developed to bring the in-class instruction on hand to the field. However, how to assist in the development of key sketching ability of interrelating conceptual architectural configurations and their perceptual spatial experiences through mobile learning remains unexplored. This study proceeds from prior action research and reports the experimental implementation of augmented reality and virtual reality to a mobile learning application. To teach architectural scene sketching, live demonstrations of on-site sketching were provided in different interactive display methods of viewing as regular video, interactive 180-degree video on handheld devices, and 180-degree VR video with VR headset. For architectural construction, augmented reality and 3D holograms, along with 3D animations of geometrical analysis, were used to create digital learning tools. Preference rating and preference scales were established from students’ responses. The results reveal that the user-friendly interactivity and functional realism enhance the mobile learning applications to assist on-site sketching training.

KEYWORDS:

• Analytical sketching
• mobile learning
• augmented reality
• virtual reality
• action research

1. Introduction

Drawing is an essential tool in architectural design. Regardless of the design scale, which ranges from urban to architectural to interior, the design process from ideation and development to realization relies heavily on graphical representations for people involved in visualization (Ching and Juroszek 2010). For effective communication, various drawing systems have been developed to describe the conceptual configuration and perceptual quality of the proposed architecture (Dubery and Willats 1972). Multiview drawings of plan, section, and elevation can precisely describe a specific configuration in a particular two-dimensional form, with multiple drawings arranged in a specific way, and holistic three-dimensional spatial relations can be illustrated. Paraline drawings, such as axonometric views or plan oblique views, fuse three two-dimensional Multiview drawings into a single three-dimensional one, allowing an intuitive understanding of their spatial configuration with corresponding proportion and scale. To envision how the architectural configuration can be experienced as architectural space, perspective drawing can transform conceptual drawings into a perceptual one, enabling us to visualize how the design proposal would appear to our eye if built (Ching 1975). Figure 1 illustrates this relationship. The drawing system was developed to draw the existing architecture from what we see to what we know and vice versa over the years, and then, in turn, can be used to depict the architecture in design through the repetitive process until the optimal solution emerges.

Figure 1. Representation and design drawing: a) Drawing to describe existing architecture; b) Drawing to envision an architectural design proposal.

The design process generally involves a series of iterations to attempt different configurations and envision their spatial qualities (Kalay 2004). Regardless of how advanced computer graphics may get, such activities continue to be performed using sketchbooks with freehand drawings that allow designers’ 3D thinking to flow freely. As a result, learning how to draw the three-dimensional configuration of an architectural construction and how it could be experienced as an architectural space in freehand remains a priority in architectural education (Ching 2015; Zell 2018).

In order to learn how to draw imaginary design proposals analytically in freehand, it is useful to first learn to draw existing structures which have already been built. Analytical on-site sketching is a practice that encourages students to sketch architecture, both its conceptual and perceptual form, from direct observation with analysis. By drawing both the inner structure and exterior appearance, it further allows one to illustrate the interrelationship of the solid structure and its enclosed void spaces and understand how to configure architectural elements to create architectural space. Figure 2 illustrates an application of this 3D thinking in an observed architectural environment. The Community Design Building is a design studio of the University of Washington located in Seattle. The on-site observation from distant views reveals the overall understanding of the uncommon form factor of the building illustrated with the interrelated plan and elevation. By applying the parallel projection of the elevation, a paraline drawing of the building demonstrates the holistic three-dimensionality; on the other hand, by projecting the vanishing lines from the same elevation, an interior space can be described. A perspective sketching of the interior architectural scene rendered with tonal value further illustrates how daylight is introduced from the void gaps created by the unfolding roof panels. Together, not only the geometrical configuration can be visualized in spatial experience, the design strategy of daylighting can also be revealed from the illustration of the cause-and-effect relationship of the conceptual architectural construction and perceptual architectural scene. Hence, the on-site analytical sketching can be an effective tool to study architectural design and eventually to enrich one’s design abilities. Analytical sketching is therefore an extremely useful and essential skill for architectural students (Dubery and Willats 1972; Ching 2015).

Figure 2. Visualization of design strategy through the on-site analytical sketching.

This paper reviews the analytical drawing method applied to on-site architectural sketching, and reveals the problems encountered while teaching analytical drawing. Action research was employed as a methodology, and various innovative computer-assisted learning tools focusing on the analytical sketching of conceptual architectural configurations and perceptual architectural scenes have been developed and tested in the teaching field. Comparisons of these mobile learning applications with interactive two-dimensional interaction and three-dimensional virtual and augmented reality mediated on handheld devices were made to determine how effectively we can use advanced computer graphics to help cultivate conventional sketching skills.

1.1. Analytical on-site sketching

Analytical drawing was first observed in a Renaissance painting, it is drawn in a transparent manner, revealing both the inner structures and exterior appearances. The resultant image is similar to computer-modeling presented in wireframe mode (Dubery and Willats 1972). In other words, analytical drawing can be considered as the construction of a three-dimensional model on a two-dimensional surface with freehand drawing. Analytical on-site sketching extends from the core concept of the analytical drawing, as illustrated in Figure 3; it is a visual note-taking of three-dimensional thinking that document one’s understanding of the tangible transformation between the architectural configuration and spatial experience and the underlying intangible design strategy unfolded through the cause-and-effect relationship. Topics of the course teaching analytical sketching thus include “perceptual sketching,” “conceptual sketching,” “seeing and knowing,” “diagramming,” and “visual notes.”

Figure 3. Visual notes of analytical analysis of an architectural environment.

A visual note of an architectural environment by analytical sketching can be of any kind depending upon the observed environment and the person’s inspired understanding; however, the key to success remains the ability to interrelate the conceptual configuration to the perceptual experience with analytical observations and drawings. Therefore, analytical sketching heavily emphasizes reading the structure’s underlying geometrical configuration. Moreover, this process requires imagination. Figure 4 illustrates examples of two different approaches to use imagined planes and volumes to develop a three-dimensional understanding of the interrelated structure of paraline and perspective views. For the Taipei New Horizon building designed by Toyo Ito, 3D thinking starts with the frontal plane, and develops into the final form with cutting and folding; for the Luce Memorial chapel designed by I.M. Pei and Chi-kuan Chen, the 3D thinking was initiated with a rectangular box, and manipulated in the same way as digital modeling by deformation, duplication, and reassembly into the final form. The two completely different approaches highlight two challenges that emerged from teaching the key ability. First, how to imagine a holistic conceptual view that is not available from the on-site observation; second, how to explain the imagined geometrical developments without visual aids. These two problems identified from reflections on teaching prompted the implementation of action research to develop digital mobile learning assistance for the Analytical Sketching course.

Figure 4. Imagined geometrical developments.

1.2. Action research in the context of teaching analytical sketching

Action research employs a repetitive and interactive process with actions applied in practice (Sumara, Carson, and Carson 1997; McNiff 2017). It is widely used in the fields of education (Sagor 2000; Mertler 2016). With the repetitive intervention of “plan,” “action,” “observation,” and “reflection,” problems can be identified, solutions can be implemented, and effect of the solutions can be revealed based on the observed and measured responses of the participants (McNiff 2013; Reason and Bradbury 2001). Application of action research in design education is not a new idea. Agouridas and Race employed this systematic reflection process to enhance knowledge management in a design studio (Agouridas and Race 2007). Kowaltowski et al. implemented action research into the third-year design studio to integrate the concept of sustainability into their design process for five years (Kowaltowski et al. 2020). In fact, for a design studio that is a course designed to develop a relatively abstract ability of design, the evaluation of the teaching effect is often difficult to quantify; however, action research can provide a methodology to evaluate the effectiveness of experimental teaching with a longitudinal framework (Agouridas 2007). As an architectural design tool, action research has also been employed in various studies in CAD teaching, particularly as computer-aided design is a field continually developing both in depth and breadth. Aspects of skill acquisition such as how well students can transition from hand drawing to CAD (Gracia-Ibáñez and Vergara 2016) or a more conceptual effect such as how CAD can inspire creativity (Winn and Banks 2012) can be systematically investigated with repetitive intervention, observation, and reflection. Action research as a set of methods is therefore an ideal research framework to investigate a core concept of the teaching objective while improving the teaching in the process.

This study in fact is a continuing study of the on-going research that aims to promote analytical sketching as a three-dimensional thinking tool. It reports the second round of the circle of action research following previous studies’ focus on using mobile learning to integrate the course as a whole.

Figure 5 illustrates the overall framework of the two rounds of action research. In the first round of action research, mobile learning materials were developed to bridge the gap in teaching activities between the classroom and field (Tai 2019, 2022). As illustrated, three major teaching activities was employed in the course of Analytical Sketching. In the classroom, slide presentations were used to introduce the theoretical foundation and general knowledge of visual processes and drawing systems, proceeding with case studies on various topics of analytical sketching illustrated with animated graphics. On the site, the instructor first demonstrates the analytical drawing process, explaining how his 3D thinking process of 3D dissembling and resembling is often done with sketching on a poster paper on a large scale. Students then go on to sketch the same view and apply the learned skill to sketch different views, while the instructor provides one-on-one feedback to ensure that the intended knowledge and skill is delivered to the participating students.

Figure 5. Framework of implemented action research.

Field studies have identified a few drawbacks in the above-mentioned method. The foremost is that as teaching activities (the computer-assisted slide explanation, on-site demonstration, and personal feedback) are scattered across different locations, it is difficult to coherently integrate the different topics in the weekly three-hour class. As a solution for this issue, two sets of mobile learning materials were developed. This included a hardcopy course sketchbook and the mobile Architectural Sketching app. The sketchbook comprised theoretical background knowledge and a stepwise tutorial of the sketching process, as well as blank sketch papers. The mobile Architectural Sketching App was developed to include three categories. In the first category, “Analytical Drawing,” related knowledge and theoretical foundations from the classroom slide presentation were transferred into interactive content. “Sketching Video” provided on-site sketching demonstration videos. In “On-site sketching,” an illustrated stepwise tutorial of the analytical sketching process was provided; this feature shows the significant stages in the hand-drawn sketching process, serving as a reference for recalling the one-on-one feedback (Tai 2019, 2022, 2021).

According to a previous study, although the video demonstration provides a more realistic representation of the sketching process, stepwise illustrations were significantly preferred by the students (Tai 2022). This is probably because the video can only reveal the sketching process on paper, falling short of providing an interactive reference of the real architectural scene. Furthermore, the mobile app solution effectively helps to sketch the perceptual aspects of architectural scenes but provides little help in applying analytical drawing to explore the conceptual aspect of architectural construction. As a result, although the app successfully anchors the class as a whole, issues of how to imagine the conceptual configuration on site and how to transform the seeing into knowing of the observed architectural environment remain unresolved. Building upon the mobile app solution, this study reports the implementation of more advanced computer-assisted tools to help students learn perceptual drawing of architectural scenes and conceptual drawing of architectural construction.

1.3. Problem to be solved

The core concept of analytical sketching is to develop students’ ability to use drawings to think three-dimensionally about an architectural design. The essential ability is being able to read and draw the interrelationship between the conceptual configuration and perceptual quality of an observed architectural environment. That is, how architectural space can be enclosed by architectural construction and how architectural construction can create unique architectural scenes. For the architectural scene perceived with the eyes, it is to be drawn using perspective techniques; on the other hand, architectural construction seen through the mind is conceptual and can be drawn using paraline drawing. To this end, this study emphasizes how to assist the on-site sketching of the visual perception and mental conception of an architectural structure; these two sketching activities are referred to as architectural scene and architectural construction. The problems identified for each activity are listed below.

For Architectural scene:

1. Depending upon the enrolment, on-site demonstrations might not be able to allow every student to have an ideal point of view to track the demonstration while referring to the real scene.

2. Conventional video can only record the drawing process on paper, but cannot provide an interactive seeing and drawing process that emphasizes analytical sketching.

For Architectural construction:

1. On site, one can only view the architectural subject from the perspective view; how to convey the geometrical analysis of its conceptual configuration is therefore more difficult than perspective sketching.

2. A paraline view of an architecture require understanding of the entire subject, it often requires exploration from many viewpoints to finally gain the understanding, however, such exploration takes time and sometime might not even possible due to the site context.

2. Method

Computer-assisted instruction utilizes programmed materials with various computer technologies to assist self-learning (Suppes and Morningstar 1969; Chambers and Sprecher 1980). Depending on the subjects being taught, computer-assisted instruction can take various forms with different innovative interactions. In this study, a single architectural environment is sketched analytically, as an architectural scene and an architectural construction. An architectural scene is a visual environment in which we are immersed, seeing it from the point of view before or within it. On the other hand, architectural construction is a holistic understanding based on collective viewing, wherein the building can be conceptualized as a structure viewed from a distant, high-vantage point. To this end, recent advancements in digital simulation and graphic representation technologies have opened up possibilities for innovative educational applications. The immersive viewing experience made possible by Virtual Reality (VR) provides a practical solution for computer-assisted instruction to sketch architectural scenes. On the other hand, augmented reality can provide a different objective viewing experience of the subject augmented with the real scene.

2.1. Virtual reality

Virtual reality refers to computer-generated simulations that allow for various degrees of interaction and sensory experience with a three-dimensional virtual environment through a digital device. The main difference between the VR and computer-simulated environments is that the VR can display the visual environment based on the viewer’s point of view to induce a perceptually realistic experience (Steuer 1992; Rheingold 1992). Various methods have been developed to provide such interactive viewing, such as updating scene information on the device screen based on eye-tracking, wearable Google glasses, or head-mount devices to induce immersive visual perception based on natural viewing motion (Burdea and Coiffet 2003).

As VR technologies have become affordable, educational applications have become common. These include the pioneer CAVE (Cave Automatic Virtual Environments) project that allows students to explore historical heritage (Ott and Pozzi 2008) and the more recent applications that deliver fundamental knowledge in health education (Claudio and Maddalena 2014; Green, Wyllie, and Jackson 2014) and aid the acquisition of surgical skills (Eve et al. 2014). VR applications are not only diverse but can vary with the depth of the learning experience of a particular field (Sakatani 2005; Häfner, Häfner, and Ovtcharova 2013). In general, educational applications utilizing VR aim to create an immersive learning environment that is otherwise unavailable in the real world, allowing users to acquire or practice knowledge or skills in this alternative environment. However, the intent of this study is somewhat different from commonly used VR educational applications. The intent is to provide an alternative visual environment recreating an architectural scene that we can refer to while sketching it in perspective. That is, the task of sketching remains a real-world activity, whereas the computer-assisted VR functions as a visual experience for reference only. In fact, the user shifts focus back and forth between the referenced space of the architectural scene and the pictorial space on the paper where he or she depicts the experienced architectural scene.

To this end, three different computer-mediated visual referencing experiences for viewing on-site demonstrations were created. Each of them differs in respect of the emphasis on the three factors of immersion, imagination, and interaction in the VR triangle proposed by Burdea (Burdea and Coiffet 2003). Depending on the nature of the intended simulated experience, digital content can be generated by computer modeling and rendering for the imaginary environment as well as through 360-degree video recording of the existing environment (Huang et al. 2017; Kavanagh et al. 2016). In this study, 360-degree video recording was employed to create two different degrees of visual experience of viewing the on-site demonstration – a 180-degree video that can be mediated on the screen and a 180-degree VR video that allows an immersive viewing experience with a VR device.

The 360-degree camera (Insta360 Evo) was used to record the on-site demonstration. This 360-degree camera can unfold the two circular fisheye lenses in the same direction to record 180-degree stereo video to be viewed in 180-degree VR mode with VR headset. It also can output a 180-degree video in a normal manner that includes the frontal half sphere that covers the field of view of the sketching process as well as the architecture before the eye. At the same time, a regular video focusing on sketching on the poster paper was also filmed using an iPhone12 equipped with a three-axis stabilizer. Figure 6 illustrates the three types of videos and their differences in immersion, interaction, and imagination.

Figure 6. Comparisons of the three types of videos for assist on-site perspective sketching.

2.2. Augmented reality for architectural construction

Sketching an architectural scene perceived in perspective requires one to view the subject from different angles in a three-dimensional environment. The visual experience offered by VR creates a complete visual perception of a simulated environment; it is therefore suitable for computer-assisted learning for the perceptual aspect of architectural scenes. On the other hand, to draw what we know about the architectural construction, a drawing system of conceptual representation of paraline drawing is required. As this type of drawing system views the subject from a far distance such that the parallel lines converge to infinity and thus remain parallel, the subject can be considered as a reduced-scale model placed on the table for review and examination. For this purpose, augmented reality provides an ideal on-site interaction that allows imaginative viewing.

Augmented reality is the creation of virtual objects augmented in a real-world environment, providing an opportunity to view the conceptual aspect of the subject on-site through a digital device (Azuma 1997). AR has also been applied in the field of education in a way that encourages users to learn from virtual information augmented by the real environment (Bacca et al. 2014; Wu et al. 2013). In particular, with the wide availability of smartphones, AR has been a great tool for on-site learning, such as for accessing supplemental information when viewing collections in museums (Yoon and Wang 2014; Jung et al. 2016). Similar to VR, in addition to displaying knowledge, AR can also provide training for skills such as drawing and drafting (Di Serio, Ibáñez, and Kloos 2013; Geroimenko 2012; Horii and Miyajima 2013).

In this study, the application of augmented reality is focused on delivering the conceptual information of the subject on site so as to allow students to compare the conceptual configuration of the subject with direct observation from a single perspective. Figure 7 illustrates the AR application. On site, after scanning the real scene with mobile devices, the architectural structure of interest appears in the foreground of the screen, allowing the user to navigate at various angles from above, which is otherwise unavailable in reality.

Figure 7. AR application to view the architecture as a distant object with the real scene.

Two other computer-assisted learning tools were developed to convey the conceptual aspects of architectural construction. Holographic projection refers to the technology to project a three-dimensional virtual object in a real scene, allowing the subject to be viewed from around as a real 3D object (Lee 1978; Yaroslavsky and Astola 2009). Holographic projections can be achieved in many different ways, and a 3D hologram pyramid is considered the most affordable. Figure 8 illustrates an example of this application. A digital model of the architectural heritage of the North Gate of Taipei City was first constructed using a computer-modeling program. Frontal views of four different orientations were generated, and the layout was arranged using an image-editing program, as illustrated in Figure 8. A truncated pyramid was constructed with translucent plastic sheets, and by placing the pyramid on a smartphone that displays the processed image, a 3D optical illusory architectural heritage could be projected within the pyramid, allowing the user to view this illusory object as a real object.

Figure 8. Holographic projection of the historical architecture.

These two methods of delivering conceptual information require additional operations from the user. Augmented reality asks to launch the mobile app of Vuforia and scan the scene to view the augmented architectural construction (Chapagain 2022). The 3D hologram requires the user to install an additional device on the phone. Therefore, a simplified computer-assisted method was developed to be incorporated into the Architectural Sketching App. This is illustrated in Figure 9. The architecture was provided with two animated illustrations, one allowing viewing from front to back and the other illustrating geometrical dissembling and reassembling. These animations can be interactively controlled with a finger swipe, allowing the user to view back and forth and stop where they prefer.

Figure 9. Conceptual animations from mobile app.

2.3. Evaluation

The teaching of analytical sketching is structured into six components including “Analytical Drawing,” “Perceptual Sketching,” “Conceptual Sketching,” “Seeing and Knowing,” “Diagramming” and “Visual Note.” In “Analytical Drawing,” theoretical foundations, base knowledge, and various techniques were introduced. “Perceptual Sketching” and “Conceptual Sketching” illustrate how to implement the analytical drawing process to sketch the perceptual aspects and conceptual aspects of the architectural structure on-site. “Seeing and Knowing” further elaborated on how to document the interrelationship between the conceptual configuration of an architectural construction and the perceptual quality of the architectural space. The “Diagramming” is a section that teaches how to use abstract drawings, symbols, notations, or text to describe the observed abstract relationships and further to link different sketches of conceptual and perceptual into a visual note to deliver one’s understanding of architectural design strategies. The final section, “Visual Note,” concludes the course with different examples of visual note-taking. Among these six components, the one focusing on developing students’ ability to analytically sketch an architectural scene and an architectural construction from on-site observation remains the most essential. To this end, different mobile learning methods have been developed to assist with this learning objective.

2.4. Mobile learning assistance

In addition to the new experimental computer-assisted learning tools, the already adopted teaching assistance methods were also included for comparison. These include the instructor’s live demonstration, the stepwise tutorial of the sketching process, and the same information presented interactively on the mobile app for both perceptual architectural scenes and conceptual architectural constructions. Figure 10 illustrates an example of these three different types of teaching assistance.

Figure 10. Already adopted teaching assistance tools.

2.5. Participants

The above-mentioned components of analytical sketching training were implemented in two courses offered by College of Design at National Taipei University of Technology – the “Analytical Sketching” course for graduate students and the “Design Drawing” course for students of the undergraduate program in Creative Design. Both courses have been taught for more than ten years. To ensure that students receive fair instruction, this study does not employ an experimental design that comprises different levels of treatment. This is a within-subject study in which all students receive the same instructions along with the same computer-assisted learning tools. Students participated in the study with no difference from their normal classes. The study employed action research to evaluate the effects of different mobile learning materials throughout the course, with the observation and course evaluation conducted at the end of the course. Fifteen graduate and 15 undergraduate students participated in this study.

Each student was provided with a print course booklet that detailed the course information binding with blank sketching papers to document their on-site sketches to track learning progress throughout the course. The Architectural Sketching App was also made available to the students. The app was developed with iOS, which is only available on an iPhone or iPad. Three spare iPhones and iPads were available for students to borrow during the class. Devices for the 3D hologram and VR headset were provided on the days when the related topics were taught. Two evaluation methods were used: a five-point preference rating scale and a forced-choice preference assessment.

2.6. Preference rating

The course evaluation employed five-point scales ranging from 1 to 5, representing “strongly disagree,” “disagree,” “neutral,” “agree,” and “strongly agree.” The items related to perceptual sketching and conceptual sketching are as follows.

For perspective sketching:

• Q1 Instructor’s on-site demonstration helps me a lot in perspective sketching.

• Q2 The stepwise illustration on the booklet helps me a lot in perspective sketching.

• Q3 The stepwise illustration on the mobile app helps me a lot in perspective sketching.

• Q4 The demo video helps me a lot in perspective sketching.

• Q5 The 180-degree demo video helps me a lot in perspective sketching.

• Q6 The VR 180-degree demo video helps me a lot in perspective sketching. For conceptual sketching:

• Instructor’s on-site demonstration helps me a lot in conceptual sketching.

• The stepwise illustration on the booklet helps me a lot in conceptual sketching.

• The stepwise illustration on the mobile app helps me a lot in conceptual sketching.

• Animated analysis on the mobile app helps me a lot in conceptual sketching.

• The AR application helps me a lot in conceptual sketching.

• The holographic projection helps me a lot in conceptual sketching.

2.7. Paired stimulus preference assessment

Perceptual preference can be more intuitively derived by the method using forced choice than rank ordering; for example, paired stimulus preference assessment has been proven to be more sensitive in various behavioral studies (Fisher et al. 1992). In this type of assessment, each item was presented with others in pairs, and the subjects were asked to pick the preferred one. The total number of preferred picks for each item can then establish a ranked order. Tables 1 and 2 illustrate the forced choice questions in the questionnaire. For the perceptual study, six learning assistances were included: “Instructor’s live demonstration on-site,” “Stepwise illustration on booklet,” “Stepwise illustration on app,” “Regular demonstration video,” “Demonstration 180-degree Video,” and “Demonstration 180-degree VR Video.” For conceptual sketching, six learning assistances were included: “Instructor’s live demonstration on site,” “Stepwise illustration on booklet,” “Stepwise illustration on App,” “Animation of Geometrical Analysis on App,” “AR Application,” and “Holographic Projection.”

Table 1. Forced-choice questions for perceptual sketching.

Among the six learning assistances, please check one from each pair that helps you better in on-site perceptual sketching
[On-Site Demo	Booklet Illustrations]	[On-Site Demo	App Illustrations]	[On-Site Demo	Regular Video]
□	□	□	□	□	□
[On-Site Demo	180° Video]	[On-Site Demo	180° VR Video]	[Booklet Illustrations	App Illustrations]
□	□	□	□	□	□
[Booklet Illustrations	Regular Video]	[Booklet Illustrations	180° Video]	[Booklet Illustrations	180° VR Video]
□	□	□	□	□	□
[App Illustrations	Regular Video]	[App Illustrations	180° Video]	[App Illustrations	180° VR Video
□	□	□	□	□	□
[Regular Video	180° Video]	[Regular Video	180° VR Video]	[180° Video	180° VR Video]
□	□	□	□	□	□

Table 2. Forced-choice questions for conceptual sketching.

Among the six learning assistances, please check one from each pair that helps you better in on-site conceptual sketching
[On-Site Demo	Booklet Illustrations]	[On-Site Demo	App Illustrations]	[On-Site Demo	App Animation]
□	□	□	□	□	□
[On-Site Demo	Interactive AR]	[On-Site Demo	3D Hologram]	[Booklet Illustrations	App Illustrations]
□	□	□	□	□	□
[Booklet Illustrations	App Animation]	[Booklet Illustrations	Interactive AR]	[Booklet Illustrations	3D Hologram]
□	□	□	□	□	□
[App Illustrations	App Animation]	[App Illustrations	Interactive AR]	[App Illustrations	3D Hologram]
□	□	□	□	□	□
[App Animation	Interactive AR]	[App Animation	3D Hologram]	[Interactive AR	3D Hologram]
□	□	□	□	□	□

3. Result

Table 3 illustrates the preference rating results for the questions regarding the stratification of different learning assistance tools for on-site perceptual sketching. For perceptual sketching, the order of the preference rating from high to low are “On-Site Demonstration” (4.77), “Stepwise illustration on App” (4.57), “180-degree Video” (4.47), “Stepwise illustration on booklet” (4.30), “Demo Video Regular” (3.97), and “Demo Video 180-degree VR” (3.73). Statistical analysis using the Friedman test further confirmed that the difference between the perception of helpfulness was significant (H = 57.709, p < .005). Post hoc comparison (using Conover’s F) further indicated that the first ranked “On-Site Demonstration” is significantly better than the third, fourth, fifth, and sixth ranked “180-degree Video,” “Stepwise illustration on booklet,” “Demo Video Regular,” and “Demo Video 180-degree VR,” respectively; but not significantly better compared to the second ranked “Stepwise illustration on App.” The second ranked “Stepwise illustration on App” is also not significantly better than the third ranked “Demo Video 180-degree,” but is significantly better than the “Stepwise illustration on booklet,” “Demo Video Regular” and “Demo Video 180-degree VR.” Among the three types of videos, the Demo 180-degree video is ranked the highest overall, following the on-site instructor’s demonstration and stepwise illustration on App. The post hoc comparison also confirms that the 180-degree demo video is significantly better than the 180-degree VR video and the regular video. To this end, we can conclude that the on-site demonstration from the instructor remains the most helpful, as it is the original instruction that all other assistances are based upon. The stepwise illustration on the app was the most helpful from the rest of the learning assistances. It is considerably better than the same information available in the print booklet, attesting that the same supplemental information can be much more helpful if mediated on a smartphone with interactive navigation. The same sketching process from the live demonstration when presented in 180-degree video mode is slightly preferred over the conventional method. This suggests that students appreciate the freedom of the field of view between architecture and the sketching process. However, although the viewing of 180-degree VR videos with VR headsets can mimic the viewing experience of standing behind the instructor while he/she is demonstrating, the immersive experience might, on the other hand, prevent students from relying on computer-assisted learning.

Table 3. Preference ratings for perceptual sketching.

	On-Site Demo	Booklet Illustrations	App Illustrations	Regular Video	180° Video	180° VR Video
S01	4	3	3	4	4	3
S02	5	5	5	4	5	4
S03	5	5	5	4	5	4
S04	5	5	4	4	4	3
S05	5	5	5	4	4	3
S06	5	4	4	4	5	4
S07	5	4	5	4	4	4
S08	5	5	5	5	5	5
S09	5	5	5	5	5	5
S10	5	4	4	2	4	2
S11	5	4	5	5	5	4
S12	5	5	5	5	5	5
S13	5	4	5	4	4	4
S14	5	4	5	4	5	4
S15	5	4	4	4	4	3
S16	5	4	4	4	5	4
S17	5	3	5	3	4	1
S18	4	5	5	3	5	3
S19	4	3	4	4	4	4
S20	4	5	4	4	5	5
S21	4	3	4	2	2	2
S22	5	5	5	5	5	5
S23	5	5	5	4	5	4
S24	5	5	5	5	5	5
S25	5	5	5	5	5	4
S26	4	4	4	3	4	4
S27	5	5	5	4	5	4
S28	4	3	4	3	3	2
S29	5	4	4	4	4	4
S30	5	4	5	4	5	4
Mean	4.77	4.30	4.57	3.97	4.47	3.73

In preference rating, students make judgments based on their subjective perceptions of items, and it is often noticed that some students might kindly to consistently rate high or even check the highest score across the items. Furthermore, as the sample size was relatively small, the differences in ratings between items were not big enough to reach statistical significance. It is evident from Table 3 that the five students who rated all six options had the same highest scores. The pair stimulus assessment forced participants to choose a preferred one from the pair, asking them to compare different items. Therefore, it is more effective to differentiate the preferences across items. Table 4 illustrates the results of the total number of preferred picks for each item from all 30 students.

Table 4. Preference order for perceptual sketching.

	On-Site Demo	Booklet Illustrations	App Illustrations	Regular Video	180° Video	180° VR Video
S01	5	2	1	2	4	1
S02	3	1	3	3	3	2
S03	5	0	4	1	3	2
S04	5	4	3	1	2	0
S05	5	3	4	2	1	0
S06	5	0	4	1	3	2
S07	5	1	2	1	4	2
S08	3	1	4	1	5	1
S09	5	0	4	1	2	3
S10	5	1	2	3	4	0
S11	5	0	4	1	3	2
S12	5	3	1	2	2	2
S13	5	3	4	1	2	0
S14	2	1	5	2	4	1
S15	5	1	3	3	2	1
S16	3	4	5	2	1	0
S17	5	2	5	2	0	1
S18	4	4	3	2	2	0
S19	5	1	2	2	2	3
S20	5	2	1	1	3	3
S21	4	3	5	0	2	1
S22	5	1	2	0	4	3
S23	4	2	5	3	1	0
S24	5	2	2	2	1	3
S25	3	1	4	4	3	0
S26	5	3	4	2	1	0
S27	5	2	3	4	1	0
S28	5	2	4	1	2	1
S29	5	1	4	3	1	1
S30	4	0	1	3	4	3
Total	135	51	98	56	72	38

Table 5 further compares the results from rank-rating of preference rating and forced-choice of preference order. As shown in the table, the two results in general agree with each other. In both rankings, on-site demonstration is ranked first, followed by “Stepwise illustration on App” and “180-degree video”, and the 180 VR video ranked the last as well. Only “Regular demo video” and “Stepwise illustration on booklet” are ranked differently in both rows; that is, they are ranked fourth and fifth, respectively, under “total picks,” but their positions are swapped under “rating.”

Table 5. Comparison between preference order and preference rating for perceptual sketching.

	On-Site Demo	App Illustrations	180° Video	Regular Video	Booklet Illustrations	180° VR Video
Total picks	135	98	72	56	51	38
Rating	4.77	4.57	4.47	3.97	4.30	3.73

The differences across the six options were also statistically significant (H = 65.405, p < .005). The post hoc pairwise comparisons has similar results as the preference ratings, with additional difference being that the “Stepwise illustration on App” significantly differed from the “180-degree demo video.” This further reaffirms that stepwise illustration on mobile phones is indeed a better help than the 180-degree demo video. Although 180-degree videos had more preferred picks than conventional videos (72 vs. 56), pairwise comparisons did not reach statistical significance. This result shows that both the 180-degree video and regular video provide good aid for learning; the additional feature of viewing the architectural structure as a reference while reviewing the sketching process is welcomed, but does not necessarily make the conventional video a worse help in comparison. However, the immersive 180-degree VR experience is not preferred by students in the process of learning. Feedback from students shows that the additional effort to use the VR headset can discourage them from using this learning assistance.

Table 6 presents the preference rating results for conceptual sketching. Similar to perceptual sketching, the on-site demonstration from the instructor remained the most preferred by the students (4.9), followed by the mobile app stepwise illustration (4.67). The animated geometrical analysis was ranked third (4.43), followed by stepwise illustration from the booklet (4.20). Augmented reality and 3D hologram were ranked fifth and last, with low rating of 3.37 and 2.63, respectively. Friedman analysis confirmed that the difference across the six items was statistically significant (H = 104.064, p < .005). The post hoc pair comparisons are all significant except two comparisons of “On-site demonstration” with “Stepwise illustration on App”; as well as “Stepwise illustration on App” with “Animated geometrical analysis on App.”

Table 6. Preference ratings for conceptual sketching.

	OnSite Demo	Booklet Illustrations	App Illustrations	App Animation	Interactive AR	3D Hologram
S01	5	3	3	3	3	3
S02	5	5	5	5	4	4
S03	5	5	5	5	4	3
S04	5	5	4	4	3	2
S05	5	4	5	5	4	2
S06	5	4	5	5	3	1
S07	5	5	5	4	2	4
S08	5	5	5	5	5	5
S09	5	5	5	5	3	3
S10	5	3	4	3	2	2
S11	5	4	5	4	2	2
S12	5	5	5	5	5	1
S13	5	5	5	5	4	3
S14	5	5	5	5	3	1
S15	5	4	5	5	3	2
S16	5	4	4	4	3	3
S17	5	3	5	5	4	1
S18	5	4	5	4	3	3
S19	5	3	4	4	4	2
S20	5	5	5	3	3	3
S21	4	3	4	4	3	1
S22	5	5	5	5	5	5
S23	5	4	5	5	3	3
S24	5	5	5	5	3	2
S25	5	5	5	5	4	4
S26	4	4	4	4	4	4
S27	5	4	5	5	4	3
S28	5	3	4	3	3	3
S29	4	3	4	4	3	2
S30	5	4	5	5	2	2
Mean	4.90	4.20	4.67	4.43	3.37	2.63

Table 7 further illustrates the results of the forced-choice preference picks and the resulting preference order. The advantage of this method that it is more sensitive to preference differences was demonstrated, as the “on-site demonstration” received a total of 136 prefer picks while the “3D hologram” received only 9. The resulting preference order aligned exactly with the ranked order of the results of preference rating, as illustrated in Table 8.

Table 7. Preference order for conceptual sketching.

	On-Site Demo	Booklet Illustrations	App Illustrations	App Animation	Interactive AR	3D Hologram
S01	5	4	3	2	1	0
S02	4	1	3	4	3	0
S03	5	2	3	4	1	0
S04	5	4	3	2	1	0
S05	5	2	3	4	1	0
S06	3	2	5	4	1	0
S07	5	2	4	3	1	0
S08	4	5	2	3	1	0
S09	5	0	4	3	2	1
S10	5	2	4	3	1	0
S11	5	2	4	3	1	0
S12	5	4	3	2	1	0
S13	5	2	3	4	0	1
S14	3	2	4	5	1	0
S15	5	2	4	3	1	0
S16	5	3	2	4	1	0
S17	4	2	4	4	1	0
S18	3	2	5	4	1	0
S19	5	2	4	3	1	0
S20	4	2	2	3	3	1
S21	4	2	5	3	1	0
S22	5	0	2	4	2	2
S23	3	2	5	4	1	0
S24	5	3	3	3	1	0
S25	5	2	4	3	0	1
S26	4	2	5	3	0	1
S27	5	2	3	4	1	0
S28	5	0	4	3	2	1
S29	5	1	3	4	2	0
S30	5	0	4	3	2	1
Mean	136	61	107	101	36	9

Table 8. Comparison between preference order and preference rating for conceptual sketching.

	On-Site Demo	App Illustrations	App Animation	Booklet Illustrations	Interactive AR	3D Hologram
Total picks	136	107	101	61	36	9
Rating	4.90	4.67	4.43	4.20	3.37	2.63

The Friedman statistical analysis demonstrates that the differences across six learning assistances are statistically significant (H = 114.279, p < .005), the post hoc comparisons are all significant between each other with only one exception of “Stepwise illustration on App” and “Animated geometrical analysis on App.” Therefore, it can be concluded that among the three experimental computer-assisted methods, the animated geometrical analysis is perceived to have greater benefit than AR applications, while the 3D hologram is considered the least helpful. Possible reasons are that the AR application and 3D hologram are essentially small virtual objects, allowing one to examine from different angles; the same information can be more quickly retrieved from the look-around animation without additional efforts, such as launching the AR application to first scan the site, or installing the 3D hologram pyramid device to finally view the object. Furthermore, AR and 3D holograms can only deliver the existing fixed properties of the subject, falling short of illustrating the abstract geometrical configurational relationship. On the other hand, 2D animation can essentially generate different types of animated processes to illustrate the conceptual information. In this study, students can swipe the finger to dissemble and then reassemble the geometrical configuration of the architectural construction to learn how to draw it analytically in a paraline view.

4. Discussion

In this study, three computer-assisted mobile learning tools–regular video, 180-degree video, and 180-degree VR video–were developed to assist students in learning perceptual sketching to draw real-world architectural scenes. Three computer-assisted mobile learning tools for conceptual sketching, including animated geometrical analysis, an interactive AR application, and 3D hologram, were developed to assist architectural constructions. The newly developed learning methods were implemented in the field of teaching to compare with three existing teaching methods in use – instructor’s on-site demonstration, stepwise illustration on booklet, and the same information presented interactively on mobile apps. The goal of the computer-assisted learning tools is to supplement the instructor’s on-site demonstration. The evaluation results indicated that on-site demonstrations were considered the most helpful. This result is in agreement with various studies of computer-assisted learning revealing that conventional face-to-face instruction remains essential in skill teaching (Rogers et al. 1998; Bloomfield, Roberts, and While 2010). One possible reason is that the instructor can provide immediate feedback during the stepwise demonstration.

The stepwise illustration documents the key developments of the analytical sketching process in the form of printed booklets and animated sequences that can be viewed interactively on mobile phones. These two learning materials, in fact provide the instructor’s stepwise demonstration in a way that allows for self-learning. Both were considered helpful, but it is interesting to note that the same information was perceived as very helpful when presented on the mobile phone with interactive viewing. This indicates that the viewing experience can play an important role in the computer-assisted learning tool. In fact, it is not surprising that many studies confirm that intuitive and easy to use learning materials can improve learning motivation and effectiveness (Gould, Terrell, and Fleming 2008; Thongmool and Phankokkruad 2014; Lei et al. 2005). The experimental computer-assisted learning methods exhibited the same trend. The animated geometrical analysis available on mobile apps is considered to be as good as the stepwise illustration on apps. AR and 3D holograms that require additional operation and even devices are much less preferred.

Based on the evaluation of conceptual learning materials, we believe that integrating computer-assisted learning on mobile phones is practical and widely welcomed. The same trend can also be observed in the three types of videos displaying on-site demonstrations. The three types of videos represent different realistic recreations of the experience of actually viewing the instructor’s demonstration. With the VR headset, the 180-degree VR video allows the user to view the visual environment where the instructor is sketching on the ground in front of the architectural structure in stereoscopic with a full field of frontal view. This was the most realistic recreation of an actual visual experience. The 180-degree VR video, on the other hand, also offers a full-frontal field of view of the live demonstration, with reduced display on the device screen. Although navigation involves different forms of interaction, such as moving the device to change the viewing or using a finger to change the point of view on the screen, it is more desirable than the full visual experience of the immersive VR experience. A conventional video that replays the process of sketching on paper provides the least realistic experience of the architectural scene to the user. However, this type of regular video is perceived as more helpful than immersive VR.

James Ferwerda defined three types of visual realism. Physical realism means that the visual information from the image needs to be exact like the real scene; photorealism, on the other hand, proposes that the image needs to stimulate the same visual response as the real scene. Functional realism, on the other hand, argues that visual information must deliver the same visual information as the real scene. Ferwerda further used an example of replacing shingles on a fence to attest that functional realistic visual information can more effectively achieve the intention than photorealistic visual information (Ferwerda 2003). The results of this study align with Ferwerda’s analysis. As teaching analytical sketching involves skill acquisition learning, 180-degree videos provide better functional realism than 180-degree VR, which provides the most physical or photorealism in this regard. We argue the same for conceptual sketching. A 3D hologram might present the most realistic viewing experience of the bird’s eye view of the architecture, followed by the reduced realism of the AR application. However, the least realistic 3D animated geometrical analysis presented on a 2D screen further delivers an abstract analysis that cannot be viewed in the real world and provides the best supplemental help to understand the taught subject.

5. Conclusion

This study first identifies that the key ability to utilize analytical sketching as a tool to study the design strategy of an observed architectural environment is to be able to imagine the interrelationships of conceptual architectural configuration and perceptual spatial experience. However, the underlying geometrical structure of architecture can be quite different from one case to another; further, the imagined process can also be difficult to convey on the site without visual aids. This study implements different computer-assisted mobile learning assistance to help visualize the three-dimensional configuration of architectural construction and different degrees of visual realism of viewing the on-site instructor’s live demonstrations. Because of the relatively small numbers of participants, the exact contribution of the implemented computer-assisted mobile learning assistance requires further study. However, students’ responses from this study do provide promising results: on one hand they demonstrate that the effectiveness of the mobile learning assistance can be enhanced with interactivity; on the other hand, it is reassuring that the instructor’s dedicated face-to-face instruction remains the most essential aspect of teaching analytical sketching.

In summary, although this study presents only one case of analytical sketching of perceptual and conceptual aspect of an architectural scene and construction, the result can be extended to many similar freehand drawings on-site, indicating that visual realism might not be the sole goal to create the learning assistance; instead, functional realism might matter the most.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the Ministry of Education in Taiwan under the Teaching Practice Research Program (PHA1090343).

Notes on contributors

Nan-Ching Tai

Nan-Ching Tai received his Master of Architecture and Ph.D. in the Built Environment from the University of Washington. He is currently a faculty member at Department of Interaction Design of National Taipei University of Technology. His teaching and research interests are in the area of visual language. He believes that sketching freehand analytically is a means by which to think and create in the field of design and thus can be considered as a native visual language. To communicate and realize a design idea, he believes computer graphics are an advanced common visual language. He notes that the development of high-dynamic-range-imaging-related technologies continue to advance visual realism. To this end, his research interests focus on developing a computational framework for creating a pictorial environment that can reflect perceptual reality and using this alternative environment to investigate the various issues related to space perception in the field of design.