A Sketch-Based Interface for Facial Animation in Immersive Virtual Reality

References

• Aaron, B., Philip, K., & James, M. (2009). Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of Usability Studies, 4(3), 114–123.
   Google Scholar
• Anjyo, K., Todo, H., & Lewis, J. P. (2012). A practical approach to direct manipulation blendshapes. Journal of Graphics Tools, 16(3), 160–176. https://doi.org/10.1080/2165347X.2012.689747
   Google Scholar
• Bae, S.-H., Balakrishnan, R., & Singh, K. (2008). ILoveSketch: As-natural-as-possible sketching system for creating 3D curve models. In Proc. Annual ACM Symposium on User Interface Software and Technology (pp. 151–160).
   Google Scholar
• Barbieri, S., Garau, N., Hu, W., Xiao, Z., & Yang, X. (2016). Enhancing character posing by a sketch-based interaction. In Proc. ACM SIGGRAPH (pp. 1–2). https://doi.org/10.1145/2945078.2945134
   Google Scholar
• Barr, A. H. (1984). Global and local deformations of solid primitives. ACM SIGGRAPH Computer Graphics, 18(3), 21–30. https://doi.org/10.1145/964965.808573
   Google Scholar
• Beeler, T., Hahn, F., Bradley, D., Bickel, B., Beardsley, P., Gotsman, C., Summer, R. W., & Gross, M. (2011). High-quality passive facial performance capture using anchor frames. In Proc. ACM SIGGRAPH (pp. 1–10). https://doi.org/10.1145/1964921.1964970
   Google Scholar
• Bentley, J. L. (1975). Multidimensional binary search trees used for associative searching. Communications of the ACM, 18(9), 509–517. https://doi.org/10.1145/361002.361007
   Web of Science ®Google Scholar
• Bernier, E., Chellali, R., Thouvenin, I. M., & Blom, K. (2012). The ICEA plug-in for virtual reality, immersive creation and edition of animation. In Workshop on Software Engineering and Architectures for Realtime Interactive Systems (pp. 36–42).
   Google Scholar
• Berson, E., Soladie, C., Barrielle, V., & Stoiber, N. (2019). A robust interactive facial animation editing system. In Proc. Motion, Interaction and Games (pp. 1–10). https://doi.org/10.1145/3359566.3360076
   Google Scholar
• Blanz, V., Basso, C., Poggio, T., & Vetter, T. (2003). Reanimating faces in images and video. Computer Graphics Forum, 22(3), 641–650. https://doi.org/10.1111/1467-8659.t01-1-00712
   Web of Science ®Google Scholar
• Brooke, J. (1996). SUS – A quick and dirty usability scale. Usability Evaluation in Industry, 189(194), 4–7. https://doi.org/10.1201/9781498710411
   Google Scholar
• Cannavò, A., Calandra, D., Kehoe, A., & Lamberti, F. (2020). Evaluating consumer interaction interfaces for 3D sketching in virtual reality. In Proc. EAI International Conference: ArtsIT, Interactivity & Game Creation (pp. 291–306).
   Google Scholar
• Cannavò, A., Demartini, C., Morra, L., & Lamberti, F. (2019). Immersive virtual reality-based interfaces for character animation. IEEE Access, 7, 125463–125480. https://doi.org/10.1109/ACCESS.2019.2939427
   Web of Science ®Google Scholar
• Cannavò, A., Zhang, C., Wang, W., & Lamberti, F. (2020). Posing 3D characters in virtual reality through in-the-air sketches. In Proc. International Conference on Computer Animation and Social Agents (pp. 51–61).
   Google Scholar
• Cetinaslan, O., & Orvalho, V. (2018). Direct manipulation of blendshapes using a sketch-based interface. In Proc. International ACM Conference on 3D Web Technology (pp. 1–10). https://doi.org/10.1145/3208806.3208811
   Google Scholar
• Cetinaslan, O., & Orvalho, V. (2020a). Sketching manipulators for localized blendshape editing. Graphical Models, 108(1), 101059. https://doi.org/10.1016/j.gmod.2020.101059
   Web of Science ®Google Scholar
• Cetinaslan, O., & Orvalho, V. (2020b). Stabilized blendshape editing using localized Jacobian transpose descent. Graphical Models, 112(1), 101091. https://doi.org/10.1016/j.gmod.2020.101091
   Web of Science ®Google Scholar
• Cetinaslan, O., Orvalho, V., & Lewis, J. (2015). Sketch-based controllers for blendshape facial animation. In Proc. Eurographics (pp. 25–28).
   Google Scholar
• Chang, E., Jenkins, O. C. (2006). Sketching articulation and pose for facial animation. In Proc. ACM SIGGRAPH/Eurographics Symposium on Computer Animation (pp. 145–161).
   Google Scholar
• Choi, B., I Ribera, R. B., Lewis, J. P., Seol, Y., Hong, S., Eom, H., Jung, S., & Noh, J. (2016). SketchiMo: Sketch-based motion editing for articulated characters. ACM Transactions on Graphics, 35(4), 1–12. https://doi.org/10.1145/2897824.2925970
   Web of Science ®Google Scholar
• Ciccone, L., Öztireli, C., & Sumner, R. W. (2019). Tangent-space optimization for interactive animation control. ACM Transactions on Graphics, 38(4), 1–10. https://doi.org/10.1145/3306346.3322938
   Web of Science ®Google Scholar
• Flueckiger, B. (2011). Computer-generated characters in Avatar and Benjamin Button. Digitalitat und Kino, 1(1), 109–137.
   Google Scholar
• Gipson, J., Brown, L., Robbins, E., Gomez, J., Anderson, M., Velasquez, J., Ruiz, J., & Cooper, D. (2018). VR story production on Disney animation’s “cycles”. In Proc. ACM SIGGRAPH (pp. 1–2).
   Google Scholar
• Gouvatsos, A., Xiao, Z., Marsden, N., & Zhang, J. J. (2017). Posing 3D models from drawings. Computers in Entertainment, 15(2), 1–14. https://doi.org/10.1145/2729984
   Google Scholar
• Guay, M., Cani, M.-P., & Ronfard, R. (2013). The line of action: An intuitive interface for expressive character posing. ACM Transactions on Graphics, 32(6), 1–8. https://doi.org/10.1145/2508363.2508397
   Web of Science ®Google Scholar
• Gunnarsson, Ö., & Maddock, S. C. (2010). Sketch-based posing of 3D faces for facial animation. In Proc. Theory and Practice of Computer Graphics (pp. 223–230).
   Google Scholar
• Hahn, F., Mutzel, F., Coros, S., Thomaszewski, B., Nitti, M., Gross, M., & Sumner, R. W. (2015). Sketch abstractions for character posing. In Proc. ACM SIGGRAPH/Eurographics Symposium on Computer Animation (pp. 185–191). https://doi.org/10.1145/2786784.2786785
   Google Scholar
• Henrikson, R., Araujo, B., Chevalier, F., Singh, K., & Balakrishnan, R. (2016). Multi-device storyboards for cinematic narratives in VR. In Proc. Annual Symposium on User Interface Software and Technology (pp. 787–796). https://doi.org/10.1145/2984511.2984539
   Google Scholar
• Igarashi, T., Matsuoka, S., & Tanaka, H. (2006). Teddy: A sketching interface for 3D freeform design. In Proc. ACM SIGGRAPH (pp. 11–es).
   Google Scholar
• Jackson, B., & Keefe, D. F. (2016). Lift-off: Using reference imagery and freehand sketching to create 3D models in VR. IEEE Transactions on Visualization and Computer Graphics, 22(4), 1442–1451. https://doi.org/10.1109/TVCG.2016.2518099
   PubMed Web of Science ®Google Scholar
• Jiang, Y., Zhang, C., Fu, H., Cannavò, A., Lamberti, F., Lau, H. Y., & Wang, W. (2021). Handpainter – 3D sketching in VR with hand-based physical proxy. In Proc. CHI Conference on Human Factors in Computing Systems (pp. 1–13). https://doi.org/10.1145/3411764.3445302
   Google Scholar
• Karras, T., Aila, T., Laine, S., Herva, A., & Lehtinen, J. (2017). Audio-driven facial animation by joint end-to-end learning of pose and emotion. ACM Transactions on Graphics, 36(4), 1–12. https://doi.org/10.1145/3072959.3073658
   Web of Science ®Google Scholar
• Kim, B. (2019). Virtual reality for 3D modeling. Beyond Reality: Augmented, Virtual, and Mixed Reality in the Library, 1(1), 31–46.
   Google Scholar
• Kim, J., & Singh, K. (2021). Optimizing UI layouts for deformable face-rig manipulation. ACM Transactions on Graphics, 40(4), 1–12. https://doi.org/10.1145/3476576.3476759
   Web of Science ®Google Scholar
• Lamberti, F., Cannavo, A., & Montuschi, P. (2020). Is immersive virtual reality the ultimate interface for 3D animators? Computer Magazine, 53(4), 36–45. https://doi.org/10.1109/MC.2019.2908871
   Web of Science ®Google Scholar
• Lewis, J. P., & Anjyo, K. (2010). Direct manipulation blendshapes. IEEE Computer Graphics and Applications, 30(4), 42–50. https://doi.org/10.1109/MCG.2010.41
   PubMed Web of Science ®Google Scholar
• Lewis, J. P., Anjyo, K., Rhee, T., Zhang, M., Pighin, F. H., & Deng, Z. (2014). Practice and theory of blendshape facial models. Eurographics (State of the Art Reports), 1(8), 2. https://doi.org/10.2312/egst.20141042
   Google Scholar
• Li, C., Pan, H., Liu, Y., Tong, X., Sheffer, A., & Wang, W. (2018). Robust flow-guided neural prediction for sketch-based freeform surface modeling. ACM Transactions on Graphics, 37(6), 1–12. https://doi.org/10.1145/3272127.3275051
   Web of Science ®Google Scholar
• Liu, N., Zhang, D., Ru, X., Zhao, H., Wang, X., & Wu, Z. (2021). Caricature expression extrapolation based on Kendall shape space theory. IEEE Computer Graphics and Applications, 41(3), 71–84. https://doi.org/10.1109/MCG.2021.3069948
   PubMed Web of Science ®Google Scholar
• Liu, X., Xia, S., Fan, Y., & Wang, Z. (2011). Exploring non-linear relationship of blendshape facial animation. Computer Graphics Forum, 30(6), 1655–1666. https://doi.org/10.1111/j.1467-8659.2011.01852.x
   Web of Science ®Google Scholar
• Mao, C., Qin, S. F., & Wright, D. (2007). Sketch-based virtual human modelling and animation. In Proc. International Symposium on Smart Graphics (pp. 220–223).
   Google Scholar
• Miranda, J. C., Alvarez, X., Orvalho, J., Gutierrez, D., Sousa, A. A., & Orvalho, V. (2012). Sketch express: A sketching interface for facial animation. Computers & Graphics, 36(6), 585–595. https://doi.org/10.1016/j.cag.2012.03.002
   Web of Science ®Google Scholar
• Müller, M. (2007). Dynamic time warping. In Information retrieval for music and motion (pp. 69–84). Springer.
   Google Scholar
• Niehorster, D. C., Li, L., & Lappe, M. (2017). The accuracy and precision of position and orientation tracking in the HTC Vive virtual reality system for scientific research. i-Perception, 8(3), 2041669517708205. https://doi.org/10.1177/2041669517708205
   Web of Science ®Google Scholar
• Osawa, N., Asai, K. (2003). An immersive path editor for keyframe animation using hand direct manipulation and 3D gearbox widgets. In Proc. International Conference on Information Visualization (pp. 524–529).
   Google Scholar
• Pantuwong, N. (2016). A tangible interface for 3D character animation using augmented reality technology. In Proc. International Conference on Information Technology and Electrical Engineering (pp. 1–6).
   Google Scholar
• Paravati, G., Lamberti, F., Gatteschi, V., Demartini, C., & Montuschi, P. (2017). Point cloud-based automatic assessment of 3D computer animation courseworks. IEEE Transactions on Learning Technologies, 10(4), 532–543. https://doi.org/10.1109/TLT.2016.2638811
   Web of Science ®Google Scholar
• Parke, F. I. (1974). A parametric model for human faces. The University of Utah.
   Google Scholar
• Sagar, M. (2006). Facial performance capture and expressive translation for King Kong. In Proc. ACM SIGGRAPH (pp. 26–es). https://doi.org/10.1145/1179849.1179882
   Google Scholar
• Serra, J., Cetinaslan, O., Ravikumar, S., Orvalho, V., & Cosker, D. (2018). Easy generation of facial animation using motion graphs. Computer Graphics Forum, 37(1), 97–111. https://doi.org/10.1111/cgf.13218
   Web of Science ®Google Scholar
• Sifakis, E., Neverov, I., & Fedkiw, R. (2005). Automatic determination of facial muscle activations from sparse motion capture marker data. In Proc. ACM SIGGRAPH (pp. 417–425).
   Google Scholar
• Singer, G. (2003). The two towers: Face to face with Gollum. Animation World Network. https://www.awn.com/animationworld/two-towers-face-face-gollum
   Google Scholar
• Thomas, F., Johnston, O., & Thomas, F. (1995). The illusion of life: Disney animation. Disney Editions.
   Google Scholar
• Thorne, M., Burke, D., & van de Panne, M. (2004). Motion doodles: An interface for sketching character motion. ACM Transactions on Graphics, 23(3), 424–431. https://doi.org/10.1145/1015706.1015740
   Web of Science ®Google Scholar
• Vogel, D., Lubos, P., & Steinicke, F. (2018). Animation VR-interactive controller-based animating in virtual reality. In Proc. Workshop on Animation in Virtual and Augmented Environments (pp. 1–6).
   Google Scholar
• Zhang, L., Snavely, N., Curless, B., & Seitz, S. M. (2004). Spacetime faces: High resolution capture for modeling and animation. In ACM SIGGRAPH 2004 Papers (pp. 548–558).
   Google Scholar
• Zou, Q., Bai, H., Gao, L., Fowler, A., & Billinghurst, M. (2022). Asymmetric interfaces with stylus and gesture for VR sketching. In Proc. IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (pp. 968–969).
   Google Scholar