New trends in art conservation, the use of lasers to clean as well as generate an augmented reality representation of an iconic public monument in bronze: The Alma Mater

Abstract

The 3.66 m tall and over nine-ton outdoor bronze sculpture of Alma Mater, created by Lorado Taft in 1929, is an iconic symbol of the University of Illinois at Urbana-Champaign. The conservation treatment of the sculpture came with some unexpected surprises, such as the discovery of failing internal ferrous nuts and bolts which needed to be replaced. The treatment involved removing the sculpture from its granite base, dismantling major elements of the sculpture and replacing the ferrous bolts with silicon bronze bolts, laser cleaning the surface with 1064 nm Q-switch scanning lasers, application of patina, applying a protective microcrystalline wax coating, and reinstallation of the sculpture. Analysis was performed with microscopy, X-radiography, and scanning electron microscopy. The conservation treatment involved cutting edge laser technology to remove corrosion from the surface, while structured light three-dimensional (3D) scanning and digital modeling techniques were used to create a new form of public interaction with the sculpture by 3D scanning the sculpture and creating a life like 3D digital model to create an augmented reality Alma Mater sculpture to temporarily take the place of the missing monument. Augmented reality is a technology which overlays a digital representation in registration with the real world giving the viewer an experience of interacting with the digital object as though it was actually present at that location in the real world. It is a longstanding tradition for students to have their picture taken with the sculpture during graduation, and thanks to this innovative use of technology, they were able to get their picture taken with a sculpture that was digitally, but not physically, present. In addition to receiving information and media related to the conservation treatment while it was taking place, the University of Illinois community and the class of 2013 was able to experience the augmented reality monument generated over the original base. This project demonstrates how augmented reality works of art can be used to replace the experience of actual objects that are removed from public view for conservation treatments.

Keywords:

• Laser ablation
• Laser corrosion removal
• 3D scanning
• 3D digital model
• Augmented reality
• Bronze

History

The 3.66 m tall and over nine-ton outdoor bronze sculpture of Alma Mater is an iconic symbol of the University of Illinois at Urbana-Champaign. The sculpture consists of the figure of the Alma Mater with welcoming open arms standing in front a throne and two figures of Learning and Labor, who are firmly shaking hands. The university's motto is ‘Learning and Labor’. Lorado Taft, who graduated from University of Illinois, created the sculpture that became a symbol of the university and a focal point for students during graduation. Students line up every year during graduations to get their picture taken with the monument. Taft said during his speech on the day the sculpture was dedicated in 1929 that he hopes the students ‘may keep the empty bronze throne polished by their visits!’ It is not often that one hears an artist directly state that he encourages physical public interaction with a sculpture and wants students to climb on it. After many years without maintenance and with continuous exposure to the elements, streaks of black and green corrosion covered the entire surface of the monument, hindering the legibility of the faces and all the intricate details of the figures. The university decided that it was time to take care of the monument and bring the sculpture back to the bronze appearance that the artist intended (Fig. 1).

Figure 1. Lorado Taft in front of the Alma Mater on the day of dedication 11 June 1929 (1). University of Illinois Archives at Urbana-Champaign.

Treatment of the sculpture

The original treatment plan for the sculpture was only surface related, and involved moving the monument to the Conservation of Sculpture and Objects Studio (CSOS) for laser cleaning, patina, and hot microcrystalline wax application. Having the monument at CSOS (Fig. 2) would allow for treatment to be efficiently performed in a secure and controlled environment that would not be subject to any weather-related delays. The sculpture was to be treated and reinstalled in time for graduation in 2013. When the sculpture was removed from its pedestal in August 2012, a major surprise was encountered. Most of the 50 elements composing the sculpture were connected by severely corroded ferrous bolts and nuts, many of which had already fallen apart. Some newer stainless steel bolts were found near the base of the sculpture in areas where they were easy to install. These newer bolts were installed when the sculpture was moved across campus in 1962, but the majority of the bolts that held the sculpture together were made of heavily corroded iron. It was visibly obvious that the iron bolts had suffered severe galvanic corrosion as a result of being in contact with the bronze for so many years. It was also observed that the interior of the sculpture was very damp due to water condensation on the internal walls. Pieces of rust from the failing internal iron armature fell out of the interior of the sculpture when it was dismantled. After this discovery the original plan of laser cleaning of the surface, patina, and wax application became a secondary issue and the treatment became a much more complex matter of restoring structural stability. A thorough investigation was required and a new treatment plan needed to be approved and executed. It was clear that reinstallation of the sculpture in time for graduation in May 2013 was no longer feasible.

Figure 2. Sculpture before treatment. University of Illinois Archives at Urbana-Champaign.

All of the visible iron bolts, which could be reached by climbing inside the sculpture from below, were documented. Team Industrial Service, Inc. was hired to perform the radiography of the sculpture to provide information on the condition of bolts deeper in the figures which could not be reached. The source used was Iridium 192, providing gamma rays at 37 curies (medium strength). Radiographic images were taken at distance of 48–76 cm and exposure was 8–13 minutes, proving that most of the ferrous bolts were severely corroded or entirely missing. Careful removal and replacement of all the ferrous bolts was a priority because the sculpture was structurally unstable. This is a sculpture that students climb onto daily, and it needed to be structurally sound.

The bolt replacement process required lifting the entire sculpture move than 0.5 m above ground to allow conservators to crawl inside the figures, removal of the heads of the three figures, and the dismantling of the figure of the Laborer to allow access to all of the bolts, some of which were in incredibly tight spaces and required significant time and creative solutions for removal. After almost a year of work, all of the approximately 1000 ferrous and stainless steel bolts in the sculpture were replaced with silicon bronze bolts, nuts, and washers. Silicon bronze is strong and is galvanically compatible with the bronze sculpture. In addition to replacing the bolts, some bronze welding was required to internally reinforce some areas of the base with reinforcing bronze brackets. Form-fitting bronze inserts were also welded in place and chased, where necessary, to fill in some holes in the monument such as areas that had corroded through completely and allowed water to flow into the sculpture. It is important to prevent water from entering bronze from above while also allowing any moisture and condensation to escape out from the bottom.

This unexpected treatment requirement meant that for the first time in the history of the University of Illinois, the Alma Mater was not going to be on campus for the graduating class of 2013. This dilemma was a motivating factor for the university to come up with a unique and revolutionary concept for preserving the campus tradition and to allow students to take pictures with the monument: replace the missing physical monument with an augmented reality (AR) sculpture. Preparation of the AR experience for the students took place while the sculpture was undergoing conservation treatment at CSOS.

Surface treatment

Samples of the surface corrosion and metal were analyzed (by MVA Scientific Consultants) with scanning electron microscopy – energy-dispersive X-ray spectroscopy (SEM-EDX). The surface corrosion was determined to be composed of copper sulfides and sulfates with chlorine, zinc, iron, lead, and oxygen present. Aluminosilicates indicating dirt and some calcium and barium were also found, most likely as a residue of graffiti paint. The metal composition was determined to be Cu 68–94%, Sn 0–7%, Zn 0–5%, Al 0–3%. Surprisingly, there was no lead found in any of the analyzed samples. Normally sculptures cast during the same period have lead in their metal composition (Scott, 2002).

Prior to treatment the goals for the final color of the sculpture were extensively discussed and evaluated based on historic images and records. The owner of the sculpture, University of Illinois, decided that the sculpture should be brought back to an original brown patina appearance after completion of the treatment. Preserving the green and black surface deterioration products did not appear as the best choice from the aesthetical or theoretical point of view. The sculpture is only 85 years old and it is known that the artists' intent was for the sculpture to be brown.¹ Another reason for removing green corrosion products was presence of copper chlorides in the surface corrosion indicating unstable nature of the green patina (Scott, 2002; Dajnowski, 2007). One more, but very important practical reason for patinating the sculpture brown was the fact that the sculpture is climbed by thousands of people every year and portions of the surface of the sculpture are abraded to bare metal and then quickly oxidize to a brown cuprite color. These polished areas visually contrast with the black and green streaks of corrosion near them. These factors led to the choice of a brown patina as the final treatment outcome rather than preserving the green and black streaks of corrosion that were merely a result of a lack of maintenance over time.

Laser cleaning was selected for removing the unwanted green and black corrosion products from the bronze surface. Chemical cleaning methods may leave behind chemical residues in surface pores and abrasive or mechanical methods inherently involve the risk of causing some level of damage to a surface (Dajnowski, 2007). Laser cleaning is precise and instantaneous (Fotakis et al., 2006). CSOS has laser cleaned many bronze monuments because a properly calibrated laser in the hands of a skilled operator offers an unmatched level of cleaning precision without causing damage to the metal surface. Laser cleaning is also relatively environmentally friendly as a cleaning technique because it avoids the use and need for disposal of chemicals and air abrasives.

The external surface of the sculpture (Fig. 3) was cleaned with high frequency and low frequency 1064 nm Nd-YAG lasers with pulse durations of 100–290 ns. The fluence was adjusted as needed from 1 to 4 J/cm². After each area was laser cleaned it was lightly cleaned with soft brass brushes to remove ablation residues from the surface. The sculpture was washed to encourage any remaining chlorides to react with the metal and laser cleaning was repeated until all visible surface corrosion products were removed. Next, the entire sculpture was hot patinated using a solution of 3% ammonium sulfide in distilled water.

Figure 3. The face of the Alma Mater half cleaned by 1064 nm laser ablation, before application of patina.

Coating options were discussed with the owner and it was determined that a hot applied mixture of microcrystalline and polyethylene waxes was the most appropriate. An acrylic coating can peel away when abraded, while a wax coating has the ability to smear and self-heal to some extent. The wax coating can be relatively easily maintained whenever needed. The interior of the sculpture also received a protective wax coating.

Re-installation

After completion of the treatment the sculpture was brought back to a very welcoming reception on campus and reinstalled on its original pedestal. The granite base was washed and a time capsule was placed inside. The entire installation process was transmitted live via a high-resolution webcam, called the Alma Cam (Fig. 4). The university sent e-mails to students and alumni about the time and date of the reinstallation of the monument with a link to the webcam so that they could watch it live. The web traffic volume exceeded 10 000 simultaneous viewers as alumni and the general public from around the world watched the event. For the conservators onsite, it was a unique experience to share with colleagues, friends, and family the historic moment of the Alma Mater returning to campus. The webcam is still active and the sculpture can be viewed live at http://illinois.edu/about/tours/almacam.html. Today the webcam also works as a protective measure. The monument is constantly monitored and is therefore less likely to be subjected to vandalism.

Figure 4. Screenshot image of the sculpture after the installation ceremony ended (transmitted through webcam by the University of Illinois) (7). University of Illinois Archives at Urbana-Champaign.

Scanning the Alma Mater

In order to develop the virtual augmented Alma Mater display, a three-dimensional scan of the Alma Mater had to be acquired (Fig. 5). This process employed two blue light Steinbichler L3D digitization systems on site at the Forest Park CSOS facility where the sculpture was located during the development of the project. Three-dimensional scanners are used when an exact digital replica is needed of a physical object (Levoy et al., 1995, 2000; Curless et al., 1996). The Steinbichler scanners project a blue structured light pattern from a high precision digital light projector while simultaneously an internal machine vision camera captures live video of the projected pattern. The precise pattern moves to computer-controlled predetermined positions and the sophisticated algorithms in the software, which control the mechanics of the system, interpret the deformation of the pattern as depth. The depth is converted into 3D space and is displayed in the interface of the software as 3D coordinates (Tao, 2007; Lakatos, 2010). This procedure of capturing one patch of data takes about three seconds, and captures an approximately 400 × 400 mm data patch with a point spacing of 0.250 mm. For a sculpture the size of Alma Mater, it required thousands of scan patches over four days to acquire the entire 3D surface. The final data set comprised approximately three billion data points, but for the sake of the production timeline only 250 million were actually processed. This created a raw merged model that was ready for the clean-up process (Levoy et al., 1994).

Figure 5. Sean Belshaw from Applied Precision performing 3D scanning of the sculpture.

Data clean-up is necessary because the physical surface quality was inconsistent, as parts of the sculpture were dark from the oxidization of the bronze while other regions were highly polished from being touched regularly. These varying surface conditions created a variety of noise and data inconsistencies. These data inconsistencies were subtle and easily removed using downstream processing software. Also occlusion due to the complexity of the surface contour prevented the 3D scanner from digitizing small portions that were hidden from view creating holes in the 3D data. This processing 3D was accomplished using 3D Systems Geomagic software through a process of removing unnecessary data, hole filing, and overall 3D mesh clean-up to improve the appearance of the model. The final 3D model is a polygon surface mesh (Levoy et al., 2002; Fabio, 2003).

In order to create the 3D model that was displayed in the augmented application a low-resolution version had to be generated which was optimized for mobile devices. This was done so that the hand-held device would display the details at a real-time rate of display. We used Autodesk tools to create the 3D model in the low-resolution format, and UV mapping tools (Zhang et al., 2005) to create a coordinate system that the 3D model uses to display color, lighting, and realistic surface properties in the augmented application. In order for the final low-resolution model to display with a high-resolution appearance, we transferred the high-resolution 3D scan data appearance onto a map that will make the appearance of the low-resolution look like the high-resolution model. This is known as a normal or displacement mapping technique (Krishnamurthy et al., 1996; Turk et al., 2004). It is a technique utilizing the difference between the high-resolution image with 250 million points, compared to the low-resolution 30 000 point polygon model, creating a difference map that through a real-time rendering displays the low-resolution model with the appearance of the high-resolution detail. This is used in video gaming applications to convey detail without the extra cost of unnecessary data straining the graphics system, allowing a smoother display, and viewing experience. (Cignoni et al., 1998; Mikkelsen, 2008). We also created the color and lighting based on the predictions of the surface appearance.

Now that the 3D model and the appearance maps are created we had to assign them to positions in space that best represented the physical 3D space they would be projected in for the AR application. This was done using the Unity 3D game engine. It was used to compile all the necessary elements, position the model in the proper space, and execute the correct data export so that the augmented application would display properly.

Augmented reality

At the University of Illinois, there has been a tradition for decades that graduating students have their picture taken, in their caps and gowns, at the Alma Mater (Steinfeldt, 2013). With the Alma Mater out for restoration, it was not clear how the students could uphold this tradition.

AR is a medium, supported by technology, in which digital information is overlaid on the real world in registration with the real world, interactive in real time (Craig, 2013). Through the use of AR, one could place a digital representation of an object, in this case the Alma Mater, at a specific location in the real world and have it appear to people as though it was actually there, at the site where the physical Alma Mater stood until it was removed for restoration. The interactivity in this case was that it allowed participants to walk around the Alma Mater and photograph it from whatever perspective they would see the actual sculpture. In other words, as you walked around the Alma Mater, they would see the Alma Mater, in place, on its (real world) pedestal, and depending on where you were standing, you would see it from the front, side, etc. Those views were continually updated as you walked about the sculpture, causing it to appear to the viewer as a solid object that you could view from all directions appropriately.

There are many different ways to carry out an AR experience. For the AR Alma Mater experience we chose to use a ‘magic lens’ paradigm (Bier et al., 1993; Craig, 2013), whereby typically a smartphone, tablet, or glasses used as a lens are used to observe the real world with the AR elements superimposed. We used a device perspective magic lens (Baričević et al., 2012). For the AR Alma Mater, we created an app for ios devices (iPhone, iPad), and Android devices. When the app runs on the device, it opens the camera on the device and the participant sees the real world on the screen through the camera lens. If the camera is pointed somewhere where an AR element should be (in this case the Alma Mater) it displays the AR object in the correct location and orientation and from the correct perspective.

In order for the application to work correctly, the device must ‘know’ where it is located in the real world, and how it is oriented. In the case of the AR Alma Mater, we chose to use computer vision. Since the camera in the device can ‘see’ the real world, it could look for features in the real world to help determine where it was, and how it was oriented. To do this properly, we needed to determine the camera's position in six degrees of freedom (X, Y, Z location and yaw, pitch, and roll orientation).

In order to provide an easily identified feature in the environment, designer Scott Paceley from the University of Illinois Creative Services created a poster that contained an image of the Alma Mater on a complex background. The image had to be carefully crafted in order to provide enough information for the computer vision algorithm to compute the position information with six degrees of freedom. The poster was then mounted on the base where the Alma Mater usually stands (Fig. 6).

Figure 6. Left – Mounting the poster that was used as a fiducial marker on the granite base, and right the AR representation of Alma Mater (photo: Magdalena Dajnowski).

Different AR development platforms were considered including Metaio, Aurasma, Daqri, ARToolWorks, and others (Marneanu et al., 2014). Ultimately Vuforia was used to create our own app. Vuforia provided robust computer vision capability and allowed us the flexibility we needed to carry out our goals. We used a version of Vuforia that could be used as a plugin to the Unity 3D Game Development environment. In order to implement the AR application, we imported the 3D model of the Alma Mater into Unity (Vuforia, 2013). It required an iterative collaboration with the laser scanners creating the 3D model in order to get the look we were after while having a simple enough model to interact with in real time on a hand-held device.

Eclipse code development environment was used to create the application which was distributed via the Apple Store and via an .apk file for Android. The 3D model of the Alma Mater was included within the app so there was no need for network communication while using the app (Fig. 7).

Figure 7. Left – Crowds lined up to try the AR Alma Mater application and have a picture taken with the virtual Alma Mater. Right – Because not everyone had a device with which to experience the AR Alma Mater, the campus set up an iPad for graduates. People with their own devices were free to walk about the area and see the Alma Mater from all perspectives in front of the poster and choose the perspective they would like for their own photo.

To experience 3D AR Alma Mater download the Alma Mater AR program by searching for ‘Alma Mater’ in the app-store, activate the app, and point your device at picture 8 and enjoy the result of our work. You can view the sculpture from different angles (Fig. 8).

Figure 8. AR Alma Mater poster. Activate the app on your mobile device and point it at the image.

Summary

The conservation treatment of the Alma Mater and the AR experience was a success. The sculpture was laser cleaned and laser scanned to create an AR stand in for the monument while it was being treated. The AR experience gave students and the general public a preview of the monument would look after the disfiguring corrosion was removed. Many students had the opportunity to try this new way of experiencing public art, and they were also able to have a memento picture taken with the AR Alma Mater. The University of Illinois utilized the newest technologies during this treatment and added live-cam experience for everyone to see the installation of the sculpture as well as take a look at the sculpture at any time. The live webcam also works as a security camera allowing the university to monitor the surroundings of the monument. This innovative project will undoubtedly influence future conservation projects and broaden the possibilities for public interaction with works of art off display and undergoing conservation treatments. From massive monuments to museum artifacts, AR can be used to allow the public to experience art in a very engaging and entertaining format.

Acknowledgements

We would like to thank: Jim Lev, University of Illinois, Melvyn Skvarla – University of Illinois, Christa Leigh Deacy – Quinn – University of Illinois, Jennifer Tapper – University of Illinois, Robert McGrath – University of Illinois, Scott Paceley – University of Illinois, Joel Steinfeldt – University of Illinois, Alex Jerez – University of Illinois, Mary Miller – MVA, Robert Bell, Applied Precision, Sean Belshaw – Applied Precision, Leo Perez – Applied Precision, Roger Machine – Methods and Materials, Kyle Brown – Team Industrial, CSOS–Robert Zarycki, Tadeusz Mlynarczyk, Christopher Ciaston, Kazimierz Parys, Luke Grim, Wladyslaw Zarycki, Rossana Ioppolo.

Notes

1 Color image of Alma Mater ca. 1945, Archives of University of Illinois.