PHOTOGRAPHY AS MACHINE VISION: THE ROLE OF IMAGING TECHNOLOGIES IN ART HISTORY

Abstract

This paper analyzes the relationship between photography and machine vision by examining its prehistory, as embodied in the art-historical photo archive. It characterizes the introduction of machine vision to the world of libraries, archives, and museums as a renewed (rather than wholly new) effort to mechanize or systematize art and its history. Assumptions and attitudes about photography embedded in the photo archive are therefore easily and often inconspicuously reproduced in efforts to study art computationally. Noting this repetition, this paper reframes the debate about art-historical applications of computer vision to emphasize the issues this technology raises about what constitutes art-historical work. The question is not whether we should use computer vision or computers for art-historical inquiry, but rather, what kinds of activities we ask computers to perform. Understanding machine vision in the context of its prehistory is critical for explicating its relationship to photography, and the role of photography and digital images in the study of visual culture more generally.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1. As compared with the field of Digital Humanities, for example, the prospect of CV’s use in disciplinary practice has prompted a more robust discourse amongst art historians and arguably, provoked more anxiety. Claire Bishop’s ‘Against Digital Art History’ remains effectively the only direct critique of digital art history or digital humanities written by an art historian. For an assessment of the uneven discourse in art history on Digital Humanities and Digital Art History, see Pugh, “Digital Art History,” 217–237.

2. PhotoTech formed the GRI’s contribution to a larger international consortium of photo archives focused on digitization and access. Caraffa et al., “PHAROS,” 2–11.

3. Research on photo archives has been led most prominently by Costanza Caraffa, director of the Photothek of the Kunsthistorisches Institut in Florenz. See Caraffa, “Florence Declaration.” Foundational texts on photography’s role in art history include Benjamin, “The Work of Art”; Malraux, The Voices of Silence.

4. For a historical overview of the new art history, see Jonathan Harris, The New Art History.

5. Maddox, “Bunched Images Begetting Ideas,” 255–256.

6. This research may be seen as part of the so-called ‘visual digital turn’ in digital humanities. Although a large majority of digital humanities research remains based in textual analysis, the last decade has seen increased interest in visual material, in significant part because of the emergence of computational methods like convolutional neural networks. Wevers et al., “The Visual Digital Turn.”

7. Han et al., “Artificial Intelligence”; Reshetnikov, “What AI can bring.” For more examples, see Cordell, “Machine Learning + Libraries.”

8. This technology, as art historian Hans Brandhorst usefully explains in relation to the iconography classification system ICONCLASS, is the basis of Google Reverse Image Search. Brandhorst, “A word is worth.”

9. di Lenardo et al., “Visual patterns discovery.” This method has since inspired the Yale Digital Humanities Lab to develop the Pixplot, a tool for sorting image collections by similarity (https://dhlab.yale.edu/projects/pixplot/).

10. Ginosar et al., “The burgeoning computer-art symbiosis,” 30–33.

11. Mansfield et al., “Techniques of the Art Historical Observer.”

12. Impett and Bell, “Ikonographie und Interaktion,” 31–53.

13. Bell and Offert, “Reflections on Connoisseurship”; Bell and Ommer, “Digital Connoisseur?”; Elgammal and Saleh, “Quantifying Creativity.”

14. Lawson-Tancred, “Can Machines Do Art History?”

15. Linguistics professor Emily Bender’s critiques of AI media hype thoroughly demonstrate that the problem extends far beyond art history. Bender, “Look behind the curtain.”

16. Drimmer, “How AI is hijacking art history.”

17. Ratzeburg, “Mediendiskussion im 19. Jahrhundert,” 27. Note that art history was still new as a discipline when the status of photography was being debated. The first university chairs representing the discipline were, for example, only appointed in the 1870s.

18. Ibid., 31.

19. Berenson, “Isochromatic Photography and Venetian Pictures,” 346.

20. Ibid.

21. Daston and Galison, “The Image of Objectivity,” 115.

22. Ratzeburg, “Mediendiskussion im 19. Jahrhundert,” 35.

23. For a discussion of Wöfflin’s references to photography in early editions of Principles, see Bohrer, “Photographic Perspectives,” 254–5.

24. Malraux, The Voices of Silence, 21.

25. Bohrer, “Photographic Perspectives,” 253.

26. For a discussion of this principle, see Nelson, “The Map of Art History,” 28–40. A lively, ongoing debate has taken place in library and archival sciences regarding library ‘neutrality,’ that is how classification can shape access and interpretation. See for example, Olson, “The Power to Name,” 639–68.

27. For more on the slide library as a site of art historical practice, see Langmead, “Art and Architectural History.”

28. Nelson, “The Slide Lecture,” 419. The side-by-side comparison of artworks persists in art history, although tools like PowerPoint have made it easier to show more than two images in a presentation at a time.

29. Ibid., 415.

30. The 1937 exhibition, curated by Beaumont Newhall, was the MoMA’s first dedicated to photography. Phillips, “The Judgment Seat of Photography,” 17.

31. McDonald, Centralizing Rochester, 3–4.

32. Crimp, “The Museum’s Old,” 7.

33. Siegal, “Masterworks for One and All.”

34. Langmead, “Art and Architectural History.”

35. For one discussion of practical issues of labor involved in processing arts collections, see ica Craig, “Computer Vision,” 198–208.

36. Saleh et al., “Toward automated discovery,” 3565.

37. Dobbs et al., “Jumping into the Artistic Deep End,” 873–89; Frank and Frank, “Complementing Connoisseurship,” 835-868; Van Noord et al., “Toward discovery of the artist’s style,” 46–54.

38. Lawson-Tancred, “Can Machines Do Art History?”; Bell and Offert, “Reflections on Connoisseurship,” 5ff.

39. For an overview of the historiography of connoisseurship in art history, see the introduction to Chapman et al., Connoisseurship and the Knowledge of Art.

40. Dobbs et al., “Jumping into the Artistic Deep End,” 875.

41. Cordell, “Q i-jtb the Raven,”188–225.

42. Nochlin’s 1971 essay “Why Have There Been No Great Women Artists?” famously explored how concepts of ‘genius’ and mastery were marked by racial and gender bias.

43. See for example Grant and Price, “Decolonizing Art History,” 8–66.

44. Trace, “Archival infrastructure and the information backlog,” 75–93.

45. See Cordell, “Machine Learning + Libraries” and the workshops organized by the AEOLIAN Network (Artificial Intelligence for Cultural Organisations, https://www.aeolian-network.net) for evidence of this interest.

Additional information

Notes on contributors

Emily Pugh

Emily Pugh received her PhD from the CUNY Graduate Center in New York City, where she focused on both postwar architecture and digital humanities. Since 2014, Pugh has led the Digital Art History department at the Getty Research Institute, overseeing research activities of projects like “Ed Ruscha Streets of Los Angeles” and PhotoTech, a project to digitize portions of the GRI’s Photo Archive. Her expertise within digital art history centers on the digital media of art history and its related infrastructures, which encompasses the digitization of physical materials, 3D scanning, computer vision, as well as collections metadata and its related workflows and processes. Her work on architecture and digital art history has been published in the Journal of the Society of Architectural Historians, Space and Culture, the Journal of Digital Art History, and will appear in the upcoming edition of Debates in the Digital Humanities. She has received funding for her research from the Center for Architecture Theory Criticism History at the University of Queensland, the Center for Digital Humanities Research at Australian National University, and the Foundation for Landscape Studies.

Tracy Stuber

Tracy Stuber received her PhD in Visual and Cultural Studies from the University of Rochester. She is currently a Research Specialist for the Digital Art History department at the Getty Research Institute, where she focuses on the use of emerging technologies such as computer vision and machine learning to discover new research possibilities within photographic archives. Before coming to the Getty, she was a Kress Interpretive Fellow at the George Eastman Museum, and her research in photography and digital humanities has been supported by the Andrew W. Mellon Foundation and Humanities New York.