A problem-based approach to the advancement of heuristics for socio-technical evaluation

References

• Ackermann, M. S., S. P. Goggins, T. Herrmann, M. Prilla, and C. Stary. 2017. Designing Healthcare That Works – A Socio-technical Approach. London: Academic Press.
   Google Scholar
• Amir, S., and V. Kant. 2018. “Sociotechnical Resilience: A Preliminary Concept.” Risk Analysis 38 (1): 8–16.
   PubMed Web of Science ®Google Scholar
• Andriessen, J. H. E. 2003. Working with Groupware. London: Springer. doi:10.1007/978-1-4471-0067-6.
   Google Scholar
• Baker, K., S. Greenberg, and C. Gutwin. 2001. “Heuristic Evaluation of Groupware Based on the Mechanics of Collaboration.” In Engineering for Human-Computer Interaction, Vol. 2254, edited by M. R. Little, and L. Nigay, 123–139. Berlin Heidelberg: Springer. doi:10.1007/3-540-45348-2_14.
   Google Scholar
• Baxter, G., and I. Sommerville. 2011. “Socio-technical Systems: From Design Methods to Systems Engineering.” Interacting with Computers 23 (1): 4–17. doi:10.1016/j.intcom.2010.07.003.
   Web of Science ®Google Scholar
• Behrenbruch, K., O. Kieselmann, M. Schuldt, M. Söllner, and L. Schmidt. 2014. “System EvaluationEvaluation.” In Socio-technical Design of Ubiquitous Computing Systems, edited by K. David, K. Geihs, J. M. Leimeister, A. Roßnagel, L. Schmidt, G. Stumme, and A. Wacker, 297–319. Springer International Publishing. doi:10.1007/978-3-319-05044-7_17.
   Google Scholar
• Bødker, S. 2015. “Third-wave HCI, 10 Years Later—Participation and Sharing.” Interactions 22 (5): 24–31.
   Google Scholar
• Booth, R. G., M.-A. Andrusyszyn, C. Iwasiw, L. Donelle, and D. Compeau. 2016. “Actor-Network Theory as a Sociotechnical Lens to Explore the Relationship of Nurses and Technology in Practice: Methodological Considerations for Nursing Research.” Nursing Inquiry 23 (2): 109–120.
   PubMed Web of Science ®Google Scholar
• Bruce, B. C., A. Rubin, and J. An. 2010. “Situated Evaluation of Socio-Technical Systems.” In Social Computing: Concepts, Methodologies, Tools, and Applications, edited by B. C. Bruce, A. Rubin, and J. An, 2211–2225. IGI Global.
   Google Scholar
• Carayannis, E. G., F. A. F. Ferreira, P. Bento, J. J. M. Ferreira, M. S. Jalali, and B. M. Q. Fernandes. 2018. “Developing a Socio-Technical Evaluation Index for Tourist Destination Competitiveness Using Cognitive Mapping and MCDA.” Technological Forecasting and Social Change 131: 147–158. doi:10.1016/j.techfore.2018.01.015.
   Web of Science ®Google Scholar
• Carroll, J. M., and R. L. Campbell. 1989. “Artifacts as Psychological Theories: The Case of Human-Computer Interaction.” Behaviour & Information Technology 8 (4): 247–256.
   Web of Science ®Google Scholar
• Cherns, A. 1987. “Principles of Sociotechnical Design Revisted.” Human Relations 40 (3): 153–162.
   Web of Science ®Google Scholar
• Clegg, C. W. 2000. “Sociotechnical Principles for System Design.” Applied Ergonomics 31 (5): 463–477. doi:10.1016/s0003-6870(00)00009-0.
   PubMed Web of Science ®Google Scholar
• Clement, A. 1994. “Considering Privacy in the Development of Multi-Media Communications.” Computer Supported Cooperative Work (CSCW) 2 (1): 67–88. doi:10.1007/BF00749284.
   Google Scholar
• Cockton, G., D. Lavery, and A. Woolrych. 2008. “Inspection-based Evaluations.” In The Human-Computer Interaction Handbook, edited by A. Sears and J. Jacko. Boca Raton: CRC Press.
   Google Scholar
• Creswell, J. W. 2013. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Sage Publications Incorporated.
   Google Scholar
• Dancey, C., and J. Reidy. 2011. Statistical Without Math for Psychology. Harlow: Pearson Education.
   Google Scholar
• Demir, F., S. M. Kim, N. Current, and I. Jahnke. 2019. Strategic improvement planning in schools: A sociotechnical approach for understanding current practices and design recommendations: Management in Education. doi:10.1177/0892020619847681.
   Google Scholar
• Desmond, D. B., D. Lacey, and P. Salmon. 2019. “Evaluating Cryptocurrency Laundering as a Complex Socio-Technical System: A Systematic Literature Review.” Journal of Money Laundering Control 22 (3): 480–497. doi:10.1108/JMLC-10-2018-0063.
   Web of Science ®Google Scholar
• Dix, A., J. E. Finlay, G. D. Abowd, and R. Beale. 2003. Human-Computer Interaction, 3rd ed. Essex: Pearson.
   Google Scholar
• Dourish, P., and V. Bellotti. 1992. “Awareness and Coordination in Shared Workspaces.” Proceedings of the 1992 ACM Conference on Computer-Supported Cooperative Work - CSCW ‘92, 107–114. doi:10.1145/143457.143468.
   Google Scholar
• Dunckel, H. 1989. Arbeitspsychologische Kriterien zur Beurteilung und Gestaltung von Arbeitsaufgaben im Zusammenhang mit EDV-Systemen. B.G.Teubner. http://dl.gi.de/handle/20.500.12116/6328.
   Google Scholar
• Eason, K. D. 1989. Information Technology And Organisational Change. CRC Press. doi:10.1201/9781482275469.
   Google Scholar
• Edmondson, A. C. 2004. “Learning from Mistakes is Easier Said than Done: Group and Organizational Influences on the Detection and Correction of Human Error.” The Journal of Applied Behavioral Science 40 (1): 66–90. doi:10.1177/0021886304263849.
   Google Scholar
• Emery, F. E., E. Thorsrud, and E. Thorsrud. 2013. Form and Content in Industrial Democracy: Some Experiences from Norway and Other European Countries. Routledge. doi:10.4324/9781315013695.
   Google Scholar
• Ferreira, F. A. F., J. P. Esperança, M. A. S. Xavier, R. L. Costa, and B. Pérez-Gladish. 2019. “A Socio-Technical Approach to the Evaluation of Social Credit Applications.” Journal of the Operational Research Society 70 (10): 1801–1816. doi:10.1080/01605682.2017.1415650.
   Web of Science ®Google Scholar
• Fischer, G., and T. Herrmann. 2015. “Meta-design: Transforming and Enriching the Design and Use of Socio-technical Systems.” In Designing Socially Embedded Technologies in the Real-World, edited by V. Wulf, K. Schmidt, and D. Randall, 79–109. London: Springer. doi:10.1007/978-1-4471-6720-4_6.
   Google Scholar
• Ghaffarian, V. 2011. “The new Stream of Socio-Technical Approach and Main Stream Information Systems Research.” Procedia Computer Science 3: 1499–1511. doi:10.1016/j.procs.2011.01.039.
   Google Scholar
• Greenberg, S., G. Fitzpatrick, C. Gutwin, and S. Kaplan. 2000. “Adapting the Locales Framework for Heuristic Evaluation of Groupware.” Australasian Journal of Information Systems 7 (2), doi:10.3127/ajis.v7i2.267.
   Google Scholar
• Grote, G., C. Ryser, T. Wāler, A. Windischer, and S. Weik. 2000. “KOMPASS: A Method for Complementary Function Allocation in Automated Work Systems.” International Journal of Human-Computer Studies 52 (2): 267–287. doi:10.1006/ijhc.1999.0289.
   Web of Science ®Google Scholar
• Hackman, J. R., and G. R. Oldham. 1975. “Development of the Job Diagnostic Survey.” Journal of Applied Psychology 60 (2): 159–170. doi:10.1037/h0076546.
   Web of Science ®Google Scholar
• Herrmann, T. 2020. “Socio-Technical Design of Hybrid Intelligence Systems–The Case of Predictive Maintenance.” International Conference on Human-Computer Interaction, 298–309.
   Google Scholar
• Herrmann, T., M. Ackerman, S. Goggins, C. Stary, and M. Prilla. 2017. Designing Health Care That Works—Socio-technical Conclusions. In Designing Healthcare That Works: A Socio-technical Approach (pp. 187–203). Academic Press (Elsevier). doi:10.1016/B978-0-12-812583-0.00011-0.
   Google Scholar
• Herrmann, T., I. Jahnke, and A. Nolte. 2016. “Evaluating Socio-Technical Systems with Heuristics – a Feasible Approach?.” Proceedings of STPIS16. http://ceur-ws.org/Vol-1604/Paper9.pdf
   Google Scholar
• Herrmann, T., G. Kunau, K. U. Loser, and N. Menold. 2004. “Socio-technical Walkthrough: Designing Technology Along Work Processes.” Proceedings of the Eighth Conference on Participatory Design: Artful Integration: Interweaving Media, Materials and Practices 1: 132–141.
   Google Scholar
• Herrmann, T., and J. Nierhoff. 2019. Heuristiken zur Evaluation digitalisierter Arbeit bei Industrie 4.0 und KI-basierten Systemen aus soziotechnischer Perspektive. http://www.fgw-nrw.de/studien/industrie16.html, FGW, Düsselsdorf.
   Google Scholar
• Herrmann, T., M. Prilla, and A. Nolte. 2016. “Socio-Technical Process Design—The Case of Coordinated Service Delivery for Elderly People.” In Blurring the Boundaries Through Digital Innovation, edited by F. D’Ascenzo, M. Magni, A. Lazazzara, and S. Za, Vol. 19, 217–229. Springer International Publishing. doi:10.1007/978-3-319-38974-5_17.
   Google Scholar
• Herrmann, T., V. Wulf, and A. Hartmann. 1996. “Requirements for the Human Centred Design of Groupware.” The Design of Computer Supported Cooperative Work and Groupware Systems 12: 77–100.
   Google Scholar
• Hertzum, M. 2010. “Images of Usability.” International Journal of Human-Computer Interaction 26 (6): 567–600. doi:10.1080/10447311003781300.
   Web of Science ®Google Scholar
• Imanghaliyeva, A. A., P. Thompson, P. Salmon, and N. A. Stanton. 2020. “A Synthesis of Sociotechnical Principles for System Design.” International Conference on Applied Human Factors and Ergonomics, 665–676.
   Google Scholar
• Interaction Design Foundation. 2021. Heuristics and Heuristic Evaluation. https://www.interaction-design.org/literature/book/the-glossary-of-human-computer-interaction/heuristics-and-heuristic-evaluation.
   Google Scholar
• Jahnke, I., M. Meinke-Kroll, M. Todd, and A. Nolte. 2020. “Exploring Artifact-Generated Learning with Digital Technologies: Advancing Active Learning with Co-design in Higher Education Across Disciplines.” Technology, Knowledge and Learning. doi:10.1007/s10758-020-09473-3.
   Web of Science ®Google Scholar
• Jahnke, I., C. Ritterskamp, and T. Herrmann. 2005. “Sociotechnical Roles for Sociotechnical Systems—A Perspective from Social and Computer Sciences.” AAAi Fall Symposium Proceedings, 68–75.
   Google Scholar
• Jahnke, I., C. Terkowsky, C. Pleul, and A. E. Tekkaya. 2010. Online Learning with Remote-Configured Experiments. Gesellschaft für Informatik e.V. http://dl.gi.de/handle/20.500.12116/4730.
   Google Scholar
• Kensing, F., J. Simonsen, and K. Bodker. 1998. “MUST: A Method for Participatory Design.” Human–Computer Interaction 13 (2): 167–198. doi:10.1207/s15327051hci1302_3.
   Web of Science ®Google Scholar
• Kirsch, C., P. Troxler, and E. Ulich. 1995. “Integration of People, Technology and Organization: The European Approach.” In Symbiosis of Human and Artifact, edited by Y. Anzai, K. Ogawa, and H. Mori, Vol. 20, 957–961. Elsevier. doi:10.1016/S0921-2647(06)80337-0.
   Google Scholar
• Kleiner, B. M. 2006. “Macroergonomics: Analysis and Design of Work Systems.” Applied Ergonomics 37 (1): 81–89. doi:10.1016/j.apergo.2005.07.006.
   PubMed Web of Science ®Google Scholar
• Kling, R. 2007. “What Is Social Informatics and Why Does It Matter?” The Information Society 23 (4): 205–220. doi:10.1080/01972240701441556.
   Web of Science ®Google Scholar
• Koo, T. K., and M. Y. Li. 2016. “A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research.” Journal of Chiropractic Medicine 15 (2): 155–163.
   PubMed Web of Science ®Google Scholar
• Krenn, B. 2015. “Robot: Multiuse Tool and Ethical Agent.” In A Construction Manual for Robots’ Ethical Systems: Requirements, Methods, Implementations, edited by R. Trappl, 11–29. Springer International Publishing. doi:10.1007/978-3-319-21548-8_2.
   Google Scholar
• Law, John. 2009. “Actor Network Theory and Material Semiotics.” Social Theory, 141–158.
   Google Scholar
• Leonardi, P. M. 2012. “Materiality, Sociomateriality, and Socio-Technical Systems: What do These Terms Mean? How are They Different? Do we Need Them.” Materiality and Organizing: Social Interaction in a Technological World, 25–48.
   Google Scholar
• Lowe, R., L. F. Chiu, and T. Oreszczyn. 2018. “Socio-technical Case Study Method in Building Performance Evaluation.” Building Research & Information 46 (5): 469–484. doi:10.1080/09613218.2017.1361275.
   Web of Science ®Google Scholar
• Luhmann, N. 1996. Social Systems (J. B. Jr & D. Baecker, Trans.; 1st Edition). Stanford CA: Stanford University Press.
   Google Scholar
• Mårell-Olsson, E., and I. Jahnke. 2019. “Wearable Technology in a Dentistry Study Program: Potential and Challenges of Smart Glasses for Learning at the Workplace.” In Perspectives on Wearable Enhanced Learning (WELL): Current Trends, Research, and Practice, edited by I. Buchem, R. Klamma, and F. Wild, 433–451. Springer International Publishing. doi:10.1007/978-3-319-64301-4_20.
   Google Scholar
• Merriam, S. B., and E. J. Tisdell. 2015. Qualitative Research: A Guide to Design and Implementation (4th ed.). John Wiley & Sons.
   Google Scholar
• Moallemi, E. A., L. Aye, F. J. de Haan, and J. M. Webb. 2017. “A Dual Narrative-Modelling Approach for Evaluating Socio-Technical Transitions in Electricity Sectors.” Journal of Cleaner Production 162: 1210–1224. doi:10.1016/j.jclepro.2017.06.118.
   Web of Science ®Google Scholar
• Muller, M. J., and A. McClard. 1995. “Validating an Extension to Participatory Heuristic Evaluation: Quality of Work and Quality of Work Life.” Conference Companion on Human Factors in Computing Systems, 115–116. doi:10.1145/223355.223457.
   Google Scholar
• Mumford, E. 1983. Designing Human Systems for New Technology: The ETHICS Method. Manchester: Manchester Business School.
   Google Scholar
• Mumford, E. 2000. “A Socio-Technical Approach to Systems Design.” Requirements Engineering 5 (2): 125–133. doi:10.1007/PL00010345.
   Google Scholar
• Nelles, J., S. Kohns, J. Spies, C. Bröhl, C. Brandl, A. Mertens, and C. Schlick. 2017. Best-Practice Approach for a Solution-Oriented Technology Assessment: Ethical, Legal, and Social Issues in the Context of Human-Robot Collaboration (pp. 1–14). Springer. doi:10.1007/978-3-662-53305-5_1.
   Google Scholar
• Nielsen, J. 1994a. “Heuristic Evaluation.” In Usability Inspection Methods, edited by J. Nielsen and R. L. Mack, 25–62. John Wiley & Sons, Inc.
   Google Scholar
• Nielsen, J. 1994b. “Enhancing the Explanatory Power of Usability Heuristics.” Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 152–158. https://doi.org/10.1145/191666.191729.
   Google Scholar
• Nielsen, J., and R. Molich. 1990. “Heuristic Evaluation of User Interfaces.” Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 249–256. doi:10.1145/97243.97281.
   Google Scholar
• Nierhoff, J., and T. Herrmann. 2017. “Data Elicitation for Continuous Awareness of Team Climate Characteristics to Improve Organizations’ Creativity.” Proceedings of the 50th Hawaii International Conference on System Sciences.
   Google Scholar
• Noehring, F., R. Woestmann, T. Wienzek, and J. Deuse. 2019. “Socio-Technical Capability Assessment to Support Implementation of Cyber-Physical Production Systems in Line with People and Organization.” In Advances in Human Factors and Systems Interaction, edited by I. L. Nunes, 299–311. Springer International Publishing. doi:10.1007/978-3-319-94334-3_30.
   Google Scholar
• Nolte, A., I. Jahnke, I. A. Chounta, and T. Herrmann. 2018. “Supporting Collaboration in Small Volunteer Groups with Socio-Technical Heuristics.” Proceedings of 16th European Conference on Computer-Supported Cooperative Work-Exploratory Papers. European Society for Socially Embedded Technologies (EUSSET).
   Google Scholar
• Orlikowski, W. J. 2007. “Sociomaterial Practices: Exploring Technology at Work.” Organization Studies 28 (9): 1435–1448.
   Web of Science ®Google Scholar
• Prilla, M., and T. Herrmann. 2017. “Challenges for Socio-Technical Design in Health Care: Lessons Learned From Designing Reflection Support.” In Designing Healthcare That Works. A Socio-Technical Approach, edited by Ackerman, M. S., S. P. Goggins, T. Herrmann, M. Prilla, and C. Stary, 149–166. London: Academic Press (Elsevier).
   Google Scholar
• Quiñones, D., C. Rusu, and V. Rusu. 2018. “A Methodology to Develop Usability/User Experience Heuristics.” Computer Standards & Interfaces 59: 109–129.
   Web of Science ®Google Scholar
• Reeves, T. C., and J. G. Hedberg. 2003. Interactive Learning Systems Evaluation. Englewood Cliffs, NJ: Educational Technology Publications.
   Google Scholar
• Reijers, H., and S. Limanmansar. 2005. “Best Practices in Business Process Redesign: An Overview and Qualitative Evaluation of Successful Redesign Heuristics.” Omega 33 (4): 283–306. doi:10.1016/j.omega.2004.04.012.
   Web of Science ®Google Scholar
• Rost, M., and K. Bock. 2011. “Privacy By Design und die Neuen Schutzziele.” Datenschutz und Datensicherheit - DuD 35 (1): 30–35. doi:10.1007/s11623-011-0009-y.
   Google Scholar
• Schmidt, M., Y. Earnshaw, A. A. Tawfik, and I. Jahnke. 2020. “Methods of User Centered Design and Evaluation for Learning Designers.” In Learner and User Experience Research. EdTech Books. https://edtechbooks.org/ux/ucd_methods_for_lx.
   Google Scholar
• Schnuelle, C., J. Thoeming, T. Wassermann, P. Thier, A. von Gleich, and S. Goessling-Reisemann. 2019. “Socio-technical-economic Assessment of Power-to-X: Potentials and Limitations for an Integration Into the German Energy System.” Energy Research & Social Science 51: 187–197. doi:10.1016/j.erss.2019.01.017.
   Web of Science ®Google Scholar
• Shin, D. 2014. “A Socio-Technical Framework for Internet-of-Things Design: A Human-Centered Design for the Internet of Things.” Telematics and Informatics 31 (4): 519–531. doi:10.1016/j.tele.2014.02.003.
   Web of Science ®Google Scholar
• Shin, D. 2019. “A Living lab as Socio-Technical Ecosystem: Evaluating the Korean Living lab of Internet of Things.” Government Information Quarterly 36 (2): 264–275. doi:10.1016/j.giq.2018.08.001.
   Web of Science ®Google Scholar
• Shin, D. 2020. “Expanding the Role of Trust in the Experience of Algorithmic Journalism: User Sensemaking of Algorithmic Heuristics in Korean Users.” Journalism Practice, 1–24. doi:10.1080/17512786.2020.1841018.
   Web of Science ®Google Scholar
• Shin, D.-H., H. Choo, and K. Beom. 2011. “Socio-Technical Dynamics in the Development of Next Generation Mobile Network: Translation Beyond 3G.” Technological Forecasting and Social Change 78 (3): 514–525. doi:10.1016/j.techfore.2010.04.017.
   Web of Science ®Google Scholar
• Shin, D., and M. Ibahrine. 2020. “The Socio-Technical Assemblages of Blockchain System: How Blockchains are Framed and how the Framing Reflects Societal Contexts.” Digital Policy, Regulation and Governance 22 (3): 245–263. doi:10.1108/DPRG-11-2019-0095.
   Web of Science ®Google Scholar
• Shin, D.-H., and J. Jung. 2012. “Socio-Technical Analysis of Korea’s Broadband Convergence Network: Big Plans, Big Projects, Big Prospects?” Telecommunications Policy 36 (7): 579–593. doi:10.1016/j.telpol.2012.03.003.
   Web of Science ®Google Scholar
• Shneiderman, B., and C. Plaisant. 2005. Designing the User Interface: Strategies for Effective Human-Computer Interaction. 4th ed. Boston: Pearson.
   Google Scholar
• Tognazzini, B. 2014, March 6. First Principles of Interaction Design (Revised & Expanded). AskTog. https://asktog.com/atc/principles-of-interaction-design/.
   Google Scholar
• Trist, Eric. 1950. “The Relations of Social and Technical Systems in Coal-Mining.” Paper presented to the British Psychological Society, Industrial Section.
   Google Scholar
• Trist, E., and H. Murray. 1993. “Introduction to Vol. II.” In The Social Engagement of Social Science. A Tavistock Anthology, 36–60. University of Pensylvania Press. https://www.upenn.edu/pennpress/book/1928.html.
   Google Scholar
• Vinck, D. 2019. “Maintenance and Repair Work.” Engineering Studies 11 (2): 153–167. doi:10.1080/19378629.2019.1655566.
   Web of Science ®Google Scholar
• Wilson, C. 2013. User Interface Inspection Methods: A User-Centered Design Method. Waltham, MA: Morgan Kaufmann.
   Google Scholar
• Wutzkowsky, J., and B. Böckmann. 2018. “Using MEESTAR to Identify Ethical and Social Issues Implementing a Digital Patient-Centered Care Platform.” EHealth, 278–285. doi:10.3233/978-1-61499-858-7-278.
   Google Scholar