Trustworthy Artificial Intelligence: Design of AI Governance Framework

Abstract

This article presents the various challenges in the current system of AI governance and the correlation between data, algorithm, technology, governance, and geopolitics surrounding its successful implementation. The focal point of the article is the Adaptive-Hybrid AI Governance framework based on technical, ethical, and societal regulatory mechanisms that models trustworthy AI and the risks associated with it. The article highlights the need for trustworthy AI and how major countries are shaping their AI regulatory mechanisms. It presents a case study on AI governance in defence that elucidates ethical AI governance through various use cases.

Keywords:

• Trustworthy AI
• AI governance
• ethical AI
• automation bias
• societal
• regulatory framework

Disclosure statement

No potential conflict of interest was reported by the author(s).

Notes

1. ‘Data Protection and Privacy Legislation Worldwide’, at https://unctad.org/page/data-protection-and-privacy-legislation-worldwide (Accessed 4 January 2023).

2. ‘National AI policies & strategies’, OECD.AI OECD.AI (2021), powered by EC/OECD (2021), database of national AI policies at https://oecd.ai/en/dashboards/overview (Accessed 15 September 2023).

3. ‘Data Protection and Privacy Legislation Worldwide’, no. 1.

4. Matt Sheehan, ‘What the U.S. Can Learn From China About Regulating AI’, Foreign Policy, 12 September 2023 at https://foreignpolicy.com/2023/09/12/ai-artificial-intelligence-regulation-law-china-us-schumer-congress/?tpcc=recirc_latest062921 (Accessed 15 September 2023).

5. “Global Tech Expert Matt Shaheen: ‘China’s Emerging Ai Governance Framework Will Reshape How The Technology Is Built And Deployed”., 13 July 2023 at https://thefederalnewswire.com/stories/645740552-global-tech-expert-matt-shaheen-china-s-emerging-ai-governance-framework-will-reshape-how-the-technology-is-built-and-deployed (Accessed 16 September 2023).

6. Paramita Ghosh, ‘Challenges for Data Governance and Data Quality in a Machine Learning Ecosystem’ at https://www.dataversity.net/challenges-for-data-governance-and-data-quality-in-a-machine-learning-ecosystem/ (Accessed 2 January 2023).

7. National Conference of State Legislatures. ‘State Laws Related to Digital Privacy’, 6 July 2022 at https://www.ncsl.org/research/telecommunications-and-information-technology/state-laws-related-to-internet-privacy.aspx (Accessed 5 January 2023)

8. ‘The Digital Personal Data Protection Bill, 2023’, Ministry of Electronics and Information Technology, 9 August2023 at https://prsindia.org/billtrack/digital-personal-data-protection-bill-2023 (Accessed 12 September 2023).

‘THE DIGITAL PERSONAL DATA PROTECTION BILL, 2022’, Ministry of Electronics and IT at https://www.meity.gov.in/writereaddata/files/The%20Digital%20Personal%20Data%20Potection%20Bill%2C%202022_0.pdf (Accessed September 12, 2023).

9. Paramita Ghosh, ‘Challenges for Data Governance and Data Quality in a Machine Learning Ecosystem’, no. 6.

10. ‘Statement on AI Risk’, Centre for AI Safety, at https://www.safe.ai/statement-on-ai-risk (Accessed 13 September 2023).

11. Cecilia Kang, ‘OpenAI’s Sam Altman Urges AI Regulation in Senate Hearing’, 16 May 2023 at https://www.nytimes.com/2023/05/16/technology/openai-altman-artificial-intelligence-regulation.html (Accessed 20 September 2023)

12. Kevin Roose, How Do the White House’s A.I. Commitments Stack Up? 22 July 2023 at https://www.nytimes.com/2023/07/22/technology/ai-regulation-white-house.html (Accessed 15 September 2023)

13. Jan-Aike Bolte, Andreas Bar, Daniel Lipinski, and Tim Fingscheidt. ‘Towards corner case detection for autonomous driving’. In 2019 IEEE Intelligent vehicles symposium (IV), pp. 438–445. IEEE, 2019.

14. Trevor Mogg, ‘Stickers on street signs can confuse self-driving cars, researchers show’, 6 August 2017 at https://www.digitaltrends.com/cars/self-driving-cars-confuse-stickers-signs/ (Accessed 12 December 2022).

15. ‘Split-Second “Phantom” Images Can Fool Tesla’s Autopilot’at https://www.wired.com/story/tesla-model-x-autopilot-phantom-images/ (Accessed 4 January 2023).

16. ‘Global AI Summit’ at https://globalaisummit.org/en/default.aspx (Accessed 13 September 2022).

17. Ben Lancaster, ‘Cyber Security in your Supply Chain’, APMG International, 2018 at https://apmg-international.com/article/cyber-security-your-supply-chain (Accessed 14 October 2022).

18. J. Fjeld, N. Achten, H. Hilligoss, A. C. Nagy, M. Srikumar, “Principled Artificial Intelligence: Mapping Consensus in Ethical and Rights-based Approaches to Principles for AI No. 2020–1, The Berkman Klein Center for Internet and Society Research Publication series, 2020 at https://ssrn.com/abstract=3518482 ; (Accessed 29 October 2022).

19. J. Morley, L. Floridi, L. Kinsey, A. Elhalal, ‘From What to How: An Initial Review of Publicly Available AI Ethics Tools, Methods and Research to Translate Principles into Practices’, Science and Engineering Ethics, 26(4), 2020, pp. 2141–2168 at https://doi.org/10.1007/s11948-019-00165-5 (Accessed 16 January 2023)

20. Ilina Georgieva, Claudio Lazo, Tjerk Timan and Anne Fleur van Veenstra, ‘From AI Ethics Principles to Data Science Practice: A Reflection and a Gap Analysis Based on Recent Frameworks and Practical Experience’. AI and Ethics, 2(4), 2022, pp. 697–711 at

https://doi.org/10.1007/s43681-021-00127-3 (Accessed 10 February 2023)

21. Matti Mäntymäki, Matti Minkkinen, Teemu Birkstedt, and Mika Viljanen, ‘Defining organizational AI governance’, AI and Ethics, 2, 2022, pp. 1–7 at 10.1007/s43681-022 00,143-x. Accessed 15 February 2023

22. T. A. Hemphill, ‘Regulating nanomaterials: A case for hybrid governance’, Bulletin of Science, Technology & Society, 36(4), 2016, pp. 219–228.

23. I. Linkov, B. Trump, K. Poinsatte-Jones,, and M.V. Florin, ‘Governance Strategies for a Sustainable Digital World’, Sustainability, 10 (2), 2018, pp. 440; Carl Gahnberg, ‘What Rules? Framing the Governance of Artificial Agency’. Policy and Society 40(2), 2021, pp. 194–210.

24. Urs Gasser and A. F. Almeida Virgilio, ‘A Layered Model for AI Governance’, in IEEE Internet Computing, 21(6), November/December 2017, pp. 58-62DOI: 10.1109/MIC.2017.4180835.

25. Kate Devitt, Michael Gan, Jason Scholz, and Robert Bolia, ‘A Method for Ethical AI in Defence’, at https://apo.org.au/sites/default/files/resource-files/2021–02/apo-nid311150.pdf (Accessed 18 November 2022); Simon Burton, Ibrahim Habli, Tom Lawton, John McDermid, Phillip Morgan, and Zoe Porter, ‘Mind the Gaps: Assuring the Safety of Autonomous Systems from an Engineering, Ethical, and Legal Perspective’. Artificial Intelligence 279 (1), 2020, 103201, pp.1–16. DOI https://doi.org/10.1016/j.artint.2019.103201

26. Matt Sheehan and Sharon Du. ‘What China’s Algorithm Registry Reveals about AI Governance’, Carnegie Endowment for International Peace, 2022, at https://carnegieendowment.org/2022/12/09/what-china-s-algorithm-registry-reveals-about-ai-governance-pub-88606 (Accessed 23 December 2022).

27. Brian T. Molloy, ‘Project Governance for Defense Applications of Artificial Intelligence’. Prism, 9(3), 2021, pp. 106–121.

28. Carlo Casalone, Luciano Floridi, Laura Palazzani, Renzo Pegoraro, Francesca Rossi and Roberto Villa, ‘Human-centric AI: From Principles to Actionable and Shared Policies’, T20 Policy Brief, September 2021 at https://www.t20italy.org/wp-content/uploads/2021/09/TF4-pb12–1.pdf (Accessed 20 January 2023)

29. Brian T. Molloy, no. 27.

30. ‘National AI policies and strategies’, OECD, AI Policy Observatory, at https://oecd.ai/en/dashboards/overview (Accessed 18 November 2022).

31. ‘OECD AI Principles Overview’, OECD.AI, 2019, at https://oecd.ai/en/ai-principles

(Accessed 19 November 2022).

32. ‘IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems’, Ethically Aligned Design: A Vision for Prioritizing Human Well-being with Autonomous and Intelligent Systems (EADe1), 2019 at https://standards.ieee.org/content/ieee-standards/en/industry-connections/ec/autonomous-systems.html

(Accessed 26 November 2022).

33. ‘US-EU Trade and Technology Council Inaugural Joint Statement’. U.S.-E.U. Trade and Technology Council, White House Briefing Room Statements and Releases, 29 September 2021 at https://www.whitehouse.gov/briefing-room/statements-releases/2021/09/29/u-s-eu-trade-and-technology-council-inaugural-joint-statement/(Accessed 24 January 2023)

34. C. Cath S. Wachter B. Mittelstadt M. Taddeo L. Floridi ‘Artificial Intelligence and the “Good Society”: the US, EU, and UK approach’, Science and Engineering Ethics, 24(2), April 2018, pp. 505–528, DOI: 10.1007/s11948-017-9901-7. Epub 28 March 2017, PMID: 28353045.

35. ‘US Defense Ethical AI Principles’, Defense Innovation Board, 2019, at https://innovation.defense.gov/ai/

(Accessed 20 November 2022).

36. Zev Winkelman, Maya Buenaventura, James M. Anderson, Nahom M. Beyene, Pavan Katkar, and Gregory Cyril Baumann, ‘When Autonomous Vehicles are Hacked, Who is Liable?’. No. RR-2654-RC, RAND Corporation, Santa Monica, 2019 at https://www.rand.org/content/dam/rand/pubs/research_reports/RR2600/RR2654/RAND_RR2654.pdf

(Accessed 12 January 2023).

37. ‘One Hundred Seventeenth Congress of the United States of America’ www.govinfo.gov at https://www.govinfo.gov/content/pkg/BILLS-117hr7776enr/pdf/BILLS-117hr7776enr.pdf

(Accessed 19 October 2022).

38. ‘Summary of AI Provisions from the National Defense Authorization Act 2023’, HAI Stanford University at https://hai.stanford.edu/summary-ai-provisions-national-defense-authorization-act-2023

(Accessed 23 March 2023).

39. ‘Artificial Intelligence Risk Management Framework (AI RMF 1.0)’, National Institute of Standards and Technology (NIST), U.S. Department of Commerce, January 2023 at.

https://www.nist.gov/news-events/news/2023/01/nist-risk-management-framework-aims-improve-trustworthiness-artificial?utm_source=Center+for+Security+and+Emerging+Technology&utm_campaign=177cedb469-Newsletter_2023_02_09_02_10&utm_medium=email&utm_term=0_-177cedb469-%5BLIST_EMAIL_ID%5D

(Accessed 25 March 2023).

40. Benjamin Cedric Larsen, ‘The geopolitics of AI and the rise of digital sovereignty’, Brookings, 8 December 2022 at https://www.brookings.edu/articles/the-geopolitics-of-ai-and-the-rise-of-digital-sovereignty/ (Accessed 23 February 2023)

41. American Civil Liberties Union. ‘The Fight to Stop Face Recognition Technology’, https://www.aclu.org/news/topic/stopping-face-recognition-surveillance. (Accessed 14 November 2022).

42. ‘What is GDPR, the EU’s New Data Protection Law?’ at https://gdpr.eu/what-is-gdpr (Accessed 28 December 2022).

43. ‘The Artificial Intelligence Act’, European Union at https://artificialintelligenceact.eu/

(Accessed 10 March 2023).

44. ‘EU AI Act: First Regulation on Artificial Intelligence’, European Parliament, 6 June 2023 at https://www.europarl.europa.eu/news/en/headlines/society/20230601STO93804/eu-ai-act-first-regulation-on-artificial-intelligence (Accessed 10 September 2023)

45. ‘Position Paper of the People’s Republic of China on Strengthening Ethical Governance of Artificial Intelligence (AI)’, November 2022 at.

https://www.fmprc.gov.cn/mfa_eng/wjdt_665385/wjzcs/202211/t20221117_10976730.html

(Accessed 13 March 2023).

46. ‘Task Force for Implementation of AI’, Press Information Bureau, Ministry of Defence, Government of India, 28 March 2022 at https://pib.gov.in/PressReleaseIframePage.aspx?PRID=1810442

(Accessed 4 September 2023)

47. Sanur Sharma ‘AI and National Security: Major Power Perspectives and Challenges’, Issue Brief, MP-IDSA, 12 September 2022 at https://idsa.in/issuebrief/ai-and-national-security-ssharma-120922#footnote19_93jpx61 (Accessed 13 September 2023).

48. ‘India takes over as Council Chair of Global Partnership on AI (GPAI)’, Press Information Bureau, Ministry of Electronics & IT, November 21, 2022 at https://pib.gov.in/PressReleasePage.aspx?PRID=1877739

49. ‘Salient Features of the Digital Personal Data Protection Bill, 2023’ Press Information Bureau, Ministry of Electronics & IT, Aug 9, 2023 at https://pib.gov.in/PressReleaseIframePage.aspx?PRID=1947264

(Accessed 15 September 2023)

50. ‘The Digital Personal Data Protection Bill, 2022’, no. 8.

51. Greg Allen and Taniel Chan, ‘Artificial Intelligence and National Security’, Belfer Center for Science and International Affairs, Harvard Kennedy School, 2017 at https://www.belfercenter.org/sites/default/files/files/publication/AI%20NatSec%20-%20final.pdf (Accessed 10 September 2022).

52. ‘Comptroller Michael J. McCord and Vice Adm. Ron Boxall Hold a News Briefing on President Biden’s Fiscal 2023 Defense Budget’, U.S. Department of Defense, 28 March 2022 at https://www.defense.gov/News/Transcripts/Transcript/Article/2980711/comptroller-michael-j-mccord-and-vice-adm-ron-boxall-hold-a-news-briefing-on-pr/

(Accessed 15 March 2023).

53. Forrest E. Morgan, Benjamin Boudreaux, Andrew J. Lohn, Mark Ashby, Christian Curriden, Kelly Klima, and Derek Grossman, ‘Military Applications of Artificial Intelligence: Ethical Concerns in an Uncertain World’, RAND Corporation, Santa Monica, CA 2020 at https://www.rand.org/pubs/research_reports/RR3139–1.html. (Accessed 12 January 2023) Also available in print form.

54. ‘Google Will Not Renew Pentagon Project that Upsets Employees’, The New York Times, 1 June 2018 at https://www.nytimes.com/2018/06/01/technology/google-pentagon-project-maven.html (Accessed 11 March 2023).

55. ‘DOD Adopts Ethical Principles for Artificial Intelligence’, US Department of Defense, at https://www.defense.gov/News/Releases/Release/Article/2091996/dod-adopts-ethical-principles-for-artificial-intelligence/

(Accessed 11 December 2022).

56. Mariarosaria Taddeo, David McNeish, Alexander Blanchard and Elizabeth Edgar, ‘Ethical Principles for Artificial Intelligence in National Defence’. Philosophy & Technology, 34 (1), 2021, pp.1707–1729.

57. Department of Defence, ‘Australia’s System of Control and Applications for Autonomous Weapon Systems’, Group of Governmental Experts on Emerging Technologies in the Area of Lethal Autonomous Weapons Systems, 25–29 March 2019 and 20–21 August 2019, Geneva at https://www.unog.ch/80256EDD006B8954/(httpAssets)/16C9F75124654510C12583C9003A4EBF/$file/CCWGGE.12019WP.2Rev.1.pdf

(Accessed 11 March 2023).

58. ‘The Protocol Additional to the Geneva Conventions of 12 August 1949, and Relating to the Protection of Victims of International Armed Conflicts (Protocol I)’, 8 June 1977 at https://ihl-databases.icrc.org/applic/ihl/ihl.nsf/INTRO/470 (Accessed 17 October 2023).

Additional information

Notes on contributors

Sanur Sharma

Dr. Sanur Sharma is a researcher in the field of New and Emerging Technologies and its geopolitical implications. Her research interests include Artificial Intelligence Governance, Generative AI, Machine Learning, Social Media Analytics and Cyber Security. She has worked as Associate Fellow in the Strategic Technologies Centre, at the Manohar Parrikar Institute for Defence Studies and Analyses (MP-IDSA), New Delhi where her research project was Exploiting AI & Data Analytics in Enhancing Battle Effectiveness. She has also worked as Assistant Professor at IILM University, Gurugram and Guru Gobind Singh Indraprastha University (GGSIPU), Delhi. She obtained her PhD in Computer Science from GGSIPU and was the recipient of a prestigious fellowship (Visvesvaraya PhD Scheme) by MeitY, Government of India.