{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"Published on May 12, 2023 by Marília Prata, mpwolke","metadata":{}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n\n#Ignore warnings\nimport warnings\nwarnings.filterwarnings('ignore')\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-05-12T15:48:35.519823Z","iopub.execute_input":"2023-05-12T15:48:35.520361Z","iopub.status.idle":"2023-05-12T15:48:36.149032Z","shell.execute_reply.started":"2023-05-12T15:48:35.520319Z","shell.execute_reply":"2023-05-12T15:48:36.147755Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#AI Ethics on Kaggle  - I knew that Philosophy was behind of all of that Ethic Behavior in AI.\n\nToward an Ethics of AI Belief\n\nAuthors: Winnie Ma, Vincent Valton - \nhttps://doi.org/10.48550/arXiv.2304.14577\n\n\n\"Philosophical research in AI has hitherto largely focused on the ethics of AI. In this paper the authors, an ethicist of belief and a machine learning scientist, suggest that we need to pursue a novel area of philosophical research in AI - the epistemology of AI, and in particular an ethics of belief for AI, i.e., an ethics of AI belief.\"\n\n\"There they took the ethics of belief, a field that has been defined in various ways, to refer to a sub-field within epistemology. This subfield is concerned with the study of possible moral, practical, and other non-alethic dimensions of belief. And in that paper, the authors will primarily be concerned with the normative question within the ethics of belief of what agents - both human and artificial - ought to believe, rather than with descriptive questions concerning whether certain beliefs meet various evaluative standards such as being true, being justified or warranted, constituting knowledge, and so on. We suggest four topics in extant work in the ethics of (human) belief that can be applied to an ethics of AI belief: doxastic wronging by AI; morally owed beliefs; pragmatic and moral encroachment on AI beliefs; and moral responsibility for AI beliefs.\"\n\n\"The authors also indicated an important nascent area of philosophical research in epistemic injustice and AI that has not yet been recognized as research in the ethics of AI belief, but which is so in virtue of concerning moral and practical dimensions of belief.\"\n\nhttps://arxiv.org/abs/2304.14577","metadata":{}},{"cell_type":"markdown","source":"![](https://leonfurze.com/wp-content/uploads/2023/01/bannerheading.png)Leon Furze","metadata":{}},{"cell_type":"markdown","source":"#Competition Citation\n\n@misc{2023-kaggle-ai-report,\n\n    author = {Bojan Tunguz, Dieter, Heads or Tails, Karnika Kapoor, Paul Mooney, Phil Culliton, Rob Mulla, Sanyam Bhutani, Will Cukierski},\n    \n    title = {2023 Kaggle AI Report},\n    \n    publisher = {Kaggle},\n    \n    year = {2023},\n    \n    url = {https://kaggle.com/competitions/2023-kaggle-ai-report}\n}","metadata":{}},{"cell_type":"markdown","source":"#The beginning of Kaggle Competitions:  Chess ratings - Elo versus the Rest of the World: 2010\n\n Chess ratings - Elo versus the Rest of the World 08/03/2010","metadata":{}},{"cell_type":"code","source":"df = pd.read_csv('/kaggle/input/2023-kaggle-ai-report/kaggle_writeups_20230510.csv')\ndf.head(2)","metadata":{"execution":{"iopub.status.busy":"2023-05-12T16:23:28.537335Z","iopub.execute_input":"2023-05-12T16:23:28.539467Z","iopub.status.idle":"2023-05-12T16:23:28.956360Z","shell.execute_reply.started":"2023-05-12T16:23:28.539424Z","shell.execute_reply":"2023-05-12T16:23:28.955150Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Flatenning Json took some time","metadata":{}},{"cell_type":"code","source":"#https://stackoverflow.com/questions/12451431/loading-and-parsing-a-json-file-with-multiple-json-objects\n\nimport pandas as pd\nmeta = pd.read_json('../input/2023-kaggle-ai-report/arxiv_metadata_20230510.json', lines=True)","metadata":{"execution":{"iopub.status.busy":"2023-05-12T16:03:29.688630Z","iopub.execute_input":"2023-05-12T16:03:29.689007Z","iopub.status.idle":"2023-05-12T16:05:53.606162Z","shell.execute_reply.started":"2023-05-12T16:03:29.688979Z","shell.execute_reply":"2023-05-12T16:05:53.604779Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#https://stackoverflow.com/questions/12451431/loading-and-parsing-a-json-file-with-multiple-json-objects\n\nmeta.to_json('new_file.json')","metadata":{"execution":{"iopub.status.busy":"2023-05-12T16:06:40.755106Z","iopub.execute_input":"2023-05-12T16:06:40.756039Z","iopub.status.idle":"2023-05-12T16:07:28.769324Z","shell.execute_reply.started":"2023-05-12T16:06:40.755980Z","shell.execute_reply":"2023-05-12T16:07:28.768342Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#The Meta: arxiv_metadata","metadata":{}},{"cell_type":"code","source":"meta.head()","metadata":{"execution":{"iopub.status.busy":"2023-05-12T16:07:35.396775Z","iopub.execute_input":"2023-05-12T16:07:35.397420Z","iopub.status.idle":"2023-05-12T16:07:35.432228Z","shell.execute_reply.started":"2023-05-12T16:07:35.397383Z","shell.execute_reply":"2023-05-12T16:07:35.431364Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"title = meta.copy()","metadata":{"execution":{"iopub.status.busy":"2023-05-12T16:10:42.789553Z","iopub.execute_input":"2023-05-12T16:10:42.790227Z","iopub.status.idle":"2023-05-12T16:10:43.490460Z","shell.execute_reply.started":"2023-05-12T16:10:42.790171Z","shell.execute_reply":"2023-05-12T16:10:43.489388Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"title = title.dropna(subset=['title'])","metadata":{"execution":{"iopub.status.busy":"2023-05-12T16:11:16.712651Z","iopub.execute_input":"2023-05-12T16:11:16.713053Z","iopub.status.idle":"2023-05-12T16:11:18.691083Z","shell.execute_reply.started":"2023-05-12T16:11:16.713023Z","shell.execute_reply":"2023-05-12T16:11:18.689987Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"title.head()","metadata":{"execution":{"iopub.status.busy":"2023-05-12T16:11:31.803006Z","iopub.execute_input":"2023-05-12T16:11:31.803414Z","iopub.status.idle":"2023-05-12T16:11:31.837088Z","shell.execute_reply.started":"2023-05-12T16:11:31.803379Z","shell.execute_reply":"2023-05-12T16:11:31.835805Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"title['title'] = title['title'].str.replace('[^a-zA-Z]', ' ', regex=True)\ntitle['title'] = title['title'].str.lower()","metadata":{"execution":{"iopub.status.busy":"2023-05-12T16:11:56.726261Z","iopub.execute_input":"2023-05-12T16:11:56.726975Z","iopub.status.idle":"2023-05-12T16:12:19.036361Z","shell.execute_reply.started":"2023-05-12T16:11:56.726913Z","shell.execute_reply":"2023-05-12T16:12:19.035164Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"title.tail()","metadata":{"execution":{"iopub.status.busy":"2023-05-12T16:12:34.842216Z","iopub.execute_input":"2023-05-12T16:12:34.842615Z","iopub.status.idle":"2023-05-12T16:12:34.877887Z","shell.execute_reply.started":"2023-05-12T16:12:34.842562Z","shell.execute_reply":"2023-05-12T16:12:34.876831Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Let's search Ethics on Json Files. Since I couldn't find that word on Competitions Write-up Titles.","metadata":{}},{"cell_type":"code","source":"title['keyword_ethics'] = title['title'].str.find('ethics')","metadata":{"execution":{"iopub.status.busy":"2023-05-12T16:13:07.295233Z","iopub.execute_input":"2023-05-12T16:13:07.295629Z","iopub.status.idle":"2023-05-12T16:13:10.007714Z","shell.execute_reply.started":"2023-05-12T16:13:07.295589Z","shell.execute_reply":"2023-05-12T16:13:10.006760Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#If the result prompt -1, then the title doesn't contained the keyword.","metadata":{}},{"cell_type":"code","source":"title.head(2)","metadata":{"execution":{"iopub.status.busy":"2023-05-12T16:13:52.145853Z","iopub.execute_input":"2023-05-12T16:13:52.146234Z","iopub.status.idle":"2023-05-12T16:13:52.171274Z","shell.execute_reply.started":"2023-05-12T16:13:52.146206Z","shell.execute_reply":"2023-05-12T16:13:52.169986Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#Finally Ethics! On Meta Json (arxiv_metadata)","metadata":{}},{"cell_type":"code","source":"included_ethics = title.loc[title['keyword_ethics'] != -1]\nincluded_ethics","metadata":{"execution":{"iopub.status.busy":"2023-05-12T16:14:39.487985Z","iopub.execute_input":"2023-05-12T16:14:39.488396Z","iopub.status.idle":"2023-05-12T16:14:41.187375Z","shell.execute_reply.started":"2023-05-12T16:14:39.488358Z","shell.execute_reply":"2023-05-12T16:14:41.186586Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"included_ethics[\"journal-ref\"].value_counts()","metadata":{"execution":{"iopub.status.busy":"2023-05-12T16:41:49.977629Z","iopub.execute_input":"2023-05-12T16:41:49.978070Z","iopub.status.idle":"2023-05-12T16:41:49.988397Z","shell.execute_reply.started":"2023-05-12T16:41:49.978036Z","shell.execute_reply":"2023-05-12T16:41:49.987653Z"},"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#By Rob Mulla https://www.kaggle.com/code/robikscube/sign-language-recognition-eda-twitch-stream\n\nfig, ax = plt.subplots(figsize=(4, 4))\nincluded_ethics[\"authors\"].value_counts().head(10).sort_values(ascending=True).plot(\n    kind=\"barh\",color= 'r', ax=ax, title=\"Data Science and Ethics Authors\"\n)\nax.set_xlabel(\"Number of Training Examples\")\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-12T17:04:05.628424Z","iopub.execute_input":"2023-05-12T17:04:05.628950Z","iopub.status.idle":"2023-05-12T17:04:05.890422Z","shell.execute_reply.started":"2023-05-12T17:04:05.628909Z","shell.execute_reply":"2023-05-12T17:04:05.889208Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#By Rob Mulla https://www.kaggle.com/code/robikscube/sign-language-recognition-eda-twitch-stream\n\nfig, ax = plt.subplots(figsize=(4, 4))\nincluded_ethics[\"journal-ref\"].value_counts().head(10).sort_values(ascending=True).plot(\n    kind=\"barh\",color= 'g', ax=ax, title=\"Data Science and Ethics on Journals\"\n)\nax.set_xlabel(\"Number of Training Examples\")\nplt.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-05-12T17:04:14.030867Z","iopub.execute_input":"2023-05-12T17:04:14.032030Z","iopub.status.idle":"2023-05-12T17:04:14.344325Z","shell.execute_reply.started":"2023-05-12T17:04:14.031992Z","shell.execute_reply":"2023-05-12T17:04:14.343130Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](https://pbs.twimg.com/media/FO3YMzDaAAI8aL4.jpg)twitter.com","metadata":{}},{"cell_type":"markdown","source":"#AI Principles, Ethics, and Key Requirements to Responsible AI Systems and Regulation\n\nConnecting the Dots in Trustworthy Artificial Intelligence: From AI Principles, Ethics, and Key Requirements to Responsible AI Systems and Regulation\n\nAuthors: Natalia Díaz-Rodríguez, Javier Del Ser, Mark Coeckelbergh, Marcos López de Prado, Enrique Herrera-Viedma, Francisco Herrera  \nhttps://doi.org/10.48550/arXiv.2305.02231\n\n\"Trustworthy Artificial Intelligence (AI) is based on seven technical requirements sustained over three main pillars that should be met throughout the system's entire life cycle: it should be (1) lawful, (2) ethical, and (3) robust, both from a technical and a social perspective.\"\n\n\"However, attaining truly trustworthy AI concerns a wider vision that comprises the trustworthiness of all processes and actors that are part of the system's life cycle, and considers previous aspects from different lenses. A more holistic vision contemplates four essential axes: the global principles for ethical use and development of AI-based systems, a philosophical take on AI ethics, a risk-based approach to AI regulation, and the mentioned pillars and requirements.\"\n\n\"The seven requirements (human agency and oversight; robustness and safety; privacy and data governance; transparency; diversity, non-discrimination and fairness; societal and environmental wellbeing; and accountability) are analyzed from a triple perspective: What each requirement for trustworthy AI is, Why it is needed, and How each requirement can be implemented in practice. On the other hand, a practical approach to implement trustworthy AI systems allows defining the concept of responsibility of AI-based systems facing the law, through a given auditing process. Therefore, a responsible AI system is the resulting notion we introduce in this work, and a concept of utmost necessity that can be realized through auditing processes, subject to the challenges posed by the use of regulatory sandboxes. Our multidisciplinary vision of trustworthy AI also includes a regulation debate, with the purpose of serving as an entry point to this crucial field in the present and future progress of our society.\"\n\nhttps://arxiv.org/abs/2305.02231","metadata":{}},{"cell_type":"markdown","source":"#Acknowledgements:\n\nMpwolke/StackOverflow https://www.kaggle.com/code/mpwolke/i-feel-love-parsing-json\n\nSalman Chen https://www.kaggle.com/salmanhiro/covids-incubation-transmission-related-articles/comments\n\nRob Mulla https://www.kaggle.com/code/robikscube/sign-language-recognition-eda-twitch-stream","metadata":{}}]}