{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## **Embracing Actionable Principles of AI Implementation: Pathways, Properties, and Human-Centric Approaches**","metadata":{"papermill":{"duration":0.006511,"end_time":"2023-06-29T13:14:57.147881","exception":false,"start_time":"2023-06-29T13:14:57.141370","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"### **Introduction**","metadata":{"papermill":{"duration":0.005478,"end_time":"2023-06-29T13:14:57.159323","exception":false,"start_time":"2023-06-29T13:14:57.153845","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"Artificial Intelligence (AI) has developed at an estounding pace particularly during the past six months, unlocking potential for societal transformations in multiple industries. To keep pace with this change, ethical considerations can no longer afford to be in the ideation stage, as we are at a point in time where ethical goals in AI implementation must translate into tangible actions with plans for implementation alongside the development of AI models themselves.\n\nIn this essay, we will explore three papers published within the past two years:\n\n- Actionable Principles for Articifial Intelligence Policy: Three Pathways by Charlotte Stix from the Department of Industrial Engineering and Innovation Sciences, Eindhoven University of Technology, Eindhoven, The Netherlands, published on February 19, 2021 in Science and Engineering Ethics\n- Meaningful Human Control: Actionable Properties for AI System Development by Luciano Cavalcante Siebert, et. al from the Faculty of Electrical Engineering, Mathematics and Computer Science, Delft University of Technology, Delft, The Netherlands, published on May 18, 2022 in AI and Ethics\n- Human-Centered Artificial Intelligence in Education: Seeing the Invisible through the Visible by Stephen J.H. Yang, et. al from the Department of Computer Science and Informational Engineering, National Central University, Taiwan, publihed in 2021 in Computers and Education: Artificial Intelligence\n\nIn the paper, Actionable Principles for Articifial Intelligence Policy: Three Pathways, three approaches were suggested to create the \"actionable principles:\n\n* Preliminary Landscape Assessment\n* Multi-Stakeholder Participation and Cross-Sectoral Feedback\n* Implementation and Operationalizability Support\n\nIn the paper, Meaningful Human Control: Actionable Properties for AI System Development, two necessary conditions for meaningful human control were brough up: tracking and tracing. The paper has also raised four properties of human-AI systems under meaningful human control:\n\n1. The human-AI system has an explicit moral operational design domain (moral ODD) and the AI agent adheres to the boundaries of this domain\n\n2. Human and AI agents have appropriate and mutually compatible representations of the human-AI system and its contaxt\n\n3. The relevatn agents have ability and authority to control the system so that humans can act upon their responsibility\n\n4. Actions of the AI agents are explicitly linked to actions of humans who are aware of their moral responsibility\n\nAnd in the paper, Human-Centered Artificial Intelligence in Education: Seeing the Invisible through the Visible, the idea of making the \"invisible visible\" through AI is suggested to not only pertain to learning processes but also extend to track and improve ethical compliance in AI systems. AI ethics monitoring tools can help make the \"invisible\" aspects of ethical violations \"visible\", providing a pathway to mitigate the issues involved.\n\nThese three papers collectively underscore he need for a multi-dimensional, comprehensive, and forward-looking approach to AI policy-making an ethics. They highlight the challenges ahead and the relevant solutions to navigate the complex landscape of AI governance and ultimately fostering the development of AI systems that serve humanity's best interests.","metadata":{"papermill":{"duration":0.005383,"end_time":"2023-06-29T13:14:57.170504","exception":false,"start_time":"2023-06-29T13:14:57.165121","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"### **Three Pathways for Actionable Principles for AI Policy**","metadata":{"papermill":{"duration":0.005377,"end_time":"2023-06-29T13:14:57.181485","exception":false,"start_time":"2023-06-29T13:14:57.176108","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"**Preliminary Landscape Assessment**","metadata":{"papermill":{"duration":0.005326,"end_time":"2023-06-29T13:14:57.192425","exception":false,"start_time":"2023-06-29T13:14:57.187099","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"A landscape assessment could address practical hurdles of actionability that result out of a lack of access to sufficiently comprehensive and relevant information. It therefore constitutes the first pillar in formulating Actionable Principles for AI. During the last few years, more than a few cases illustrate how a lack of advanced lanscape assessments can constrain the downtream impact of AI Ethics Principles. In 2017, New York City enacted bill \"Int. 1696\", establishing an Automated Decision Systems Task Force. The goal of this task force was to develop recommendations reviewing New York City's use of automated decision-making systems, including individual instances of \"harm\". However, the final report was subject to critique, which homed in on its weak recommendations that were partially related to the lack of resources the ADS has intially been allocated to conduct an appropriate landscape assessment.","metadata":{"papermill":{"duration":0.005502,"end_time":"2023-06-29T13:14:57.205023","exception":false,"start_time":"2023-06-29T13:14:57.199521","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"**Case Example - Biases in Loan Approval**","metadata":{"papermill":{"duration":0.005376,"end_time":"2023-06-29T13:14:57.216027","exception":false,"start_time":"2023-06-29T13:14:57.210651","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"Let's take the case of AI automated loan approval system. In order to ensure the fairness of the system, we will need the following information in our landscape assessment:","metadata":{"papermill":{"duration":0.005327,"end_time":"2023-06-29T13:14:57.226920","exception":false,"start_time":"2023-06-29T13:14:57.221593","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"**The Issue:**","metadata":{"papermill":{"duration":0.005367,"end_time":"2023-06-29T13:14:57.237970","exception":false,"start_time":"2023-06-29T13:14:57.232603","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"The purpose of the system is to automate the loan approval process in a bank, which uses customer data such as credit score, income, and employment history to make approval decisions.","metadata":{"papermill":{"duration":0.005508,"end_time":"2023-06-29T13:14:57.249070","exception":false,"start_time":"2023-06-29T13:14:57.243562","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"**Identify Stakeholders**","metadata":{"papermill":{"duration":0.005351,"end_time":"2023-06-29T13:14:57.260113","exception":false,"start_time":"2023-06-29T13:14:57.254762","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"Stakeholders can include:\n- Customers applying for loans\n- The Bank\n- Regulatory bodies for the banking industry\n- Society at large, as falsely approved loans can result in misuse of the loan proceeds for undesirable purposes","metadata":{"papermill":{"duration":0.005464,"end_time":"2023-06-29T13:14:57.271369","exception":false,"start_time":"2023-06-29T13:14:57.265905","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"**Potential Ethical Implications**","metadata":{"papermill":{"duration":0.0057,"end_time":"2023-06-29T13:14:57.282780","exception":false,"start_time":"2023-06-29T13:14:57.277080","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"Fairness: The system must not discriminate against certain groups of customers over others\n\nTransparency: The decision-making process of the AI system should be explainable and understandable\n\nAccountability: There must be a mechanism to hold the system accountable for its decisions\nPrivacy: the system must respect the privacy of customer data","metadata":{"papermill":{"duration":0.005634,"end_time":"2023-06-29T13:14:57.294341","exception":false,"start_time":"2023-06-29T13:14:57.288707","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"**Practical Case: Loan Approval System**","metadata":{"papermill":{"duration":0.005793,"end_time":"2023-06-29T13:14:57.306395","exception":false,"start_time":"2023-06-29T13:14:57.300602","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"Below is an example case using the German Credit data to show fairness in the loan approval process. You can clearly see that single male individuals have more loans granted than any other personal trait categories. The full discussion on this topic is [here](https://open.substack.com/pub/luckypen/p/detecting-biases-in-loan-approvals?utm_campaign=post&utm_medium=web).","metadata":{"papermill":{"duration":0.006023,"end_time":"2023-06-29T13:14:57.319782","exception":false,"start_time":"2023-06-29T13:14:57.313759","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"
\n
\n \n \n \n
Loan Approval Rate by Personal Trait
\n
\n
","metadata":{"papermill":{"duration":0.006024,"end_time":"2023-06-29T13:14:57.331849","exception":false,"start_time":"2023-06-29T13:14:57.325825","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"**Multi-Stakeholder Participation and Cross-Sectoral Feedback**","metadata":{"papermill":{"duration":0.005534,"end_time":"2023-06-29T13:14:57.343215","exception":false,"start_time":"2023-06-29T13:14:57.337681","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"The predominant method of soliciting external expertise is to conduct a public consultation. Most commonly, consultations are conducted prior to the drafting stage. One issue with this kind of consultation is they shift the burden of knowing what could constitute \"useful\" information on the person or entity submitting informtion. They equally force the stakeholder group soliciting that information to already have a clear sense of the type of information they neeed, by virtue of asking the 'correct' questions or call for input. This can lead those submitting information to submit overly generic feedback, and the stakeholder group to ask overly broad questions. In the end, feedback received may be lacking information that is actually relevant.\n\nThe consultations prior to the drafting stage would also benefit from supplements from consultations during the drafting stage and after publication stage, which unfortunately are rarely made use of.\n\nConsultations during the drafting stage can serve to stress test an initial proposal and steer the work towards actionability, while consultations after publication can help in understanding reasons for a potential lack of implementation, or pinpoint omissions, outdated assumptions, or areas that require revision since the moment of publication.\n\nThrough the consultation process, the end goal is to achieve consensus on how AI should be implemented. However, it may be impossible to reach consensus despite broad stakeholder inclusion, as various groups depending on their background can end up using the same terminology, yet denoting different content spaces resulting in varying interpretations for their resolution. This creates the appearance of agreement at the drafting stage while causing difficulties at the implementation stage.\n\nAn example would be over 81% of AI Ethics Principle-type documents contained some recommendations in favour of 'safety' and 'security', in a study conducted by Fjeld et al. ([2020](https://doi.org/https://doi.org/10.2139/ssrn.3518482)). He argues that perceptions of what constitutes safety thresholds differ among people and societies. This would entail that AI would need to pass the median between all individual safety thresholds distributed across society, in order to be perceived as safe even if the concept is understood by all to denote the same thing.\n\nBelow is a sample survey in Python that would accomodate multi-stakeholder participation with multiple rounds of feedback entry.","metadata":{"papermill":{"duration":0.005584,"end_time":"2023-06-29T13:14:57.354611","exception":false,"start_time":"2023-06-29T13:14:57.349027","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"
\n
\n \n \n \n
Multi-Stakeholder Feedback Form
\n
\n
","metadata":{"papermill":{"duration":0.005515,"end_time":"2023-06-29T13:14:57.365891","exception":false,"start_time":"2023-06-29T13:14:57.360376","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"**Implementation and Operationalizability Support**","metadata":{"papermill":{"duration":0.005369,"end_time":"2023-06-29T13:14:57.376974","exception":false,"start_time":"2023-06-29T13:14:57.371605","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"In order to account for the broad impact of AI models, it is necessary to consider and provide both technical and non-technical tools and measures.\n\nTechnical measures can include explainable AI research, which can help to illuminate some of the underlying decision making processes within some AI systems, increasing an individual's ability to understand and challenge the AI's output.\n\nThey can include:\n\n- privacy-preserving measures, which serve to adequately protect and secure data used to develop and maintain a given AI system's functionality\n- requirements for sufficiently representative data sets (Flournoy et al. [2020](https://cset.georgetown.edu/wp-content/uploads/Building-Trust-Through-Testing.pdf)) to ensure both adequate functioning of an AI system in a real-world environment as well as to tackle bias introduced through inadequate data sets; or, describe testing and validation protocols and procedures that would support the fulfillment of the principles in question by a given AI system. Methods used to ensure privacy-preserving measures may cover:\n\n - adversarial testing by dedicated red teams (Brundage et al. [2020](http://arxiv.org/abs/2004.07213)), evaluation\n - evaluation of out of distribution robustness (Lohn [2020](http://arxiv.org/2009.00802))\n - repeat testing within reasonable time spans once deployed, or\n - denoting adequate requirements for audit trails at all stages of the lifecycle of a given AI system (AI HLEG [2020](https://ec.europa.eu/digital-single-market/en/news/assessment-list-trustworthy-artificial-intelligence-altai-self-assessment))\n\nNon-technical measures can provide requirements and methods to increase society's participation in decision-making processes surrounding the development and deployment of AI systems, emppowering them to flag and halt the development and application of AI systems with potential negative impacts. The dialogue with society could be carried out by developing and provisioning of relevant algorithm impact assessments. An example would be the [Canadian government's algorithmic impact assessment](https://open.canada.ca/aia-eia-js/?lang=en) or the [AI HLEG’s revised assessment list for trustworthy AI](https://digital-strategy.ec.europa.eu/en/library/assessment-list-trustworthy-artificial-intelligence-altai-self-assessment).\n\nThe provisioning of mechanisms for implementation can contribute to an actionability of underlying goals of an AI system documentation, thus it directly builds and expands on the earlier two elements: langscape assessment and multi-stakeholder engagement strategy. The three pathways together would form a prototype framework across the lifecycle of Acionable Principles - inception, development, post-publication.","metadata":{"papermill":{"duration":0.005504,"end_time":"2023-06-29T13:14:57.388226","exception":false,"start_time":"2023-06-29T13:14:57.382722","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"### **Meaningful Human Control: Actionable Properties for AI System Development**","metadata":{"papermill":{"duration":0.005353,"end_time":"2023-06-29T13:14:57.399278","exception":false,"start_time":"2023-06-29T13:14:57.393925","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"The third pathway above, implementation and operationalizability support for AI models, the paper Meaningful Human Control: Actionable Properties for AI System Development tackles the topic of AI implementation and operationalizability on an even more practical level. It identifies, through an interative process of abductive thinking, four actionable properties for AI-based systems under meaningful human control, using two application scenarios: automated vehicles and AI-based hiring:\n\n1. A system in which humans and AI algorithms interact should have an explicitly defined domain of morally loaded situations within which the system ought to operate.\n2. Humans and AI agents within the system should have appropriate and mutually compatible representations.\n3. Responsibility attributed to a human should be commensurate with that human's ability and authority to control the system.\n4. There should be explicit links between the actions of the AI agents and actions of humans who are aware of their moral responsibility.\n\nThe above when implemented appropriately should achieve the following two conditions for meaninful human control:\n\n1. Tracking: a human-AI system should be responsive to the human moreal reasons relevant in the circumstances\n2. Tracing: a human-AI system's behavior, capabilities, and possible effects in the world should be traceable to a proper moreal and technical understanding on the part of at least one relevant human agent who designs or interacts with the system.\n\nBelow is a visual representation of the above abductive thinking framework:\n\n
\n
\n \n
Framework of Abductive Thinking (Dorst, K.: The nature of design thinking. In: Design Thinking Research Symposium (2010). DAB Documents)
\n
\n
\n\nLet's dive into each of the four properties below:\n\n**Property #1: The human-AI system has an explicit moral operational design domain and the AI agent adheres to the boundaries of this domain**\n\nAs the human-AI system has to be \"responsive to relevant human moral reasons\", designers should consider a larger design space, one that captures the values and societal norms that must be considered and respected during both design and operation.\n\nThe concept of operational design domain is often used in the context of automated driving and refers to a set of contextual conditions under which a driving automation system is designed to function. Specific contextual properties of the automotive operational design domain include factors such as road structure, road users, road obstacles and environmental conditions, as well as human-vehicle interactions and expected vehicle interactions with pedestrians.\n\nAs simple as a hammer should be used to hammer nails and not to injure others, this moral concept may not be so clear cut when implemented in a complex human-AI system.\n\nIn the automated vehicle vase, the moral operational design domain should contain moral reasons representing safety (e.g. avoid road accidents), efficiency (e.g. reduce travel time), and personal freedom (e.g. enhance independence for seniors), among others. In the AI-based hiring context, moral reasons could include, from the employer's side, reducing discrimination or increasing the number of applicants in the recruitment process, while for the applicant's side, autonomy over self-representation could be considered very relevant.\n\nBelow are example codes for autonomous vehicle and AI-based Hiring\n\n [![image.png](attachment:bb2f3f08-e97a-41b5-8b0e-dea30fe7d80e.png)](https://colab.research.google.com/drive/1ZpzleiiY9_6mKSx2VTH_lnbDZA9U47TC?usp=sharing)\n\nThe assessment of whether and how an AI agent is confined to the moral operational design domain is not a binary check, but rather a contextualized and deliberated analysis of the interaction between the AI agent, human agents, and the social, physical, ethical and legal environment surrounding them. Humans therefore should have an understanding of such blurry boundaries of the moral operational design domain and their responsibility to meaningfully control the AI agent in this process, **which includes the possibility of deciding that the use of an AI agent is not acceptable in certain contexts.**\n\n**Property #2: Human and AI Agents have appropriate and mutually compatible representations of the human-AI system and its context**\n\nRepresentations that are mutually compatible between human and AI agents within the system allow the agents to have appropriate understanding of each other, the task, and the environment (Jonker, C.M., Van Riemsdijk, M.B., Vermeulen, B.: Shared Mental Models. In: De Vos, M., Fornara, N., Pitt, J.V., Vouros, G. (eds.) Coordination, Organizations, Institutions, and Norms in Agent Systems VI, pp. 132–151. Springer, Berlin (2011)), which facilitates agents to cooperate, adapt to changes, and respond to relevant human reasons.\n\nTo ensure safe operation of the system, agents should also have a shares representation of each other's abilities and limitations. Specifically, the AI agent shosuld acocunt for humans' inherent physical and cognitive limitations, while human agents should account for the AI agent's limitations to avoid issues such as overreliance (Lee, J.D., See, K.A.: Trust in Automation: Designing for Appropriate Reliance. Human Factors, 31 (2004). Crucial to achieve meaningful human control, these shared representations hdoul include the human reasons identified in the moral operational design domain, which can change over time anc across contexts.\n\nIn order for the agents' shared representations to facilitate the system's tracking of relevant human reasons, the system designers first need to define which aspects of the system and its context each agent should ahve a representation of. A useful approach to determine the necessary representations and to translate the concept into practical design requirements is co-active design ([Johnson, M., Bradshaw, 2014](https://doi.org/10.5898/JHRI.3.1.Johnson)). Specific to building and maintaining shared representations, this approach provides guidelines on how to establish observability and predictability between the human and AI agents, including what needs to be communicated and when (Jonker, C.M., Van Riemsdijk, M.B., Vermeulen, B.: Shared Mental Models. In: De Vos, M., Fornara, N., Pitt, J.V., Vouros, G. (eds.) Coordination, Organizations, Institutions, and Norms in Agent Systems VI, pp. 132–151. Springer, Berlin (2011)).\n\nRepresentations can include practical matters such as task allocation, role distribution and system limits, but also understanding of how humans perceive the AI agents, human acceptance of and trust in the human-AI system, human values and social norms.\n\nThis should also include determining the appropriate level of representation. In the case of automated vehicles, in order for the vehicle to intereact with a pedestrian, the designers need to determin whether is suffices for the vehicle to have a representation of just the location of a pedestrian on the road and their movement trajectory , or also the height and age of that pedestrian, their goals and intentions.\n\nIn the case of AI-based hiring, a key aspect requiring shared representation is the meaning of competence. In particular, the meanings of soft skills, such as teamwork and creativity, are highly fluid, context-dependent, and contestable. Therefore, aligning the job-specific meaning of competence among job seekers, employers, and any AL agent involved in the hiring process is critical.\n\nOnce the representations required for each agent are define, the design and engineering choices need to sufficiently take these into account. Ecological interface design can also leverage knowledge on human information processing to design human-AI interfaces that are optimally suited to convey complex data in a comprehensive manner ([Vicente, K.J., Rasmussen, 1992](https://doi.org/10.1109/21.156574)).\n\nFor the AI agents to have appropriate representations of human agents, the assumptions about human intentions and behavior adopted by AI agents, implicit or explicit, need to be validated. This can be aided by incorporating theoretically grounded and empirically validated models of humans in the interaction-planning algorithms of AI agents ([Schürmann, T., Beckerle, 2020](https://doi.org/10.3389/fpsyg.2020.561510), [Siebinga, O., Zgonnikov)](https://arxiv.org/abs/2109.13077), or by augmenting bottom-up, machine-learned representations with to-down symbolic representations ([Van Bekkum, M., de Boer, M., Van Harmelen, F., Meyer-Vitali, A., ten Teije, A.](https://arxiv.org/abs/2102.11965), [Marcus, G.](https://arxiv.org/abs/2002.06177)).\n\nAn alternative approach, value alignment (Gabriel, I.: Artificial intelligence, values, and alignment. Mind. Mach. 30(3), 411–437 (2020)), aims to mitigate the problems that arise when autonomous systems operate with inappropriate objectives. In particular, inverse reinforcement learning, which is often used in value alignment, aims to infer the \"reward function\" of an agent from observations of the agent's behavior, also in cooperative partial-information settings (Hadfield-Menell, D., Russell, S.J., Abbeel, P., Dragan, A.: Cooperative inverse reinforcement learning. Adv. Neural Inf. Process. Syst. 29 (2016)).\n\nBelow is an example code for inverse reinforcement learning:\n\n
\n
\n \n \n \n
Inverse Reinforcement Learning
\n
\n
\n\n**Property #3: The relevant humans and AI agents have ability and authority to control the system so that humans can act upon their responsibility**\n\n**Relevant humans should not be considered just mere subjects to be blamed in case something goes wrong**, but rather they should be in a position to act upon their moral responsibility by influencing the AI system through its operation and bring the system back to the moral operational design domain if needed.\n\nThis is only possible when the distribution of roles and control authority between humans and AI is consistent with their individual and combined abilities, including reasonable mechanism for overruling the AI agent through intervention and correcting behavior, setting new goals, or delegating sub-tasks.\n\n**In human-machine systems, the ability to control should not be smaller than control authority, and control authority should not be smaller than responsibility** (Flemisch, F., Heesen, M., Hesse, T., Kelsch, J., Schieben, A., Beller, J.: Towards a dynamic balance between humans and automation: authority, ability, responsibility and control in shared and cooperative control situations. Cognit. Technol. Work 14(1), 3–18 (2012)). The **ability of a human or AI agent** includes their skill and competence to perceive the state of a system and the environment. This also includes a way to acquire and analyze relevant information, to make a decision to act, and to perform that action appropriately. ([Parasuraman, R., et al., 2000](https://ieeexplore.ieee.org/document/844354)) Ability also includes the resources at their disposal, such as tools or time.\n\nThe understanding of an (AI or human) agent's ability is intrinsically related to the socio-technical context in which the system is embedded. Approaches such as the nature of activities (Santoni de Sio, F., et. al.: Who should enhance? Conceptual and normative dimensions of cognitive enhancement. HUMANA.MENTE J. Philos. Stud. 7(26), 179–197 (2014), [Santoni de Sio, F., 2016](https://doi.org/10.1007%2Fs11948-015-9715-4)), under the umbrella of Value Sensitive Design ([Friedman, B., Hendry, D.G., 2019](https://doi.org/10.7551%2Fmitpress%2F7585.001.0001)), can support the understanding of which set of tasks should be delegated or shared with AI agents, and which should be left exclusively to humans.\n\nThe second component mentioned above is **control authority**, which is the degree to which a human or AI agent is enabled to execute control. According to the above rule in ability to control not being smaller than control authority, which is not smaller than responsibility. An agent should be responsible only for tasks they have authority to perform, and they should have authority only over tasks they are able to perform. A key implication of this consistency is that control is exerted by the agent that has sufficient ability and authority, and more responsibility is carried by the agents that exert more control. While ability and authority are attributes that both human and AI agents possess, an important point to note is that **responsibility is a human-only quality**.\n\nConsider our autonomous vehicle case and AI-based hiring case in the context of authority to control, the driver may have final authority over a fully autonomous vehicle, but the driver's loss of situational awareness, or even skill degradation as a result of systematic lack of engagement in the driving task, will limit the driver's ability to exert control authority ([Heikoop, D.D., et. al., 2019](https://doi.org/10.1080/1463922X.2019.1574931), [Flemisch, F.et. al., 2012](https://doi.org/10.1007%2Fs10111-011-0191-6), [Kyriakidis, M., et. al., 2019](https://doi.org/10.1080/1463922X.2017.1293187)). The same may happen for a hiring manager with final authority over who to hire, if they merely sign off on the hiring recommendations of the AI agent, without substantively engaging in the aseessment process themselves. **Humans must be empowered to act upon their moral responsibility.**\n\n**Property #4: Actions of the AI agents are explicitly linked to actions of humans who are aware of their moral responsibility**\n\nTo trace any consequence of the human-AI system’s operation to a proper moral understanding of relevant humans, there should be explicit, explainable and inspectable links between actions of the system and corresponding human morally loaded decisions and actions. As such, we argue that **moral understanding of the system’s effects should be demonstrated by, at least, those humans who make decisions with moral implications on the design, deployment, or use of the system, even if the actions that bring a human decision to life are executed by the AI agent**.\n\nThe links between actions of the AI system and the corresponding human morally loaded decisions and actions need to be explicitly identifiable in two ways:\n\n**Forward link**: Human should be aware of their moral responsibility associated with that decision\n\n**Backward Link**: For any consequence of the actions of the human-AI system, the human decisions and actions leading to that outcome should be readily identifiable.\n\nTo aid the link mapping process, values hierarchies ([Van de Poel, I, 2013](https://doi.org/10.1007/978-94-007-7762-0_20)) provide a structured and transparent approach to map relations between design choices and normative requirements. It visualizes the gradual specification of **broad moral notions**, such as moral responsibility, into **context-dependent properties or capabilities** the system should exhibit, and further into **concrete socio-technical design requirements**. Such a structured mapping can equip stakeholders with the means to deliberate design choices in a manner that explicitly links each choice to relevant aspects of moral responsibility.\n\nIn the autonomous vehicle case, consider an automated vehicle which slows down and pulls off the road after it recognizes a car accident ([Mecacci, G., 2020](https://doi.org/10.1007/s10676-019-09519-w)). The vehicle should immediately remind the driver of their duty to provide assistance to possible victims in the accident.\n\nIn the context of AI-based hiring, explanations of assessment scores in language that directly links observed job seeker performances to job-specific meanings of competence can help employers, job seekers, designers, and developers better outline the boundaries of moral operational design domain during design phase. This can help identify if there is misalignment between conceptions of competence among the human agents and the AI algorithm.\n\nIn complex socio-technical environments, he establishment of links between human moral awareness and actions of a human-AI system is complicated by the “problem of many hands”, which happens when more than one agent contributes to a decision. In case of unintended consequences of the AI agent’s actions, this complexity can hinder the backward link. This challenge calls for systemic, socio-technical design interventions that jointly consider social infrastructure, physical infrastructure, and the AI agents that are part of these infrastructures.\n\nMethods from social sciences, such as Actor-Network Theory (Latour, B., et al.: Reassembling the Social: An Introduction to Actor-Network-Theory. Oxford University Press, Oxford (2005)), can support the development of tracing networks of association amongst many actors, which can help understand how, for example, humans may offload value-laden behavior onto the technology around us. The paper “sociology of a door closer” (Johnson, J.: Mixing humans and nonhumans together: the sociology of a door-closer. Soc. Probl. 35(3), 298–310 (1988)), describes how we made door closers the element in the assembly that manifests politeness by ensuring the door closes softly and gradually, even as the human actors may barge through without any action to regulate the door.\n\nAlthough establishing explicit links between human decisions, human moral awareness, and actions of the AI agents is challenging, they allow appropriate post hoc attribution of backward-looking responsibility for unintended consequences, helping to avoid responsibility gaps and prevent similar events from repeating in the future. It also facilitates forward-looking responsibility by creating an incentive for the relevant humans to proactively reflect on the consequences of their decisions (design choices, operational control, interactions, etc.).\n\nBelow is an example code for the link between AI actions and human responsibility, obviously over-simplified for demonstration:\n\n
\n
\n \n \n \n
Action Responsibility Link
\n
\n
\n\n**Conclusion**\n\nFor the purpose of summarizing the paper, Meaningful Human Control: Actionable Properties for AI System Development, some final points are made in achieving meahingful human control in AI-based systems:\n\n- Designing for meaningful human control requires designing for emergence\n\nThe socio-technical complexity of human-AI systems and the inherent uncertainty of some aspects of their operation call for designing for emergence ([Pendleton-Jullian, A.M, (2018)](https://doi.org/10.7551%2Fmitpress%2F11801.001.0001)), where the focus shifts to designing the social, physical, and technical infrastructures that jointly provide favorable conditions for interactions between agents to lead to emergence of desirable system properties and behaviors.\n- Meaningful human control is necessary but not sufficient for ethical AI\n\n**Meaningful human control does not prevent humans from consciously designing and operating the human-AI system in an unethical way.** Therefore, meaningful human control must be part of a larger set of design objectives that collectively align the human-AI system with societal values and norms.\n\n- Transdisciplinary practices are vital to achieve meaningful human control\n\n**While this is undoubtedly a challenge, it may prove to be a rewarding opportunity for socially inclusive innovation that puts human moral responsibility front and center.**","metadata":{"papermill":{"duration":0.005494,"end_time":"2023-06-29T13:14:57.410421","exception":false,"start_time":"2023-06-29T13:14:57.404927","status":"completed"},"tags":[]},"attachments":{"bb2f3f08-e97a-41b5-8b0e-dea30fe7d80e.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAlUAAAB8CAYAAABAOkVlAAAgAElEQVR4nOy9eXhV5bk2fq89z8nemSEhQIAECAFkUqBCcW6dKMFax6PWWSsyiMoBqSg4AY6nxaJe1CIoFJWq1LGoODFVQCBgIJCQBDLted57re+P8Dx51yYK5yvn1/P5W/d1cZHs7PWud1rrud9nlBRFUaBBgwYNGjRo0KDhX4Lu390BDRo0aNCgQYOGnwI0UqVBgwYNGjRo0HAaoJEqDRo0aNCgQYOG0wCNVGnQoEGDBg0aNJwGaKRKgwYNGjRo0KDhNEAjVRo0aNCgQYMGDacBGqnSoEGDBg0aNGg4DTgpqUqn00in0wCAZDLJnweDQf45M9VVIpFAKpXinwmyLCMajfLv0WiUvxcKhQAAkUiEvyvLMl8fCAT4/slkEvF4XHXPaDQKRVEgyzL3J5VKQVEUbpvaSqfTqj7T+MQ26TPqb3dtUH9SqRRSqRT3PZ1Oo6WlBZMnT8bdd9+NSCQCWZZ5XAAQi8X4XqFQiPuzbt06PProo3wv+l4ymVSNS5zXzDHR3NF1dA2NUZZlTJ8+Hf3790dZWRkeeeQRbkdc4+7WncYoftZdn8T9Aaj3iM/nU/WJ/k5zDAAdHR3dXr9ixQoMGjQIvXr1QnV1tep76XSa1zAcDp8wDhGZ/RP7Iu7Rk4H6FYvFIMsyrrjiChw6dAiyLHe7r6jtYDDI95RlGclkUrVHutvf1E9q1+fz4e6774bX60U8Hlddkzl/YhuZe5fao7EoisJ/z+wH9Y/2G12TSCS4HXHuqZ3unj0tRZ4GDRp+ajgpqXr++efRu3dvDBw4EOXl5aioqMCLL74Ip9PJ38kU4iaTCQaDAalUCo2NjZg6dSoSiQR0Oh2sViuAzpe+1WqFwWAAADgcDsTjcdhsNm5Tp9PBZDKho6MDLpcLRqOR/0bX0cvaarUiHA5Dp9NBkiS0trbCYDBAkiQ4HA4oisIERa/XIxqNIpFIwO/388vdYDCwENHr9dxuIBCAyWTqnDCdDh0dHdDr9TAajYjFYjAYDDAYDLDZbOjo6EAymUR+fj6SySTMZjNfF4/HmaRS/9PpNPePyAYJJZPJBIvFAgAwGo3czz/84Q/46KOPkEgkkEwmkU6nodfrIUkS30uSJCQSCRiNRvj9fr6fJElYu3YtFEXB5s2bUVNTgwceeIDXhMYUDod5DiRJgt/vBwBeH4PBgHA4zILUYDBAURQmXeL+SKfT3Ld4PI7s7GzVHAcCAciyDIfDgUAggFgsBo/HA6BL8EqShO+//x4ffvgh1q5di/r6eqxevRput5tJrV6v5zbtdjuSySRisRj0ej0WLVqEl156CbIsI5VKcf+o/7IsIxaLQVEU3qOZxLE7EDGKRqO48847cd1116GwsBA6nQ56vR6NjY24++67UVpaiuLiYnzyySeIx+Ow2+38jEiShHQ6zYcAnU4Hs9mMVCqFaDSKZcuW4ZxzzsHOnTthMBiYbDudTpxzzjl45plnEAwGea+lUil4PB6EQiHuXzKZhNVqRSKR4L0bj8d5zrxeLyRJQiAQgCRJiMVi+I//+A8MHToUhYWFWLRoEa+VTqeDLMuQJInX1WQyQa/XIxaLwW6344033kCPHj3Qr18/VFZW4q9//Svi8Th0Oh2vq0aqNGjQ8FPDSUlVKBTCggULsHfvXhw4cAA1NTW46aabAHSdznU6HQtdIldAp6AlMiJqr9LpNDweDws4ug8JBRIwAOD3++HxeBAIBAB0Eg6z2Qy9Xs9kh4Sfw+FAIpFAOBxGXl4e9yMWi0GSJLhcLqTTaQSDQdhsNphMJmRlZTHh0Ov1MJvNrDGRZRnhcBgulws6nQ6pVAoGg4H7DgAWi0WlofJ4PLBYLPD7/XjnnXewcOFCJiJmsxmSJKmEv16vZ2GTnZ2NVCqF4uJinjPqfzAYZIHU0tICv98Pk8kEo9HIbdHYSVCbTCYoioKsrCwkk0lEo1EYjUY0NDRg7NixcLvdMBqN3C7102KxwGq1IhKJMGl0uVwqrYUkSbDb7Tz2eDwOs9kMm82m0t4QqYnH44hGozCbzUxwJUlCJBKBy+WCXq9HIpGAy+XiuY1Go5AkCaFQCH6/H21tbXC73Rg0aBD3i9ZEJKvxeBzt7e0wGo1MSk0mE7Kzs6HT6ZhcU998Ph9MJhOvDwA0NTUxif8xEAn94IMPYLfbMWHCBFgsFsiyDJ/PhyeffBKjR4/G4cOHsWnTJjz22GPYvHkzdDodWltbmXBaLBaYTCbEYjFed6/Xi1tuuQUmkwk9evRAbm4uwuEwrFYrLBYL4vE4LrroIgDA559/zgSR9oPFYmESZjQakU6n+Xmk542eM7fbrbp29erVGDZsGGpqarB9+3bs2bMHr7/+Ou8R6mMymUQkEuE1T6fT+OCDD3DgwAHU1dXh8OHDeO2117B+/Xo0NDQgHo/z/qR9p0GDBg0/FZz0rWa1WllrQFoRQjKZxIMPPoidO3fi4osvRkVFBWbMmIHW1lb4fD5ceumlGD16NPbv349BgwahtLQU5513Hg4fPswmjtmzZ6OgoACjRo3CU089xaYHi8WCe+65B42NjVi0aBGqqqowfvx4HDt2TKWdArqICdB5Srfb7QCA5cuXo6ysDJWVlZgwYQL2798PvV4Pp9PJppBkMolwOMzaof3792PBggWoq6uDoiiw2+1ob2/Hzp07MX36dL7utddeQ1FREfr27YvbbrsNra2tTBLXrFmDoUOHoqKiAi+88AKAzpM5aX/279+Pn/3sZyguLka/fv0wduxYjB07FslkEolEAs3NzXjyyScxcOBAlJSU4PDhw3A6nXjttddQXFyM119/HXPnzsWAAQMwcOBAPPbYY+jo6IDJZILdbleZTUnrIEkSrFYrotEoampqVOQiGAxCURRYLBbMnDkT/fr1Q58+fXDppZfC6/WiubkZkiRh9uzZ+Pzzz7FgwQKUl5dj3LhxTEBFAUlrYTabYbFYkEgkYDabsXHjRvTu3RvFxcUYOnQoNm3aBKvVimQyiWAwiPvuuw8FBQUYM2YMbr31VpUWMysrC42NjUgkEgiFQirNIQB88sknGDBgAEpLS3HjjTcyOVq7di169uyJxYsXY/78+cjJycH48eOxaNEimM1mvP/++3jggQcQCoWYzK9evRqvvPIK2tvbT+ERAlpbW/HRRx9hxowZ3C9JkrB3714cPXoU1dXVAIDevXvjggsuwMaNGyHLMvLy8mA2m5FMJtHa2spaMpPJBK/Xi/Xr12PevHkYM2YMfD4f0uk0a2SBThJsNptx7bXXYsOGDXzwkCQJXq8XBoMBa9euxcSJE1FXVwe9Xq86gKTTaRiNRhUxttvtqKurw7Jly3DxxRcjmUyiR48euPzyy/HJJ59AURSk02nY7XbWbJKGFujUELa0tDDZT6fTqKqqYk2axWJh4iaaCTVo0KDhp4CTkioyidCpXlEUNp9FIhFs2rQJzz//PP7617/i448/RkdHB44ePYrs7Gy88cYb+OabbzBs2DBs374dDQ0N+PDDD5Gbm4t0Oo3nnnsOffr0wbFjx/Dll1/iyJEjWLNmDSwWC5OdcePGYdCgQTh06BAuuugibNiwASaTCbIss0ksFovxqZtO0nv27EF5eTlqampQW1uLu+++G3/6058QDAYRi8VY+BmNRtjtdiZoeXl5qKmpgdVqZSGUk5MDWZZht9uRTqexdetWvPPOO9i/fz8OHjyIkSNHYvHixdDpdDAYDLj88stx6NAhzJs3jwUzCaJAIIDXX38dv/3tb3HkyBGsWLECNpsNK1euhNFoRHZ2NlatWoXCwkIcOHAAzz77LBYuXIh0Oo2pU6fiyJEjmDp1Kh5//HHs3r0bu3fvxv333w+Px4N4PM7mV9JSRSIRWCwWBAIBXHjhhaioqMD777+P+fPno6KiAhdddBEaGxshSRLWrFkDvV6P7du34/Dhw7jhhhtwxx13oLCwEF6vF6lUCldffTVGjBiBXbt24ZJLLsEf/vAHJJNJ1tjQnAJQmYY2b96MN998E1999RUaGxuxb98+jB8/HpIkwWg0Ytu2bbjzzjvR1NSEbdu2weVyYd26dYjFYnjwwQcxYMAA3H777fjqq68wbtw4lJaWYuXKlZAkCVu2bMHjjz+OXbt24cCBA7j66qsxd+5c+Hw+XHzxxWhsbMTMmTOxYMECtLe344svvsADDzyARCKBwsJCtLS0oLm5GQaDAclkEp9//jmKioqQk5Nz0geINIAWiwU5OTm8ZyRJQmNjIyorK1lb9uKLL2L16tUIhUIqQmE0GpGXlwdJktiU6Xa7cdVVV6FPnz7Izc1FQUEBIpEIm9Cj0ShisRgikQjKysqQSCTQ0tLCZDo7O7tbX7JMyLIMi8WCdDrNmuLm5mace+656NevH4xGI7Zv3445c+YgEAjwPpBlGR6Ph03WOTk5PBfV1dVYvnw53n77bSSTSSxcuBDl5eV82CFiR79r0KBBw08FJyVVqVQKM2fOxODBgzFo0CCMHj0aGzZsgNVqhdPpxNixY1nw9uzZExUVFdixYwdrm/x+P5Mkcop1uVyIxWJob2/HhRdeCKDT/FBdXY2vv/4aQKeJy+PxYO7cubjssssAAD//+c+xa9cuNq2QGYFOvgaDgf27+vXrh1GjRrEpacCAAWhqaoLZbGa/EtE5mTQVbrcbffr0gSRJ2LVrF2677Ta0tbVh//79sNlssNvtWL9+PaZPnw673Q5FUXDjjTeivb0dBw4cUDlrm81mlJSUAOjyDTp27Bi++OILnHfeeYhGo6iqqmJNWzgcRjAYxO23345rr70W8XgcZ511FhRFYUGWSCTg8XjY9Cc6VRNRpDFLksSC0uPx4O9//zv279+Pa6+9FjfddBMOHTqEN998E3379kVNTQ0++eQTXHLJJexvdPHFF8Pj8WDbtm1wOp3w+XyYO3cuLrroIuj1elx44YVIJpNMHEWTJvWX+terVy/s378fLS0tkGUZTU1Nqj02YsQI9O/fn68dP3489u3bB4vFgoULF2L79u1Yvnw5Jk6ciA8++AANDQ246qqroNPpsHHjRjz//PNsshwzZgwikQhCoRATSvKhEx2lzWYzhg4dikmTJmHLli2QZZm1qJMnTz7Zo8FzvWPHDhQVFfGBg0hlLBaD1WpFXV0dCgsLcejQIaxcuRItLS0nBGyIzuLUP/I5rK+vR2trKxMx8tcStT7Dhg3DV199xeZw8neqrq7Gxo0b0adPH9ZM0X4nTZJOp1MRMFmWYTKZ8Pnnn+Pcc8/Fgw8+iBUrVqCkpAShUIhNxsFgENnZ2bzfjEYj+23V1NTgyy+/xBlnnAGHw4HFixcjOzsbLS0tMJvNiEaj3TrBa9CgQcP/yzCc7As5OTl48sknce6558JoNMJkMkGSJESjUQQCARw9ehQmk4mdpV0uFwv8eDzOzuLkMwJ0qv2bmprw2Wef4cMPP0RDQwNyc3MhSRLOOuss+Hw+eDwe6HQ6TJo0iR1zx44di6FDh7LvkqIobFpKJBLswE4ah3nz5uHdd99lDc6ECRPg8/mQn5+PUCgEh8PB/7tcLh5zUVER6urq0NbWhsGDB6OtrQ0mkwlDhgxhH5/LLrsMLpcLiqLA7/ejd+/ecDgcPEag0+/kyJEjqvns1asXFEXBV199herqamzduhXZ2dkoKiqC3W5ns1wsFoPFYkFTUxNSqRSTI6ArOiscDp9w+hf9gMh5GwDq6+vh8XjgcDjg9/sxYsQIAGAtSu/eveH1epGTk8PEw2QyqfzHPB4PBg0axFGWVVVVGDZsGN+PTHF0vRiYYLVa8dRTT2Hy5MkwGAz45S9/iUcffZTnKx6Po7q6Grt374bNZkMymeTISZvNBoPBwCSCzNGpVAo6nQ779u3DH/7wBzYXRqNRDBw4EO3t7SguLobL5eJx6vV6OBwO/p4kSRg5ciTWrVuHyy+/HN9//z37m7W3t59UW2U2m3Ho0CF2LqfnAOg0zz366KM4cuQIjh49imQyic2bNyMvL4/7sGDBAqxZswZ1dXUYMGAA7rnnHlx33XXcPhF9UXMmkhHaF263G0ePHuX9QeuRSqWYaBmNRiQSCZhMJg4MoevpMKLT6WCxWLBs2TLs27cPb775JpxOJ+rr61FbWwuHwwGdTgdFUeB0Onm88Xic/fja29sxfvx4vPzyy/jP//xP3Hvvvbj00kuxfv165OfnIxqN8r7QoEGDhp8STkqqyEmYhADQ+VK3Wq38wibHcRLsdOolHwoK46e/EXkYNGgQFi1ahPz8fH7Bi47HwWAQkUiEzVkA2ASXTqf5RA10ammIxPn9fqxcuRLxeBzbtm2D3W5HQ0MDHnjgAY5cIp8Sh8PB9ySBU1xcjJqaGni9XowZMwZNTU2oq6vDmWeeiWAwiEQigfXr12PUqFEqIUqmEBqLJEls6jQajRxhNnLkSEyfPh1z5sxBJBLBO++8g6ysLCQSCVitVnZEBoDc3FxkZWXhyJEjKCoqYk1OMplkTZmiKCoyRWkliGTEYjH06tWL+0bmT+pnJBLhNQoGg6zxi0ajnB6AnJ2J8Igkj4QkRX8R+aE0AWazGVlZWTjjjDPw/fffAwCWLVuGGTNm4JlnnkE8Hsf111+P6667Dr/+9a+h1+uxevVqNDQ0sOO7xWJBKBSCz+djgiBJEsLhMPLz87Fq1SoMHz6cTcOiViaZTEKv1zPpI2dvEux9+vRBQ0MDDh8+jC+//JJJzamY/wCgZ8+eqK+v571ARDAnJwe/+MUv8NBDD8Hr9cLtdqOjowNZWVm8Z+bOnYtZs2axNpdMpvR9SZJgMBjQ3t7OwR4094qicJRna2srysvL+ZkEwPsaAGs1rVYrPzt0GBEd+ltaWpCXl4cLLrgAs2fP5rYOHTqEPn36MKGNRCKw2+08DpGcrV69Gtdffz1GjhwJAHjuuefwyCOPYMWKFbj66qt53sVnR4MGDRp+Cjil8BsSpAA4gkskIuTHRE6v4guWXuJbt24F0Kkp6OjoQFlZGVwuF/785z+zoBAJFSGdTrNJjcx19CI2Go3cLxISQOcL3+l0wuPxsFB58sknVZFHYnv0HZPJxGHt5PTdr18/bNmyBfv370fPnj3hdDpxwQUX4OGHH4bX61XlVjKbzdyPjo4O2Gw2RCIR7q9Op8Pu3bvh9/tRX1+PrVu3oqGhAeXl5Xx/ckgWQ+F9Ph9ycnKYQFVWVmLjxo0IhUKqKCpyIhbnCAATW1GbSMKU0ljo9XqMHDkSb731Fnw+H3Q6Hf72t7+hsLAQVVVVsNlsrEETozQBqLQOJJxpfYi0EahfTqcTTqeTnarNZjMqKiqg0+lQX1+PpUuXcqSgJElMpERybzAYYLVaceaZZ+Lhhx+GTqdDMpmETqdjEgt0Hgz69OmDbdu2cSoP0bE+JycH1dXVuP/++zl9iJhH7GQoLi5mR+1YLAabzYZUKoWqqiqEQiG89NJLyMrKgtfrxcqVK/GLX/yC9xzNGa0tfe52u5FIJHj8BoOBIyTJDEjzmkql0Nraip49e6rMzwaDAe+//z4qKytRV1encnInwkkRh4T8/HwAQP/+/bFkyRK+x1tvvYXBgwdzxKXRaMTmzZvRr18/PP744zzXpN1raGjgvrS1teGDDz7AmDFjOCcX0H3+MA0aNGj4fxknJVUmkwn33XcfhgwZgoqKClRVVeHPf/4zdDodwuEw50ECuiLvotEoQqEQdDod8vPzcdddd+Gee+5Bz549MXr0aNYkLFiwADU1NTj//PNht9sxdOhQvPHGG5AkCfF4HE6nE3l5ear8VJmgv1F/YrEYioqKMHDgQHz55ZcYMmQIevTogWHDhiGVSqGwsJDbJ3MNRd2Rlmbo0KG4/fbbMXz4cBQUFLD/Bzmsn3feebj++utxxhlnYOLEiejZsyduu+02xONxJBIJzJo1C0OGDMG9996LVatWITs7G0uXLkU8HseYMWNgNptRVlaGCRMmYOTIkXC73di1axeCwSAcDoeKWJIjvc/nY1PqmWeeidzcXFRWVqK0tBRPPPEE5xeiYAIaFwkw0iZGIhFun0xWPp8PLpcL9913H7Kzs1FZWYnevXtj3bp1nBKiubmZnZHFfF8ngyzLsNlsWLVqFUpKSuByuVBeXo6//e1vmD17NoxGI3r16oU777yTI0inTZuGK6+8UmXmIi2N2+1WJTlVFAWXXnopLrzwQvTs2ROjRo1CQUEBZs6cyRo2t9uNs88+G4FAAOeeey769u2LpUuXcp4tIpSHDh3C4MGDAajJ4clwxhlnYMuWLdi2bRsMBgPPscvlwrJly/D1119j4MCB6NevH2bNmoU+ffowqROTb4qmY0rBUF5ejokTJ+KLL77AxIkTcd1116G1tZXzQcXjcdTV1aG1tRUlJSV8wKE9XV9fryKJonaJiDuR4ra2NgQCAZSWluKuu+5CYWEhhg0bhsGDB2P48OG48cYb4fP5WENMebCIaAGdgR4XX3wxQqEQBg8ejLKyMpx//vlYvnw5evXqxa4BXq9XNV4NGjRo+ElAOQni8biSTCYVRVGUQCDAn4dCIUVRFEWWZUVRFCWRSCiKoiixWIy/EwwGlfb29hPaTKfTSjQa5d/p2ng8rsTjcdV3/X4//yy2TX2ia6LRKH9G9xDbpr6mUikeB7VNY1AURYlEIkoqlTqhnZaWFr4vXR+JRJRYLMbf6W6c9Dca11dffaXMnDlTOXz4sJJOp5VwOKz885//VK688kq+B/Wnra1NSSaT3EZbWxu3La4F3SuRSKjGQtfJsqxEo1Gei46ODp5LWZZVc9TdWCKRiKIoyglrJq7Nj8Hr9aquy2w3EAhw216vVzX/mT8fO3aMf4/H40oikeDvUHu0DzL/l2VZkWWZ10Js++DBg8rtt9+uHD58mK8R99MPIZFIKMlkUvn73/+uzJo1S1GUzn2fOVZFUe/f7uYulUqp1o/6Fw6HT/iu3+/nvy9ZskT529/+xnuX2qD/M+8l9oPmXRxrIBBQ0uk0z6fP5+O/0f4Q90l380T3TiaT3P+Ojo4TxqtBgwYNPyWckqaKTBJiCLRocgK6NEbkJB6LxeBwOODxeDgPEiEej8NisahyRdG96PRKfyMHctHpWrwvXUOJDoFOEwyZ5egE3d7ezpmryYREDvZkDmxra4PVamXfqEgkwtfn5ubyqdzpdHJkl9ls5rkgzQhFmVG7dC8AqK2tRSKRYId1m82GDz/8EDabDXl5eUin09z3nJwcGAwG9gMjHx/S/gBg/xgyeVG/yQwGgAMF6Fq3283mVkppQGug0+kQjUZV0Wk01xTNSf5KonP/j0HU+pD2SRFMXU6nk02KDocDer2etZn0M61/fn4+R0OS1iMYDLKpORAIcALQzKz7tP6ZGjav14ulS5fi7LPPZt8zSZK61Yxmwmg0wmAw4IwzzkBzczNWr14Nh8OBtrY2leM8mc0pUpJMsKIJjKLxaL5Fh/dIJMLf9fv9nDB15cqVOHjwIM477zzO+UX7Tjke0ED3onJJooZIDC4QzbmKkFk+KysL8XgcwWCQfbioLBTdJxaLcQb4RCKhyu5vs9nY4Z7Mf4FAQPOn0qBBw08Op+RTRaVXdDodlxSh8HV6ycqyzP5D5EsDdJa1IN8ZAJzAEOgyO5BfRyqVYqHbXXQQCSmRoFGpGRKAqVQKFosFLpcLXq+XHepzcnL4HmRysVgs0Ol0aGlpAdBJnBRF4Qg4yrouZvYmwkN+T6KJisrqUMkOUShT2Hx1dTUkSUJeXh6GDx+OwYMHo6GhAf/1X//FYe5OpxMdHR1IpVIIh8Oq8h7ku0P+NcrxKDv6LtAphIlsAFDlj4pGowiHwwiFQjAYDIhGoyfUJSQnbspRJjpGk0O6oiinFBJPBDqVSnGkJa0bZfkGwBGlRIhkWeYxUtkdv9/PJk3RJyk7O5ud1onoidnCbTYbEyuxhI9er8fMmTNRVVWFAQMG4LLLLmNTtl6vPyXzFBGEvLw8vPjii/jHP/6BvXv3cjQrzackSby3Mmse0p4kokn5qgKBAM+xzWbjz7OyshCNRtHc3IyDBw9i2rRpqoSrALhkk+g7ZbPZVOOnDOtE+ACw6ZJ8H2nv6HQ61XNst9u5P+ToTtGeYtZ2nU6Hw4cP854BwIRc86nSoEHDTw2SIjKUbkBRSJmgSCLluCMt0CWUSVCIDs0kIE0mE9cPczgcJ4RXi5FaJIRJ6yOeqoHufXqI7FDqB0o7kHkfcqjP9C+hsinxeBxGoxGhUIgFNWmEAHCUGwBO2UCCJpFIqDRMYokbGiMJu8yoR0Lm70BndCHVzQsEAir/mO5A9yZSeKp+QlQTTrw/aVBEYitq4n4M4lhIQ2G32xGLxVSZt8U9Qt8V6x9mIrMPFNbv9XpRUFBwwlwAXZqZzP0g7nPyNyNt5MlA80zaHtL+xeNxJlyEQCDQrYaP/MOI9IggbSRpFKlts9mM1tZW1nBSShEiqNR3n8/HmfaTyaTK2T9zLkjDRz5b4v4Vg1Lo2aT1ouTARqORCR0daDL3DEVHatF/GjRo+KnhpKQK6IrAo9MmkSX6jKKzxDB7QiY5EHMrZd6D8t1ktkP5iIikAF3C0e/3czqCSCTCpEM57qwt1ocT702aEtKAZGVl8f0URUE0GmUBkkql2JxJpg/6fnfkh0DpIEh7RoQvk0QRQSONFBEu0iBEIhGVIMy8J2kMSTCS+YU0QSS4xALBlLOIssDT/UQNDdV0IzMbgTRkmcSrOyjHo9pIWHdXT08U6lSGhkL3gS7TFM0ltSuSCxLUYh9J2JtMphOILYHIFJHsUCiErKwsFYE+VYh5rTKJJOXaEpFJVGlcpB2i6Fq73c7tiUSUxt7W1obc3NwT+vxDzxodesgcSs+b+ByIz6G4Pl6vl8vj0BiIFFNKEABcw5AIpKIoPD+SJCEYDKqKbmvQoEHDTwEnJVV0gibBLApp8WVLP4vfp1M1EVfeplIAACAASURBVBwiYECXsNXpdCykM1M1kHCn0z8JgcyTr/hdMkWKWiTKx0NCloSSqOmg+1IuKTIZiaRDbE/sP/3MkyqMEQCH+hOhoPtS1vFMM1M8HmdNG60B3Uf0C6LyKKJplfydRG0H+SSJZEkkIJkaA7Gvmf0SE1yeCvEQhS5pYZTjyUPFNkTyQ/uJcn2JYxGJQiahItMgmaJIcNM9KMElETLaE6KAJ4Ir9vfHQG2IcyFGxIl7+YeeHeV4EtsfIqnkT0ZzBuCEuSfNEZlsKZUJzYdITMXnhdpPJpPcH9KmifcQr6HPxWdK9Nmjdac9lEmgupszDRo0aPgp4KRvNFGokYo/M7kkJVMkEkJmBrEGHBEC8v2g7wLgBKL0XboXgQQFmTWIrNALnBJZklZF1GiR0BWFlSjYqB3RsVYs8wKo8+lQAVz6G5mcSBNCglhsj4geQcyPRcKFiAZpDoh00fjpPoDaSZ8IK42L2hP9dkiY0vdEQpWJSCSiclyne1HmehozjU30qxLnify79Ho9m/yoVh1pNkVSTUSIggTIDynT74bq3omgfaTX62Gz2dhZmjJ+i2tD2jIxD5jT6VTVSaTx0nyLuchonwFd5VxEMyuZ4Oh/+h71TxwHaS7JKV7co6JWkUx64vViIlOgKxDCYrGonkVxPPQdsR26r7i2LpeL50ssn0MQSTG1J2oQaT8DnXuCiK14Pa19ZjBAZg4u6iNd093zTxDzu8XjcV4r8qfL3DenEoigQYMGDf8dnJRUkVACwFmtSWtC/hOiSY0+p795vV4AXQRF1DJRlBoJUrqeXuxA58uRfId8Ph87y4ZCIRVZoT4BYGJFkXSUjJE0FQBUxZeplpqiKKrxkMMu+fpEIhE2G8ZiMc7z1NbWppozigCcOnUqbr31VlUEJWnKKGM53Z+I2fr16/H000+zuSwcDrO5JRAI8JwAXRoD0gqSICaNBwkY8g0TkzH+7ne/Q+/evVFaWoqnnnoKyWSSE4HSNYFAgNsgDQfQScrIoVs01dIcA52OzPR90lJYrVbWKFksFv475TySZRn19fWorKxEUVERzjrrLDQ0NHDiUgBYvXo1JkyYgJycHNx4443o6Ojgth555BGMGDGCnadFzZC47qTtJLNmOp3GgQMHMGXKFCxatIj7GgwGVT5IlPiUDgeiZiocDrMfHe0BMokRseno6FBFP4rPAzmMi3voVEBZ9b/77jvccsst/LwRKaQ9IRI8MYkoPZfJZJIJPeVEI+0ZjZvKRen1ehw4cAAPP/wwAoEAa6PFvWcymeD3+1XzQXNCc0jPEBVipvGQ6T0Wi/G+pIMKRTgCne8mui6RSPAzQ4couo9IOIFO4iaOXYMGDRpOF05KqhYvXow+ffqgoKCAa72tWrUKOp2OS1WQJsnhcLD5DAAnJ7zuuuv45JlOp9lcQFFq2dnZHPlFL0MiHSS0vV4vsrOzVcSNHGFJUIindzKZiOkTyKwHdL78qYgykTsA/CIGwOVb6HqKBqQi0QTxHnTC93q9XJLH6/WqIuyIqJKjP0VOAp3CjQo/0xwCUAnnF198EatWreIM9gCYyFIUGdCV3qKtrQ0Oh4Pna+vWrZAkCTU1Ndi7dy9mz57NmeDD4TD79bhcLibHpJWJxWLsU0akltZbjLgTQSSE5pJIoBgkQGVaHn74YaxYsQKHDx/G119/jT59+nBAQW1tLd577z288MILaG9vx6uvvsrzQwTJ6XSitbWV/cMoAlI0JVN/Kb1BJBJBLBZDOp1GPB5nzZTT6WRiS2tDe5ESqJJZ1W63qzR8pHkTyajH42ECQponStIp7l+as1NFY2MjHnroIcyfP5/nw2azYdOmTRg4cCB69+6N888/H//85z85SS71mfYfFTkGOqMpyY/w0ksvxfTp09kkTdrn/Px8uN1uvPjii3zQoHESgcrKymIttdls5qhYmk/az2IBahr7b37zG1RWVmLgwIH45ptveB2pv6IZn0BEzGQy4fHHH0dZWRn69++POXPmcLt+vx92u521fxo0aNBwOnFSUmUwGPDUU0/h2LFj2LlzJ2pra/Gb3/wGQBdREXNOkd+Uoijw+XysgaDTKEVztba2Auh6kRLRIE1VKpXisH6g6wVMQspsNnNWZtJKEWGhlyahra1NpZkioUk5kYjcUTkQSltgt9v5vvRSj0ajTB6BLkLQ0tLC2gqDwYCCggKsXbsWixcvhtvtZmFKJWFIo5DpV2K321X+J6QdI1Mf9Z+EiRidJpY1AboEUG5uLs+xwWDAli1bMHLkSCa2BDFqUtQkkfaA2rBYLEyyMos9kymWfqa5NpvNqvJFtLdozxgMBjQ0NMBgMCA/P5/3lmgGamtrg8vlwtChQwF0pbAgzJgxAxs3bkS/fv2YJDidTs6xROSB5oycyp1OJyorK/Hee+9h/vz5cDgcvAdoT8iyzOuSnZ3NGkwiV0QUaHx+v581RUTEqc9ExkKhEIxGI6+/SExsNpvKtCr6g4kIBoNYvHgxqqurUVRUxNG1ra2tePLJJ/Hee++hrq4ON9xwA5599ln4/f5uc5wlEgk4HA7el3v27MFVV12FCy64AFarlfcv7YOsrCxcd911aGpqwo4dO3iuaIxk2qNSSqKWmPzygK7n32q1wmAwQFEU/O53v8Mvf/lL1NbW4v3338f06dNRU1PDe9piscBisfDzT1GiqVQKwWAQ8+fPh81mw549e7Bnzx48+uijTHazsrKY5IlrpkGDBg2nAyclVZRaAACXnwmHw5yT6qGHHsJHH32EX/ziFyguLsaMGTPYF+aWW27BhAkTsGfPHlRVVWHw4MEYNWoUmpub4fF4kE6nMW/ePOTk5ODMM8/EU089BUmSWCDOmjULu3fvxu9//3uMGDECw4cPx8GDB7k/brdbVc6FCvgCnZFMa9asQVVVFc466yxcdNFF2LJlCwsPepmTky6Z92pqavDYY49h//79PAfpdBo1NTW45557OE/Tm2++iV69emHYsGGorq5GdnY29+v111+H2+3G8OHD8dhjjyGdTsPn87GQPHDgAK666irk5eWhb9++qKiowM0334za2lp0dHTA7XZj3rx56Nu3L8aMGYMdO3ago6MDb7/9Nnr06IGXX34Z99xzDwYPHoyBAwdi0aJF6OjoYE2CqBUi7RGZbYBOciJGNlLkpsFgwLPPPovKykqUlJRgwoQJOHDgAJ/u7733XtTW1mLhwoUYMmQIzjrrLDQ2Nqr8y8S1ADqJq9/vh6IoeO+991BQUIChQ4di6tSpnAeJtFsNDQ2QJAklJSXwer0q0pdOp9Ha2sqEmUxDkiTh22+/xejRozF06FBccMEFqK2thV6vZ/IkyzIaGxtRUVGBrKwsjB49GsuXL2cT2BNPPIGysjKUlZXhlVdeAdCpVaJ+fffddxg8eDD69euHIUOGYN++fVw257XXXsP777+PF198ET179sS4cePw6aef8r2NRiPWrFmDYcOGYcCAAejZsye+/vprGAwGPPjgg1ixYgXPXzQaxYwZM7Bu3ToEAoFuo2lF7YqiKGhubkY4HMaVV17Jf3O5XHjmmWfw85//HAMGDAAAXHLJJYhEIti8eTOnNSCTt6hFdbvdaG5uxpo1a/D2228jNzcXkUiETcGklU0mk3C73Zg6dSrWrl2Ljo4O1ozSweKNN97Ar3/9a+zdu5dLJAFdZnWgK6Esff7ll18iFArhN7/5DaLRKMrKynD11Vfj448/VvlABYNBuN1uVcoTnU6H7777Dh6PB9OmTYPZbOZnksppEckTD1kaNGjQcLpwUlJFTr6hUAhWqxUOh4PzMbW3t6O+vh7PPvss3nnnHWzevBkHDx7E9u3bAQCvvvoqvvnmG1RVVWHHjh04ePAgtmzZwtGBTz/9NPLz89He3o7NmzejqakJr776KjvZKoqCKVOmYOzYsfj6669x4403sulR9CsST8Dk70QapZ07d+Lrr7/GTTfdhPXr16v8jsQwfzLv5eTkoKWlRRV5R9oWWZbR3NyMTZs24YMPPsA333yDzZs34+KLL8aiRYs4c3x1dTW8Xi9mzpzJJMzlcrFGa8mSJbj55pvR2tqKNWvWwGAwYN68eejXrx969OiBhQsXYvz48dizZw9mz56NdevWwWw244orrkBTUxN+97vfYcWKFdi+fTv27t2LBx54AB6Ph01X5OtFfiukabnssstQUlKCDRs2cC3Gc889F0eOHAEA/OlPf0JDQwO2bNmCuro63H///ZgxYwYsFgvP6fnnn4+Kigrs3r0bF110EV599VU+8RMJEdNCAJ2arr179+K5555DbW0tdu/ejZtvvhnz5s2D2WzGO++8g969e+O6667Djh07UFhYiCFDhmDp0qUAgHnz5mHIkCG455578NJLL2HQoEEoKyvDe++9h1QqhWHDhmHz5s3YsGED+vbti549ewIAk//9+/fjmmuuwYcffogjR45gy5Yt+O1vf8smsGnTpuHAgQO47bbb2Emb6kDW1NTgiSeewJtvvomdO3firbfewty5c1kraLVacdNNN6FXr15obGzEtGnT8Omnn/Jc/P3vf8fTTz+Nuro67Nq1C42NjRg/fjyCwSDGjh2LnTt3sjN1a2srGhoaMGjQoG5zWWUSqnQ6jVdeeQW//OUvWfNCGtVgMIizzz6bs6AvWLAAGzduhNfrRSKR4OeAzNViuoqioiLMnTuXNXEej+eElB5EaCsqKnD48GE2pYoaXJPJBK/Xe0KNQdEXKp1OIysri9NL7Nu3D2PGjEE8HofVasWKFSvw2muv4ciRI+zcTr6VIsmj4Izdu3fDbrdj7NixGDp0KEpLS/HZZ58B6CT4lBZFDE7RoEGDhtOFk5IqEjpVVVUoLy9Hr1698OGHH3L0kizLeOGFF+BwONCjRw+MHz8e9fX17BtFTqEkZFOpFHr06MGE7OabbwbQae647LLLsGPHDi4FEwgEcOutt2LSpElwuVwYM2YM2traOPKQEiKmUik+BZOTcmFhISZMmMDlXc4880w0NTXxS5kiDinsnISMx+NB3759EY/HsXfvXvz2t79FVlYWGhsbUVxcjKKiInz66ae48cYbUVRUBKvVimuuuQZ79uxBa2sr+8qEw2EubUMkMRgMoqOjA21tbRg2bBh8Ph8GDRqEIUOGIBAIIJFIoLa2FjNmzMA555wDi8WCMWPGoLm5WWUGa29vh8/nYy0UaevMZjMcDgebasgPi7Kav/3229izZw8uv/xyLFmyBI2NjfjHP/6BkpIStLe34/PPP8c111zD83veeeehT58+2LZtG5tVp0+fjilTpqCjowOXX365ykeJNCvd+VW98cYbeOaZZ7jEz8iRIxGPx7FlyxZMnjwZ3333Hd555x307dsXR44cwZEjR3DvvfcC6CRVu3fvxpIlSzBlyhTs2bMHjY2NOPfcc2EwGDhQIBAIqEoU2e126HQ6vPzyy5g9ezbcbjcHOgQCATalUX9pPxkMBjb77ty5E9dccw0qKiq4lJDH48HOnTt57z3wwAOYMmUKAKC8vBxtbW3w+/3wer144okn8NJLL7EfE62VTqfDlVdeiWPHjmHv3r3Q6XTYsWMHxo0bh9LS0hPmLzO1hFi6hzS2drud14Cej5dffpk1nsuXL0d9fT30ej2ysrIQiUQ4mpISvoraRpqbeDyuqlpAzzYRstLSUk4wK2bfnzx5Mj799FMMGTJEVX1ATB0h3lOSJD5k7d+/H/369cOuXbvw9NNPI5VKobm5GQBUuehorSnaePfu3fj000/xwQcfYPv27Vi9ejUef/xx7Nq1C3q9nveKyWTSzH8aNGg47fjxzI3o9OlZtGgRrr76ahbUlKeJ/KMoVxT5UZBvBJ0G8/PzVYkOk8kkOjo6sHnzZpxxxhkcDabX6/Gzn/0MOp0O7e3tcDqdOOuss9h0MHr0aIwePVqV44dOxBTaTj5dDQ0N+OMf/4iVK1dyyHdVVRUSiQQ7xpMAJVMERf95PB4EAgGEw2FUVFTg22+/hU6nQ3FxMcLhMNrb21FdXc2kzGAwwG63Iy8vD7FYDLFYDNnZ2eyXQ8Lb6XQiJycHiqJg69atuOKKK/Dll18imUyib9++UBQF/fv3x9dff82ReKQNoXnT6XQoKChQ+eEAat8sMQqS1oBKDDmdTs54DXSVkSGzYY8ePTi6j4gzaapcLhcmTpwIr9cLj8cDp9OJ2bNnc0QfzSMJf9EXLBKJYOzYsTAajcjLy0MgEOAACIqwo3scPXoUxcXFqpxhVPvO7XarojRTqRT7jNF6kraSIibb2tqQk5PDWiiDwaDKkk/jdblc8Pv9KlJYW1uLV199FbfeeiuTLZ1OhxtuuIEJEpnE7XY7kwKXy4UjR47A7XbDYDBwmRnRFy6VSmHo0KE4cOAAysvL8cUXX+C222475Szusixj165duOSSS/gZsdls7Eg/ZcoULFmyBN999x1sNhuefvppFBUV8fMwa9YsfPTRRxwk8u6776J///6q1CCUyoB84sgvksaRlZUFg8GAvXv3oqqqiiMu6TnMrAEIdOWwUhRFlTSY/AfXrl2L5uZmJpvfffcdLBaLSltGOcookpZSd+Tk5KC6uhoOhwPJZBIjRoxAaWkpa8fooPNDme01aNCg4V/BSUkV+R6Q0BYj7AwGA6xWKzo6OlBcXAxAXbwWAIdk02d0otTpdKisrMTChQuRl5cHWZZZs0LlMMTSKpSM02QyqaLmukuyKEkSli1bhmg0in379kGv1+P777/HY489xkI+swQJCRKXy4X8/HzU1dXh6NGjGDduHI4ePYpt27Zh8uTJsFqt7K81evRong+xfAidpEmoiCRDlmUMHDgQ06dPx/333490Oo21a9ciJycH7e3t7EgspgIgx2aK9hNzU1Geo+4ix8hcQqBIPjJ30rXUt0gkgmPHjqFHjx5wOp0IBoOor6/naMuWlhY2AVN7os8KgfouFsf2+/34+OOP2cmcxkZRpCRkY7EYcnJymCBQ3ilKi0HFunkDH9fIkNCmuaHAA5fLhezsbK7rKM6HmHzV6XTC5/OhT58+bHomx/AFCxagurqaBTmttZjOgyLpQqEQsrOzYTAY4Ha7OWKV/AyJrFAfJ02ahJdffhnl5eUwm828dzITZmaaqmjuy8vLeT7EwsW5ubm4++67WYPW0dGBw4cP4+yzz+axvfDCCzgZiEgRUadnhrKvh0IhBAIB9O3bV9VPMt+JpZnEDO1UfooCGyRJgs1mQ0lJCQYMGIAlS5bwntqzZw9KS0tPSPoqltAxmUzIysri/pLWkaIExWhdkVSfTohBIho0/G8A7Uk6wIjJrLX9+j+Dk5r/KDIuMwkjaXhEB2gSHqJQLywsRFtbGw4ePAgA7CTbu3dvGAwGrFu3Dl6vVyXsiBgRgSCillknTEwwCIDD8n0+H/r27YuCggJOVvrqq68iGo2ivr4ekiRxnTfyQaEcQ36/n3PdmM1mVFRUYNOmTaqT96WXXoonn3xSJdxpzGQSSqfTKCgoYKLn9/thMpnQ1NSE5uZmHDhwAAcPHsThw4dRWVkJoDMajQgICSAylRDpIRPUxx9/zBoG6puY/FCcF1Eg0gNG19J9cnNzcfbZZ+Mvf/kLGhsbodPp8Nlnn2HAgAHo27cvpxcIBoMs7CiC88cQj8fhdDoxdepUzJgxg9MX0ANNecQAsKAU0wtQFCiBQvPtdjubbCmEn7RZpDWjyNTzzz8f9957L4LBIAthWmci2clkEjabDYFAgNMxSJKEyZMn44knnsDevXthNpvR3t7O4xezv1M/HA4HQqEQYrEYm5JfeeUVtLW1cZtEIv1+P3r37g2z2YzHHnsMU6ZMgcfj4YSkpwKHw4HDhw/zulNk6ZQpU/DMM8/g/fffBwDs27cPdXV1KC4uVpFw8bmmgA0C7ZPc3Fx2aidzW1ZWFkd+kpaOcnfRmq1ZswajRo3C/v37YTab+XkhUk7PJq0pAPTt2xc2mw0PPfQQAODYsWNYuXIlxo8fz8TebDZj0aJFKC8vxyeffKIq4PyrX/0KL7zwAvfz22+/RW1tLWw2G6xW62k3+WVWUiCzrJZYVMP/BtB7gfK1kRIimUx266ah4V/HSUmVyWTCtddei9LSUpSXl6OsrAyrV6+GxWKBz+fj/EdAp+aBTCiUgM9oNGLhwoW46KKLYLfbMXHiRPaveP7557Flyxacf/75yM7OxpAhQ/DGG290duy45iE7O5sJg8iqKdSdzDwAOKIpOzsbQ4cOxapVq1BZWYny8nKUlpZCr9ejtLSUBTAJfIo+CofDyMrKQkFBAebOnYsLL7wQOTk5cDgc2LhxIwoLCyFJEi6++GJcf/316NWrF8rLy9G3b19MmzYNkUgEoVAIM2bMQEVFBa688kr88Y9/RHl5OZYvX462tjaUlZXBZrNhyJAhqKio4Ou/+OILzm1Fddzo5awoCux2O5vZLrzwQo4uLC0txYIFC9DS0sIELBqNqsr5yLLMfiuRSAQFBQWQJAk+nw/JZBItLS0AgFmzZsFoNOLSSy/F4MGD8cILL2DGjBlwu93w+XzIz89XOTmLhYJ/COQw//Of/xx33nknKisrUVVVhT59+mDGjBlMlH0+H1paWpCdnY3W1lYW+jQGr9fLfj9kVjUajYhGo3j33XcxcuRIXHLJJdi0aRPOPfdcXHDBBbzPLrvsMtxxxx0YOnQoz/umTZsQi8Vw7NgxXH311Rg+fDiee+45zJ8/Hz/72c+wfPlyAEC/fv3w5JNP4rLLLkPPnj0xfvx4XHLJJWhpaYGidNbQIy0t+elQmL/BYMCcOXPQo0cPjBw5Er1790ZJSQlqa2sRCASQlZXF/n4dHR2c4sFoNLL/0Mnwq1/9Ch9//DH8fj8ThuzsbOTl5bFTfXFxMZ544gm8/vrryMnJ4b1BByI6uJAGs6GhAXPmzGGH9aVLl2LSpEmYM2eOKs2DwWDAZ599hsLCQvTu3ZvJJhEzWZbh8XiYNIvXxmIxrpBAaGlpQU5ODp577jkcPXoUvXr1wqRJkzB//nyMGjUKoVAI+fn5LBT0ej169OiBpqYmAJ1kraysDLfccgv69++PiooKzJ8/nyN1gU4/OzE/2ekAHWZIc0YHNE1oafh3Q7QAiImARSuDhtMM5SQIBAL8cyKR4J87OjoURVGUaDSqKIqiBINBRZZl1bWxWExRFEVJpVKqz+vr67v9XjKZVNLptBIOhxVFUZR4PK4Eg0G+N7WTTCZV13u9XiUej/Pvzc3N/DO1Rf8risL9TCaTSiQSUfWB7qcoiuLz+U4YE40nFAqp2k2n0yd8T5ZlJRqNqsb/xRdfKNOmTVOCwaCSTqcVWZaVTZs2KTfccAP3OxQK8VzHYjHug3gPcV1+CN2NWVEUpb29XbWW1DatpSzLSjKZ5P8VpXP+29vb+fter/eENn4I4vhTqZSqL7FYTDUu6oOidK4FrQtBHLc4Pvo5Go0qsiwr6XSa7yu2Qdc3NDTwZ62trdyHVCql+P1+RVE654T2VeYeFttsbW1V7RsaA+0R2mPUrizLvMZtbW3Ku+++qyxdulTJROY9u0N9fb1yxx13KGvWrFEUpesZovv6fD7V/Ij9iUQiP7qPxHUKhULcltfrVfx+vxKLxZT7779f+eijj1TX0b7w+/38nqA1URTlhDUVn5FYLMZ/TyQSvC/FvUdrEo/HuU36u/jcEBobG5VIJKLEYjHVPPxPIp1On/LzoUHD/yRkWT5BjqVSKdW7VsPpw0l9qsQiyeTXA4DNZ/S76ERK/hZiIWOgK9lgSUkJgK7itQLBUzm0kgM6gbQXmap1p9OpMvd4PB52jrbb7fD5fMjOzmbfDYrCEstekEaE+pNKpeByuSBJEmdzF82aYuZqGptyPFWEWIhZrL3m8/nQ2trK2iSHw4F4PI5vv/0W6XRapUUgfxCLxYJUKoXW1lYUFRVBlmUEAgEeD82PctyXiE4joj+S1WplLR6FyAPg0z75qFBfQ6EQ18OjeTEajXC5XOyI7HK5TqkYLhX6Jb8cmi/K0k7RgGKWeRq3mBMpHo9zslYaEyXIpMAJuheg9kES+0l/Ly4uZl88cnQHwE7WdB35/JAzPd1PTGQqXk+O3DRn9H2gM38U7UUq4xONRrFs2TLcf//9AICmpiY2W2cWuu4OJSUluOuuuzBv3jyMHj0ahYWFHChCPo+0H0RfJECdOFaMzBPrV9Icmc1m3vPk9/X000+jV69eGDduHEciSpLEe5f8lpTjpl5qT/S10+v1vMfFhLI0fzTHtGcpoEA5HihDefQ8Hg/7UQHgIAWj0YgePXqcMG+iD+e/AtLwZdYJpXnUzIAa/p0g30YCPafie07D6YWkKBkpmk8CehHSy49qeQHgF3BmlnDxd/oOOeKGw2F+sRKU4+YIcoynch5imDfZhkXneVGoisJWdEqn+xLZoygqg8Ggul4kg4R4PM6Fein/E6CuqyaOWyRXhFgshmnTpuHjjz/m7NDjxo3DsmXLVPNHxImEsDj/NBYaIxFEMfqP2sp0RKRxxeNxyLJ8QqSZOG4yG2ZmbRcLOYtj+yEcPXoUhYWFAMD18KhNErji+maCnPVJlU3kW3T8FkHjj8VinFZB3CtUysdoNPL8krk1c81F4k9mJyJ+Yl05MlFn7tXMftH4Dx06hOuvvx7hcBjLly/H2LFjec3EeTkZaM99/vnnWLZsGV599VXV4YeiPzPbEotH0/fF3Gw03yIpFOekqakJL774Iu644w706tWLyQP57NFepPp+RLYy+w10BXmIe5v845xOJ2eeF1MxAODPqX90oKD8bESCxaz1mc/o6YIYtADgv/V8aNDw/wW6k0+Z7zsN/zpOSqooQuyHWK1Y5iKTQJCwV45HG1HElHLcATYzRJoc1EXBSVokcmAW70malkzCBkAl2EStlSisiFCRsKVrMgUwvaTppA50CVtyrABzvgAAIABJREFUzhdJnpiiQZIktLS0ID8/H4B6I5PDMgUC0Iv5h9IkJJNJPqGTNilzHTJBmb/F+4hrKZIkUfBmhpzT3NBaZkYc/hjonvF4nIlkJBLh+4mkTexfOp1WRfJlgtZAFK6Z8yCOSyQIpA2h/SKuuagZIw0kaVHk4zUpxb2QSCRYO5T5nNB4xH6Ic05rQ+tP+1ks7H0qc5sJkUy1t7erIioziZJIiMjZOpOIkb8W9emH7ivOqzhmIlc0h5FIRJW4k94hmeRL/Ls4J+IYqC/ioUj8rhhJSQl/T9cpvbvnjt5fAP5HCJwGDf8dUACJ6POYKas1nD6ckqaKiJBYAFdMZQCoNUNU8Z5Mh6RVyHyRxeNxRKNRlSaGukMvd5FUiA7rFPZOL23qCwlAiiyikH2bzcYvXXJ4ztQgUXkNkdSJbdNckBaJ2stk/JnXEyikVTS5iZ8RcaECuyK5EE1xhExNnWh6oN8zBbo4T3SSpr6ImoJMkNqY1kZck1NBpvZGXFu6v2ju7O6B704zJe47UUOTqeETCZgkSSpyQFFsmdoF8X5koqTriGyLJiegS4BHo1HO4SaOn/Zi5hjp8CBqvE4VHR0dHDVIpDtzbAS6P4HWkLTOIuEgokcReuL1tMfpvUCf07MgEmk6LIla4cz+iRpBMmNTNK8YfUyBCmQOpHmk68Vnm7K+k7ndbDZzEl6n0/mj2tFThZhe47PPPsPKlSvx0Ucfoa2tjfeyBg3/LogBKRTV7nK5MHnyZFxzzTUYN27cv7uLPzmc9BhFgptyvwBg8x2F+gNd2bRDoRDnkiKGDKDbU6/BYEB2drYqzJk0VQBUtfzIr4USFxLoxZ4pCEVYLBYoigKHw8GpEjKFFhEqErrdEapAIACdTscvb0owKBY7pjFQZBFp5QCwCYNMpqIfFtAVNUXJFclURA8E+b2QsOvOvEdtAmD/I1FbIJJgRVFUflmZGqPM+QG6IqZIqyD2W7yGwuepL+RnA3SRS9Ig0t6hdRLXl0x1NA9UR5D2BWX2Jg2HWF+OIGaWNxqNXCCa5l7M6C1qGVKpFPfFYrGcMCfiHqI1IcJBUY8E0vLRNSToCeFwmM2gdH/xfmSuJVAqEACqAt8mk4mfm0zNGZn1xD6LyTkzn4tMsyDQ5UNIJIn6Srns6P7UjjgP9JyK+5P2nngwEHNakQmSDgdAZ9UDsV4gJbCl62ifUTJY6oeYY4/GQHNL/aP5ETXHPwYxl9b111+Pv/zlLzh69Cjy8/NV66tBw78D9H4XD/J6vR5/+ctfcMUVV2Dr1q38XbIMiL9r+L/ASRzZOZqIosAyI+8IFOmkKOqoLfH7YgQCRbrRZxQJRRCj+aLRqCqaJ5VKqf4uRo+lUin+PbOv9Hk0GuUIqMyoPYoaSyaTHJnW3t6uuh8hmUxyhE9HR4cqAkxROqOrxDE3Njbyz16vV9VnMdJLlmVVW+l0mvtJ7Yn9CYfDPB76mxh5lBk1KUZSiQiHwz+4vkePHlX1R5blE+YwFotxRIm4XolEQtU/MUKNEAqFTojMorHSfFDb9H8qlVIikQjPReb14u/ifIl7NbOv4XBYFTmYCfqcIlPFSFKxz+J9aD9lQpblE9YiM6pUUdQRjy0tLfwz7VERYsQigSLw6J7i+Gmf0P+Z14vrRs9dMplU7VkxGlb8Lo1ZbIN+Fvsh7gXxZ5/Pd8I+ESOZfD4fP2M/FGknvrdkWeZ5pfcP7eX/W8TjcSUSiSi//vWvFZPJpMycOVMZMWKEas41aPjfANrr33//vTJ37lzFaDQqN910Ez+H4ruku2h2DaeGUzL/Ufb0TDU9lT2hnDuA2lmdfE8yzWPi70qGCUn8PdMcJfq6AF3mLdFcIjqMysfLnBA7p+SGZMIRHWTpRP5jZpd4PM5lbOh65bhmRTSpZJpYMn1YaFzUb/qdIomoFBCBIp5CoRCvw6n4apCJjdrPNHeIWeDJeV0MGBBL+dDvpGUT26E1S6fTXF6HPqd/mWY7MqM6nU42O5H2CujUIpBm0GazneBonJlxXPTLoevJbETXku8TrXEymUQoFILb7Vb5vdF3aV0pH5uorRPXVzRlUn62TAdQ0upIksR9FOea8n/R3iSVPc1zd6ZZMj/HYjHWktG805xSX8n8RT5q4jNIP4s+SYFAAFartdv9kgnR1CruARqv2He6F5mTxSoI4vfIp4+CJUirJQa2dNcfv98Pp9OpMvuLUBQFra2tvNaZ7zOXy8X7JNN37sfQu3dvhMNh/POf/8SoUaNOOc+YBg3/kxDfp6Kc/P777zFp0iQkEgnU19fDbDar5NTpMI3//xUnlcz0oqdJF52sKcQ+KysL4XCYEx6KCSvFhH9A54tLzIBMwpPMGcrxaD0ATAQo3NpoNLKjLfUhEyJHJIFMqk8y/dB3aNNYLBYW6lTzj8w+ZPaksROhIsJBpk4AXMyXMnNXV1fj5ptvZidboJOgislSxUi4VCqFlStXYs2aNZytXJZlZGVlcfoBq9UKnU7HWcmpn6lUCuFwWDV+SltADwqRlng8jptvvhkjRoxAWVkZnn32WciyzMKF+kplRihZotVqhd1uZx8SIrBiNm9R8BPZoFD7UCjEhEKn0yE7O5uFpRgtRlFcLpcLDoeDiSHQZWokQkV19ygbPqXhIFOgmMWcyDXtG8qs39bWpiJUXq8XJpMJNpuN6zgSIQwGgyyofT4f9u3bh8svvxyFhYW48soreTzpdJoTzNJ46X+LxaIiVF6vF/fddx9Gjx6N/v37Y86cOarUBDT3NN/hcBjxeJz3ssViYfMbmRGJFIjkhg4aynG/K7EiwIYNG7B48WKuYuByuWA0GvngIFZOoD5TNCwRKq/Xy+VmqHAx9Z3MeiKhkiQJtbW1uOWWW5hQeb1ePkSI80XvDLvdzqZYen7b2tp4TuhwpxxP30F7mfZPPB7HsmXL0NjYyH5bqeNFz2lP0b49FUIVCAQQiUTQ1taGYDCI4uJilTmWnhPtn/bv3/FPDDiJxWL8TiopKUEwGOTDFr0PCXTw0vDfx0lJ1e9//3v07NkTJSUlqKqqwsiRI/Haa6/x6Zf8qux2O5/yRK1Ea2srbrvtNhZkDoeDfWkcDgc6Ojrgcrk4zQG9RIGuvFXkbAdAFcVAL8FMrY34O93XbDZzJm6R6NHGI78cl8sFu93Ojq5EYkT/EyrDoSgK2tvbVWHk9Dl9t6CggAUgEVTy1yDtGG1eyiB/8OBBlSMu0FVDMZ1OY/Hixfj8888575Ysy1zUmcZMpU5IkNIJPx6P469//Svy8vLwySef4MCBA5g2bRpr/ICuCK9QKIRQKIScnBwcOXIEQJedXZZlFVEQ/ZLEuSWClE6n4XA4VNoRESS4rVYrZyQntLa2Mlkj3ygivg6Hg/dLVlYW+wUQoSD/IlEDRJ/T3s3NzVURdMrBJta8I+2P0+mELHcKzOzsbAwYMABvvfUWvvrqK+Tm5uLYsWMIBALQ6/Ww2Wyqk5+4L5PJJN9zw4YNkGUZ27ZtwzfffINHH32UtUoWi4UPHPQM2O12BINB1cuSUjyIRIDyk0lSZw1NeoHSWpG2J5lMorS0FF6vl8dK60OaTCokTeVf3G73CT5YtN9isZgqdxfQVZKGxi5JEjZv3owFCxbgrrvu4n3hdrvhdDoxZcoU9OzZE4WFhVi/fv0JfpSSJCEej3NBbTr4AZ37cc6cOTjnnHP4cEL7OxaLcYUD2mOkWaT1pvfLqZS0cblcsNlscDgcrA0gMkp90f5p//5d/+gQTe8NeifRs0/RuGLAxSkYrzT8CE5Kqnr06IFly5ahrq4Oe/fuxVdffYWrrroKQFetNgDsOEovP52uU/1OxIsczYmckIMxnUjFumM+nw8AVOY4ug/QSRpEJ+LuItAURUFbWxtrFfx+PzweD1pbW1XkSKfrynFFoFM2ORen0+kTzHnUp5ycHCZmOl2XViaZTGLNmjV45JFHWGAcPXoUQJcDOaUCENWs7e3t7GCcPl5wlsZjsVg4dNzn87FQ0+l0KgJD9RplWWZzFJE6i8WCxsZGDB8+nAUfzQe1Te04HA5Vskygy8Gaoj/pOlFoio70hGAwqHrARSde0ljSgw6A90UkEuFyOMpxB3VKRCmmQ6AXgVhPjtaQygcBapMbzTPNGX1GY6BaiTRuCpKgMWQ61Pv9fhQVFZ1gchJrYiaTSS5qTd/bt28fxo8fz4lrCUQenU4nmyKJYBERJK2UGGlIZEM0xba3t8NqtaqSk/6f9u48Osry+gP4N5NlZjIJk52QMJCFhCQQQNbaoOyFA7IpICIuFfCwSLEiKAUOYCkULVCkHvRwxFKCKGVRtOKhQmmVskRkJyEYCISErCQzWWaSSWZ+f8T75J2AP5BOD/3j+zmHwyEkb2beeZf7Ps997tUG4xUVFaioqFBTrbJftFPCLpdLjcJIgC+9BrXkc5R9KZ+pXC/ks9mxYweGDx+O7t27w9fXF+Xl5bDb7fjlL3+Jxx9/HIWFhThz5gxWrVqF48ePA4A6f1sXMpUAqLCwEEOGDEGvXr3UAw7QsvijTZs2GDduHEwmEw4ePKiuNUBLb0w5D+5lpMrlah5trqmpQUREhHptck4SPUhyPMtIsfRJBVoGH7QFbB0Ox4/22qV7c9egqry8XN2YdbqWUSIZSpw/fz7Onz+PX/ziF0hOTsbKlStVMDV58mQMGjQIx48fR9euXZGQkIBRo0ahqKhITV/Nnz8fCQkJSElJwYYNGwBA9X+bN28ejh49igULFmDQoEEYP348jh07pp4KtSuJAM/RE6vVisuXLyMxMREdO3bEmDFjUFxcrJ6wAXiMgMkURk5ODn73u98hNzcXQEspge+++w6zZs1CQUEBTCYTtmzZgqSkJLRv3x5PPfUUrly5oqbKdu3ahUGDBsFiseCNN95QNzGLxQKXy4Xc3Fw89thj6Nu3L+Li4hAWFoann34atbW1CAwMRJs2bbBkyRJ07doVcXFxOHnyJNxuN3bu3ImYmBjs3LkTr732mupp+Oabb6oATZ6W5Y9er1cBhU7XPKJUXFzsMfWjDUreffdd9O7dG23btkVGRgbOnj0LoLmx7bx583D8+HGsW7cOaWlpGDJkCE6dOqX2ZesnHJnCKioqwpEjR/Dll19i9erViI2Nxc9+9jP8+9//BtAcvJWWlmLZsmWIi4tDx44dsWLFCo/cslmzZuH48ePYuHEjIiMj8eijjyI7OxtAcwC0a9cuxMfHIyEhAV26dMGFCxc8aitNnDgRcXFxSEtLw9SpU1XQ6XA4MGvWLFy6dAl9+vRBdHQ0Zs2apd6LwWDAl19+CYvFgpSUFKSkpODcuXPq90pgrg14tblXLpfrtiKQcrO22WxwOp2wWq0wGAweeTgOhwPff/89Fi9ejBMnTmDgwIFIT0/Hjh07VEBqNBrxyiuvICEhAcnJyVi8eDFcLpea2rt+/Tri4uIQHh6OF154AUVFRR7133bt2oWYmBikp6djwoQJqi+mTBuPGzcOXbt2RUxMDJYtWwabzYaKigocOnQI77zzDiZOnIjx48fj22+/RXp6OtavX68CCXkgAXDHc/TEiRNwuVx4/vnnVWASERGBK1euwOl0YvDgwQCaR66mTJmCb775BjabDZGRkWrKXPosanPR9u/fjx07diA9PR06XfMIszzwyTkQEBCA0aNH4+9//7tqDC2Bvdvtxp///GdMnToV58+fx93I9iQHT/vUL9ciogdFHhwlP1Pu4zJTAsDjoV6O59a5znTv7nrWt2nTRj1pAi3TPwaDAQ6HAzdu3MDKlStx4MABZGVl4dy5czh//jyMRiP27duHY8eOoXv37jh37hwKCgpw4MABmM1mmEwmbNy4EXFxccjLy8OZM2dgs9mwd+9e1NTUqNGJZ599FhkZGfjXv/6FXr16qRvxnYYptVN0NTU1KCgowOXLl3Ht2jW8+OKLWLFihfpZ+X7t6IjL5UL79u1RWVnpccBJgrbZbEb79u1x6NAhHD58GFlZWSgoKMDEiRPxzjvvwOFwIDw8HCNGjMCZM2fwpz/9SVWNF7W1tVi3bh1effVVnDx5Etu3b0dGRgZWrFih8kDWr1+Pn//85zh16hTWrl2L7du3w+FwYNKkSbh+/TqeeeYZrF27Frm5ubh27RoWLlyoWnfIPpCTSJJ66+rqMGnSJCQnJ2P//v1YsmQJ4uLi0L9/fxQWFsLHxwe7d+9GQUEBjh49ipKSErz00ktYtWqVR7Pf8ePHo0ePHrh48SKGDRuGb775RgVtcmzIiBLQHLDFxMTAarVi5syZ6Nq1KwoKCrBjxw6sWbMG5eXlcDgc+PTTT+Hn54fCwkJkZ2fDx8cHu3btUnWJgoOD8dRTT6nRxiFDhuDAgQMAmkcvPv/8c5w9exaXLl1Cbm4u0tPTodfrcfPmTSxfvhyTJ09Gfn4+jh49iuTkZCxatEjlV5WWluJXv/oVDh48iOvXr6OyshJff/01gOZgsra2FgUFBcjJycHWrVvx5ptvquROyT2QJGdtbqCM+mhHG2W/ZGdnY/bs2UhKSsLu3bsxd+5cjBw5Er1790ZeXp6qG7Z//36sWbMGu3btwr59+7Bz504UFhYCAFavXo0OHTrgwoULyM3NhdPpxPbt29UT6ezZs7F//35UVFTg17/+NZ577jkVdJ89exYffvghcnNzce7cOezduxdutxuVlZUwGo14+eWX8eSTT+Ly5cvIzs5GTU0NNm3ahPDwcBQVFeGLL77AG2+8gU6dOmHv3r3IzMzEpUuXVHNuoOVJVwIfAGrK9ciRI+jfv78KZCQP7fTp0+jbt6863vbs2YM//vGPKC4uvi2XT0ZftQV/n3nmGURFRakcOh8fH5VnpZ3ijYmJQWBgII4ePapG66QciclkQllZ2W0jjneiHR3WjrgyH4X+F8gIudSh0+l0Ku8Z8Jyt0RZa1n6dfpq77jW3243FixcjISEBiYmJSElJweHDh5t/+Icn9HXr1sFsNsPf3x8DBgxQ+TeyuspsNqun68bGRphMJtTW1iI/Px/Tpk2Dj48PgoKCMHr0aBw6dAh+fn4oKSlBSEgIpk+fjnHjxsHf3x/Dhg1Dfn6+mj5x/1BoVLYrX6upqUH79u0xadIk6HTNK8IeffRR2O12lJaWehws2p/T6Zpr9cTFxcHpdOLq1auYOnUqnE4n8vPz1QjZiRMnMG/ePISGhsLHxwePPPIIysvLce3aNTVFArRMKZpMJjgcDjQ2NqK0tBR5eXno0aMHjEYjkpOTYTQacePGDQQFBaGmpgYzZ87EyJEjYTAYkJaWphKAZbrG5XKpm7eMhAAtQ73aBHbJ5dHr9di9e7cKhtauXYv8/Hx8/fXXiI2NRUVFBTIzMzFlyhQEBgaiqakJI0eORGhoKPLz81We0CuvvIJRo0ahsbERgwcPRmFhocpNkTl87Unr5+enVofOmTMHEyZMQH19PTp27IiQkBCUlpbCYDAgJycH06dPVyf2iBEjkJWVherqapXo/PLLL+Ppp58GAAwdOhT5+fmw2+1oaGhAdnY2CgoK4O/v7xHYVFRU4MSJE3jssccANN/8xo0bB6fTicLCQrjdbhiNRrz33nswGAzQ6/VISUlR76lt27Z44oknVD5eamoqzGYzGhoa1HuUoElbX0n7lKctmir/n5qaiszMTFy5cgWDBg3CunXrcOrUKfzzn/9EfHy8mirs0KEDVq5ciaioKLRr1w7p6emorKxEVVUViouLMXPmTPVwMX78eGRlZUGn02H//v2YPXs2OnfujKamJgwcOBBJSUk4ffo0fH19sXXrVixatEiN8MhDUmBgII4cOYLq6moMGDAAdrsdZrMZM2bMQG5uLmw2GwIDAzFo0CDEx8ejrq4OL774IsLDw1FWVqbaEWmnd+Uck4cLl8uFq1evIi4uzmOqW/aTwWBAVlYWOnXqhCNHjuDAgQOorKyE3W5XxzIAj+Nem1bgcDhgNpvVQ4b2IUoEBAQgMTER169fV7mJkks2ceJEHDlyBB06dMDdaKegpT6XdlqZ6EGTdABJX5BrtOSbamuzyXErDzD00921Tr3D4cD69esxduxYj9yI2tpa6PV6j6dQWWkkNzV/f3+VLCtD/HIDysvLw/nz59G5c2cEBwerXIQJEyYgICAAERERyMvLw/Tp01X+RL9+/dClSxePXAdZTaQ9SCSomTFjBo4dOwar1Yr6+noMGTLEo0VK65VqMv0m/cYKCwvx8MMP4+LFiwgLC0NqaioqKytRWlqKSZMmqWF+k8mEiIgINeQv7WeCg4ORk5MDoKV4ZGxsLOLi4vD5559j2rRpuHnzJux2OzIyMlBXV+exjF0aU/v5+aG8vBwxMTEqJ6Zdu3YeZQXkM5DRKaDlZqZ9jzKsK+/XZDKpXBOTyeTRF1FGFHU6nVrV1adPn+YDx88P/fr1Q/fu3T1ah2jLVQQEBKC6ulqVg5ApR0nittvtsNlsOHnyJA4dOoRPP/1UJVHm5eXhpZde8ii6mZqaqgLihx9+WFUDjouLw7vvvovhw4ejqqoK8+fPx/Lly+F0OlFeXo5u3bqpVV5ysSgqKkJ4eLgKJLTFUENCQlQOAgBs2LABmzdvVu9JegXKZy3HjLRCampqUuUJ7nZzra2thcVi8VhdK8Pzer0eCQkJSE1NVcfQ8uXLYTabcebMGVy4cAFJSUlqZWBTUxMmTZoEm82GwsJCrFu3Dr/5zW9QVFQEi8WCxsZGNc2s7TggU2R+fn4q2dzX1xdRUVGqLIO8roaGBpWwr9Pp1Ao7k8mE2NhYVZpCm4clgYeM/DgcDnXcu1wuj30UFRWFmTNn4oUXXkBOTg78/Pzwj3/8A+3atVMdCFavXo2PP/5YBdyLFy/GE0884REg6vV6WK1Wdf0JCgryOLb1ej3Cw8NRUlKi8julAPCPlcW4E+0xYbfb1bVFHmY4hUIP2p16Y8pUtVzfAdw2oq5Np6F7d9egSm408oQoT8U6nU5VVZcEdLk4atvTlJWVwd/fH2azWdVC0uv1MBqN6Ny5Mz755BOPG5LVaoVO19xzz2w2o127dh6vIygoyKOvl1y0GxoaVCVrAFi6dCnCwsLwxRdfwGKxIC8vT62qkuBPDhz5eUmO7ty5Mw4ePAij0YiePXuitLQUFy5cwPDhw9US/I8//hi9e/eG0+mEw+FAcHCwqp4uCa92u11N/0mwJ4HYmjVr8Pbbb6O6uhrbt29HdXU1wsLC0NjYiFu3bsHpdMJsNqOsrAyVlZUeN2in04mKigq13yQxHWjJAZOcM1nxJCePVGE3GAxqSb6/vz9KS0vVijTJEbl58yYuXrwIoPnElFElSWz09fVVn4OUmBASsGqXqcvoWUBAAIqKimAwGNC2bVsAwMCBA7Fo0SJERkaipKQE0dHRHlXZ6+rqPFaUaXsuNjU1oWfPnsjOzkZQUBBWrlyJGTNmYPPmzQgPD1f5VXKjl+NVRijkhipTeZIr5evri927d6sRpOjoaBQVFWHJkiUqSVpazkiCv1yYZMrpbrSrFbVJ3DLKou1nWFdXp7YbGhqK2NhY7Ny5U02VyfaA5kT2t99+GxMnTlTvUY6LsrIyVfpBjkdZbVhdXa1Gd6S0QWBgIG7evKlWXEq3BJlSkMBB+xnL6JH2oizBvMFgQFRUlJq+k5yPhoYGmM1mjB49GsuWLVPBWGFhoXp4AYAFCxZg6dKlanvyO6Svn8PhQFVVFdq0aXNbz07t4pPS0lJ06NBBvRcZVdX2vLwbWalaW1urzlE5DxhQ0YOmrTfVusajLOSQY1+mCmXhFetU3Z+7hqEBAQHqxgW05CPJCiy5WMvUj/Yp1uVyoXfv3igpKVEJzXKTiIiIgF6vx6ZNmwBA3eDlplFeXo7w8HDcvHnTo0eerIIDWgo0yio5bdmEqKgoREZGwmKxoKmpCVu2bMH169dVhC7D/XLhlsTX2tpaVFdXIyoqCgEBAUhNTcXly5eRl5en6kVNmDABGzduxJUrV+Dv73/bgSoHpiT8mUwmNXV4/vx5hIWF4cyZMzh37hyuX7+ORx55BGFhYSrQk1EvGVlq27Yt/P39VR2w5ORkfPvttx41r2R/aBMMZRqidZNru93u0ebE6XSiTZs26NWrF7Zt26bqNJ09exZpaWmIi4tDYGCgOhakWKOsZHO5XKoUgnZVpnw+VqsVJpNJfbYOhwP79+9XN9ewsDDU19cjMzMTPj4+iI6Ohs1mU8v4ZSTNx8dHvW69Xq9WP0qwKX0dU1JSEBwcDKvVisjISAQFBeH06dNoampCbW0tPvjgAzz//PNq9EpKOch0tvwOyYfS6XQqcPnwww+Rk5OjahvJSkhZ0i8BC+DZSubHBAYGoqKi4rbPUobqZam/toWE5GuFhYVh27ZtAJpHvOx2u9of4eHh2Lx5M77//nuP81bOtfj4eOzduxf+/v7Izs7Gq6++iqqqKgQHB+Ohhx7C1atXcebMGVVS4aOPPsKYMWMQHR2NmzdvqoBWVvAaDAZUVlaqUTwhx6KsBpR92q1bN+Tk5HjURgsICEBaWhpsNptatHLjxg387W9/w9ixY9XxK8Gh7BegpWmsNtDXFq6V75Ebh5+fH65cuYL27dur0VGgOYd0+/btGDBgAPLy8u76+QEtrZJkFFlW2RI9aNoFIxIkSYqFjFJL8rq2vRVHqO7fXfecy+XCggULEBMTg4SEBCQkJGDbtm1qJZkUwJSLlU6nw61bt1BVVQWdrrno5euvv47p06cjMTERGRkZahnzH/7wB1y9ehU9evRAdHQ0EhMT8de//hUulwtt27ZFcXGxukBKMqu2zIIUBbTZbGq6JiwsDA6HA0OHDsWOHTvQrl07WCwWxMbGIjEx0aMXniyNl/yZwMBAmEwmdOrUCYsWLULfvn0RGhqK6upqVFVVwWKxQKfTYcyYMRg2bBgef/xxJCQkwGKxYM6cOSqYmDPNqYsjAAALIUlEQVRnDpKSkrBw4UK8//77sFgs+O1vfwugedqquLgYDz30kFqt1qFDBxw9elTd4GUqR1ZbVlZWwuFwqIKMQ4cOhcvlQlJSElJTU7F69WrcunVLFb2sr6/3qMXVukCoxWJRo4LymRkMBixcuBCRkZHo378/IiMjsWnTJixduhQhISHIz89Hu3bt4HA4YLPZ1FSadvhYbvxykkqyo9lshtPpxMaNG9GxY0d07twZJ0+exHvvvaeCrY0bN6KkpEQFw/369cOePXvg6+uLsrIyNVIpI18SFAPNyczt27dHmzZtkJ6ejm3btuG1116D2WxGdHQ03nrrLaxatQq9e/dGcnIy0tLSMHjwYPj6+kKv1yM4ONijWKWM2gBARkYGmpqa0L17d0RFRSE8PBxdunRRDXP37NmDjh07YtCgQTh8+DAGDhyIyZMno7Cw0KPi+4+pr69HbGysqizvdDrV/tXr9airq1PBgmyvqakJERERWLVqFU6dOoXExET06dMHvXr1wieffAIAGDZsGDZs2IBRo0YhPj4eXbp0wejRo1Xx0hEjRqCwsBDx8fFYvXo11q9fj+TkZJSUlCA4OBgfffQRfv/738NisSAtLQ1dunTBlClT4HQ6ER8frx5O9Ho9KioqUF5ejtjYWAAt0wzaaT1tn8r6+nokJSXhs88+U4V/tQ9oW7duVf3zhg4dinnz5qFTp07qvcvqRpkql1w2t9uNv/zlL0hOTsbQoUORlZWF3r17Y/z48bhw4YJHl4ITJ07AZrMhISFBHU9CHt7upV6PJMRLPiTQkk/G3n/0v0AegGUUvXU/X7l+Sz6g/JvNwO+T+y5a9wCS/kANDQ1up9OpetRJryttzzLp0eZ2e/b/ku9xuVwePdZk2zU1NWq7VVVV6v/q6urcTU1NHtt13aFvl7Y/mtvtvuPv0L6v+vp6j+/R9uVraGhQvdDkPVitVrfb7VY90OS9ymuxWq239eoThw4dcs+dO1e9BofD4T59+rT7ySefdFdVVbmdTqfqVybbkN5v2t9xp15zDofDXV1d/aN90ERZWZnHv2/dunVbjzf5W7tftP3r5PN3u5v3n8Ph8Oi72HpbmZmZ7nXr1nl8Xft7Wh8LcozI17SfyZ1617ndnvtCvq793pqaGrU9+X+n06m2bbfbPY7fmpoa9dnLMaXtj9W6756Q/XGvysrKPPa/lrxOOQa0+124XC53U1OT2ob8re0Jqe1xJ9uSY03Opzv1DZT9Je/H5XKp/dW6P6J8XV6jbEuOCe3nXVlZ6Z47d647MzPTYxvyeWnPcaF9nXfqC6jdH9r9UFNT49H3z+12u9966y33Z5995rFtOR9bv/d7ERQU5A4JCXG73W53UlKS2+123/U8JPpva30Oas+Z0NBQt9FovO38FHc6B+nu7jpSJbkcstxYnuBlaF2mvuQpWpJ+JXdCEoC1UxvaIXtJmJVqz/LUGhQUBKvVqpJbZfm6jKoALW1YtEula2trERkZqYpr/hA4AmgugCjRuoxOuX8YAZNtaFtdyPJw+Rl5D5J4La9dti9PuzL9KVXahdVqVb3JtMVOjx49CpPJpKaSfH19UV1drZ4YgoODVY8y7fJYyf2R/SlPzDLkK9Nv8r7kZyQ3SX42NDQU/v7+qh2N5LdIv0R5fy6XC1VVVXA4HOrzd/+QkyVTcT4+PqitrVU5RtoiklarVdWG0hYLlVwc7bSRHDeyj0JCQtQUl4xeahPxpcK4fObS90/7vSaTCT4+PrBarern/Pz8EBISompFSb6dfL98BlLiQ/ICZX+Lqqoq9VplmvReSH0mo9Gopgu1y/Fl2lfOL+2Ul7wvmRo3Go1qBEcS8OWYvXXrlloMIsefJGbL+aQdcWw92iT1l3x8fFQiv7wO6ckpX5djQ/tkrP3b19cXISEhmDZtGvbt26cWcwDNU5Ny7bDZbCqvrry8XG1PRj+FVJSXz0WOAWl7I5+7XJu2bduG4uJiZGRkqNEkmWrW6/Wq9cy9ruCTxRiBgYG4cOHCHev/ED0Ick2Vc1v+/u6776DX69VqXblnC7kv0k93TxOncqOXBGe50EmSNtASfAEttTEAqFoxUoFZemxJjSgJSiRpLjg4WN1cpEyDNJuVat4AVGscWUkmgZ8kvxqNRgQFBaklpFL9HIDq5yZBj/xeWdEogUFUVJS6MErrFwAqKVUCioaGhtuSoIGWYE72kdlsxogRI1BZWYmOHTuiZ8+eSE9PR1ZWFt5//31ER0ervChJfJd9KqvgtM2ttbWoWp8QgOfNXZuLArQk9cq/Gxoa1Go4CVbkM3Jr5uRDQkLU16RPn2xDkpYlKNYWig0NDVU3rtbNcKWqvLYcgrSK0VbFNplM6kSX1w8036xlf0mQrw3oJFiVKVWZjpQVkADUsdJ629oCsXK86PV6VZlYAsKQkBC12hGAR4/B/4+0iQBw23Sh5KdJ4rTcvLUFXX18fFQeT11dnTo/ZR9L4BwREaGai0vhP23wJlMBUnlce9zI4gt/f3/V9kUuuhJEyWuQKQR5/YBnHqZ2kUD37t2xcOFCbNmyRU3xSoDY1NSkcrWkFY2cVzabTeVFyns1GAzqeJIHscbGRoSGhnq0BHI6nTh79ixef/11hIaGwmg0qnQEeUiTAOleSJCakZGB+vp6fPDBB7DZbPc8fUj03yT3Je2DWHFxMXbv3o26ujp069ZNBVraTiOsU3X/fNw884mI7pvL5cKJEyfw7LPPoqCgAI2NjejUqRNycnLUwwHRgyAt4qQItLRUCggIQHh4OL766itERUV5rNLWDorQT8egiojoPmnLu3z11VfYs2cP9uzZo9IZmOxLD5rD4UBAQACMRiNqampgNBoxduxYPPfcc+jatSuCg4PVqL5bs3qcdaruD4MqIqL/kNyMZIpY28ya6EGStAdtmsadCtNK+oY2t5KdAX46BlVERPdJgilppF1XV4ewsDD1f0QPkrZmoXaBEQBV9BNoKYejrfunHbWie8egiojoPt3pxiMtubSLPIgeBOkaol2Jarfb1eIw7RSfLCjh6NR/hkEVEdF/oK6uTrXv0TaiZT4K/a/Q9obVav1QIKtWmax+/xhUERHdJ+0UCtCShyJlXHhjogepurrao6abNG2XvD9trT3Ruu4i/TQMqoiI/gNut1vVrpMgi0vT6X+F1KqTAEpqVmkL+wIcWfUWBlVEREREXsDQlIiIiMgLGFQREREReQGDKiIiIiIvYFBFRERE5AUMqoiIiIi8gEEVERERkRcwqCIiIiLyAgZVRERERF7AoIqIiIjICxhUEREREXkBgyoiIiIiL2BQRUREROQFDKqIiIiIvIBBFREREZEXMKgiIiIi8gIGVURERERewKCKiIiIyAsYVBERERF5AYMqIiIiIi9gUEVERETkBQyqiIiIiLyAQRURERGRFzCoIiIiIvICBlVEREREXsCgioiIiMgLGFQREREReQGDKiIiIiIvYFBFRERE5AUMqoiIiIi8gEEVERERkRcwqCIiIiLyAgZVRERERF7AoIqIiIjICxhUEREREXkBgyoiIiIiL2BQRUREROQFDKqIiIiIvIBBFREREZEXMKgiIiIi8gIGVURERERewKCKiIiIyAsYVBERERF5AYMqIiIiIi9gUEVERETkBQyqiIiIiLyAQRURERGRFzCoIiIiIvICBlVEREREXsCgioiIiMgLGFQREREReQGDKiIiIiIvYFBFRERE5AUMqoiIiIi8gEEVERERkRcwqCIiIiLyAgZVRERERF7AoIqIiIjICxhUEREREXkBgyoiIiIiL/g/n2VDaoH/WdwAAAAASUVORK5CYII="}}},{"cell_type":"markdown","source":"# **References**","metadata":{"papermill":{"duration":0.005651,"end_time":"2023-06-29T13:14:57.422066","exception":false,"start_time":"2023-06-29T13:14:57.416415","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"1. Actionable Principles for Artificial Intelligence Policy: Three Pathways: \n https://link.springer.com/article/10.1007/s11948-020-00277-3\n\n2. Principled artificial intelligence: Mapping consensus in ethical and rights-based approaches to principles for AI:\n https://doi.org/https://doi.org/10.2139/ssrn.3518482\n\n3. Building trust through testing:\n https://cset.georgetown.edu/wp-content/uploads/Building-Trust-Through-Testing.pdf\n\n4. Toward trustworthy AI development: Mechanisms for supporting verifiable claims:\n http://arxiv.org/abs/2004.07213\n\n5. Estimating the brittleness of AI: Safety integrity levels and the need for testing out-of-distribution performance:\n http://arxiv.org/2009.00802\n\n6. The assessment list for trustworthy artificial intelligence (ALTAI):\n https://ec.europa.eu/digital-single-market/en/news/assessment-list-trustworthy-artificial-intelligence-altai-self-assessment\n\n7. Meaningful human control: actionable properties for AI system development: \n https://link.springer.com/article/10.1007/s43681-022-00167-3\n\n8. Coactive design: designing support for interdependence in joint activity:\n https://doi.org/10.5898/JHRI.3.1.Johnson\n\n9. Ecological interface design: theoretical foundations:\n https://doi.org/10.1109/21.156574\n\n10. Personalizing human-agent interaction through cognitive models:\n https://doi.org/10.3389/fpsyg.2020.561510\n\n11. Validating human driver models for interaction-aware automated vehicle controllers: A human factors approach:\n https://arxiv.org/abs/2109.13077\n \n12. Modular design patterns for hybrid learning and reasoning systems: a taxonomy, patterns and use cases:\n https://arxiv.org/abs/2102.11965\n\n13. The next decade in AI: four steps towards robust artificial intelligence:\n https://arxiv.org/abs/2002.06177\n\n14. A model for types and levels of human interaction with automation:\n https://ieeexplore.ieee.org/document/844354\n\n15. When Should We Use Care Robots? The Nature-of-Activities Approach:\n https://link.springer.com/article/10.1007/s11948-015-9715-4\n\n16. Value Sensitive Design: Shaping Technology with Moral Imagination:\n https://direct.mit.edu/books/book/4328/Value-Sensitive-DesignShaping-Technology-with\n\n17. Human behaviour with automated driving systems: a quantitative framework for meaningful human control:\n https://www.tandfonline.com/doi/full/10.1080/1463922X.2019.1574931\n\n18. Towards a dynamic balance between humans and automation: authority, ability, responsibility and control in shared and cooperative control situations:\n https://link.springer.com/article/10.1007/s10111-011-0191-6\n\n19. A human factors perspective on automated driving:\n https://www.tandfonline.com/doi/abs/10.1080/1463922X.2017.1293187?journalCode=ttie20\n\n20. Translating Values into Design Requirements:\n https://link.springer.com/chapter/10.1007/978-94-007-7762-0_20\n \n21. Meaningful human control as reason-responsiveness: the case of dual-mode vehicles:\n https://link.springer.com/article/10.1007/s10676-019-09519-w\n\n22. Design Unbound: Designing for Emergence in a White Water World: Ecologies of Change\n https://direct.mit.edu/books/book/4151/Design-Unbound-Designing-for-Emergence-in-a-White\n \n23. Human-centered artificial intelligence in education: Seeing the invisible through the visible\n https://www.sciencedirect.com/science/article/pii/S2666920X21000023","metadata":{"papermill":{"duration":0.005595,"end_time":"2023-06-29T13:14:57.433615","exception":false,"start_time":"2023-06-29T13:14:57.428020","status":"completed"},"tags":[]}}]}