|
Why? Without a doubt, the first cause is the gap between the research community and business sectors, impeding the full penetration of the newest ML models in sectors that have traditionally lagged behind in the digital transformation of their processes, such as banking, finances, security and health, among many others. In general this issue occurs in strictly regulated sectors with some reluctance to implement techniques that may put at risk their assets. The second axis is that of knowledge. AI has helped research across the world with the task of inferring relations that were far beyond the human cognitive reach. Every field dealing with huge amounts of reliable data has largely benefited from the adoption of AI and ML techniques. However, we are entering an era in which results and performance metrics are the only interest shown up in research studies. Although for certain disciplines this might be the fair case, science and society are far from being concerned just by performance. The search for understanding is what opens the door for further model improvement and its practical utility. 如引言中所述，可解释性是AI在实际执行中面临的主要障碍之一。无法解释或完全理解最先进的ML算法表现如此出色的原因，这个问题的根源在于两个不同的原因，图2从概念上说明了这两个原因。 毫无疑问，第一个原因是研究界和商业部门之间的差距，阻碍了最新的ML模型在传统上落后于数字化转型过程的部门的全面渗透，如银行、金融、安全和健康等。一般来说，这个问题发生在严格监管的行业，有些不愿实施可能会使其资产面临风险的技术。 第二条轴是知识。人工智能已经帮助世界各地的研究人员推断远远超出人类认知能力范围的关系。每一个处理大量可靠数据的领域都很大程度上得益于人工智能和ML技术的采用。然而，我们正在进入这样一个时代，结果和性能指标是研究研究中唯一显示的兴趣。虽然对某些学科来说，这可能是公平的，但科学和社会远远不是只关注成绩。对理解的探索为模型的进一步改进和实际应用打开了大门。 Who? Domain experts/users of the model (e.g. medical doctors, insurance agents) ? Why? Trust the model itself, gain scientific knowledge. Who? Users affected by model decisions. Why? Understand their situation, verify fair decisions…? Who? Regulatory entities/agencies. Why? Certify model compliance with the legislation in force, audits, ... Who? Data scientists, developers, product owners... Why? Ensure/improve product efficiency, research, new functionalities... Who? Managers and executive board members Why? Assess regulatory compliance, understand corporate AI applications... 谁?该模型的领域专家/用户(如医生、保险代理人)? 为什么?相信模型本身，获得科学知识。 谁?受模型决策影响的用户。 为什么?了解他们的情况，验证公平的决定…? 谁?监管单位/机构。 为什么?认证模型符合现行法律，审计，… 谁?数据科学家、开发人员、产品负责人…… 为什么?确保/提高产品效率，研究，新功能… 谁?经理和执行董事会成员 为什么?评估法规遵从性，了解企业AI应用…… Figure 2: Diagram showing the different purposes of explainability in ML models sought by different audience profiles. Two goals occur to prevail across them: need for model understanding, and regulatory compliance. Image partly inspired by the one presented in [29], used with permission from IBM. The following section develops these ideas further by analyzing the goals motivating the search for explainable AI models. 图2:显示了不同受众需求的ML模型中可解释性的不同目的。有两个目标贯穿其中:对模型理解的需求和法规遵从性。映像部分受到[29]中提供的映像的启发，在获得IBM许可的情况下使用。 以下部分将通过分析推动探索可解释AI模型的目标进一步发展这些想法。 2.4、What for? 到目前为止，围绕XAI的研究活动暴露了不同的目标，从一个可解释的模型的成就。几乎没有一篇被审查的论文完全同意描述一个可解释的模型应该强迫的目标。然而，所有这些不同的目标可能有助于区分执行ML可解释性给定练习的目的。不幸的是，很少有人试图从概念的角度来定义这些目标[5,13,24,30]。现在，我们综合并列举了XAI目标的定义，从而为本次综述中涉及的所有论文确定了第一个分类标准: (1)、Trustworthiness: several authors agree upon the search for trustworthiness as the primary aim of an explainable AI model [31, 32]. However, declaring a model as explainable as per its capabilities of inducing trust might not be fully compliant with the requirement of model explainability. Trustwor- thiness might be considered as the confidence of whether a model will act as intended when facing a given problem. Although it should most certainly be a property of any explainable model, it does not imply that every trustworthy model can be considered explainable on its own, nor is trustworthiness a property easy to quantify. Trust might be far from being the only purpose of an explainable model since the relation among the two, if agreed upon, is not reciprocal. Part of the reviewed papers mention the concept of trust when stating their purpose for achieving explainability. However, as seen in Table 1, they do not amount to a large share of the recent contributions related to XAI. (2)、Causality: another common goal for explainability is that of finding causality among data variables. Several authors argue that explainable models might ease the task of finding relationships that, should they occur, could be tested further for a stronger causal link between the involved variables [159, 160]. The inference of causal relationships from observational data is a field that has been broadly studied over time [161]. As widely acknowledged by the community working on this topic, causality requires a wide frame of prior knowledge to prove that observed effects are causal. A ML model only discovers correlations among the data it learns from, and therefore might not suffice for unveiling a cause-effect relationship. However, causation involves correlation, so an explainable ML model could validate the results provided by causality inference techniques, or provide a first intuition of possible causal relationships within the available data. Again, Table 1 reveals that causality is not among the most important goals if we attend to the amount of papers that state it explicitly as their goal. (3)、Transferability: models are always bounded by constraints that should allow for their seamless transferability. This is the main reason why a training-testing approach is used when dealing with ML problems [162, 163]. Explainability is also an advocate for transferability, since it may ease the task of elucidating the boundaries that might affect a model, allowing for a better understanding and implementation. Similarly, the mere understanding of the inner relations taking place within a model facilitates the ability of a user to reuse this knowledge in another problem. There are cases in which the lack of a proper understanding of the model might drive the user toward incorrect assumptions and fatal consequences [44, 164]. Transferability should also fall between the resulting properties of an explainable model, but again, not every transferable model should be considered as explainable. As observed in Table 1, the amount of papers stating that the ability of rendering a model explainable is to better understand the concepts needed to reuse it or to improve its performance is the second most used reason for pursuing model explainability. (4)、Informativeness: ML models are used with the ultimate intention of supporting decision making [92]. However, it should not be forgotten that the problem being solved by the model is not equal to that being faced by its human counterpart. Hence, a great deal of information is needed in order to be able to relate the user’s decision to the solution given by the model, and to avoid falling in misconception pitfalls. For this purpose, explainable ML models should give information about the problem being tackled. Most of the reasons found among the papers reviewed is that of extracting information about the inner relations of a model. Almost all rule extraction techniques substantiate their approach on the search for a simpler understanding of what the model internally does, stating that the knowledge (information) can be expressed in these simpler proxies that they consider explaining the antecedent. This is the most used argument found among the reviewed papers to back up what they expect from reaching explainable models. (5)、Confidence: as a generalization of robustness and stability, confidence should always be assessed on a model in which reliability is expected. The methods to maintain confidence under control are different depending on the model. As stated in [165, 166, 167], stability is a must-have when drawing interpretations from a certain model. Trustworthy interpretations should not be produced by models that are not stable. Hence, an explainable model should contain information about the confidence of its working regime. (6)、Fairness: from a social standpoint, explainability can be considered as the capacity to reach and guarantee fairness in ML models. In a certain literature strand, an explainable ML model suggests a clear visualization of the relations affecting a result, allowing for a fairness or ethical analysis of the model at hand [3, 100]. Likewise, a related objective of XAI is highlighting bias in the data a model was exposed to [168, 169]. The support of algorithms and models is growing fast in fields that involve human lives, hence explainability should be considered as a bridge to avoid the unfair or unethical use of algorithm’s outputs. (7)、Accessibility: a minor subset of the reviewed contributions argues for explainability as the property that allows end users to get more involved in the process of improving and developing a certain ML model [37, 86] . It seems clear that explainable models will ease the burden felt by non-technical or non-expert users when having to deal with algorithms that seem incomprehensible at first sight. This concept is expressed as the third most considered goal among the surveyed literature. (8)、Interactivity: some contributions [50, 59] include the ability of a model to be interactive with the user as one of the goals targeted by an explainable ML model. Once again, this goal is related to fields in which the end users are of great importance, and their ability to tweak and interact with the models is what ensures success. (9)、Privacy awareness: almost forgotten in the reviewed literature, one of the byproducts enabled by ex- plainability in ML models is its ability to assess privacy. ML models may have complex representations of their learned patterns. Not being able to understand what has been captured by the model [4] and stored in its internal representation may entail a privacy breach. Contrarily, the ability to explain the inner relations of a trained model by non-authorized third parties may also compromise the differential privacy of the data origin. Due to its criticality in sectors where XAI is foreseen to play a crucial role, confidentiality and privacy issues will be covered further in Subsections 5.4 and 6.3, respectively. (1)，可信度:多位作者同意将可信度作为可解释AI模型的主要目标[31,32]。然而，根据诱导信任的能力来声明一个模型是可解释的，可能不完全符合模型可解释性的要求。信任度可以被认为是一个模型在面对给定问题时是否会按照预期行为的信心。尽管它肯定是任何可解释模型的属性，但这并不意味着每一个值得信赖的模型都可以被认为是可以单独解释的，也不是一种容易量化的属性。信任可能远远不是可解释模型的唯一目的，因为两者之间的关系，如果达成一致，就不是互惠的。部分综述论文在陈述其实现可解释性的目的时提到了信任的概念。但是，如表1所示，它们在最近与XAI相关的贡献中所占的份额并不大。 (2)，因果关系:可解释性的另一个共同目标是寻找数据变量之间的因果关系。一些作者认为，可解释的模型可能会减轻寻找关系的任务，如果它们发生了，可以进一步测试相关变量之间更强的因果联系[159,160]。从观测数据推断因果关系是一个长期以来被广泛研究的领域[161]。正如研究这一主题的社区广泛承认的那样，因果关系需要广泛的先验知识框架来证明观察到的结果是因果的。ML模型只发现它学习的数据之间的相关性，因此可能不足以揭示因果关系。然而，因果关系涉及到相关性，因此可解释的ML模型可以验证因果推理技术提供的结果，或在可用数据中提供可能因果关系的第一直觉。表1再次表明，如果我们注意到有大量论文明确将因果关系作为目标，那么因果关系并不是最重要的目标之一。 (3)、可转移性:模型总是受约束的，应该允许他们的无缝转移。这就是为什么在处理ML问题时使用训练测试方法的主要原因[162,163]。可解释性也是可转移性的倡导者，因为它可以减轻阐明可能影响模型的边界的任务，允许更好的理解和实现。类似地，仅仅理解模型中发生的内部关系，就有助于用户在另一个问题中重用该知识。在某些情况下，缺乏对模型的正确理解可能会导致用户做出错误的假设和致命的后果[44,164]。可转移性也应该落在可解释模型的结果属性之间，但同样，不是每一个可转移模型都应该被认为是可解释的。如表1所示，认为模型可解释的能力是为了更好地理解重用它或提高其性能所需的概念的论文数量是追求模型可解释性的第二大原因。 (4)信息性:使用ML模型的最终目的是支持决策[92]。然而，不应该忘记的是，模型所解决的问题并不等于人类所面临的问题。因此，为了能够将用户的决策与模型给出的解决方案联系起来，并避免陷入错误的陷阱，需要大量的信息。为了这个目的，可解释的ML模型应该提供有关正在处理的问题的信息。在回顾的论文中，发现大部分的原因是提取模型内部关系的信息。几乎所有的规则提取技术都通过对模型内部功能的更简单理解来充实它们的方法，说明知识(信息)可以用这些更简单的代理表示，它们认为这些代理可以解释先行项。这是在那些被评审的论文中发现的最常用的论点，用来支持他们期望得到的可解释模型。 (5)、置信度(Confidence):作为稳健性和稳定性的概括，置信度(Confidence)总是应该对一个期望可靠性的模型进行评估。在控制下保持信心的方法因模型而异。如[165,166,167]所述，从某一模型进行解释时，稳定性是必须具备的。可靠的解释不应该由不稳定的模型产生。因此，一个可解释的模型应该包含关于其工作机制信心的信息。 (6)公平:从社会的角度来看，可解释性可以被认为是ML模型中达到和保证公平的能力。在某些文献中，可解释的ML模型显示了影响结果的关系的清晰可视化，允许对现有模型进行公平或伦理分析[3100]。同样，XAI的一个相关目标是强调模型所暴露的数据的偏差[168,169]。在涉及人类生活的领域，算法和模型的支持正在快速增长，因此，可解释性应该被视为避免不公平或不道德地使用算法的输出的桥梁。 (7)，可访问性:在被评审的贡献中，一小部分认为可解释性是允许终端用户更多地参与到改进和开发特定ML模型的过程中的属性[37,86]。显然，当非技术或非专业用户不得不处理乍一看似乎无法理解的算法时，可解释的模型将减轻他们的负担。这一概念在调查的文献中被认为是第三大目标。 (8)，交互性:一些贡献[50,59]包括模型与用户交互的能力，这是可解释ML模型的目标之一。再一次地，这个目标与最终用户非常重要的领域相关，他们调整和与模型交互的能力是确保成功的因素。 (9)，隐私意识:在综述的文献中几乎被遗忘，在ML模型中解释性的副产品之一是其评估隐私的能力。ML模型可能对其学习到的模式有复杂的表示。如果不能理解模型[4]捕获了什么并存储在其内部表示中，可能会造成隐私泄露。相反，由未经授权的第三方解释训练模型的内部关系的能力也可能会损害数据源的差异隐私。由于其在XAI预计将发挥关键作用的部门中的重要性，保密和隐私问题将分别在第5.4和6.3小节中进一步讨论。 This subsection has reviewed the goals encountered among the broad scope of the reviewed papers. All these goals are clearly under the surface of the concept of explainability introduced before in this section. To round up this prior analysis on the concept of explainability, the last subsection deals with different strategies followed by the community to address explainability in ML models. 这一小节审查了在广泛范围内的审查论文中遇到的目标。显然，所有这些目标都隐藏在本节之前介绍的可解释性概念之下。为了总结之前对可解释性概念的分析，最后一小节讨论了社区在ML模型中处理可解释性的不同策略。 |
|
|
