Digital Approaches to Legal Evidential Reasoning: A Comparison of Vlek et al. and Hepler et al.
Written and Edited by: Ahmet S. Sakrak, M. Graziella Brevi, Elia Calderazzi, Maria Decio, Simona Pescaru.
Introduction
In all legal cases, lawyers, judges, and juries alike must handle and consider evidence that will inevitably affect the outcome of the case. In this paper we will compare two texts concerning evidential reasoning approaches and the digitalisation of such approaches in the legal field. The first text is a research paper published in 2007 and written by Hepler et al. entitled ‘Object-oriented Graphical Representations of Complex Patterns of Evidence’; the second, is a 2013 research paper by Charlotte Vlek et al. entitled ‘Modeling Crime Scenarios in a Bayesian Network’.
Before delving into the two papers, some background knowledge with regards to the two primary tools utilised in the text — Wigmore charts and Bayesian networks — will be provided. The paper will then proceed to compare the methodologies of the two papers, which will lead to a discussion of the papers themselves. Here, the strengths and weaknesses of the two sources, and their respective implications, will be discussed. Thirdly, our paper will provide an account of some further literature related to the same topic and the results that that literature presents. Finally, the paper will conclude with a brief summary of our findings.
Bayesian Networks and Wigmore Charts
Both texts employ Bayesian networks (BNs) to graphically represent legal evidence. A Bayesian network is a probabilistic graphical model representing different sets of variables and their conditional dependencies. In the legal field, Bayesian networks are especially used to help judges and juries in their evidential reasoning for they grant “a good overview of the interactions between relevant variables” and prevent “tunnel vision by comparing various scenarios” (Vlek et al. 2013, 150). Thus, Bayesian networks allow one to calculate the probability of any event taking place in light of the pieces of evidence provided and their possible interactions. The use of expert knowledge in the construction of these networks, as well as the implementation of control mechanisms for the making of predictions render Bayesian networks a relatively objective alternative to the use of machine learning models which are often deemed as ‘black box’ models due to their complex nature.
Whilst Vlek et al. only make use of Bayesian networks, Hepler et al. also use Wigmore charts. These were devised by John Henry Wigmore, a legal scholar who identified ‘the problem of proof’, whereby proof differs from admissibility; whereas the former is directly related with “the ratiocinative process of contentious persuasion”, the latter involves “the procedural rules devised by law and based on litigious experience and tradition” (Wigmore 1913, 77). Wigmore charts may be seen as an early attempt at the creation of a model for qualitative reasoning processes as they can be utilised to describe and organise all available evidence in a legal case. They are constructed by firstly listing and defining the relevant details of all the available facts of a case, secondly, assigning them various identifiers, and finally, connecting facts of the case and details by means of arrows representing the flow and degree of inference.
Applications of Wigmore charts and Bayesian networks to complex patterns of events indicate how complex legal cases can be reconstructed in a simplified manner. Indeed, the main goal of Hepler et al. is to showcase that certain qualitative features of Bayesian networks can be used as graphical tools to reorganise a myriad of evidence (2007). By automating the probabilistic calculations through Bayesian networks, the authors consider several alternative configurations of the network so that various possibilities are taken into account in the evaluation of complex criminal cases. Their findings are enhanced by their adoption of an object-oriented approach, which makes their outputs more accessible to judges and juries as they are characterised by greater clarity. This is a result of the grouped sentence feature of the approach, which structures and organises in a logical manner the representation of evidence.. Similarly, Vlek et al. try to facilitate the decision-making process by combining the narrative and probabilistic reasoning approaches. They quantify the variables in the Bayesian network (2013) and develop a formal method which they hope will serve as the foundation for the production of a software that will help deal with evidential reasoning on legal cases. The main goal of this work is to be able to compare how the probability of several scenarios can be modelled and compared as opposed to the calculation of absolute probability.
Methodologies
By combining the narrative approach and the probabilistic reasoning approach, the authors Vlek et al. create a framework within which multiple scenarios can be included in just one Bayesian network. The authors also introduce two new idioms to the already existing legal idioms: these are the scenarios idiom and the merged scenarios idiom. Legal idioms are substructures that tend to occur rather frequently in legal cases. The aim is to connect all scenarios for a legal case to the available evidence in a single Bayesian network.
In the paper ‘Modeling Crime Scenarios in a Bayesian Network’ the authors provide an example of a typical crime pattern, whereby a suspect is in a fight with the victim and after the victim has insulted him/her, the suspect uses a knife that was lying on the counter to stab the victim. This example represents a sequence of events that results in a coherent scenario. Its coherence will be captured in a scenario idiom once all states and events are united into one single scenario node, which is in turn connected to a guilt hypothesis stating that a suspect is either guilty or innocent of a crime. Whether a suspect is guilty or not depends on the veracity of the node. It is also possible to have scenarios without having a guilt hypothesis; if we were to refer back to the example provided, this would mean saying that the victim is guilty, perhaps because they tripped on the knife. In such cases, the guilt hypothesis node is excluded from the scenario node altogether.
The construction of a Bayesian network entails four steps (Vlek et al. 2013).
Firstly, all relevant scenarios must be collected and it must be decided which guilt hypothesis these scenarios support; the hypothesis must either be equal or mutually exclusive. As mentioned above, there is the possibility that a guilt hypothesis may not exist. The second step is to construct a scenario idiom from a scenario. This step requires the inclusion of a binary node — that is, a node that is either true or false — for each event of the scenario and the design of an arrow connecting each scenario to all its states or events. Given that the authors are constructing a Bayesian network, the independence of all those nodes that are untouched by an arrow shall be double-checked. A probability table is added to the scenario to highlight how plausible a scenario is without taking in consideration the evidence. The third step entails the merging of scenario idioms, resulting in the replacement of guilt hypothesis nodes which when connected to different scenario nodes express equal hypotheses, by just one hypothesis node connected to both scenario nodes. This serves to declutter the representation but also means that the remaining guilt hypotheses are mutually exclusive. Once the merging has been completed, there may still be additional dependencies between state or event nodes in different scenario idioms. Whenever this happens, we assume that if a state or event is influencing the scenario, it is already included in the scenario. At the end of this complex process, only two dependencies may occur between scenario idioms: nodes describing equal states and nodes describing conflicting states. The fourth and final step to construct a Bayesian network is to add to the current structure all relevant evidential nodes.
For the sake of demonstrating how this process can be implemented in practice, the authors, Vlek et al., construct a Bayesian network for a case study concerning a burglary. In this case, a window was broken and the fingerprints of a suspect were found on that window. The suspect attempted to justify the presence of his fingerprints by stating that a couple days earlier he had climbed on the windows. Yet between the day on which he claimed to have climbed the window and the day of the burglary, those same windows had been cleaned so his fingerprints would have been removed. The items stolen from the burglary were found with another suspect, and later investigations revealed that the two suspects had engaged in criminal activity together in the past. This allowed the authorities to convict them both. =
Object-oriented Bayesian Networks
The paper ‘Object-oriented Graphical Representations of Complex Patterns of Evidence’ by Hepler et al. is based on two graphical aids: Wigmore charts and Bayesian networks. Both of these offer solutions to evaluate mixed types of evidence in legal cases and their interrelationships. The two methods share some similarities but differ in terms of the information that is represented on the graphs. Hepler et al. endeavour to compare the two tools and explore how they may interact in a case study, namely the 1920 Sacco and Vanzetti case where two payroll guards, Alessandro Berardelli and Frederick Parmenter were attacked, robbed, and murdered by two other men. Nicola Sacco and Bartolomeo Vanzetti were arrested shortly after the fact and convicted of first-degree murder, ultimately being sentenced to death. Neither one of the two ever admitted to being guilty.
The first method employed by the paper utilises Wigmore charts to support the organisation and description of existing evidence in the case. The chart consists of a diagram with diverse symbols and arrows. Every symbol represents a proposition, connected by arrows which indicate the flow and degree of inference between the propositions. The chart aims to present masses of evidence and facilitate the reasoning process required to approve or deny different hypotheses during trial. The legal hypotheses are referred to as a probanda, and it is imperative to have an ultimate probanda along with a set of penultimate probanda in order to prove the ultimate probanda. Legal scenarios are fitting for this kind of chart analysis because in the legal field, once one knows what overarching hypothesis they have to prove, they also know which sublevel hypothesis (penultimate probanda) must be proven, too.
The first symbols in the Wigmore chart will be the ultimate and penultimate probanda, and to continue the diagram a ‘key list’ containing the relevant case details, called trifles, has to be established. The relevance of the trifles results from the direct or indirect link of the details to one of the penultimate probanda. The inferential linkages will be represented by arrows in the chart after the trifles and the two probanda are introduced.
The Bayesian Network method described is another tool that can be used to graph the connection between different sources of evidence, weigh them, and draw inferences from them. The research paper emphasises the qualitative features of a Bayesian Network or marshalling masses of evidence. As previously discussed, Bayesian Networks include relevant hypotheses represented by nodes and connected by means of arrows which in turn display their probabilistic dependence.
One of the similarities shared by Wigmore charts and Bayesian networks is that both result in graphical representations of masses of evidence. Nonetheless, they differ in that Bayesian networks are less schematic in comparison to the Wigmorean charts. Moreover, Bayesian networks automatically create probabilistic calculations, whereas Wigmore charts require further analysis once they have been drawn up. While Wigmore charts may be considered more limited due to the fact that they include exclusively binary propositions, a common criticism of Bayesian networks is that they call for specifications of conditional probabilities, which are often hard to draft in a precise manner from legal scenarios. The act of combining both methods leads to an object-oriented approach, applied in the article in the context of Bayesian Networks.
An object-oriented Bayesian network uses small modular networks to facilitate hierarchical construction of a case (Hepler et al., 2007). It enables a ‘top-down’ approach for which the lower-level modules are not required from the beginning of the process, but rather can become known later on. The authors also mention that an object-oriented design is advantageous because it allows for multiple instantiations of a module or fragment, a fact that becomes especially helpful when analysing masses of evidence. In fact, it is not uncommon to observe patterns of evidence that crop up both within one individual case or amongst a series of cases, (for example, eyewitness testimonies are frequently used in criminal trials).
So, although both articles by Vlek and Hepler aim to strengthen and advise courts’ decisions in criminal cases by making use of graphical aids, the former only operates with Bayesian networks while the latter operates with Bayesian networks and Wigmore charts. These two methodologies differ in themselves, as explained previously. The two papers not only share a common aim, but also share a similar structure: they both begin by introducing their model in theoretical terms and then showcasing its practical application through the use of case studies. This goes to show how much one builds from the other. The case studies are quite different in their details, but are both criminal cases, which also goes to show how apt these methodologies are to cases of this kind. Unlike other literature in the field, such as Dalhman’s 2019 paper, the authors of these two papers do not focus extensively on how to calculate the likelihoods of evidence and hypotheses, but rather create the grounds on which these calculations can be made.
Results
Once again, the goal of both papers analysed is to create a graphical structure of Bayesian Network that could be easily and efficiently applied to, potentially, any real-life case. Hence, the methods previously described were applied to past criminal cases in order to give concrete examples of how Bayesian networks can help judges and jurors have a more coherent view over complex inquiries.
The paper written by Hepler et al. analysed the renowned and very much discussed Sacco and Vanzetti case. This 1920 case concerns the robbery and murder of two payroll guards in Massachusetts. The anarchists Nicola Sacco and Bartolomeo Vanzetti were charged and sentenced to death after a long-lasting trial.
The authors applied an objective-oriented Bayesian network model to the case. This procedure began by highlighting the ultimate probandum, which, in this case, was a felony degree murder. In order to prove the ultimate probandum, three penultimate probanda had to be demonstrated. In this case, they were the following: was Berardelli murdered? Was payroll robbery committed? Is Sacco (and Vanzetti) the culprit? (p.283). Based on these probanda, the authors decomposed the probanda into modules. For instance, the latter was divided into four modules, with each of them being able to give a partial answer to the probanda. Each of the modules were then in turn divided into other sub-modules. This process of expansion keeps evolving, until all the relevant ancillary pieces of evidence are added to the model.
The second paper, written by Vlek et al., develops on the use of Bayesian networks in legal cases, by adding interesting tools to the previous models. By doing this, the authors report a Dutch case about a burglary in a house, in which two suspects were accused: the former, X, left fingerprints on the window that the thieves broke to access the house, the latter, Y, was found with the stolen items.
The authors apply the four steps procedure that they previously discussed. Firstly, they formulated all the possible scenarios that could apply to the case. Then, they connected the scenarios to the guilt hypotheses, which, differently from the scenarios, were either the same or mutually exclusive. Secondly, the scenarios were connected to their scenario idioms. Thus, event nodes were taken into consideration and their prior probability, i.e. the plausibility of a scenario without evidence being taken into account, was calculated. Thirdly, the multiple scenarios were merged using the merged scenario idiom. This step creates constraint nodes, which allow decision-makers to have a clear overview over conflicting or overlapping states and events. Finally, all the relevant pieces of evidence were added to the model, with some of them supporting different scenarios.
As the two papers are compared, it becomes evident that they both attempt to support and advertise the use of graphical representation methods of legal cases, (Bayesian networks in particular), in the decision making process. They both follow the same line of theory, and the latter paper openly attempts to elaborate on the former, by adding new concepts, such as the scenario idioms and the merged scenario idioms (p.151). Although different types of cases are analysed (one being renowned, the other representing a relatively less complex and non-detailed situation), they follow the same structure, as they first introduce their representation model, followed by the application of said model to the cases. Both papers supply decision-makers with concrete and detailed instructions on how to break down complex cases so that every piece of evidence is valued. However, contrary to other lines of literature, such as the paper by Dahlamn (2020), the authors don’t focus extensively on how to calculate the probabilities of evidence and hypotheses, yet, they create the grounds on which these calculations can be made.
Discussion
The weaknesses of the papers stem from the weaknesses of the models themselves. The models are, in fact, complex and even more complex for a jury with no background in the field of law or AI to understand. (Hepler et al., 2007)
Another weakness of both models is their use of numbers to give an appearance of objectivity. On the one hand, this makes them extremely appealing to the general public, on the other hand, it leads to a complete dismissal of the many statements made by most social scientists, with the exception of a few, that an objective version of reality does not exist. For instance, the scenario Nodes developed by Vlek et al. (2013) construct, by means of binary choices, mutually exclusive hypotheses which are intended to systematically analyse the case and provide numerical values for the likelihoods of each hypothesis being true. One could argue that in a similar way one would prefer to use qualitative survey interview style over a quantitative one, for a qualitative method would allow to go further deep between (metaphorical) lines without having to stick to a script, allowing the exploration of potential patterns; having a more qualitative approach to the evaluation of evidences would allow for more space, for both the jury and the judge, to understand the case in a less structured but more fitting way to the case at hand. This qualitative approch, which does not necessarily overfixate on the idea of objectivity, certainly does leave much room for discrimination and inconsistency of the law. Notwithstanding, it may still underscore the notion of equity over equality. Many examples can be given to testify to the importance of subjectivity in courts; one such example could be the case of Cyntoia Brown, a young POC woman with a history as a victim of rape and abuse. At the age of sixteen, Ms. Brown was sentenced for the murder of a man who was using her as a prostitute. Despite her being a minor and claiming she was acting in self defence, she was sentenced as an adult to a life in prison until 2019, when, after mass media shed light on her case, the Tennessee Governor ended her sentence. These types of cases should not only make us question why objectivity is priotized to such an extent, but also remind us how objectivity can have discriminatory consequences to the disadvantage of non-white people. Admittedly, this statement may sound contradicting, yet ironically it exposes the contradicting racist nature of the American, (in Cyntoia Brown’s case, but overall global) judicial system, which seems to prioritise objectivity and heavily sentence non-whit individuals. Of course, we are not proposing an across-race objective judicial system that would sentence a white sixteen-year-old with the same history of Cyntoia Brown in the same way she was, but we are just trying to demonstrate that, although understanding the timeline of events that brought to a murder or a burglary is essential, it should not be a central part of how a case is ruled on.
Furthermore, it is important to note that the Wigmorean chart and the Bayesian network are quantitative tools aimed at increasing objectivity in court rulings, that nonetheless still rely on individuals and the algorithms created by such individuals who relied on datasets from cases the outcomes of which were determined by historically racist and colonial judicial systems. In fact, Dahlman (2020) admits that behind these assigned percentages which stand for the probability that a piece of evidence or an event is true, there is the evaluation and decision of individuals, who by definition cannot be objective. Similarly, when these methods are automated through algorithms in softwares, the latter are inevitably built by individuals, once again falling into the same fallacy. When these algorithms make decisions, these are made with the use of databases, which are also not objective. A documentary by Shalini Kantayya, Coded Bias (2020) investigated and discovered how Artificial Intelligence for facial recognition was unable to recognize people with dark skin tones because it was built on a database of people with light skin tones. Likewise, it can be questioned how, if Bayesian Networks and/or Wigmore charts also pull data from previous cases and base probability estimation on these, can we ensure that those previous cases are a good enough sample when they are the product of a judicial system deeply rooted in institutionalised racism? On the other hand, as Dahlman (2020) writes, it is inevitable to project personal ideologies on the outcome of the case, but by making them explicit through quantifiable percentages those ideologies are made explicit, which also makes them contestable. A middle ground could be found where Wigmore charts and Bayesian Networks are made by a panel of experts with a diverse and representative sample of people.
Conclusion
To conclude, this paper has provided in-depth explanations about the function of Wigmore charts and Bayesian networks, along with the steps entailed in their construction. Bayesian networks are used as a tool by both Vlek et al. and Hepler et al., but only the latter make additional use of Wigmore charts. Thus, whilst there are similarities between the two texts, there are also some differences. The focal similarities are their use of Bayesian networks and the structure of their research — both papers commence with a theoretical analysis of Bayesian networks and Wigmore charts and then move on to demonstrate their application to case studies. Furthermore, both papers focus on the use of said tools in the context of criminal cases, thus going to show that these tools are particularly apt for that specific field of law. There is also some similarity in the two papers that is reflected by the similarities between the two tools employed; that is, Wigmore charts and Bayesian networks share the characteristic of being graphical representations of masses of evidence with the aim of helping judges and juries navigate through that evidence and facilitate efficient evidential reasoning. However, some differences between Wigmore charts and Bayesian networks have also been pointed out, such as the fact that the latter are less schematic.
This paper has also engaged in a critical discussion of the use of Bayesian networks and Wigmore charts by warning about the possible racial and discriminatory implications of the use of the aforementioned tools. It has shed light on the problem of objectivity and how the individual creator’s lack thereof, when considered in combination with the systematic racism that is embedded in many judicial systems and that has influenced the outcome of previous cases, could lead to Bayesian networks and Wigmore charts predicting verdicts that are particularly detrimental to non-white people.
Reflection
The group has proven to be quite productive from the first week onwards. We successfully managed to split up the tasks equally, hence why we were able to work efficiently in a coordinated manner. Ahmet was charged with writing the introduction and the initial explanation of Bayesian networks and Wigmore charts. Simona looked into the methodologies of the paper and reported on them. This also included providing an explanation of object-oriented Bayesian networks. Elia detailed the results of our findings, and Graziella engaged in a fascinating discussion of the papers studied. Finally, Maria took care of the conclusion and of proofreading and editing our paper. We then got together one last time to approve final details, such as citations. As a result, every member contributed equally, meaning that everyone contributed to 20% of the paper.
Bibliography
Brachman, Ronald J, and Hector J Levesque. 2004. “Chapter 12 — Vagueness, Uncertainty, and Degrees of Belief.” In Knowledge Representation and Reasoning, 237–66. Elsevier Inc. https://doi.org/10.1016/B978-155860932-7/50097-2.
Dahlman, Christian. 2020. De-biasing Legal Fact-Finders with Bayesian Thinking. Topics in Cognitive Science, 12:1115–1131.
Du, Cheng-Jin, and Da-Wen Sun. 2008. “4 — Object Classification Methods.” In Computer Vision Technology for Food Quality Evaluation, edited by Da-Wen Sun, 81–107. Food Science and Technology. Amsterdam: Academic Press.https://doi.org/10.1016/B978-012373642-0.50007-7.
Hepler, Amanda B., A. Philip Dawid, and Valentina Leucari. 2007. “Object-Oriented Graphical Representations of Complex Patterns of Evidence.” Law, Probability and Risk 6 (1–4): 275–93. https://doi.org/10.1093/lpr/mgm005.
Kadane, Joseph B., and David A. Schum. 1997. “A Probabilistic Analysis of the Sacco and Vanzetti Evidence: Joseph B. Kadane & David A. Schum, (Wiley, New York, 1996).” Journal of Statistical Planning and Inference 64 (1): 171–91.https://doi.org/10.1016/S0378-3758(97)00105-5.
Shalini Kantayya, 2020. Coded Bias. Netflix.
Vlek, Charlotte, Henry Prakken, Silja Renooij, and Bart Verheij. 2013. “Modelling Crime Scenarios in a Bayesian Network.” In , 150–59. ICAIL ’13. ACM. https://doi.org/10.1145/2514601.2514618.
Wigmore, John H. 1913. “Problem of Proof.” Illinois Law Review 8 (2): 77–103.
Ziegel, Eric R. 1998. Review of Review of A Probabilistic Analysis of the Sacco and Vanzetti Evidence, by Joseph B. Kadane and David A. Schum. Technometrics 40 (2): 166–166. https://doi.org/10.2307/1270674.
