The statistical case against the GLC 2019 admissions

In delivering a ruling on a case that bordered on elementary rules of contract law in 2009, Justice Date-Bah was clearly dismayed that a tawdry matter like that had to travel to through the lower courts to the Supreme Court. In his own words, the issue called for “…resort to the basic analytical tools for determining the formation of contracts… which are to be found set out in the earlier chapters of textbooks, and practice books, on the law of contract. To our mind, the outcome from the application of those tools is clear and self-evident.”

What it meant was that a very clear and self-evident matter had eluded lawyers and judges at the High Court and Court of Appeal. What should we make of this? Is it that the lawyers and judges who tried the case of NTHC v Antwi were subpar? Did it mean that prior to and after that case they never got anything right? Probably not. It was not because they never read those “earlier chapters”, rather, what this should tell us is that in resolving any legal matter rigour proves more useful than knowledge of extant rules. It is often said that a good lawyer is not one who knows the law but where to find it. This also raises important questions about how law should be taught and how law students must be assessed.

The General Legal Council’s (GLC) Independent Examinations Committee (IEC) conducts an entrance exam to determine entry into the Ghana Law School. The Council argues that this has become necessary because of the increasing number of law faculties in the country. To write the IEC’s entrance exam though, a student must fulfil the faculty requirements and graduate successfully in order to be considered for admission into Ghana Law School. In 2019, there were 1820 students who sat the exam.

Out of this, the GLC declared that 128 (7%) passed to be considered for admission. The council arrived at this number without considering the numbers they require, this number is simply because only 128 of the students scored 50 and above. Further, the council will have us believe that they made provision for increased numbers but simply cannot admit people who scored below 50. It is this claim that we must subject to critical and statistical analysis.

The case of NTHC v Antwi shows how even seasoned professionals can get a basic question in law wrong on any given day not for want of knowledge of extant rules but because of failure of legal rigour on that particular question. Which makes it very problematic that a two-hour test meant for law students on one afternoon is dispositive of the question whether a person is suitable to be a lawyer. Granted that the students did poorly on that test, just like the lawyers and judges in the NTHC Case, it does not tell us anything about how they will do in the next test with different questions or how they will perform in the school and even as lawyers. More importantly, we cannot take the Independent Examination Committee’s (IEC) assessment as final on this matter.

Especially where the IEC does not teach the students and does not provide any curriculum for students to study. There is no reason why we must prefer the IEC’s assessment of Ghanaian law students on contract and constitutional law to the accredited faculties who have trained and assessed these students for years. The IEC’s exam is and ought to remain a placement exam and not an evaluative one. In other words, its value is in ranking students for admission purposes not passing a verdict on who is worthy of becoming a lawyer. The IEC will have that decision to make when the students have been admitted into the Ghana Law School and have been trained and assessed by it.

Because the IEC does not provide a curriculum and does not instruct students, it cannot have the power to determine in a qualitative sense, who is a fit law student and who is not. It is also in this vein that a pass mark of 50 is just arbitrary and does not tell us anything about the students or the exam. A student’s performance on a test involving constitutional law and contract tells you nothing about their grasp of Succession Law and Law of Taxation.

The IEC’s entrance exam is a competitive exam very much unlike a semester course where a lecturer grades students according to how much they have performed in a course he or she has taught and is in a position to assess and as such can set a pass mark. Competition exams only make sense when students are assessed in comparison with each other. This form of testing is known as the norm reference testing.

In this model, you are only as good or as bad as the larger population being examined says you are. So if you score 80 where most people score between 85 and 90 you have not done so well, yet, if you score 49 in an exam only about 25% score between 40 and 67 you may have done well. By its nature, nobody fails a norm reference test because there is no arbitrary pass mark. But what we do have is a metric to determine among a population who belongs in what percentile. And so for admission purposes, students are ranked according to their percentiles and the required number of students could simply be taken. With this model, we would not have space as the GLC claims and no students to fill them.

The raw scores released by the IEC this past September shows how arbitrary the pass mark is and how it says nothing about the ability of the students. The topmost score was 67 and 128 students (7%) scored between 67 and 50 below which they ruled a ‘red line’ right above someone with 49.5. On a normal distribution curve, where 50 is part of 7%, a person with 49.5 is up there around the top 8% of the population.

By setting a pass mark at 50 and in declaring someone in the top 8% of the population to have failed, the GLC is putting its very short and narrow test above what accredited faculties of law say about students they have reviewed for years. Yet this is the precise thing it is not equipped to do. How is a person in the top 8% of a national law student population not good enough? It would have been understandable if there is room for only 7% but we are told that is not the case.

The IEC’s exam, at best, can tell us in a quantitative sense who is first and who is last among the population on that exam; nothing further. Thus it can only be used where it matters; where there are more students than the school can take and the council needs a metric to eliminate. It cannot be used to deny students the opportunity to further their legal education in spite of an opportunity to do so yet this is what is happening.

Setting a threshold is to suggest that the entire population was to demonstrate some particular form of knowledge the IEC did not give notice of and did not provide any instruction to the students on. Interestingly, the faculties, whose requirements needed to be fulfilled before a student could take the IEC exam deemed each of the 1820 to have demonstrated satisfactory understanding of each of the courses required to take the IEC exam.

In other words, the IEC is in an inferior place compared to faculties to determine who can enter law school and by extension who can be a lawyer. What it can and ought to have done was to assess the students in comparison with each other and to that end a pass mark is nebulous. Because the IEC was testing nothing in particular known to the students, its sole basis for failing students is nothing more than they did not analyze a legal issue(s) to their satisfaction, but that is a dangerous prospect. For every day in court, one lawyer has that very fate of having his analysis rejected by a judge. Should such lawyers lose their licenses then?

The statistical problem before us relative to the mass failure is that the IEC and the GLC have turned what ought to have been a quantitative process (ranking) into a qualitative one (evaluation). It is also one in which it is uniquely unequipped to undertake. The Council must accept the verdict of the accredited faculties that these are fit and proper candidates and allow its IEC’s exam to be limited to determining by a quantitative metric who gets in because of limited space and to that end, assessing on a curve (norm reference testing) will be its fairest tool.

It will also help us avoid the unfortunate fate of penalizing students in the top 8% to 14% of a national law student population. Statistically, the first 16% of a population on an exam are the best and brightest. The IEC’s exam can only aid us in identifying that 16%. What it cannot tell us and we should not rely on it for is an assessment of who has what it takes to be a lawyer. If the GLC has a problem with the training of law graduates, it must take that up with the law faculties and not on students who have done everything they are supposed to do.

Therefore, if the GLC is honest about having space, they should give up the pass mark which is arbitrary and allows the IEC to pretend to make a qualitative assessment of students it is not equipped to make and peg the maximum admission number at the corresponding percentile of students who have taken the exam.

That will be assessing the students in comparison to each other and not in comparison to an arbitrary threshold. There will be some justice there. A country in which being in the top 8% of students in your year group is not good enough is a scary one; we must do better than this.

#OpenUpLegalEducation