Latest Results Of CBSE And ISC Demonstrate The Perils Of Our 100-Mark System

The sanctity and relevance of India’s school-leaving examinations have been destroyed, in no small measure, thanks to our emphasis on absolute or raw scores in examinations. This would have been fine if the entire country had one school board, but that is not the case. Many of our central universities, such as Delhi University, compare board marks literally, while deciding who makes the cut at the time of admissions. So, 99 per cent in one board is considered better than 98.8 per cent in another board.

Many jobs and degree programmes specify absolute cut-offs such as 60 per cent or 75 per cent. This has created a situation, where Indian school exam boards are trapped in a vicious cycle of score inflation, simply because of perverse incentive structures, which make schools and students defect to boards which dole out marks generously.

History Of inflating Scores By Central Board of Secondary Education (CBSE) And Indian School Certificate (ISC) In The Name Of “Moderation”

In the past, CBSE had been caught on multiple occasions, 'moderating' marks, by adding as much as 15-16 marks to the actual scores of students in 100-mark exams. However, they would top the moderated marks at 95. This led to a situation, where the student, who genuinely scored 95 in mathematics and the student, who scored 85, both saw exactly the same score of 95 on their marksheets. This is extremely unfair to those who score high marks on their own merit, as this kind of tampering with marks, disturbs the relative ordering of students.

The data mined for this year shows that marks have again been manipulated at a very large scale, by the CBSE board, which has once again inflated its scores in an arbitrary manner. These curves for such a large number of test-takers are expected to be bell-shaped curves, largely resembling gaussians – though there will be some skew and a few aberrations around the pass mark. Here is the histogram of Class XII mathematics scores.

It was also clear from ISC’s scoring trends that their scores were being inflated as they kept soaring every year. In addition, there was a glitch in their system because of which certain marks such as 84, 85, 89, 91 (and several others) had never really been attained by any of their students – in any of the subjects. So, histograms of ISC and Indian Certificate of Secondary Education (ICSE) marks, had a large number of gaps in them. This went on for at least 20 years. A viral blog on Quora led to a big hue and cry about this in 2013, but the board responded with predictable silence.

In 2017, they fixed this glitch silently. For the first time in several decades, ICSE and ISC candidates received scores like 81, 82, 84, 85, 89, 91 (and others) which had never been awarded in any subject, for a very long time. In the 2018 examinations, a surprisingly large number of ISC students have scored centums in subjects like computer science, biotechnology and physical education this year, despite the board being strict with evaluation standards in physics, chemistry, mathematics and English.

What is vexing is that all of this has happened even after all the boards got together in 2017 and came to a consensus that they would not inflate or misreport scores. This reflects on the incompetence of the CBSE head as well as the school education secretary.

There are several takeaways from our current assessment debacle.

Absolute Marks Are Meaningless

We do need to decide on some minimal standards while deciding on who passes the examination. But scoring a 95 per cent in a trivial high school physics paper is often far easier than scoring 50 per cent in a physics olympiad paper. In the IIT JEE examination of 2016, scoring approximately 65 per cent was sufficient for a candidate to rank within the top 100. While the exact marks were never disclosed at that time, in some of the tougher IIT JEEs conducted between the mid-1990s and the early-2000s, a score of 50 per cent was sufficient for test-takers to rank within the top 50.

What matters in an examination is primarily the information it provides about the relative ordering of candidates. Grade cards or marksheets need to provide columns containing information about the relative standing of students, indicated through the percentile score or the z-score.

An absolute score of 95 in CBSE or ISC exams was a far more stellar score around 50 years ago, than it is now. But one can say with a lot more certainty that the capability of those scoring in the 95th percentile, whether in 2003, or in 2018, is quite likely to be similar.

Unstable Scores Are The Hallmark Of A Poor Assessment System

The pass rate in the Class XII ISC examination for mathematics was over 97.6 per cent in 2014, which was an era when score inflation was at its peak. This year, the pass rate stands at 86.41 per cent. This is just an example of many stunning swings in scores and pass rates, across different boards. Given that the academic capability of the student population doesn’t change in such a short span of time, this is a reflection of how the assessment system is unstable. A hallmark of a good exam is reliability. A student of similar calibre, taking the exam in 2014 or 2015 or 2016, should receive a score or a grade, which is largely similar, each time.

The 20th percentile mark in the ISC examinations from 2012 to 2018 have been: 419, 425, 443, 439, 427, 438 (out of 500). Noise and fluctuation in the assessment have caused this mark to yo-yo all over between 419 and 443. Similar instability was noticed in CBSE’s Class XII scores.

If a score of 90 in one year is considered stellar, it should continue to indicate stellar performance even five years later. The repeatability of examination scores or grades is a critical feature of assessment design.

The Meaning Of 64 Students Scoring 99 per cent Or More, In ICSE And ISC

Schools and boards often highlight their toppers using metrics, which conceal the reality and make the scores more marketable.

For example, for ICSE exams, there are 15 students who scored either 99 per cent or 99.4 per cent. These scores, however, have been computed using only English plus the best four subjects. For ISC, there are 49 students who scored 99 per cent or more. The scores have been computed using English plus the best three subjects.

The best three (or four) subjects often skip the scores in subjects like mathematics or science (or physics and chemistry) where ISC/ICSE evaluation standards have been strict in recent years. Instead, they end up factoring in scores in subjects like physical education, art, computer science, biotechnology and geometrical drawing where the board’s assessment has been a farce: centums are literally doled out on a platter in these subjects.

Scores Or Grades Should Not Be Strikingly Dissimilar Between Subjects

The Dhirubhai Ambani International School has a sensible analysis of data and they have uploaded charts from which one can see the percentage of students scoring 80 per cent or more, on a subject-wise basis. They benchmark students of their school with the nationwide percentage of students scoring 80 per cent or more. You may observe the bar charts for ICSE 2016, 2017 and 2018 examinations here.

Notice how the percentage of high scorers (80 plus) in English, mathematics and science is relatively under control for each of the years and generally 25-40 per cent of the total examinees. However, in subjects like art, Hindi, technical drawing and physical education this ratio is often between 75-95 per cent.

Dramatically different scoring trends in elective subjects, also sets up problematic incentives which make students opt for subjects for inflated scores rather than academic interests. As per ICSE 2018 data, computer applications has around 60 per cent candidates scoring 80 or more, while in the case of art, this percentage is over 90 per cent. This sets up a wrong incentive structure where students will opt for art, because of the inflated grades.

Similarly, at the Class XII level of both ISC and CBSE, there has been a large scale abandonment of academic electives like economics and Hindi, in favour of physical education, where centums abound.

One subject being graded far more liberally than the others, also means, that the final rank list will often see a far higher correlation of overall ranks with ranks in subjects with liberal marking or inflated scores – than the correlation of overall ranks with ranks in subjects which are strictly graded.

This is because, we use a simple linear sum of scores across subjects, without considering the dramatic variations in the shapes of statistical distributions of those scores.

How Does One Interpret Scores From Different Boards?

Had boards catered to similar demographics, it would have been easy to directly compare percentiles or other indicators of relative standing, across different boards. It would have been fairly safe to assert that the candidate at the 91st percentile in CBSE would have been ahead of the candidate in the 90th percentile in ISC and so on.

The real problem here is that different boards cater to vastly different socio-economic strata. International schools cater homogeneously to the affluent. ICSE and CBSE cater to a mix of affluent and middle-class layers. State boards cater to everyone, including first generation learners.

So, a 95 per cent in ISC or CBSE might be far easier to attain than the West Bengal or Uttar Pradesh Board Class XII exam. However, the student at the 95th percentile in CBSE or ISC is likely to be academically ahead of the student in the 95th percentile of the merit list of UP or West Bengal board.

What India needs is a general Scholastic Aptitude Test or SAT-like exams for students wishing to apply to central universities or universities outside the state. This can be a lightweight test, focusing on basic reading, writing, numeracy and logic skills. The results from such a third-party test will enable us to do a cross board comparison and benchmark how the average student in the 90th percentile of CBSE’s merit list, compares with his counterpart at the 90th percentile in a state board merit list.

Right now a board can push more of its students into Delhi University or Mumbai University, by diluting exams, grading liberally and/or inflating marks. Such gaming can be thwarted if one only sticks to (a) using the relative standing within a board and (b) curves from the result of the common SAT-like exam to correct for board-to-board variations.

Once the relevance of absolute scores reduces, the dilution of standards will also reduce, as scoring trends will become agnostic to the dilution of exams or the inflation of marks.

In the race to inflate absolute scores, what has suffered the most, is practical work and project work. Students are awarded full marks by schools. Boards need to crack down on this practice by ensuring that scores of internal components (like practicals and projects) are in sync with the scores attained by students in the external examination.