THE FALLACY OF "HARD" TESTS

(Ithacaunleashed.blogspot.com)

A great deal of fuss is often made about failing the bar exam. The news a few weeks ago was that Governor Patakis daughter passed the exam, but it is always mentioned that it was her second try. Similarly, John Kennedy, Jr. failed the New York bar exam twice, before finally passing it on his third try.

As one who took several medical licensure and specialist exams, and the Virginia bar exam, passing all, I might be inclined to pat myself on the back, but my former background as a mathematician won’t let me do that. I do remember, however, some remarks from a noted orthopedic surgeon about his own specialty exam: “It was a hellishly hard test, and went on for hours,” he said, ”but I’m really glad I passed the first time I took it. Only about 35 percent who took it passed the exam.”

He was describing, with only the slightest tinge of boastfulness, the qualifying exam for specialists in orthopedic surgery. Passing the exam entitled one to join the “college” of orthopedic surgeons, and list oneself as specialist.

“Was it all multiple choice?” I asked. “And how did they grade it?” I was thinking of my own exams. “Did they count only the right answers.?”

When he said Yes to all the questions questions, I did not have the heart to tell him what I knew as a mathematical certainty—that the exam was, like most graduate medical exams, and large parts of legal licensing bar exams in most states , virtually a complete fraud.

The reason these tests are fraudulent—and the harder they are, the more they are fraudulent—is that for an extremely difficult test graded in that way, guessing tends to count much more than knowledge.

A simple example will describe why this is the case. To illustrate this, consider an extreme case.

Suppose you and I take a test, and you know twice as much as I do. For simplicity (this is the extreme case) suppose the test consists of 100 questions, each True or False, and moreover (this is the key point), let us agree that the test will be graded by only counting the number right.

Naturally, both of us will guess at an answer for those questions that stump us.

Now suppose the test is very hard. As hard as it could be actually. Suppose the test is so hard that I, with lesser knowledge, can only answer one question based on actual knowledge. I answer that question, and guess at the other 99. You, who know twice as much as I, can answer two questions based on knowledge. So you guess at 98 answers.

As you can readily imagine, the odds of you getting a higher grade than I are very slight. In fact, over 45 percent of the time, in repeated trials, I would outscore you, even though my knowledge is half that of yours.

I chose a True-False test for this example, but it doesn’t make any real difference were the test to be multiple choice with several choices in each question. The only thing that makes a difference is how hard is the test. Your advantage would grow substantially as the test was weakened.

For further example, if the test was so easy, and you so well-versed in the subject that you could get a perfect score, and I knew half as much, I would answer 50 questions based on knowledge, and guess at 50. In the long run, I would get half of those 50 correct, for a final score of 75. So you get 100, and I get 75, on the average.

Were the test to be multiple choice, with four choices for each question, and your knowledge was also 100 percent and mine half that, I would then (guessing at 50) get a score of 50 + (1/4 times 50), or 62.5. on the average.

These extreme cases demonstrate the point, that truly hard multiple choice tests, graded by counting only the number right and ignoring guessing, are fraudulent.

But suppose the grading attempts to adjust for guessing. There is no way of knowing what is in the mind of the test-taker, so the customary is to subtract, from the number correct, some fraction of the number wrong.

For True-False exams for example, the number subtracted would most likely be (Number Wrong ÷ 2). Let’s see how that would work out, for the sample case above. You, answering two questions correctly and guessing at 98 would be likely, on the average, to get 49 wrong, and so have a final score of 2 + 49 - (49 ÷ 2), or 75.5, while I, again on the average. answering only 1 correctly and guessing at 97, would get a final score of 1 + (97 ÷ 2) - ((97 ÷ 2) ÷ 2)), which comes out to be 25.25. Here there is a substantial difference between our scores, closer to the two-fold difference in our actual knowledge.

The situation is only a bit more complex for multiple choice tests with four or five questions, and you can readily calculate the variation between the knowledgeable you, and the ignorant me. As an old math teacher might say, we leave that for the reader to work out by himself or herself.

THE SIMPSON-GOLDMAN MURDERS SOLVED !

The Dog That Didn’t Bite, Or
The Simpson-Goldman Murders Solved
(ithacaunleashed.blogspot.com)

by A. N. Feldzamen

During the night, the dog, who should have barked, was silent. Very few of those who were on the scene paid attention to this, but it was not overlooked by the detective—the most famous detective in the world. It was this curious occurrence that led him to solve the mystery in Conan Doyle’s story, SILVER BLAZE.

What might Sherlock Holmes have made of another criminal mystery, also involving a dog, an occurrence whose importance has also been largely ignored? In this case, also at night, the dog in question was found wandering with blood on his paws. The finder was a passerby out for a stroll. The dog led him to the crime scene, where two bloody corpses were found. A knife wielding murderer or murderers had slaughtered them brutally.

But this dog with bloody paws was not an ordinary breed. It was an Akita, a type of dog nurtured and admired for centuries in Japan, where it is designated as a national monument. While the Akita’s characteristics are not generally known among the American public, this breed does have its admirers here too, and also a club devoted to its special virtues.

The Akita Club of America states these “are large, powerful dogs with substantial bone and musculature. . . Typically the male Akita is substantially larger than the female. The males range in weight from about 100 to 130 pounds, while the females range from 70 to 100 pounds. . . the Akita is very intelligent, extremely loyal, and can exhibit aggressive tendencies. . . [and] a very well developed guarding and protective instinct.

“Akitas also have a high and well developed prey drive. . . . The loyalty and devotion displayed by an Akita is phenomenal. . . Your Akita lives his life as if his only purpose is to protect you and spend time with you. . . Akitas are natural guardians of the home and do not require any training to turn them into guard dogs. When there is a reason to protect family and property, your Akita will act to do so.”

This large, powerful dog, so prized in Japan, is noted as being “dignified, aloof and with a fearless temperament . . . a no nonsense protector of family and home.” In fact, it is a tradition in Japan to award a small Akita statue as a gift upon the arrival of a firstborn child to signify health, happiness, and a long life, or to wish a rapid recovery to someone who is ill.

One Akita is especially famous throughout that country. This dog, named Hachiko, was born in Odate in 1923 and moved to Tokyo two months later. Hachiko’s owner was Dr. Ueno, a professor at Tokyo University. Dr. Ueno passed away when Hachiko was only a year and a half old. But for the next ten years, rain or shine, Hachiko continued to go to the railroad station every evening to wait for his master even though his master had died. He did this for ten full years until Hachiko himself died in 1935. To commemorate this loyalty, a statue of the faithful dog was erected, and is known to all Japanese.

In our double murder case, Sherlock Holmes might have pondered on the time when the roaming Akita was let out of the house. Surely it could not have been let loose or freed from any restraint after the killings had occurred. Why would the perpetrators have done that, released the dog after the brutal murders?

But if this fearsome animal was out and about at the time of the crime, it was likely close to its Nicole Simpson, its mistress, given its protective temperament at the time of the crime. Or at least, not far away. Had an outsider attacked her, surely the dog would have responded with a rapid and ferocious defense. A charging 100 pound Akita could stop any physical attacker.

Why then didn’t the dog act in such defense, when its mistress was being murdered?

There is only one conclusion, one logical answer to that seeming mystery. There is only one reason. The dog could not have acted in defense of its mistress by attacking its master.

But then, what could have been a motive for such an attack? Further research into the relation between the two victims elicits evidence that they had become close, and according to a CNN article, a close friend and confidante of the woman noted that on the night of the killings, the two, Nicole Simpson and Ronald Goldman, “were giggling that they were going to ‘do it’” because the lady wanted to find out if he “was good in bed.” In fact, the confident had been told that the pair were planning add another woman to their sexual romp, for a classic menage a trois.

Certainly, knowledge of this could have enraged a jealous ex-husband, a noted former world class athlete, a man of great muscular strength and speed, physically capable of committing the crime. And probably the only person in the world, being the dog’s master, to be immune from any sort of attack by an adult Akita, a type bred for centuries to protect its master and mistress.

Elementary, my dear Watson, the great detective might have concluded. This was the dog that didn’t bite. There can be no doubt as to the identity of the killer.

Go to TOP


Dr. A. N. Feldzamen
3 Arrowood Lane, Ithaca, New York 14850-9793
607-257-8080
alfeld@twcny.rr.com