Newcomers to this blog are advised to begin with the first two posts, Just the Facts, Ma'am and Case Solved, which explain in very general terms why I believe I've solved this case. Some important questions are answered in the following post, Misunderstandings, Misconceptions, Misdirections. After that feel free to browse whatever topics might interest you (see blog archive).

NB: If anyone has trouble posting a comment, email it to doktorgosh (at) live.com, and I'll post it for you.

Notice to readers of my Kindle book: I recently noticed that, on certain devices (though not all), the Table of Contents begins with Chapter One and omits the Introduction and Preface. Since the Introduction is especially important, I urge everyone to make sure to begin reading at the very beginning of the book, not the first chapter in the Table of Contents. Thank you.

Wednesday, September 7, 2016

Has the Author of the Ransom Note Been Unmasked? Part 2

(.....continued from previous post.)

I don't think Mr. Berger will mind my calling him Tom. Over the last week or so we've exchanged several emails and by this time I see him as a friend. Not that we always agree. But I find him to be affable, open-minded, reasonable and extremely helpful. Also very knowledgeable. As I mentioned last time, when Tom did me the favor of double-checking his previous result with a different set of exemplars, he came up with a completely different "suspect," drawn from the large pool of Enron texts used as a control. Did someone who once worked at Enron write the Ramsey ransom note? I seriously doubt it. The Enron "hit" can be regarded as an artifact of the methodology, designed to control for confirmation bias by including a large sampling of texts from a presumably innocent source.


At that point, puzzled by the new result, and after consulting a respected text on the subject, Tom decided that his samples were not long enough to satisfy a statistically oriented system:
1--You need 6500+ words to TRAIN ie your suspects and corpora
2--You need 500+ words to TEST ie ransom note.
(Tom has also alluded to problems stemming from an attempt to compare verbal with written texts. Ideally they should be treated separately -- but that would cut down on the sample size, as the pool of written texts by Patsy and John is limited.) He decided to toss out the relatively short samples from Haddon, Thomas, and Leopold-Loeb, and possibly he managed to beef up the size of the samples from Patsy and John -- though I doubt he was able to find anything close to 6500 words from each. (The sample I sent from Patsy was less than 3000.) Then, trying one more time, using his favorite Baysian classifier, he once again got Patsy:
As I said, I added more enron to try to get over 6000 words per author.
Ran it again--This time Patsy at 100%.
At this point I see a potential problem. If it's simply a matter of feeding texts into a black box and getting a result, based strictly on probabilities based on prior research, that's one thing. But where we have a situation where there are all sorts of knobs and dials that can be adjusted, such as sample size, number of sample texts, feature sets, and classifiers, we open the door to confirmation bias. If you give the software a spin and it doesn't give you what you want, then you can decide there was a problem with your inputs, adjust them and try again: until you get your desired result. I'm not saying anyone does that deliberately. And I'm not accusing Tom of doing anything like that either. I honestly believe he didn't. Nevertheless, it is a concern.

Tom added some reassuring comments, based on his considerable experience with this sort of methodology:
I dont see it like you---the automatic software is not proof positive, its a strong indication ie red flag. . . . These are not tools to convict but to point the investigation, get more info, direct resources etc.. . .
What this means is that you dont have to give up your theory, because even with my result, the probability is realistically maybe 70-75% because the training sample of patsy and john were lower than recommended. We both look at the results differently--You are disappointed because you want certainty, I am delighted with 70% (a guess) under the circumstances, so we are both happy....your theory still stands! But I have a small window of opportunity too. Thats how I see it anyway.
 Very gracious. But I am nevertheless troubled. Why would this software settle on Patsy out of all the many Enron samples?

There's more, however, because shortly after this report I get the following email:
Hi. . . Things get weirder... I tested your patsy exemplars against 4 patsy texts I have, and it is not similar, according to Jstylo. In fact, john exemplars is a close match to patsy exemplars...In other words, 4 patsy texts I have do not match your patsy exemplars----your john exemplars is a close match to patsy exemplars. Make sense?? me neither.
In other words, the Jstylo software, designed to pick the author of an anonymous text out of a wide array of known exemplars, was unable to match two different texts from the same author. I suspect that, once again, the problem is sample size. Algorithms of this kind are based on probabilities and probabilities are based on "the law of large numbers." The smaller the sample, the less amenable it is to this sort of test. Which brings me to the ransom note itself, weighing in at roughly 370 words (according to an online word counter). According to the text Tom consulted, 500 words are the absolute minimum for a test document, such as the ransom note. We must also take into consideration that some of the ransom note text is drawn from external sources:

"At this time, we have your daughter in our possession. She is safe and unharmed. . ." "You will withdraw $118,000 from your account. $100,000 will be in $100 bills and the remaining $18,000 in $20 bills." Standard ransom note language, obviously no help in identifying anyone.

"If we catch you talking to a stray dog, she dies. If you alert bank authorities, she dies." "Don't try to grow a brain . . . "  Obviously lifted from movie scripts.

"Use that good, Southern common sense of yours." Generally thought to be a deliberate reference to a pet phrase of Patsy's, which should be eliminated from consideration to avoid obvious bias.

That's 66 extraneous words that should be eliminated from the note before attempting a match with Patsy, John, or anyone else. Leaving a text of only 304 usable words, while a length of 500 or over is considered necessary for a meaningful search.

As should now be clear: 1. the Jstylo software has some limitations, possibly due to sample size issues, possibly due to more fundamental design flaws; 2. regardless, the Ramsey ransom note is much too brief to be meaningfully evaluated by statistically oriented methods; 3. the brevity of the note suggests that we need not rely on statistics at all, but can evaluate its content directly.

I recall a saying I heard recently: "If you have a talking dog, you don't need statistics."

In a blog post titled Johnisms, I've already identified several key words or phrases drawn from various utterances of John Ramsey that match similar words and phrases from the note in a manner that, if nothing else, makes one wonder. If anyone thinks he or she can do something similar with Patsy, I invite them to give it a try. Of course, for Tom anything other than a statistical comparison based on distinctive linguistic features is far too crude to be meaningful. That's one point on which we can agree to disagree. When dealing with a text of several thousand words, then obviously a statistical analysis is the only meaningful option -- but when the text is only 300 words or so, it seems to me that statistics are both irrelevant and unnecessary.
----------------------------------------------------------------------------
Postscript:

Despite the many problems with the Jstylo methodology, due to sample size or some other issue, I find it puzzling that Patsy would turn up as a match to the ransom note at all, and for whatever reason. To me, there is nothing whatever in the style of that note that comes close to Patsy's style, either verbal or written. So what is it in the sample Tom found that led to Patsy as author of the note? One thing I can think of is his inclusion of the Christmas message authored jointly by Patsy and John. Was it the "and hence" that triggered the match? Or some other features of that message, possibly authored by John rather than Patsy? Or possibly the phrase "good southern common sense," probably a deliberate reference to Patsy, intended as sarcasm? Possibly the match was just a coincidence? Or maybe she wrote it after all, and I've been wrong. Hard to say. And food for thought.

As for Jstylo and other methods of that type, I find them extremely interesting and very promising, especially where sizeable texts are involved, such as long essays or books. I looked for references to Jstylo via Google, by the way, and could not find any dating from later than 2013. And my efforts to contact some of the people involved in that project have drawn a blank. I have a sneaking suspicion that further testing by independent researchers failed to replicate their results. But I have no doubt that any such problems will be overcome in time, and an exciting new forensic tool, comparable to fingerprinting or DNA identification, will emerge.




16 comments:

  1. Wow, I did not think that you would delve this deeply into this when I posted it but much applause to you for investigaing it so deeply. I only read the basics as to how it works but thought that 1 day it will be refined and will be a wonderful investigative tool as well. I am curious of this though, has this swayed your thinking at all about who wrote the note or no ?

    ReplyDelete
    Replies
    1. Well, first of all I want to thank you for finding it and bringing it to our attention, as it's an important piece of sleuthing, regardless of what it might mean. As I said, this finding is the first such bit of evidence that stopped me cold. I have a lot of respect for this approach to content analysis, as it has the potential to be completely objective and free of confirmation bias.

      But as we see, there was still room for confirmation bias to creep in nevertheless, probably in the choice of examples. Where I think Berger may have gone wrong was his selection of the Xmas message as a sample of Patsy's writing, whereas the more I study it the more the whole thing looks like it was written by John.

      I'd love to do more research with this software myself, but I just haven't been able to get it to work for me. Hopefully I'll get some help from its designers, but so far I haven't heard from them.

      To answer your question: if I'd been able to query this program myself, using a variety of different examples from different sources, and Patsy consistently came out as the most likely match, regardless of what examples of her verbal and/or written language I used, that would have really rocked my boat, yes. I don't know if I'd ever be willing to concede that Patsy wrote the note, because to me that's just impossible. But I would probably have simply thrown up my hands and given up, finding something better to do with my time. And maybe in the long run I'd have been better off, because this is starting to look like a road to nowhere.

      Delete
  2. I am glad you became so interested in this. I think that after the kinks are worked out and it gets refined it will be an awesome tool and may even lead to a couple of other computer facilitated identification programs as well. As far as this case goes I have felt it was a road to nowhere for over 10 years, yet I will read any information on it that I can get my hands on. Unfortunately, I think nothing definitive will come out of any of these shows and they may even muddy the waters even more than they were before. I have studied alot of murder cases, and when I say alot I mean it would be hard for you to bring 1 up that I do not know anything about. I always get a definite feeling in my mind for who the guilty party is and I am usually right, not always but usually. I do, however always get a definitive feel. There have only been a few cases where it took quite awhile to get that definitive feel, the Darlie Routier case and the West Memphis Three are 2 that come to mind. I have never had any, even close to baffling me as badly as the this case does. This case is a giant rabbit hole of epic proportions that you can not escape! I think that because we cannot definitively figure it out in our minds, is the reason you, I and many others are so obsessed with it.

    ReplyDelete
  3. The problem I see with the test is that if Patsy wrote the note and John dictated in part, we then have two sets of comparison in one. If you take out Patsy's letter with the"and hence" and added Johns verbal "and hence" it would probably come up with John as the writer. As you say Doc, some bias of input could come into it.

    ReplyDelete
    Replies
    1. The more I study that Christmas message, the more it looks like John all the way. The phrase "we, as a family" was used elsewhere by John, and the overall style strikes me as his. I really wish I was able to use the Jstylo software as I'd love to see whether that message comes up as closer to John or Patsy.

      Delete
  4. From what I have read PR was really into Christmas, decorating and sending out cards etc. It also appears JR was always gone and busy and did not have his hand in much as far as doing things with the kids and family etc. From what we know I would think that JR had nothing to do with writing out Christmas cards. The same as he did not do much with the kids, the planning of parties etc etc. JR just doesnt seem very involved in much other than his work, so writing out Christmas cards would seem most likely to be PR's job.

    ReplyDelete
    Replies
    1. That seems like speculation, to me. I don't think we know how much John helped at Christmas -- no one that knew them well has spoken about it. But I will suggest this, since I'm very much into Christmas, included the decorating and sending out cards, social events, etc. My husband is very busy with his job, too (as I am I). However, we're both off from work for a week or two at this time of year, and I delegate tasks to the hubby. Obviously, helping with the cards is easy relative to the other work at hand, so he takes that one. I have the family photos done, I order the cards, I select the printed message and I print out mailing labels. He composes the short, personal message. He usually writes it, but he is slow and sometimes I will help finish them up, but you can bet I just write exactly what he wrote because its easiest for me and no one will claim we wrote more to one relative than we did to another. As far as John goes, the "and Hence" is a pattern of his and is not found in Patsy's style of writing.

      Delete
    2. Well I do know this, Patsy would hire people to decorate the house for Christmas as an indicator.

      Delete
  5. That's what I thought, too. Plus, after living with someone for a number of years, a person sometimes unconsciously picks up some phrases and a manner of speaking from their spouse. At least I have.

    ReplyDelete
  6. That was me... Minnesota Linda

    ReplyDelete
  7. Maybe it was John, I really dont know. This case will never be solved so we can speculate forever. thats all we will ever get.

    ReplyDelete
  8. There might be some credence to the idea that both John and Patsy are responsible for the note. I have heard it suggested that Patsy wrote while John dictated at times. This might account for the similarities to both parents' writing/speaking style.

    It is interesting how predetermined opinion can influence perception. Had the results of this Jstylo software said it was 75% John, would you not have jumped for joy and used that as further evidence for your theory?

    You have some great ideas and this is a very well thought out resource. I know a lot of your opinion is based on the fact that Patsy would not have called he police if she were involved and that she did so without John's knowing. I can't help but think that it's possible that Patsy's first reaction would have been to run up and tell John before phoning the police. Especially since the note is addresses to him. There are also other explanations for why the two parents were in on it and left the body in the house. Maybe they couldn't bare to leave her elsewhere? Maybe they thought it too risky? Maybe they wanted her found quickly? It's all speculation.

    ReplyDelete
    Replies
    1. "Had the results of this Jstylo software said it was 75% John, would you not have jumped for joy and used that as further evidence for your theory?"

      Good question. I suppose I'd have welcomed it, yes. But I'm a born skeptic and I'm especially skeptical of any approach to this case where we're expected to accept findings based on some cut and dried system rather than critical analysis. I feel sure I'd have wanted to get hold of this software and see if I could replicate the results myself.

      Also I hope everyone reading here understands that as far as I'm concerned, I'm not here to defend "my theory." In a sense I have no theory. My conclusions are based on my attempt to make sense of all the facts and evidence, based on a logical analysis, not a predetermined theory. If conclusive evidence were to be presented that contradicted my theory and pointed in some other direction, I would follow it.

      Delete
    2. "There are also other explanations for why the two parents were in on it and left the body in the house. Maybe they couldn't bare to leave her elsewhere? Maybe they thought it too risky? Maybe they wanted her found quickly? It's all speculation."

      No, it's not all speculation. Not that part of it, anyhow, not the basics. As I see it there are no other explanations for why two people going to such trouble to stage a kidnapping would agree to call the police on themselves with the body still in the house and then hand them evidence that could be used against them in the form of a patently phony ransom note.

      If they had second thoughts about their original plan, or wanted the body found quickly then they would not have reported a kidnapping and handed the police an obviously phony "ransom note." Once the body was found in the house, it was clear to the police (at least most of them) that it was an inside job. The only reason John wasn't indicted was the decision to rule him out as writer of the note, which was a huge mistake.

      Delete
    3. I'm thinking the ransom note was to explain why JB was killed instead of kidnapped. Then the call to 911 would be deliberate, to trigger the "kidnapper" to "kill" JB. Like the RN threatened. And to match JB being dead.

      Also, he, she or they probably didn't want to leave JB's body somewhere. They probably wanted to bury her in a nice ceremony.

      So if Patsy was in on it, that's why she would call 911. To pretend the killer was in the basement with JB alive. Then they called the cops, and everyone else, so the kidnapper killed her and left her there.

      But there was no reason to call them that early. Unless they had been thinking about it all night and were anxious to get on with it.

      Dona

      Delete
  9. I would like to say that this site is one of the best I've come across on the subject. Everyone seems to articulate their meaning of certain evidence in a very respectful and responsible way. This case sometimes keeps me up at night going over certain scenarios. One question I'd like to ask ...on many of the televised interviews it seemed to me that JR had coached PR on certain ways to respond to questions. I also observed that even in some instances where a question was asked to her and as she's answering that if you look at him and see that he's watching her closely and his mouth/lips are at same time she's answering are saying answer in tandem. Kind of like when you've worked with your child on an important speech for school and when the day arrives you sit in audience and can mouth the words coming from their lips! Another question I'd like to ask is...Am I correct that there were reports of cigars in or near basement? And a crafting area? Any glue gun or perhaps the device used to burn design on wood? Just a thought because this is what this case tends to make one do! To the point of exhaustion in fact! Thank you

    ReplyDelete