1/10/2006

Able Danger Data Mining. How good was it ?

Data Mining Possible? Impossible! Says Buckland at redstate.com. It matches with my experience with it -- law

Probably the only data available [was] immigration data. When he entered or exited, type of visa.. [w/ that] there's just no way to correlate the attributes of a terrorist and assign probabilities to the event ...Prior to 2001 terrorist data included ex army guys from Kansas and 60's era protesters. Picking out an Egyptian student as a terrorist? Just can't happen.
"

Data Mining Possible?

"Now, this information was not obtained through human sources, radio intercepts, or any other confidential communication. Able Danger operated a data mining operation. It accessed 'publicly available information from government immigration agencies, from Internet sites and from paid search engines like LexisNexis.' In other words, Atta's name must have come up as data through this mining, presumably repeatedly in some sort of pattern in order for his name to have any significance to the miners.

"So there must be data referring to Atta then — right? If so, where is it? There must be documents where Atta's name came up, frequently enough so that Atta would stick out among all the other names which come up in through the data mine. And curiously, not one document with Atta's name has yet come to light." [11] (http://martinirepublic.com/item/2102)
[edit]
Impossible!

Buckland, a statistician trained in data mining, posted August 16, 2005, on RedState.org (http://buckland.redstate.org/story/2005/8/16/143637/027):

"Probably the only data available to the government that shows Atta for sure is immigration data. When he entered or exited, type of visa, etc. That's pretty barren ground for predicting interesting stuff like terroristic activities. I have serious doubts that data from Egypt (Atta's homeland) would have been either forthcoming or interesting as it would present horrendous integration issues, and real data miners tend to try to stay away from those. I also doubt that integrating credit bureau stuff would have been worth the hassle, as most of the hijackers seem to have been reasonably well off financially.

"Airline data would have been even less useful. Prior to 9/11 there was no single repository that housed airline data, each airline keeping it's own data separate. Also airline data is extremely hard to work with. There's no identifier in airline data to identify a passenger beyond name. Matching millions of people and their visa data with airline travel patterns is just something that isn't going to happen with a team of 11 guys. That in itself is a project for years and a large team."

No Training Set: "However prior to 9/11 there was almost no record of terrorism by foreigners. Without a large number of actual terrorist events in this country there's just no way to correlate the attributes of a terrorist and assign probabilities to the event. No way to train the model. Prior to 2001 terrorist data included ex army guys from Kansas and 60's era protesters. Picking out an Egyptian student as a terrorist? Just can't happen."

Buckland concludes: "I don't doubt that a group calling themself Able Danger existed, and they may have played a little with data mining techniques. However nothing presented leads me to think that they would have had any success in finding terrorists, and the fact that people are talking without any supporting documentation tells me that it may not exist. A more likely scenario is that some staffers produced some names (any data mining software will produce results). How many names were on the list? Sixty is a number that I've heard, but that hasn't been confirmed. Would a list of 60 names have been meaningful? What about 60,000? With the number of people entering and exiting this country the difference in those list is a rounding error. The 'propensity to terrorism' of the 60,000th name would have been virtually identical to the 60th. That's just the way picking very rare events work."

Able Danger - SourceWatch: "lawn"

1 Comments:

Anonymous Anonymous said...

The Former Able Danger Co, Capt. Phillpott is not a good person. I understand that he is currently under a Naval Inpsector General investigation for becoming an accessory to violation of the civil rights of a African-American sailor onboard the USS Leyte Gulf.

I also understand that The NAVY IG's and COMNAVSURFLANT are trying to White Wash the results to help Capt. Phillpott and ruin the African American sailor to save any reputation that Capt. Scott Jon Phillpott has left.

If he commited the crime then why are the NAVY IGs and COMNAVSURFLANT committing perjury for him. Double standards are illegal.

8/29/2007 02:31:00 PM  

Post a Comment

Links to this post:

Create a Link

<< Home