Conclusion
The case for investing in the nationwide deployment of biometrics has not been made.
Background
In their 7 May 2013 report Performance Lessons from India’s Universal Identification Program one of the lessons that Alan Gelb and Julia Clark (G&C) draw from UID (also known as "Aadhaar") is that ...
Some evidence of reconsideration
But that wasn't their conclusion.
Their conclusion was that the usefulness of biometrics to the US and other countries has already been "shown" or demonstrated or established by Aadhaar.
They're not holding to that.
Now, it transpires, the evidence of Aadhaar is insufficient. Something more is needed – careful study – before the usefulness of today's biometrics to the US is established. We cannot yet say, pace G&C's earlier report, that its usefulness has been demonstrated.
What was G&C's original conclusion based on if not careful study?
Audit
In his comment, Mr Gelb ignores the point about the need for an audit of the biometrics performance figures published by UIDAI, the Unique Identification Authority of India.
A striking omission, G&C are endorsing India's investment in biometrics and recommending the same for the US without first getting an independent expert audit of the performance figures. That would be imprudent behaviour for a responsible investment manager.
G&C are convinced that Aadhaar will be beneficial to the millions of Indians whose prospects of escaping poverty are limited for lack of an official identity. Why are they convinced? Is it any more than a hunch or a hope?
They're not convinced because of any government programmes which depend on Aadhaar – as Mr Gelb says:
The Indians and the Americans and everyone else would be well-advised to insist on an audit before any more of their money is invested in biometrics.
Statistical control
G&C cite a paper by three world-class experts, Messrs Wayman, Possolo and Mansfield (WP&M), which argues that the study of biometrics is out of statistical control – biometrics isn't a scientific discipline.
Their case rests on audits of biometrics systems that the three of them have conducted.
You can examine all the test results you like, WP&M say, but those results will tell you nothing about how biometrics systems will perform in the field, in operational use.
They discuss the implications for US homeland security. The National Institute of Standards and Technology (NIST) has a duty under the USA PATRIOT Act to audit biometrics systems and to certify them. The best NIST can manage is to say that the results of the tests they performed are the results of the tests they performed. They can't predict how the systems will perform in the field. No benefits to homeland security can be assured.
The same audit report on Aadhaar's performance figures would dissipate the will to invest in biometrics, whether in India, the US or anywhere else.
G&C rest their pro-investment case on the Aadhaar figures for False Positive Identification Rate (FPIR) and False Negative Identification Rate (FNIR). It is on the basis of two statistics that they recommend investment in biometrics, a technology which WP&M say is out of statistical control.
Look again at the back end of the quotation above:
You can't measure FNIR in the field. For the reason noted in the DMossEsq blog post – impostors don't come back and tell you that they fooled the system.
So where does UIDAI's figure of 0.0352% for FNIR come from?
They tell us. In their report, Role of Biometric Technology in Aadhaar Enrollment. On pp.18-19. It's the result of a laboratory test:
What Mr Gelb calls "field data" three times in his comment is, in each case, laboratory data – data which WP&M say tells us nothing about how Aadhaar will work in the field.
It's not just WP&M who cast doubt on these statistics. So do G&C themselves, when they note that UIDAI have to "relax" the FNIR to keep the FPIR down to manageable proportions, to avoid "drowning in a sea of false positives". With their butcher's thumb on the scales, UIDAI can make the meat weigh whatever they want. Or, dropping the butcher analogy, by varying the matching threshold, UIDAI can choose whatever FPIR they like.
Whatever these FPIR and FNIR statistics are, one thing is clear – they're not a benchmark. UIDAI have chosen 0.057% for the FPIR and they're sticking to it. It doesn't matter how well Aadhaar performs or how badly, the FPIR will always be 0.057%.
Maths
Mr Gelb says in his comment:
He also says:
Mr Gelb's aim is to prove that the number of false positives generated by Aadhaar is and will remain manageable. There's no need to do any maths to prove that – not when you know that UIDAI have already decided that the FPIR is and always will be 0.057% and therefore is and always will be manageable. It's a management decision and not a scientific observation.
Multiple identities
G&C acknowledge that there is a trade-off between FPIR and FNIR.
In his comment, Mr Gelb says that:
It means that Aadhaar's population register will be crammed full of people with multiple identities.
If any government programmes do start to rely on Aadhaar, then some individuals will be entitled to multiple votes, multiple food rations, multiple fuel allowances, multiple temporary jobs and multiple bank accounts. And if the banks start to rely on biometrics alone to authorise payments, then some individuals will be entitled to multiple benefit payments.
Cyberattack
That means fraud. Large-scale multiple identities in Aadhaar means large-scale fraud. If Mr Gelb is right about the statistics, then Aadhaar is a machine to automate corruption.
The Indian media openly acknowledge the high incidence of corruption in India's current food security and other welfare programmes. Not just the Indian press. The Economist, too. In a staggeringly awful article they wrote:
Along with the Economist's contempt for the Indians goes a crippling naïvety. Why would Aadhaar make corruption harder? Aadhaar could simply automate corruption. It could increase the incidence of corruption, not reduce it.
At the limit, with their butcher's thumb on the scales, UIDAI – or whoever controls Aadhaar, perhaps a cyberattacker – could choose whatever party they like to be the winner of a general election. Please see for example this cautionary tale in the Washington Post – Hacker infiltration ends D.C. online voting trial.
Investment
It is wrong to insist on 100% accuracy, Mr Gelb says:
This looks like a call to be pragmatic.
This is the case you make for investment when you have had to abandon all the unconvincing statistics and unfulfilled promises that bedevil the biometrics industry.
Risk
There is no need whatever for G&C to take the risk of endorsing biometrics. So why take it?
Their report is published by the Center for Global Development (CGD). What are G&C committing CGD to?
Publishing the bald assertoric statement "UID shows that countries with large populations can implement inclusive, precise, high-quality identity systems by using existing technology" opens CGD to the risk that biometrics salesmen will plant stories in the press with lurid headlines like:
This one isn't – Paper highlights positive biometrics role in developing countries:
Planet Biometrics is a marketing organisation for the biometrics industry. CGD is already being co-opted, thanks to G&C's product endorsements, into the worldwide (planetary?) promotion of the biometrics industry.
"Caught in a dragnet", said the headline, 17 July 2011:
Two problems with that.
Firstly, to the mainstream media and the general public, not to mention legislators and public administrators, a biometric is a biometric is a biometric – the distinction won't come across.
Second, US-VISIT uses face recognition and fingerprints, not iris scans. How long before you see the headline:
----------
Updated:
5 June 2013, 19:02
Remember what Mr Gelb said, quite rightly:
18 June 2013
Premature: Computerworld Honors 2013: ID program empowers citizens in India
Not for India either: The Indian experiment is not for us
The case for investing in the nationwide deployment of biometrics has not been made.
Background
In their 7 May 2013 report Performance Lessons from India’s Universal Identification Program one of the lessons that Alan Gelb and Julia Clark (G&C) draw from UID (also known as "Aadhaar") is that ...
... restated a page later as ...
UID’s performance suggests that accurate, biometric-based, identification is quite feasible for large countries, including the US. (p.8)
In his 12 May 2013 blog post Biometrics: will the Center for Global Development reconsider? DMossEsq suggested that this conclusion of G&C's needs to be qualified in at least six ways and should read "the US could safely deploy an identity management scheme based on biometrics":
UID shows that countries with large populations can implement inclusive, precise, high-quality identity systems by using existing technology. (p.9)
- "subject to an annual audit"
- "apart from the possibility of cyberattack"
- "and as long as we've got our maths right"
- "and as long as you realise that it's not identity that's being managed"
- "and as long as you're relaxed about the fact that anyone could have any number of entries on the population register"
- "and the fact that the discipline of biometrics is out of statistical control"
... we hold to our conclusion that the data released provides a very significant benchmark on the capabilities of biometric systems in developing country conditions and one that should be studied carefully by other countries.
Some evidence of reconsideration
But that wasn't their conclusion.
Their conclusion was that the usefulness of biometrics to the US and other countries has already been "shown" or demonstrated or established by Aadhaar.
They're not holding to that.
Now, it transpires, the evidence of Aadhaar is insufficient. Something more is needed – careful study – before the usefulness of today's biometrics to the US is established. We cannot yet say, pace G&C's earlier report, that its usefulness has been demonstrated.
What was G&C's original conclusion based on if not careful study?
Audit
In his comment, Mr Gelb ignores the point about the need for an audit of the biometrics performance figures published by UIDAI, the Unique Identification Authority of India.
A striking omission, G&C are endorsing India's investment in biometrics and recommending the same for the US without first getting an independent expert audit of the performance figures. That would be imprudent behaviour for a responsible investment manager.
G&C are convinced that Aadhaar will be beneficial to the millions of Indians whose prospects of escaping poverty are limited for lack of an official identity. Why are they convinced? Is it any more than a hunch or a hope?
They're not convinced because of any government programmes which depend on Aadhaar – as Mr Gelb says:
Their conviction relies exclusively on the enrolment of people into UIDAI's population register, where they are identified by their biometrics:
It is far too early to assess the UID program record in delivering more effective and inclusive services.
Without an audit, how do G&C know that India's excluded millions really are being granted an identity? Has a benchmark been established? The US doesn't have the same social exclusion problem as India according to G&C so why the interest in using biometrics to identify all Americans?
... we see the data that it [UIDAI] has released on inclusion and accuracy as a very significant benchmark for biometric systems in developing countries, and a major advance on the use of laboratory data. These appear to be the most extensive field data released so far.
The Indians and the Americans and everyone else would be well-advised to insist on an audit before any more of their money is invested in biometrics.
Statistical control
G&C cite a paper by three world-class experts, Messrs Wayman, Possolo and Mansfield (WP&M), which argues that the study of biometrics is out of statistical control – biometrics isn't a scientific discipline.
Their case rests on audits of biometrics systems that the three of them have conducted.
You can examine all the test results you like, WP&M say, but those results will tell you nothing about how biometrics systems will perform in the field, in operational use.
They discuss the implications for US homeland security. The National Institute of Standards and Technology (NIST) has a duty under the USA PATRIOT Act to audit biometrics systems and to certify them. The best NIST can manage is to say that the results of the tests they performed are the results of the tests they performed. They can't predict how the systems will perform in the field. No benefits to homeland security can be assured.
The same audit report on Aadhaar's performance figures would dissipate the will to invest in biometrics, whether in India, the US or anywhere else.
G&C rest their pro-investment case on the Aadhaar figures for False Positive Identification Rate (FPIR) and False Negative Identification Rate (FNIR). It is on the basis of two statistics that they recommend investment in biometrics, a technology which WP&M say is out of statistical control.
Look again at the back end of the quotation above:
That is simply false.
... we see the data that [UIDAI] has released [as] ... a major advance on the use of laboratory data. These appear to be the most extensive field data released so far.
You can't measure FNIR in the field. For the reason noted in the DMossEsq blog post – impostors don't come back and tell you that they fooled the system.
So where does UIDAI's figure of 0.0352% for FNIR come from?
They tell us. In their report, Role of Biometric Technology in Aadhaar Enrollment. On pp.18-19. It's the result of a laboratory test:
UIDAI's figure of 0.057% for FPIR is also the result of a laboratory test (p.18).
False accept (FNIR): To compute FNIR, 31,399 known duplicates were used as probe against gallery of 8.4 crore (84M). The biometric system correctly caught 31,388 duplicates (in other words, it did not catch 11 duplicates). The computed FNIR rate is 0.0352%. Assuming current 0.5% rate of duplicate submissions continues, there would only be a very small number of duplicate Aadhaars issued when the entire country of 120 crores is enrolled.
What Mr Gelb calls "field data" three times in his comment is, in each case, laboratory data – data which WP&M say tells us nothing about how Aadhaar will work in the field.
It's not just WP&M who cast doubt on these statistics. So do G&C themselves, when they note that UIDAI have to "relax" the FNIR to keep the FPIR down to manageable proportions, to avoid "drowning in a sea of false positives". With their butcher's thumb on the scales, UIDAI can make the meat weigh whatever they want. Or, dropping the butcher analogy, by varying the matching threshold, UIDAI can choose whatever FPIR they like.
Whatever these FPIR and FNIR statistics are, one thing is clear – they're not a benchmark. UIDAI have chosen 0.057% for the FPIR and they're sticking to it. It doesn't matter how well Aadhaar performs or how badly, the FPIR will always be 0.057%.
Maths
Mr Gelb says in his comment:
He is saying that the number of matches rises with 0.5*n*(n-1) and that it rises with n². Since 0.5*n*(n-1) is not equal to n² that must be false.
To correct the record, we do not assert that the number of bilateral comparisons is the square of the population, n. It is 0.5*n*(n-1) which rises (as we note) with the square of n.
He also says:
Why 1 billion? Why not 0.8 billion? Or π/5 billion?
...since no identification system will cover 100% of population, we rounded n off to 1 billion for India.
Mr Gelb's aim is to prove that the number of false positives generated by Aadhaar is and will remain manageable. There's no need to do any maths to prove that – not when you know that UIDAI have already decided that the FPIR is and always will be 0.057% and therefore is and always will be manageable. It's a management decision and not a scientific observation.
Multiple identities
G&C acknowledge that there is a trade-off between FPIR and FNIR.
In his comment, Mr Gelb says that:
Hard to understand, it looks as though he is saying that there will be only 7 false positives for every trillion matches. That can't be what he means but, roll with it for the moment, if he is saying that false positives will be at any sort of rock bottom level like 7 per trillion, then he must accept that false negatives will be sky high. That's what the trade-off means.
If we accept the field estimate of 0.057% false positive rate against a data base of 84 million, the rate for a 1:1 comparison would have to be very small, in the range of 7 in one trillion.
It means that Aadhaar's population register will be crammed full of people with multiple identities.
If any government programmes do start to rely on Aadhaar, then some individuals will be entitled to multiple votes, multiple food rations, multiple fuel allowances, multiple temporary jobs and multiple bank accounts. And if the banks start to rely on biometrics alone to authorise payments, then some individuals will be entitled to multiple benefit payments.
Cyberattack
That means fraud. Large-scale multiple identities in Aadhaar means large-scale fraud. If Mr Gelb is right about the statistics, then Aadhaar is a machine to automate corruption.
The Indian media openly acknowledge the high incidence of corruption in India's current food security and other welfare programmes. Not just the Indian press. The Economist, too. In a staggeringly awful article they wrote:
Indian fathers are feckless? Emigré Indians are clever and the stay-at-home ones are dim? Poor people don't need privacy the way Economist journalists, for example, do? "Today's filthy system"? This is the case for Aadhaar put by someone who despises India.
Armed with the system [Aadhaar], India will be able to rethink the nature of its welfare state, cutting back on benefits in kind and market-distorting subsidies, and turning to cash transfers paid directly into the bank accounts of the neediest. Hundreds of millions of the poor must open bank accounts, which is all to the good, because it will bind them into the modern economy. Care must be taken so mothers rather than feckless fathers control funds for their children ...
Mr Nilekani [UIDAI chairman] harnessed the genius of Indians abroad, including a man who helped the New York Stock Exchange crunch its numbers and one of the brains behind WebMD, an American health IT firm ...
India plainly needs better data-protection laws, but even if the existing rules remained unchanged, the threat to liberty would be dwarfed by the gains to welfare: to people who live ten to a room, concerns about privacy sound outlandish.
Some of the resistance is principled, but much comes from the people who do well out of today’s filthy system. Indian politics hinge on patronage—the doling out of opportunities to rob one’s countrymen. [Aadhaar] would make this harder. That is why it faces such fierce opposition, and why it could transform India.
Along with the Economist's contempt for the Indians goes a crippling naïvety. Why would Aadhaar make corruption harder? Aadhaar could simply automate corruption. It could increase the incidence of corruption, not reduce it.
At the limit, with their butcher's thumb on the scales, UIDAI – or whoever controls Aadhaar, perhaps a cyberattacker – could choose whatever party they like to be the winner of a general election. Please see for example this cautionary tale in the Washington Post – Hacker infiltration ends D.C. online voting trial.
Investment
It is wrong to insist on 100% accuracy, Mr Gelb says:
The question is not whether it works or not ...
On multiple identities, no system will be able to guarantee 100 percent accuracy. Certainly not the systems in place in the rich countries where identity theft is hardly unknown! The question is not “whether it works or not” but the precision of one system versus another and relative cost-effectiveness. For some applications, such as access to a health insurance program, one might accept a modest level of duplicate or false identities. For others ...
This looks like a call to be pragmatic.
This is the case you make for investment when you have had to abandon all the unconvincing statistics and unfulfilled promises that bedevil the biometrics industry.
Risk
There is no need whatever for G&C to take the risk of endorsing biometrics. So why take it?
Their report is published by the Center for Global Development (CGD). What are G&C committing CGD to?
Publishing the bald assertoric statement "UID shows that countries with large populations can implement inclusive, precise, high-quality identity systems by using existing technology" opens CGD to the risk that biometrics salesmen will plant stories in the press with lurid headlines like:
"The time has come for the US to do its duty and deploy biometrics for all", biometrics experts Gelb and Clark, of the internationally respected Capitol Hill Center for Global DevelopmentTo be clear, that headline is invented to make a point.
This one isn't – Paper highlights positive biometrics role in developing countries:
That article appeared on the Planet Biometrics website on 15 February 2013 and, to be clear again, it concerns an earlier report by G&C, not the one being discussed here.
The research underpinning the paper was performed by Alan Gelb and Julia Clark at the Center for Global Development. According to Gelb and Clark, civil registration systems are often absent or cover only a fraction of the population. In contrast, people in rich countries are almost all well identified from birth. This “identity gap” is increasingly recognized as not only a symptom of underdevelopment but as a factor that makes development more difficult and less inclusive.
Planet Biometrics is a marketing organisation for the biometrics industry. CGD is already being co-opted, thanks to G&C's product endorsements, into the worldwide (planetary?) promotion of the biometrics industry.
"Caught in a dragnet", said the headline, 17 July 2011:
With CGD's name associated with biometrics, next time the headline could read:
John H. Gass hadn’t had a traffic ticket in years, so the Natick resident was surprised this spring when he received a letter from the Massachusetts Registry of Motor Vehicles informing him to cease driving because his license had been revoked ...
It turned out Gass was flagged because he looks like another driver, not because his image was being used to create a fake identity. His driving privileges were returned but, he alleges in a lawsuit, only after 10 days of bureaucratic wrangling to prove he is who he says he is ...
At least 34 states are using such systems. They help authorities verify a person’s claimed identity and track down people who have multiple licenses under different aliases, such as underage people wanting to buy alcohol, people with previous license suspensions, and people with criminal records trying to evade the law. Lisa Cradit, a spokeswoman for L-1 Identity Solutions, the largest developer of the software, said it can reduce fraud by 80 percent.
Caught in Center for Global Development biometrics dragnetYou may say that that won't happen. G&C/CGD endorse composite fingerprint/iris scan biometrics, not face recognition. They're quite different propositions.
Two problems with that.
Firstly, to the mainstream media and the general public, not to mention legislators and public administrators, a biometric is a biometric is a biometric – the distinction won't come across.
Second, US-VISIT uses face recognition and fingerprints, not iris scans. How long before you see the headline:
"India has better security systems than Uncle Sam", Center for Global Development. Napolitano eruptsNo doubt CGD has enough staff to defend its reputation if and when the tulipmania passes and the world falls out of love with biometrics. But why get involved in the first place?
----------
Updated:
5 June 2013, 19:02
Remember what Mr Gelb said, quite rightly:
That hasn't stopped the IT magazine ComputerWorld going for broke in the product endorsement stakes:
It is far too early to assess the UID program record in delivering more effective and inclusive services.
ComputerWorld have jumped the gun. UIDAI are getting an award for doing something they haven't done yet. Aadhaar hasn't empowered the citizens of India. UIDAI promise that it will, one day, in the future. Even they don't claim that it already has. What possessed ComputerWorld?
Computerworld Honors 2013: ID program empowers citizens in India
Government program, the 21st Century Achievement Award winner for economic development, uses biometrics to assign unique identity numbers, allowing residents of India to participate more fully in society.
18 June 2013
Premature: Computerworld Honors 2013: ID program empowers citizens in India
Not for India either: The Indian experiment is not for us
