Monday, 5 September 2016

RIP IDA – "wildly unrealistic expectations"

No need to say it, it goes without saying, it should be obvious to all but,
just in case it isn't obvious to all,
IDA is dead.

IDA, now known as "GOV.UK Verify (RIP)",
is the Cabinet Office Identity Assurance programme.
And it's dead.

"If Verify is the answer, what was the question?"

Take a look at New GOV.UK Verify [RIP] chief sets out stall after departure of Janet Hughes. That's a Civil Service World (CSW) article, 23 August 2016, and there's something in there for everyone.

"If there's a tricky job facing the Government Digital Service (GDS), or indeed an impossible job, what do they do? Call for Janet Hughes". That's what we said. Several times. Now the heroic Janet has left GDS.

Can she be replaced?

A, B, C and D below say no, she can't be. E, F, G and H are waiting in the wings.

Are GDS advertising for a replacement? You take a look.

A Jess McEvoy is standing in as interim programme director of GOV.UK Verify (RIP). CSW say: "According to McEvoy, Verify has now been used to verify more than 800,000 individual identities, with more than three quarters of users reporting that they are either satisfied or very satisfied with the service". Is she right?

GDS have three ways of measuring user satisfaction on the GOV.UK Verify (RIP) performance dashboard. 84.11% of respondents say that they are satisfied or very satisfied from the point of view of security. That figure falls to 72.17% from the point of view of certified companies and 64.97% from the point of view of verification.

Never mind what these categories mean – security, certified companies and verification – in each case there have been about 11,100 respondents out of 821,000 or so GOV.UK Verify (RIP) accountholders. That's a 1.35% response rate.

About 821,000 accounts have been created. They have been used about 844,000 times, i.e. about once each:
  • How many accountholders are there? GDS provide no answer. If each accountholder has seven accounts, one with each of the remaining "identity providers", we could be talking about only 118,000 people, not 821,000.
  • Are these people using GOV.UK Verify (RIP) or just trying it out once and then going back to the Government Gateway? That is, is Ms McEvoy right to refer to them as "users"?
  • Why are the user satisfaction statistics four weeks out of date at the time of writing?
  • Less than three-quarters of respondents are satisfied or very satisfied in two of the user satisfaction categories, where does Ms McEvoy get more than three-quarters from?
  • Do the respondents constitute a representative cross-section of the population from which it is legitimate to extrapolate? Or would it be better to say that 64.97% of 1.35% = 0.88% of users, if that's what they are, are satisfied or very satisfied with GOV.UK Verify (RIP) from the point of view of verification?
  • Do the respondents understand the question? Do they each understand the same thing by the question? Or are they all answering different questions?
GDS are meant to be the experts in data analytics. If this is how they handle statistics, they are in danger of suffering the same fate as the pollsters who get referendum and general election results hopelessly wrong – no-one will believe them.

B Ms McEvoy is supported in the CSW article by Jessica Figueras, chief analyst for technology consultancy Kable: "Figueras said it 'should not come as any surprise' if HMRC was considering other options for identiy verification, because the original plan for Verify was for it 'to provide low to medium security ID assurance for citizens, and this hasn’t changed' ...". Is she right?

Presumably Ms Figueras is talking about low-to-medium assurance, not low-to-medium security. GDS claim to offer nothing but unqualified high security.

Take a look at the 9 October 2014 IPV Operations Manual published by GDS. That document covers identity-proofing and verification for GOV.UK Verify (RIP). Para.5 on p.5 specifies registration requirements at both identity assurance level 2 (civil courts) and level 3 (criminal courts). Para.58 also includes level 3 requirements. So does para.71. So much of paras.87-91 has been blanked out that it's impossible to know for sure but it looks as though GDS are talking about more than low-to-medium security. And so on, para.113, para.118, ...

Ms Figueras appears to be wrong. If GOV.UK Verify (RIP) is now required to provide only low-to-medium assurance as to people's identity, then, surprisingly, the specification has been quietly changed since October 2014.

Suppose for a moment, though, that she's right. CSW talk about GOV.UK Verify (RIP) "allowing drivers to tell the DVLA about their medical conditions and allowing mortgage deeds to be signed through the Land Registry". They talk about "offering the service to NHS trusts and local authorities, as well as private sector organisations". Are DVLA and the Land Registry and NHS trusts and local authorities and private sector organisations happy to accept low-to-medium assurance as to people's identity?

"The fact is", says Ms Figueras, "that Verify is an incredibly ambitious programme and the fundamental concepts behind it were untested". Incredible? Untested? Is that meant to increase the confidence of DVLA, the Land Registry and the rest?

"Figueras said the main problem faced by Verify had been the 'wildly unrealistic expectations for roll-out' ...". Wildly unrealistic? With support like this, GOV.UK Verify (RIP) doesn't need any detractors.

C She is also supported by Daniel Thornton of the Institute for Government, who "explained why HMRC might opt for its own verification system". He's quite right, of course. GOV.UK Verify (RIP) can at best only verify the identity of natural persons, not legal persons like companies and partnerships and trusts and, as Mr Thornton says, that's no use to HMRC, they need "something that will work with businesses as well as individuals".

Was that always meant to be the case?

No. GDS used to hold out the prospect of their scheme verifying the identity of legal persons, please see Good Practice Guide 46, published by GDS on 18 October 2013: "This guide deals with proving the authenticity (identity) of a legal organisation, such as a business, partnership, charity, government body or other public sector organisation".

GOV.UK Verify (RIP) is shrinking and it is pointless to pretend otherwise. As it shrinks it is of interest to fewer and fewer organisations.

D Talking of untested fundamental concepts, is it feasible to verify millions of people's identities on-line and only on-line to a level of assurance satisfactory to the likes of the NHS and local authorities? The US National Institute of Standards and Technology (NIST) raise that question. Their answer seems to be no. They consider the identity-proofing work done in GOV.UK Verify (RIP) to be pointless. They class it as no better than self-certification.

Connect.gov in the US has been terminated. GOV.UK Verify. 821,000 self-certifications. RIP?

----------

Updated St Patrick's Day 2017

Verify service manager sought to lead GDS expansion ambitions, we read on 15 March 2017, and yesterday GDS to expand Verify team as pressure to increase user numbers mounts.

Last September we said "Jess McEvoy is standing in as interim programme director of GOV.UK Verify (RIP)", please see above. Isn't it Ms McEvoy's job to "lead GDS expansion ambitions" and to "increase user numbers"? Presumably not.

GOV.UK Verify (RIP) hasn't had a named senior responsible owner since Mike Bracken left GDS in September 2015. And it hasn't had a permanent programme director since Janet Hughes left. It's an orphan programme, unwanted and abandoned.

In the circumstances, how is some poor unfortunate service manager supposed to add 24 million verified GOV.UK Verify (RIP) accountholders in three years flat?

No comments:

Post a Comment