Thursday 16 January 2014

"The cloud is a giant security and reliability disaster waiting to happen"

Computer Weekly magazine:
Banks should never use the cloud

By Karl Flinders on January 15, 2014 2:44 PM

I have been working on a feature today and going through my interviews have found some interesting stuff.

This one comes from an unnamed source within banking IT. This is what he said when asked about the cloud's role in banking.

"None at all hopefully. The cloud is a giant security and reliability disaster waiting to happen. Banks should keep their systems safely locked away in their own data centres and do all they can to protect the infrastructure and physical security. I hope the cloud is only used for holiday snaps and music. Banks should not go there. We have to remember there are bad guys out there trying to crack into these systems millions of times a day around the world. And they only have to get it right once to cause a major disaster! I would not bank with a firm using the cloud to operate my account or hold my details."

So that's pretty clear then.

I recently wrote this article after an event about the cloud in banking: Is cloud computing almost too good to be true for banks?.
So who should use the cloud? For whom doesn't it matter that the cloud is a giant security and reliability disaster waiting to happen?

"The cloud is a giant security and reliability disaster waiting to happen"

Computer Weekly magazine:
Banks should never use the cloud

By Karl Flinders on January 15, 2014 2:44 PM

I have been working on a feature today and going through my interviews have found some interesting stuff.

This one comes from an unnamed source within banking IT. This is what he said when asked about the cloud's role in banking.

"None at all hopefully. The cloud is a giant security and reliability disaster waiting to happen. Banks should keep their systems safely locked away in their own data centres and do all they can to protect the infrastructure and physical security. I hope the cloud is only used for holiday snaps and music. Banks should not go there. We have to remember there are bad guys out there trying to crack into these systems millions of times a day around the world. And they only have to get it right once to cause a major disaster! I would not bank with a firm using the cloud to operate my account or hold my details."

So that's pretty clear then.

I recently wrote this article after an event about the cloud in banking: Is cloud computing almost too good to be true for banks?.
So who should use the cloud? For whom doesn't it matter that the cloud is a giant security and reliability disaster waiting to happen?

Tuesday 14 January 2014

Whitehall schizophrenia – the cartoon

We have noted before that Whitehall is at one and the same time advising individuals and businesses (a) that the web is dangerous and (b) that we should put all our personal data on-line in the cloud. Please see The on-line safety of the mooncalves, 4 July 2013.

Six months later and it's happening again.

The nice Dr Jekyll at the Home Office issued a press release the other day, New campaign urges people to be 'Cyber Streetwise':
A new campaign to change the way people protect themselves from falling victim to cyber criminals has been launched by the government.

The ‘Cyber Streetwise’ campaign aims to change the way people view online safety and provide the public and businesses with the skills and knowledge they need to take control of their cyber security. The campaign includes a new easy-to-use website and online videos.
Meanwhile, thanks to all the nasty Mr Hydes, Whitehall departments are shunting their systems into the cloud as fast as possible with our data in them. No more efficient way of losing control of our data has yet been discovered.

We're used to the schizophrenia.

That's now been joined by infantilism.

We already have a grown-up website giving us advice how to protect ourselves on the web, Get Safe Online. The new website seems to have been designed for children, Cyber Streetwise.

What did we do in our previous lives to deserve this:


----------

Updated 14.2.14

It transpires that the National Cyber Security Programme is spending £4 million on the Cyber Streetwise campaign, which is said to "look like a site aimed at children rather than adults and business owners". The Office of Cyber Security & Information Assurance (OCSIA) whose campaign it is comes under Francis Maude.

Whitehall schizophrenia – the cartoon

We have noted before that Whitehall is at one and the same time advising individuals and businesses (a) that the web is dangerous and (b) that we should put all our personal data on-line in the cloud. Please see The on-line safety of the mooncalves, 4 July 2013.

Six months later and it's happening again.

The nice Dr Jekyll at the Home Office issued a press release the other day, New campaign urges people to be 'Cyber Streetwise':
A new campaign to change the way people protect themselves from falling victim to cyber criminals has been launched by the government.

The ‘Cyber Streetwise’ campaign aims to change the way people view online safety and provide the public and businesses with the skills and knowledge they need to take control of their cyber security. The campaign includes a new easy-to-use website and online videos.
Meanwhile, thanks to all the nasty Mr Hydes, Whitehall departments are shunting their systems into the cloud as fast as possible with our data in them. No more efficient way of losing control of our data has yet been discovered.

We're used to the schizophrenia.

That's now been joined by infantilism.

Seven professors and a virtuous circle

Interoperability between central and local government identity assurance schemes

The project highlighted the issue of accurate data matching, specifically the matching of names and addresses originating from different sources. (p.9)

The complexity of data matching may present a significant barrier to implementation by Service Providers. (p.10)

The project has highlighted shortcomings in the user journey arising from the technical implementation of the IDA Scheme. (p.10)

... considerably more thought needs to be applied in this area [stepping up from LoA1 to LoA2] if it is to become a viable proposition going forward. (p.10)

... at the time of this project, the functionality required to deliver user data directly within the IDA Scheme [to create a new account] had yet to be developed ... The consequence is that the user is faced with a convoluted process when using the IDA Scheme for the first time. (p.11)

User experience testing was performed in a laboratory environment and involved 5 [sic] users on a one-to-one basis with an experienced research facilitator provided by GDS. Each user had extensive experience of online services including internet banking, government services and social media such as Facebook and Twitter ... The feedback from the small sample of users was generally fairly consistent. (p.12)

Most users would be very reluctant to use their social media accounts with a government site, the prevailing view being that their social life is distinctly separate to doing “business” with government. The issue of privacy and the feeling that government would be able to “see my social life”, or that government transactions would appear in their social media profiles, was of concern. (p.12)

The Hub ... users often struggled as they sought to understand how this method of signing in to government services worked. The Hub service provided the user with a link to a video clip that described the scheme and its purpose ... (pp.12-3)

Users were not clear why private sector companies were being used to carry out identity assurance on behalf of government. (p.13)

Some aspects of the registration processes proved annoying to the users ... (p.13)
GDS, the Government Digital Service, used Warwickshire County Council to alpha test IDA, the identity assurance system they have been putting together for some years now.

The alpha was reported on by OIX, the Open Identity Exchange. A selection of their findings is reproduced alongside.

Certain words and phrases stand out. "Significant barrier", for example, and "shortcomings". "Considerably more thought needs to be applied", "convoluted process", "reluctant" and "struggled". "Not clear" and "annoying".

The alpha was also reported on by David Rennie, a member of GDS, in Steering Collaboration, 26 November 2013. He says:
The alpha project was used to test integration between identity providers and the identity assurance hub and provides insights about how users of local authority services respond to the concept. The alpha found that identity assurance will support the move to digital by default, simplify and improve the customer experience and make service providers more efficient. In short, a virtuous circle of reduced effort, reduced cost and improved customer satisfaction.
You wouldn't know he was talking about the same test, would you?

The disconnect is total.

What's going on?

In their book The Blunders of Our Governments Professors Anthony King and Ivor Crewe talk about several of the causes of failure in government projects. Among them, group-think, which they blame for the Poll Tax, for example.

Group-think was given its first academic treatment apparently by Irving J Janis, a US psychology professor. Messrs King and Crewe have this to say about it (pp.255-6):

According to Janis, whose views are now almost universally accepted, group-think is liable to occur when the members of any face-to-face group feel under pressure to maintain the group's cohesion or are anyway inclined to want to do that.

It is also liable to occur when the group in question feels threatened by an outside group or comes, for whatever reason, to regard one or more outside individuals or groups as alien or hostile.

Group-think need not always, but often does, manifest itself in pathological ways. A majority of the group's members may become intolerant of dissenting voices within the group and find ways, subtle or overt, of silencing them. Individual group members may begin to engage in self-censorship, suppressing any doubts they harbour about courses of action that the group seems intent on adopting. Latent disagreements may thus fail to surface, one result being that the members of the group come to believe they are unanimous when in reality they may not be.

Meanwhile, the group is likely to become increasingly reluctant to engage with outsiders and to seek out information that might run counter to any emerging consensus. If unwelcome information does happen to come the group's way it is likely to be discounted or disregarded. Warning signs are ignored. The group at the same time fails to engage in rigorous reality-testing, with possible alternative courses of action not being realistically appraised.

Group-think is also, in Janis's view, liable to create “an illusion of invulnerability, shared by most or all the members, which creates excessive optimism and encourages taking extreme risks”. Not least, those indulging in group-think are liable to persuade themselves that the majority of their opponents and critics are, if not actually wicked, then at least stupid, misguided and probably self-interested.

It's not just the Warwickshire County Council alpha test. Once you've got the group-think idea in your head, the examples start to multiply.

For example, it is a year now since four professors published their draft review of GDS's digital strategy. They were not impressed. GDS's response? They have ignored the professors' criticisms. They have "discounted or disregarded" them.

Is that a problem? Or is it a "virtuous circle of reduced effort, reduced cost and improved customer satisfaction"?

Seven professors and a virtuous circle

Interoperability between central and local government identity assurance schemes

The project highlighted the issue of accurate data matching, specifically the matching of names and addresses originating from different sources. (p.9)

The complexity of data matching may present a significant barrier to implementation by Service Providers. (p.10)

The project has highlighted shortcomings in the user journey arising from the technical implementation of the IDA Scheme. (p.10)

... considerably more thought needs to be applied in this area [stepping up from LoA1 to LoA2] if it is to become a viable proposition going forward. (p.10)

... at the time of this project, the functionality required to deliver user data directly within the IDA Scheme [to create a new account] had yet to be developed ... The consequence is that the user is faced with a convoluted process when using the IDA Scheme for the first time. (p.11)

User experience testing was performed in a laboratory environment and involved 5 [sic] users on a one-to-one basis with an experienced research facilitator provided by GDS. Each user had extensive experience of online services including internet banking, government services and social media such as Facebook and Twitter ... The feedback from the small sample of users was generally fairly consistent. (p.12)

Most users would be very reluctant to use their social media accounts with a government site, the prevailing view being that their social life is distinctly separate to doing “business” with government. The issue of privacy and the feeling that government would be able to “see my social life”, or that government transactions would appear in their social media profiles, was of concern. (p.12)

The Hub ... users often struggled as they sought to understand how this method of signing in to government services worked. The Hub service provided the user with a link to a video clip that described the scheme and its purpose ... (pp.12-3)

Users were not clear why private sector companies were being used to carry out identity assurance on behalf of government. (p.13)

Some aspects of the registration processes proved annoying to the users ... (p.13)
GDS, the Government Digital Service, used Warwickshire County Council to alpha test IDA, the identity assurance system they have been putting together for some years now.

The alpha was reported on by OIX, the Open Identity Exchange. A selection of their findings is reproduced alongside.

Certain words and phrases stand out. "Significant barrier", for example, and "shortcomings". "Considerably more thought needs to be applied", "convoluted process", "reluctant" and "struggled". "Not clear" and "annoying".

The alpha was also reported on by David Rennie, a member of GDS, in Steering Collaboration, 26 November 2013. He says:
The alpha project was used to test integration between identity providers and the identity assurance hub and provides insights about how users of local authority services respond to the concept. The alpha found that identity assurance will support the move to digital by default, simplify and improve the customer experience and make service providers more efficient. In short, a virtuous circle of reduced effort, reduced cost and improved customer satisfaction.
You wouldn't know he was talking about the same test, would you?

The disconnect is total.

What's going on?

In their book The Blunders of Our Governments Professors Anthony King and Ivor Crewe talk about several of the causes of failure in government projects. Among them, group-think, which they blame for the Poll Tax, for example.

Group-think was given its first academic treatment apparently by Irving J Janis, a US psychology professor. Messrs King and Crewe have this to say about it (pp.255-6):

According to Janis, whose views are now almost universally accepted, group-think is liable to occur when the members of any face-to-face group feel under pressure to maintain the group's cohesion or are anyway inclined to want to do that.

It is also liable to occur when the group in question feels threatened by an outside group or comes, for whatever reason, to regard one or more outside individuals or groups as alien or hostile.

RIP IDA – Warwickshire County Council

No need to say it, it goes without saying, it should be obvious to all but, just in case it isn't obvious to all, IDA is dead.

IDA is the Cabinet Office Identity Assurance programme. And it's dead.

----------

"Happy new you", says Steve Wreyford.

He's the Government Digital Service man (GDS), you'll remember, the sexton, digging the grave for IDA, GDS's identity assurance programme.

"Identity Assurance gets closer to market", he told us over 18 months ago, on 25 May 2012. Four days later we learned from him that "Identity Assurance goes to Washington", which is all very well, but was IDA coming to the UK?

The answer wasn't clear but, next best thing, OIX – "Cabinet Office joins the Open Identity Exchange". That was 14 June 2012. Then there were months of silence before Mr Wreyford claimed that IDA was "Less About Identity, More About Trust" (4 October 2012). Our privacy would be respected by IDA and we would be in control of our data. How? No answer.

Roll forward to 14 November 2012. "Identity assurance – Stepping Up A Gear". And about time too. After all, at this stage, the promise was that IDA would be ready for 21 million benefit claimants by March 2013. Anyway, at least Mr Wreyford now had seven "identity providers" (IDPs), with the promise of an eighth one coming soon. And indeed, only two months later, on 17 January 2013, PayPal joined the IDP fold, "To Identity and Beyond".

But what's this we read? "Of course, this is just the beginning of the process. The real work of realising our ambitions for identity assurance services can now begin." Were the 21 million benefit claimants going to get IDA by March?

No.

Instead, GDS went off to South Yorkshire "to test the theory that if you make it easy for people to establish their identity when accessing digital public services, people will choose to access them digitally rather than pick up the phone or go to a branch", please see "Identity Alphas", 12 March 2013.

Three people are believed to have taken part in that test, although the figure was later revised upwards to 15. But not to 21 million.

Sextons need a sense of humour and on 15 April 2013 Mr Wreyford wrote "Delivering Identity Assurance: You must be certified", in which he answered the question how do we members of the public know if we can trust a digital service. Answer, the supplier will be certified by tScheme, an organisation that claims to have worked out how to measure trust. But how do you know if you can trust tScheme?

Further, as one fusspot commenting on Mr Wreyford's post said: "Possibly a silly question – and I may not be reading the article correctly – but shouldn't the [identity] providers be certified *before* being appointed?".

Public performers know how to handle this sort of contumely. Quick as a flash, Mr Wreyford organised the "Digital Identity Summit", 15 October 2013, where GDS were joined by "Australia, Canada, Denmark, Japan, New Zealand and Sweden (sadly the United States were unable to join us due to the shutdown)".

Which may be why there was no time to conduct the HMRC PAYE on-line trial which had been promised for October 2013.

Luckily, while he was busy with the summit, CESG helped by producing "Good Practice Guide 46", 18 October 2013, which good practice may or may not have been implemented in the ID hub which appeared from nowhere on 30 October 2013, "A hub is born", which, in turn, brings us, finally, to "Happy new you", where we started: "Before the end of this year [2014] you’ll be able to use our service to prove that you are who you say you are online. A whole new you, if you like".

We've heard that before, of course. IDA was meant to be live in 2013. And in 2012. Will 2014 be any better? Is there a whole new Steve Wreyford, perhaps? A whole new sexton?

It's wrong to concentrate on Steve Wreyford. He's just the sexton. The senior responsible owner of the graveyard is ex-Guardian man Mike Bracken CBE.

He's the one who said all those years ago on 1 March 2012 that: "GDS has been working closely with DWP to revise the OJEU and agree it with other Departments. In the first instance, IDA digital services will be used to support Universal Credit and the Personal Independence Payment, which from 2013 will replace DWP’s current benefit system". That was in "Identity: One small step for all of Government". In the event, DWP missed their deadline. So did GDS.

And he's the one who, on 16 October 2013, gave the Code for America Summit 2013 the impression that IDA already provides proof of identity to 45 million users.

Is there any reason to believe that 2014 will be the year?

We don't know what happened in the South Yorkshire trial. That can't give us any hope. We know that relations with DWP are rocky. Also with the Electoral Commission. We know that the HMRC trial planned for October 2013 didn't take place. And we know that CloudStore keeps falling over ever since GDS took charge of it – we can't have the ID hub falling over, life would stop.

There is one other potential source of hope.

Warwickshire.

Interoperability between central and local government identity assurance schemes

The project highlighted the issue of accurate data matching, specifically the matching of names and addresses originating from different sources. (p.9)

The complexity of data matching may present a significant barrier to implementation by Service Providers. (p.10)

The project has highlighted shortcomings in the user journey arising from the technical implementation of the IDA Scheme. (p.10)

...  considerably more thought needs to be applied in this area [stepping up from LoA1 to LoA2] if it is to become a viable proposition going forward. (p.10)

... at the time of this project, the functionality required to deliver user data directly within the IDA Scheme [to create a new account] had yet to be developed ... The consequence is that the user is faced with a convoluted process when using the IDA Scheme for the first time. (p.11)

User experience testing was performed in a laboratory environment and involved 5 [sic] users on a one-to-one basis with an experienced research facilitator provided by GDS. Each user had extensive experience of online services including internet banking, government services and social media such as Facebook and Twitter ... The feedback from the small sample of users was generally fairly consistent. (p.12)

Most users would be very reluctant to use their social media accounts with a government site, the prevailing view being that their social life is distinctly separate to doing “business” with government. The issue of privacy and the feeling that government would be able to “see my social life”, or that government transactions would appear in their social media profiles, was of concern. (p.12)

The Hub ... users often struggled as they sought to understand how this method of signing in to government services worked. The Hub service provided the user with a link to a video clip that described the scheme and its purpose ... (pp.12-3)

Users were not clear why private sector companies were being used to carry out identity assurance on behalf of government. (p.13)

Some aspects of the registration processes proved annoying to the users ... (p.13)
Warwickshire County Council conducted a trial of IDA – a very limited, primitive trial:

  • They worked with GDS and three of GDS's IDPs – Mydex, PayPal and Verizon (p.7). We don't know when this trial took place. It must have been some time before 3 September 2013 when we learned that PayPal are no longer in the IDA "framework", they've pulled out. One other thing we know is that none of these three is certified by tScheme – as Steve Wreyford has warned us, they are therefore not to be trusted.
  • P.5 of OIX's report on the trial says: "The IDA Scheme will eventually support the four recognized levels of identity assurance (as set out in GPG45). In most cases in local government, online services will require a Level of Assurance 1 or 2 (LoA1, LoA2). ... LoA2 is significant in that it is a level of assurance that would be expected to stand up in a Civil Court of Law in England and Wales, but not in a Criminal Court". That's all they tested. This trial can tell us nothing about IDA being proof against fraud, a criminal offence, obviously.
  • There is some mention of privacy (p.6) but the extent to which users can be confident that they are in control of their own data was not tested. In the absence of any such test users would be well advised to assume that they can have no confidence on that score.
  • "The principle [sic] areas of investigation were ... Users’ acceptance of social media IDs as a means of obtaining personal information for transactions requiring low levels of trust" (p.8). As anyone could have predicted, the sensible people of Warwickshire weren't having any of that nonsense.
  • The idea was to see what happens if users try to use IDA to avail themselves of services provided by (a) Warwickshire County Council and (b) DVLA, the Driver and Vehicle Licensing Agency. But not the real Warwickshire County Council and not the real DVLA: "The project utilised two Service Providers. The first was a mocked-up central government agency, the Driver and Vehicle Licensing Agency (DVLA) and the second a mocked-up WCC site". Again, the results of the trial must be of limited value.
And what were those results?

They are shown in the sidebar alongside. And they are not flattering to IDA.

The Cabinet Office has had since at least 20 September 2010 to work on IDA. Over three years. And there's clearly a long way to go still, before they have a product.

Not a shining example of agile software engineering.

Nor of the "simpler, clearer, faster" motto of GOV.UK – not if users have to stop and watch a video before they can use the ID hub. "The feedback from the small sample of users was generally fairly consistent", we learn. Consistently what? Hostile? Baffled? Incredulous?

Would the Major Projects Authority give IDA an amber/red rating? Or would they go straight to red?

What is it with GDS and data-matching? Why can't they do it? That was one of the problems they had working with the Electoral Commission.

Warwickshire County Council were hoping that IDA would help them to save money, in view of the cuts being made to local government budgets. In the event, they can have no idea how much IDA would cost to operate. For all they know after this trial, it might be cheaper for DWP staff, for example, to visit people at home.

"Cuts in numbers and pay restraints have combined with mounting evidence of unfitness, for example in commissioning and contracting, together with the profound unwillingness of serving civil servants to think outside the Whitehall box – which increasingly resembles a coffin". That was David Walker writing in the Guardian yesterday.

Coffin?

We've got just the man.


RIP IDA – Warwickshire County Council

No need to say it, it goes without saying, it should be obvious to all but, just in case it isn't obvious to all, IDA is dead.

IDA is the Cabinet Office Identity Assurance programme. And it's dead.

----------

"Happy new you", says Steve Wreyford.

He's the Government Digital Service man (GDS), you'll remember, the sexton, digging the grave for IDA, GDS's identity assurance programme.

"Identity Assurance gets closer to market", he told us over 18 months ago, on 25 May 2012. Four days later we learned from him that "Identity Assurance goes to Washington", which is all very well, but was IDA coming to the UK?

The answer wasn't clear but, next best thing, OIX – "Cabinet Office joins the Open Identity Exchange". That was 14 June 2012. Then there were months of silence before Mr Wreyford claimed that IDA was "Less About Identity, More About Trust" (4 October 2012). Our privacy would be respected by IDA and we would be in control of our data. How? No answer.

Sunday 12 January 2014

Agile is the opposite of waterfall – no

The Iguazu Falls (healthy/agile)
The Department for Work and Pensions have written off millions of pounds spent on developing IT for Universal Credit and we expect the write-off to rise into the hundreds of millions.

How can we stop intelligent organisations from wasting money like this?

Over and over again we are told that the answer is "agile".

Use "agile" software engineering methods and the waste will be minimised.

How? What problem is "agile" solving?

Over and over again we are told that "agile" is to be contrasted with "waterfall". Waste is endemic in "waterfall" software engineering methods. That's the problem. And "agile" will solve it – that's the suggestion.

That's the suggestion made by ex-Guardian man Mike Bracken CBE, for example, when he was over in the US telling the Americans how to do government IT last October:
What is your reaction to HealthCare.gov and what you're reading and seeing regarding failures of what was meant to be an Expedia shopping for health coverage?

Yeah ... I'll say this with no sense of enjoyment whatsoever, but it feels a bit like Groundhog Day to where we were three or four years ago. Hundreds of millions of dollars, large-scale IT enterprise technology, no real user testing, no real focus on end users, all done behind a black box, and not in an agile way but in a big waterfall way, which is a software methodology. And basically not proven good value, and I'm afraid to say I've got example after example in the U.K. in the past where we've had that experience. So it looks just like one of those.
As a further example, that's the suggestion in Richard Bacon MP and Christopher Hope's Conundrum: Why every government gets things wrong and what we can do about it, pp.240-1:
The traditional approach to software development is often known as 'waterfall' development: that is, you plan, build, test, review and then deploy, in a relentless cascade. But some IT industry players regard this practice as the chief problem ...A rather different answer which has emerged in the last ten to fifteen years has been what are called 'Agile Systems', perhaps best described as a philosophical movement in action within the software industry.
Niagara Falls, January 2014 (unhealthy/DWP)
The suggestion is nonsense. There's nothing wrong with the "waterfall" method. You can't get away from the "waterfall" method. All the "waterfall" method says is that you can't deploy a system until you've coded and tested it and you can't code and test it until you've designed it and you can't design it until you've analysed the requirements.

If you reject the "waterfall" method, you must believe that you should start designing a system before you know what's required and if you believe that then you're no use to the benefit claimants who need Universal Credit.

All that its proponents tell us about "agile" is that it's not "waterfall". But the "waterfall" method is right. It's the only method there is. It must be. There it is right bang in the middle of the "agile" method professed by ex-Guardian man Mike Bracken CBE's Government Digital Service – first you have a discovery phase, then an alpha and a beta, then you go live. That's a waterfall.

You can iterate, of course. You can have one release/deployment after another as the system is maintained and enhanced. But what you're iterating is a waterfall. There is no escape from the waterfall. And no need to escape from it. What would you be escaping from? From the belief that you must analyse before you design? But that's madness.

Madness won't solve DWP's problems. And those problems are not diagnosed simply by saying "waterfall" as though that's all bad and "agile", whatever it is, as though that's all good.

This distinction being touted between "agile" and "waterfall" is false.

----------

The picture of Mr Boehm's spiral has been added, as have the hyperlinks in the four citations at the end. but otherwise unchanged here's an extract from someone's essay written 12 years ago in February 2002 as part of his MSc in software engineering taken after 25 years of doing the job. For what it's worth:

Waterfall

... After that, it is pleasant to come back to the chirpy school magazine style of Requirenautics Quarterly [[1]]. There is not a megalomaniac or religious fanatic in sight and it is packed with sensible articles: a quick review by Ian Alexander of Barry Boehm's WinWin including a copy of every IT man's favourite cartoon; a long article by Ralph Young on how to help everyone; a good contrarian contribution by Richard Veryard in praise of scope creep; some helpful thoughts on abstraction and scenario-building by Ian Alexander; and a long series of book reviews contributed by the same man.

One of the book reviews covers Ralph Young's book, Effective Requirements Practices. The review is witty, sensible, fair and acute but there is one sore thumb sticking out of it. Quite out of the blue, Mr Alexander writes (p.12):

... Something that they both agree on [Michael Jackson and Ralph Young] is that the waterfall model is inadequate: engineering development certainly does not follow a straight line path.
Why does he say this? It adds nothing to his review. It is gratuitous cruelty. The waterfall model is like some unfortunate dog that no-one can pass without kicking. Some people will even go to the trouble of crossing the road just so that they can kick it: Mr Alexander, for example; and Messrs Bowen and Hinchey in their Ten Commandments of Formal Methods [[2]] – you would think that they had enough on their plate but, no (8th commandment):

... System development is by no means a straight-forward one-pass process. Royce's 'Waterfall' model of system development [[3]] was abandoned because of the simplistic view it held of system development ....
People have been kicking it for years; kicking it seems to be an 11th commandment; but there's something funny about this dog – it's still there, it's always there, it doesn't matter how much you kick it, it just won't go away. Why?

The waterfall method is supposed to be all wrong. How silly to think that a specification can ever be finished or that development can ever end! Well, yes, it is silly, but who ever thought that? The merest charwoman knows that she must dust over and over again, the dust keeps coming and she must keep dusting it away.

Are we to suppose that once upon a time, in some dark age or some foreign country, users and system developers didn't realise this, unlike charwomen, and thought that requirements could be fixed finally and forever? Show me one of these system developers. Introduce me to one of these mythical users who would sign off a specification and then wait patiently and confidently for six years to have the perfect system delivered. I don't think they ever existed. Attacks on the waterfall method are attacks on an Aunt Sally.

"What about Government contracts, then", you may ask? "They always insist on quoting a fixed fee for a fixed specification." Do they? Show me. I'd be interested. I've never seen a government contract. Perhaps they are as stupid as you suggest. Silly old government. But perhaps they aren't. Perhaps the contracts do allow for change.

If you attack the waterfall method, does that mean that you think that you shouldn't analyse and design before you start coding? No, it is generally recognised that actually that is a rather sensible order to perform these tasks in.

You can play games with the topology, of course:

·  You can say that the evolutionary method is better than the waterfall method. But all you've done then is to string out system development into a long series of ... waterfall cycles, each iteration with a bit of requirements capture followed by a bit of design followed by a bit of implementation and, yes, we'd probably better have a spot of acceptance testing before going into production. You're still using the waterfall method and, frankly, it would be surprising if you weren't.

·  Alternatively, if you really can't stand straight lines, spirals may be more your bag, with quick access from one iteration to the next, optionally cutting out a few phases of the waterfall cycle. But you're still actually acknowledging that the waterfall model is there, wrapped up in one of Mr Boehm's spirals. You can't get away from it and there is no reason to try.

What does this achieve? It reinforces the point that all the processes involved in system development have to be performed iteratively and that progress is incremental. That should always have been clear anyway. It doesn't add anything to the original model and it doesn't make the original model wrong.

So, I think that it may be time now for us to stop kicking the poor old dog. We should give it a shower, start feeding it properly and take it out for walks with us. The waterfall model should be treated as the long-suffering, worthy and faithful member of the software engineering family that it has always been. This dog deserves our warm regard and respect.

After 30 years of this sort of vilification Mr Royce, the dog's breeder, has probably suffered his own Darkness At Noon [[4]] and brain-washed himself into believing that the waterfall model is a legitimate target for any passing boot. He should be rehabilitated.



[1] Requirenautics Quarterly, BCS RESG, Issue 24, July 2001.
[2]Bowen, Jonathan P. & Hinchey, Michael G., Ten Commandments of Formal Methods.
[3]Royce, W.W., "Managing the Development of Large Software Systems", Proc. WESTCON'70, August 1970.
[4]Koestler, Arthur, Darkness At Noon, Jonathon Cape, 1940.
__________________________________________________________________ 

Agile is the opposite of waterfall – no

The Iguazu Falls (healthy/agile)
The Department for Work and Pensions have written off millions of pounds spent on developing IT for Universal Credit and we expect the write-off to rise into the hundreds of millions.

How can we stop intelligent organisations from wasting money like this?

Over and over again we are told that the answer is "agile".

Use "agile" software engineering methods and the waste will be minimised.

How? What problem is "agile" solving?

Over and over again we are told that "agile" is to be contrasted with "waterfall". Waste is endemic in "waterfall" software engineering methods. That's the problem. And "agile" will solve it – that's the suggestion.