Why large projects usually fail

Discussions on the future of databases, computing , the meaning of life ........

Moderators: Phil Winkler, Graham Smith, Pete Tabord

Why large projects usually fail

Postby Pete Tabord » Mon Nov 04, 2013 6:29 am

What most people, and especially those who commission large projects, fail to understand is the ease of making hard-to-find errors in computer system design and programming. Such errors range from slips like simple failures of logic or coding, up to complicated interactions between modules of different authorship, especially when there is also interaction with the external human world. When such errors arise from rarely occurring circumstances in real time involving real people, debugging them can become a nightmare. The rate of occurrence of such errors increases with the size of the project (raised to a power of two or greater). The virtually immutable rule is – large, publicly funded, top-down designed computer systems never work. That they still persist with them is to confirm the definition, attributed to Einstein – Insanity: doing the same thing over and over again and expecting different results.


Quote from John Brignell, of Number Watch.

When you couple this with the fact that the modern large so-called consultancy companies - and indeed the customers themselves - point blank refuse to budget for or fully participate in proper QA, or proper documentation either, then clearly we are on a road to disaster.
Peter J. Tabord
Head of Development
Database Software Ltd.
ptabord@ffenics.com
Pete Tabord
 
Posts: 1881
Joined: Fri Sep 07, 2007 12:48 pm
Location: Caernarfon, Gwynedd, UK
Has thanked: 0 time
Been thanked: 3 times
 

Re: Why large projects usually fail

Postby Graham Smith » Mon Nov 04, 2013 1:15 pm

Pete Tabord wrote:refuse to budget for or fully participate in proper QA, or proper documentation either, then clearly we are on a road to disaster.

This is a problem we have faced since the start of our business. And even when there is cooperation, there are still going to be some problems because true testing only starts when everyone is using it full time.

As for documentation, as a programmer, I can document what the program does but not how people use it. And since business rarely make their employees create their own job documentation, when one person leaves, the person taking over doesn't know how to use the program or decides to use it differently with unexpected consequences.

Proper testing and documentation of the system is not something I, as a programmer, can adequately do and companies are reluctant to pay for it anyway. They just expect that somehow, it will magically work on delivery day. Surprise!!!
Graham Smith
DataSmith, Delaware
"For every expert there is an equal and opposite expert.", Arthur C. Clarke (1917 - 2008)
"X-Clacks-Overhead: GNU Terry Pratchett"
User avatar
Graham Smith
 
Posts: 2501
Joined: Fri Sep 07, 2007 11:31 am
Location: Delaware, USA
Has thanked: 0 time
Been thanked: 1 time
 

Re: Why large projects usually fail

Postby Fred Kingston » Mon Nov 04, 2013 2:03 pm

Pete... We want our nationalized health care system to cost more than yours... so we're starting out right... :wink:
Fred Kingston
 
Posts: 281
Joined: Sun Aug 01, 2010 10:54 pm
Has thanked: 0 time
Been thanked: 0 time
 

Re: Why large projects usually fail

Postby Pete Tabord » Tue Nov 05, 2013 10:30 am

Hi Fred - yes, I've noticed.

30 years ago our NHS was something to be proud of - constant interference by government and civil servants who know zip about health care have reduced it in some places to an over-expensive uncaring factory farm. But you can now watch TV in bed as you starve to death in your own bodily waste.

And now is when you decide to copy it...
Peter J. Tabord
Head of Development
Database Software Ltd.
ptabord@ffenics.com
Pete Tabord
 
Posts: 1881
Joined: Fri Sep 07, 2007 12:48 pm
Location: Caernarfon, Gwynedd, UK
Has thanked: 0 time
Been thanked: 3 times
 

Re: Why large projects usually fail

Postby Pete Tabord » Tue Nov 05, 2013 10:39 am

Graham Smith wrote:As for documentation, as a programmer, I can document what the program does but not how people use it. And since business rarely make their employees create their own job documentation, when one person leaves, the person taking over doesn't know how to use the program or decides to use it differently with unexpected consequences.

Proper testing and documentation of the system is not something I, as a programmer, can adequately do and companies are reluctant to pay for it anyway. They just expect that somehow, it will magically work on delivery day. Surprise!!!


The thing is both documentation and proper QA require people with specific professional skills which are not those of a programmer. (Though I think for QA some programming experience helps.) This appears to no longer be recognised by either the majority of software development companies or by customers. And since quotes have to be competitive to get the business it gets worse and worse, especially for public sector bodies in the UK where I believe they are obliged by law to take the lowest quote.
Peter J. Tabord
Head of Development
Database Software Ltd.
ptabord@ffenics.com
Pete Tabord
 
Posts: 1881
Joined: Fri Sep 07, 2007 12:48 pm
Location: Caernarfon, Gwynedd, UK
Has thanked: 0 time
Been thanked: 3 times
 

Re: Why large projects usually fail

Postby Ed Paice » Wed Jun 10, 2015 10:16 am

Can I suggest that very often the problem is not the technology, or the project team, or the developers, but is in fact a combination of the Business Leads and the Suppliers. The Suppliers do not insist on reasonable budgets (either money for the Supplier to supply, or for the Business to own and deliver themselves) for documentation and proper testing - instead they want to win the business and hence price accordingly. Similarly the Business Leads frequently do not allow sufficient time and effort for thorough documentation and testing including unit, golden thread, stress etc. - oten because they don't have the experience on which to call.

The glimmer of hope is the growth in Project Assurance - which has certainly made a difference when used. One recent project I know of had 189 formalised, documented, tests (the client team wrote the tests themselves based on their primary working scenarios) which were then executed by multiple people each and the test results logged in a database (btw... a great vertical market application for someone!) together with re-tests when there was a "fail". It was the Project Assurance guy who pushed the client into this (the supplier certainly didn't)...and guess what, the go-live was painless!
Ed Paice
 
Posts: 5
Joined: Tue May 29, 2012 8:58 am
Has thanked: 0 time
Been thanked: 0 time
 

Re: Why large projects usually fail

Postby Graham Smith » Wed Jun 10, 2015 1:19 pm

Since this thread is active again, let me add in and old and a recent experience having to do with the same customer failing to grasp the scope of the project from the outset.

Many moons ago, I started supporting a company in California who had a set of DfD databases which ran their business. These were semi-vertical market apps and they had been semi-integrated and highly customized. The company who wrote the software had moved on and we (PLM) were supporting the apps. The owner decided to switch to the software that the original programming company had changed to, and after a number of conferences and assurances, the project commenced. Everyone hated it. There was no integration, nothing worked the same, pieces were missing, and it was much harder to use.

A few months after that, they asked me to come out for a few days and discuss what could be done. Their existing system was bogging down because of a slow network and the fact that DfD really needed Novell to work at peak efficiency and that was becoming impossible to support. I did what the prior developer and their own network consultant had failed to do, I spent time with the people who used the systems to find out what their biggest issues were. Turned out that part of the problem was that the heaviest users were on a branch of the network with a string of cheap network hubs and old wiring. But a bigger problem is that their business had changed and they were trying to use the software to do things it couldn't easily do.

I took all my notes from the trip and spent a week digging through the DfD program and gave them a proposal to basically rewrite the software in DfW using their old forms and reports as a guide. This was an expensive process, but it was less than they would have spent on their replacement system if it had worked. The new system worked better than they had hoped and was used for a number of years, then upgraded to Ff along with some system tweaks.

Fast forward to 2012. At this point, the company needed a much greater web presence than they had. They worked with a local developer to create a web database that was fed nightly from their Ffenics app. At this time, I was already looking 2 years ahead to a potential retirement and I encouraged them to find a local company that could either take over the app or, more properly, build a new app that better suited their changing needs. About this time, the company was absorbed into a larger organization, but it wasn't until the end of 2013 that they got serious about the upgrade.

Whereupon, they made precisely the same mistake that the prior owners made. They let the project be run by someone who didn't understand the day to day operations of the business, rather, they were going to change the way things were done to suit their own ideas. Everyone hated the evolving new system. There was no integration, nothing worked the same, pieces were missing, and it was much harder to use. People started looking for new jobs.

I understand from someone who is still there that management has changed everything. Maybe that's what was needed, but I've had a glimpse at how this new company did things and when it comes to data management, they don't know squat.
Graham Smith
DataSmith, Delaware
"For every expert there is an equal and opposite expert.", Arthur C. Clarke (1917 - 2008)
"X-Clacks-Overhead: GNU Terry Pratchett"
User avatar
Graham Smith
 
Posts: 2501
Joined: Fri Sep 07, 2007 11:31 am
Location: Delaware, USA
Has thanked: 0 time
Been thanked: 1 time
 

Re: Why large projects usually fail

Postby Pete Tabord » Thu Jun 11, 2015 2:07 pm

Ed Paice wrote:The glimmer of hope is the growth in Project Assurance - which has certainly made a difference when used. One recent project I know of had 189 formalised, documented, tests (the client team wrote the tests themselves based on their primary working scenarios) which were then executed by multiple people each and the test results logged in a database (btw... a great vertical market application for someone!) together with re-tests when there was a "fail". It was the Project Assurance guy who pushed the client into this (the supplier certainly didn't)...and guess what, the go-live was painless!


But Ed - that's what Quality Assurance used to mean! User test cases, those were called, although they represented only one weapon in the QA armoury. Retesting on a fail is regression testing. That's part of how Trina and I were trained back in the mainframe days. I won't go into all the other test weaponry here.

I'm glad you had someone on the team for that project who insisted on doing it properly, but it is quite unusual to meet a client who will allow the budget or staff commitment for such testing. Or a software company, for that matter - as a rough guide, at least a third of any development budget ought to go on QA.
Peter J. Tabord
Head of Development
Database Software Ltd.
ptabord@ffenics.com
Pete Tabord
 
Posts: 1881
Joined: Fri Sep 07, 2007 12:48 pm
Location: Caernarfon, Gwynedd, UK
Has thanked: 0 time
Been thanked: 3 times
 

Re: Why large projects usually fail

Postby Graham Smith » Thu Jun 11, 2015 3:23 pm

Pete Tabord wrote:<snip> Or a software company, for that matter - as a rough guide, at least a third of any development budget ought to go on QA.

Unless you are Microsoft and then you can get your users to pay to do half of your QA by enrolling them in TechNet and giving them "free" preview (e.g. beta) copies of software. It's good to be the King.
Graham Smith
DataSmith, Delaware
"For every expert there is an equal and opposite expert.", Arthur C. Clarke (1917 - 2008)
"X-Clacks-Overhead: GNU Terry Pratchett"
User avatar
Graham Smith
 
Posts: 2501
Joined: Fri Sep 07, 2007 11:31 am
Location: Delaware, USA
Has thanked: 0 time
Been thanked: 1 time
 

Re: Why large projects usually fail

Postby Adrian Jones » Tue Jun 16, 2015 8:25 am

Graham Smith wrote:
Pete Tabord wrote:<snip> Or a software company, for that matter - as a rough guide, at least a third of any development budget ought to go on QA.

Unless you are Microsoft and then you can get your users to pay to do half of your QA by enrolling them in TechNet and giving them "free" preview (e.g. beta) copies of software. It's good to be the King.


Actually, it's not that bad a deal for the users either, at least those who make a living out of MS software...
User avatar
Adrian Jones
 
Posts: 2000
Joined: Tue Sep 11, 2007 2:38 pm
Location: Cornwall, UK
Has thanked: 5 times
Been thanked: 4 times
 

Re: Why large projects usually fail

Postby Gil Fleming » Mon Jun 22, 2015 1:02 pm

I think this sums up large scale (especially public) computer projects:

"You know we're sitting on four million pounds of fuel, one nuclear weapon and a thing that has 270,000 moving parts built by the lowest bidder. Makes you feel good, doesn't it?"

Actor, Steve Buscemi, about to launch into space in Armageddon (1998)
Gil Fleming
Director
Fleming Technical Limited

You can't think about what you don't know - Mike Fidler
If you can't fight, wear a big hat - John S Fleming
The best way to have a good idea is to have lots of ideas - Linus Pauling
Gil Fleming
 
Posts: 546
Joined: Tue May 15, 2012 10:26 am
Location: Liverpool, UK
Has thanked: 1 time
Been thanked: 2 times
 
 

Return to Philosophical Discussions

Who is online

Users browsing this forum: No registered users and 2 guests

cron