The Novopay inquiry

June 4th, 2013 at 1:46 pm by David Farrar

Steve Joyce has released the report of the Ministerial Inquiry into Novopay. Major findings:

  • The problems with have affected public trust and confidence in the Ministry of Education, and also the wider public sector
  • Weaknesses in project governance and project leadership allowed Novopay to go live with a number of significant risks which the Ministry of Education and its vendors, including Talent2, were over-confident of managing
  • These risks resulted in service issues and the Ministry and Talent2 were unprepared and overwhelmed by their nature and scale
  • The School payroll is overly complex due to an accumulation of historical changes
  • There was extensive customisation of the Novopay software
  • There was a failure to involve the users of the Novopay system in the schools and appreciate their requirements
  • There was no overall accountabilityfor Independent Quality Assurance
  • The project has cost $23.9 million more than estimated for a total cost to date of $56.8 million
  • Ministers were not well served by the information they were given on the project. Reporting to Ministers was inconsistent, unduly optimistic and sometimes misrepresented the situation.

Joyce comments:

“This report makes for sober reading and, while it confirms the view that there is a lot of blame to go around for the problems with Novopay, it provides a greater understanding of the level of fault between the organisations involved,” Mr Joyce says.

“There are substantial lessons to be learned by the Ministry of Education in a number of areas which the Acting Secretary of Education is taking steps to address.

“There are also lessons to be learned by the public service and the wider State Sector on the design, delivery and oversight of major ICT projects.

“As the report notes, these problems are not unique with issues identified in the Ministerial Inquiry into the police computer system INCIS 13 years ago also evident here.

“The Government will be carefully considering the findings of the Ministerial Inquiry into Novopay. It intends to act on all the recommendations. It is critical these problems are not repeated again.

In the report itself:

Work commenced on the requirements for the schools payroll project in October 2008. This  process was lengthy, and was never actually completed. Even after Go Live, new requirements  were being discovered. There was little direct customer (boards of trustees) or user (principals  and school administrators) involvement in the definition of the requirements, and Datacom’s involvement was minimal.

And that’s where it would all have started to go wrong. You must finalise the requirement and have customer input into them.

Also:

A 5 June 2012 paper, which invited Ministers English, Parata and Foss to approve the continuation of the project following Confidence Point Two, misrepresented its state.

In more detail:

We were particularly concerned about the 5 June 2012 document, which invited Ministers to approve the Ministry’s Go Live decision. This report misrepresented the situation in two significant ways. It suggested that the Go Live decision was supported by three members of the ICT Council, which was not the case; and it stated that Confidence Point Two criteria had been met, when in fact a number either had not been met or were “deemed” to have been met (a lower criterion). These factors overrepresented project readiness and its chances of success.

The report is very interesting reading. There are even questions over the original selection decision.

Tags:

26 Responses to “The Novopay inquiry”

  1. wreck1080 (3,798 comments) says:

    You will see job applications for government IT jobs virtually requiring a pHD and 1000 years of experience…

    So, what happened with all these rocket scientists?

    I find it interesting they were testing/designing at the same time.

    Full integration testing is simply not possible when the system design is not static.

    Vote: Thumb up 5 Thumb down 0 You need to be logged in to vote
  2. Poliwatch (335 comments) says:

    As with most (if not all) disastrous IT projects, it is because of a lack of good specification. How could a project like this get started without the Requirements Document being completed? And who was responsible for signing off the document before development started?

    Vote: Thumb up 5 Thumb down 0 You need to be logged in to vote
  3. berend (1,673 comments) says:

    DPF: “There are also lessons to be learned by the public service and the wider State Sector on the design, delivery and oversight of major ICT projects.

    One lesson not in the report: why should our state manage every school, and do every payroll?

    An intermediate step could be a fixed sum per child, and let schools buy the payroll provider on the market.

    But we can’t have that.

    Vote: Thumb up 3 Thumb down 2 You need to be logged in to vote
  4. berend (1,673 comments) says:

    Pollwatch: How could a project like this get started without the Requirements Document being completed?

    There are known knows, but also unknown unknowns…

    The approach you suggest is not more likely to lead to good outcomes, and most companies don’t work the way you suggest they do.

    Vote: Thumb up 2 Thumb down 1 You need to be logged in to vote
  5. rosina (9 comments) says:

    “The School payroll is overly complex due to an accumulation of historical changes”

    This should have been sorted before any new computerised payroll was commissioned and is probibly the biggest bunny in the woodpile.

    Vote: Thumb up 3 Thumb down 0 You need to be logged in to vote
  6. Paulus (2,556 comments) says:

    Once the original specification is put out any alterations should be deferred until the original is completed.
    Biggest problem is interference, changes and sub committees.

    Vote: Thumb up 1 Thumb down 2 You need to be logged in to vote
  7. Dave Stringer (186 comments) says:

    The reality is that design by testing and testing by implementation are conditions in the system development life-cycle that have been around for several decades. They appear, in every case, for the same reason – the “users” don’t put the necessary effort into stating at the beginning what their reqirements are.

    Implementation of a new “system” whether or not it is automated or computer assisted, should be used as an opportunity to ensure that hangovers from past decisions that are no-longer appropriate are eliminated and a “clean page approach” to the task(s) being systematised taken. Sadly, this requires time, effort, commitment and (dare I say it – yes, I will,) imagination on the part of the people whose working lives will be affected by the implememtation of the new system. From the very top of an organisation down to the very bottom, there needs to be engagement in the process. Simply stating that “we want what we’ve already got” is not doing the job properly, and amost always results in significantly more effort being expended in correcting activities than would have been required had the right approach been taken in the first place.

    Vote: Thumb up 3 Thumb down 0 You need to be logged in to vote
  8. wreck1080 (3,798 comments) says:

    “One lesson not in the report: why should our state manage every school, and do every payroll?”

    Perhaps because the teachers award system is so complex we’d end up with 1000 different implementations of the law?

    Vote: Thumb up 1 Thumb down 0 You need to be logged in to vote
  9. wreck1080 (3,798 comments) says:

    10 years from now, we’ll be saying the new IRD computer system has been a total failure for largely the same reasons outlined above.

    Vote: Thumb up 1 Thumb down 0 You need to be logged in to vote
  10. F E Smith (3,314 comments) says:

    Reporting to Ministers was inconsistent, unduly optimistic and sometimes misrepresented the situation.

    I presume that there have been, or will be, resignations over this factor?

    Vote: Thumb up 2 Thumb down 0 You need to be logged in to vote
  11. PaulL (5,981 comments) says:

    I’d come at it a different way. Failed projects often have common attributes. But many of those common attributes are also problems on successful projects. Lots of projects that are very successful do elements of design and test in parallel – that’s the problem with users, they always want change. It’s about how you control and manage it, not about whether it exists.

    I’ve seen plenty of projects, I’ve seen plenty of audits. It’s pretty easy to write an audit listing all the things that weren’t perfect. What is harder is to articulate which of them were the inflexion points that caused the failure, as opposed to just being there at the same time a failure happened. It’s about as useful as observing that the project had a bunch of Indian programmers and failed. Since just about every project has a bunch of Indian programmers (there are a lot of Indians, and their universities turn out more computer scientists than most of the rest of the world combined), and some of them don’t have great English skills. But everyone else deals with that, it’s not a cause of a project failing.

    Not taking away from the report – no doubt one or more of those items did cause the failure. But I reckon just listing them all and stating it failed is different than pointing to it as an actual cause.

    Vote: Thumb up 3 Thumb down 0 You need to be logged in to vote
  12. Dirty Rat (383 comments) says:

    and here i was thinking it was okay due to DPF’s previous analysis.

    So its good and bad, depending on who you shift the blame to.

    Vote: Thumb up 1 Thumb down 0 You need to be logged in to vote
  13. berend (1,673 comments) says:

    From the report: The project had by this stage shifted from implementing a configured package software solution towards a heavily customised solution, and was therefore increasingly moving away from the original strategy, business case and basis for procurement.

    Very typical.

    Also: During the service design and development phase, the intended pilot and phased rollout of the
    service were removed from the project plan.

    The persons responsible for that decision should hand in their resignation or be sacked.

    Vote: Thumb up 3 Thumb down 0 You need to be logged in to vote
  14. mikenmild (11,246 comments) says:

    I see reports that two staff members are facing employment investigations. I wonder if that will go wider.

    Vote: Thumb up 3 Thumb down 0 You need to be logged in to vote
  15. Judith (8,372 comments) says:

    Start at the top. When you take the top pay, you get to take the responsibility as well.

    If their advisors weren’t doing their job properly, bigger fool them for employing the wrong people.

    There has to be accountability – and the buck stops at the top. That is the nature of earning the top dollar – you get to be responsible for every lower plebs stuff ups – there shouldn’t be any question over it.

    No doubt they will do the usual – and pass it down until they reach the cleaning lady.

    Vote: Thumb up 1 Thumb down 2 You need to be logged in to vote
  16. mikenmild (11,246 comments) says:

    Well, the person at the top already lost her job and has gone back to England. Whether others are held to account remains to be seen.

    Vote: Thumb up 3 Thumb down 0 You need to be logged in to vote
  17. berend (1,673 comments) says:

    And then we have this:

    During this stage the intended pilot and phased rollout of the service were removed from the
    project plan. These decisions were to have major ramifications later on. The Ministry had earlier
    told the Parliamentary Education and Science Select Committee that both would happen, but
    did not advise of their removal.

    That’s a direct ministerial responsibility. Going, going, going….

    Vote: Thumb up 1 Thumb down 0 You need to be logged in to vote
  18. mikenmild (11,246 comments) says:

    Not necessarily any ministerial responsibility there, if it was the Ministry (officials) telling a select committee one thing and then neglecting to correct they story. One would have to show that a Minister knew of that omission and did nothing about it.

    Vote: Thumb up 3 Thumb down 0 You need to be logged in to vote
  19. KevinH (1,149 comments) says:

    Despite the hand wringing admissions this report completely fails to acknowledge that a prima facie case of criminal negligence exists involving misappropriation of millions of taxpayer dollars. Refer this matter to the SFO for investigation.

    Vote: Thumb up 0 Thumb down 1 You need to be logged in to vote
  20. Dirty Rat (383 comments) says:

    I’m a bit confused here

    David Farrar was telling us that the errors were statistically insignificant, then he is saying that “there are more questions about the original selection decision”

    I think there are more questions about your logistics and statistician qualifications….

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  21. berend (1,673 comments) says:

    Another quote:

    Some important areas of functionality were not fully tested prior to Go Live. Some types of
    testing were not completed to the original scope, on the basis that testing could be completed
    after Go Live, or that the risks of not doing the testing had been adequately mitigated. Not all
    System Integration Testing criteria were met.

    If a whole series of senior staff + minister is not sacked for this, lessons won’t be learnt…

    Vote: Thumb up 1 Thumb down 0 You need to be logged in to vote
  22. PaulL (5,981 comments) says:

    Dirty Rat – I seem to recall that allegations against DPF himself is one of the few things that get you banned around here. Which I can see – why would you want to provide for free a platform for other people to attack you.

    Vote: Thumb up 0 Thumb down 1 You need to be logged in to vote
  23. Dirty Rat (383 comments) says:

    If people want to attack me…I honestly couldnt give a shit, I dont keep a drawer for of those who have offended me, nor do I have this childish small cock NFWAB rubbish…

    I honestly couldnt give a shit..

    But I do care when someone tells lies and changes opinions according to the spin shit that they are sent

    Vote: Thumb up 0 Thumb down 1 You need to be logged in to vote
  24. Dirty Rat (383 comments) says:

    PaulL (5,206) Says:
    June 4th, 2013 at 8:32 pm

    BTW, it was me calling Bill English a cunt that got me banned, I and I still reiterate that he is a cunt for having that rort with his flat in Wellington,…the one that you and me paid for…

    but DPF will defend that misuse of taxpayers dollars

    Vote: Thumb up 1 Thumb down 0 You need to be logged in to vote
  25. Honeybadger (186 comments) says:

    Dirty Rat (222) Says:

    June 4th, 2013 at 10:46 pm
    BTW, it was me calling Bill English a …..
    You didnt! I just would have called him a thieving twat myself….

    Vote: Thumb up 0 Thumb down 0 You need to be logged in to vote
  26. berend (1,673 comments) says:

    And my last list of interesting quotes from this document:

    Some important areas of functionality (including service accumulation and End of Year/Start of Year) were not fully tested prior to Go Live. Some types of testing were not completed to the original scope, on the basis that testing could be completed after Go Live or that the risks of not doing the testing had been adequately mitigated. Inconsistencies were also observed between
    the tested software and the software actually deployed.

    And then we have a member of the Ministry’s Audit and Assurance Committee, which has an advisory role to the Chief Executive:

    One member of the Committee could not think of anything that could have been done differently.

    The entire committee should be sacked, as it was useless (it completely failed the objective it was set up for, while the members were drawing their 6 figure salaries). But won’t happen either.

    The entire governance was a mess:

    We found no evidence of the sustained and focused attention on the project from the Ministry’s leaders that we would have expected. The Leadership Team did not clarify its
    own role in relation to the project.

    And the lefties can’t get away with: only if they had more money than every problem would be solved! There was money enough. Way too much actually.

    And then the actual lying:

    We were particularly concerned about the 5 June 2012 document, which invited Ministers to approve the Ministry’s Go Live decision. This report misrepresented the situation in two significant ways. It suggested that the Go Live decision was supported by three members of the ICT Council, which was not the case; and it stated that Confidence Point Two criteria had been met, when in fact a number either had not been met or
    were “deemed” to have been met (a lower criterion). These factors overrepresented project readiness and its chances of success.

    This document was provided by the MoE!!! It’s recommendation was to proceed to Go Live.

    Then we have another party putting their lives on the taxpayer, the State Services Commission. Apparently these guys get paid to prevent such a mess:

    SSC has acknowledged that it lost its detachment from the project.

    Then the Treasury, also supposedly being paid to make sure the taxpayer gets value for money:

    We are surprised that the financial management of this project seems not to have attracted
    greater attention.

    This is just a sobering report. The complete absence of ownership, the breakdown of any resemblance of governance, the failure of basically every government party involved indicates a systemic problem.

    What I would like to know why other IT projects mentioned in this report as successes, were successes. That would be interesting. It’s not because we had an SCC or Treasury or PwC. That would provide us some valuable lessons. And could save us a lot of money as we can do away with the parties that don’t matter.

    I realise it’s government, so the result of this report will be to establish even more procedure and committees, but no taxpayer reading this report will come away with the conclusion that more oversight committees and procedures would have mattered in the slightest.

    Vote: Thumb up 1 Thumb down 0 You need to be logged in to vote

Leave a Reply

You must be logged in to post a comment.