Help! Your project has been selected for an audit--what now?

Introduction

“You've been selected for an audit.” are not the words most project managers want to hear. Their fears are understandable. An audit means scrutiny. Coordination and time is required when the project manager's plate is often already full. There are concerns about the outcome and its effect on the team and current work as well as careers and advancement. As with many topics related to project management, the solution to overcome this apprehension is proper planning and preparation. A project manager who knows what the audit process entails is more likely to have a positive audit report. This paper will present an overview of what a project audit is and describe a case study to illustrate key points.

Why the Interest in Project Audits?

Two events have contributed to more organisations conducting project audits than in the past: regulatory compliance and pressure on corporate profits. Laws such as Basel II, the EU 8^th Company Law Directive (84/253/EEC), and Sarbanes-Oxley mandate organisations to put effective risk management processes and internal controls in place. Both Sarbanes-Oxley and the 8^th Directive require publicly listed companies to have an independent Audit Committee and monitor the effectiveness of internal controls. Sarbanes-Oxley goes further than the 8^th Directive in that it requires internal control deficiencies be reported to the Audit Committee and that an internal control report to the company's shareholders be issued. Although Sarbanes-Oxley is a U.S. law, it affects international companies that are listed on U.S. stock exchanges. In 2004, the New York Stock Exchange required all listed companies to, “maintain an internal audit function to provide management and the audit committee with ongoing assessments of the company's risk management processes and system of internal control.” (NYSE, 2003, p.1) Management is increasingly using a project audit as verification that they have such a system in place. An audit examines areas where management has the most concern: risk management procedures and the business benefits case.

Scrutiny of project benefits is another reason why project audits may be occurring more regularly. For-profit companies are under pressure to provide earnings growth of 15% and higher. Projects are required to contribute to that growth and show a return on investment. At the same time, executives are keeping a closer watch on spending, especially for information technology projects. There is also an increasing emphasis on good governance, in both for profit and not-governmental entities, and project audits are viewed as a useful tool in these efforts.

Three W's: What, Who, When

What is a project audit? A Guide to the Project Management Body of Knowledge (PMBOK^® Guide) defines it as, “a structured independent review to determine whether project activities comply with organisational and project policies, and procedures.” (Project Management Institute, 2004, p. 189) In short, it is a quality management tool. There are several variations of a project audit: in-process quality assurance review, gateway review, project management audit and post-implementation audit. The project manager should realise that each can have a different set of objectives. By ascertaining what these objectives are, the project manager will better understand the direction in which the auditors are headed – and better anticipate the types of questions the auditors will ask. We'll discuss two methods of determining the audit focus, the announcement letter and the audit programme, later in this paper.

Who will conduct the audit? This depends on the organisation and project. To ensure the audit is unbiased, the reviewers should have no conflict of interest and be independent, i.e. not related to or controlled by the party being audited. Organisations may use a combination of project management office staff, internal audit staff, external auditors and/or external third-party experts. The audit team should include functional as well as subject matter experts, e.g. if the project is information systems related, the business viewpoint must be represented as well as information technology. For post-implementation reviews, some businesses include team members from the project implementation team, but then ensure the audit team lead is independent.

A third party firm may be used when the internal audit function lacks bandwidth or expertise in a particular subject matter area. Some prescient organisations even plan for such possibilities as part of their procurement – see the case study for details.

When the audit is conducted often depends on the type of audit. Gateway reviews are conducted at the end of a project phase and prior to progressing on to the next phase. A project audit may be conducted at any time, but is often timed so that sufficient deliverables are available for review, or when a project sponsor seeks an independent assessment of project progress. Post-implementation reviews occur after the end of a project, but the exact timing could be from a few weeks to a year, depending on what is to be examined. It can be difficult to assess benefits unless enough sufficient time has elapsed and the proper benefits realisation processes built. Ideally, the auditors should be able to examine the already implemented benefits measurement procedures and compare initial benchmarks with ongoing results.

The Audit Process

The processes by which an audit is conducted usually consist of the following steps:

1. An announcement letter
2. Documentation requests
3. Opening conference
4. Self-assessment forms
5. Field work
6. Draft report of findings and recommendations, and a conclusion/opinion
7. Closing conference
8. Final report issued including management's response
9. Action plan and follow-up

These processes are carefully documented in indexed, cross-referenced work papers.

If the internal audit function is responsible for auditing projects, an “audit risk universe” defining the areas to be covered is often used to determine what is to be audited as part of the overall plan. Most internal audit departments try to maintain some flexibility so they can react to requests from management. For the project is to be audited, the first step is often to notify the project sponsor and project manager of the intent to audit and the proposed timing. Whether this is communicated informally first via email or telephone and then followed with a formal announcement letter depends on the organisation. If the project manager is approached informally, he should consider the proposed dates carefully. If the timing would negatively affect the project, the project manager should be prepared to explain the reason and to propose alternative dates. Whether such a request is successful depends on the business reasons presented, the negotiating skills of the project manager, and the amount of flexibility available to the auditors. Once a formal announcement letter is sent, the dates are generally set and are usually difficult to change.

The announcement letter contains key information for the project manager: the objectives of the audit, the scope, and the audit team. The objectives are the areas on which the audit will focus and can vary substantially. Here are objectives from a gateway review of a warehouse management system of a wholesale distribution company:

1. To ensure the local project is operating within this remit, decision making is appropriate and project governance/risk management meet the project requirements;
2. To ensure best practices are being followed in project management and that, when appropriate, the organisation's global guidelines are followed, enforced and are effective.
3. To assess the pilot project's effectiveness and the work performed to date with an eye to future rollouts

Gateway reviews are typically conducted throughout the project, concentrating on these areas in succession: strategic assessment, business justification, procurement strategy, investment decision, readiness for service and benefits evaluation.

Here are the objectives from a project audit the author conducted at a U.S. federal agency:

1. To determine whether the organisation's Enterprise Resource Planning (ERP) project management controls are adequate to provide reasonable assurances that the project has identified all relevant Human Capital requirements under federal law, including Office of Personnel Management guidance, regulations, and requirements, as well as the organisation's internal regulations, guidance, and standards; and
2. To determine whether the organisation's Office of Human Resources Enterprise Resource Planning project management plans and technical system implementation plans, processes, and actions are adequate to meet federal and the organisation's human capital requirements.

The objectives of a post-implementation audit of an Information Technology (IT) project (Information Systems and Control Association, 2005) might include:

• Ensure that the intended objectives of implementing the IT solution are met and aligned to meet the business objectives of the organisation
• Evaluate the adequacy of procedures and controls over input, processing and output to ensure that information captured is complete and accurate, information processing complies with required business rules, and information generated is accurate, reliable and timely
• Verify the accuracy of financial and management reports generated by the IT solution
• Ensure the adequacy of application-level access control enforced by the IT solution.

When assessing the project management portion of the audit, the standards are generally based upon a recognised framework, such as PMBOK^® Guide .

The auditors may use Control Objectives for Information and related Technology (COBiT) guidelines and examine the project management methodology used for the following items:

• business management sponsorship for projects
• programme management
• project management capabilities
• user involvement
• task breakdown, milestone definition and phase approvals
• allocation of responsibilities
• rigorous tracking of milestones and deliverables
• cost and manpower budgets, balancing internal and external resources
• quality assurance plans and methods
• programme and project risk assessments, and
• transition from development to operations. (IT Governance Institute, 2000)

Auditors will evaluate the deviations from standards that were to be applied, and if deemed sufficiently important, will make it a finding. They will also look to an organisation's internal standards and its policies and procedures to verify these are followed. Standards issued by recognized bodies as the Institute of Electrical and Electronics Engineers (IEEE) and the International Organization for Standardization (ISO) may be referenced.

For a construction audit, areas reviewed are often procurement related. The objective is usually to verify all costs to the organisation are legitimate, complete, and appropriate in accordance with the final contracts and all related change orders. Areas examined may include construction bid process, change orders, project management services, contractor compliance, liquidated damages, and cost verification of major equipment and construction components.

Auditors often look to see how changes are managed. They examine:

• Whether an impact assessment made
• If changes grouped and prioritised when feasible
• If changes are approved, and
• If implementation of changes is done in an orderly organised fashion.

Although these objectives may initially seem to cover quite a bit of territory, there are some recurring themes: internal controls, project management controls, risk management, security, following policies and procedures, and quality assurance planning. The project manager should know what is included in each and the types of documentation that will be requested and questions asked.

Internal Controls

The term “internal controls” is frequently used by an auditor, but a project manager may be unclear as to what these words mean. They wouldn't be alone as historically there has been a lack of consensus as to what constitutes “internal control”. The confusion was the catalyst for the formation of the Committee of Sponsoring Organization (COSO). This group defined an internal control framework, and their model is widely recognised as the definitive standard. It defines internal control as, “a process, effected by an entity's board of directors, management and other personnel designed to provide reasonable assurance of the achievement of objectives in the following categories: effectiveness and efficiency of operations, reliability of financial reporting, and compliance with applicable laws and regulations.” (COSO, 2006)

In an effective internal control system, the following five components work to support the achievement of an entity's strategies and related business objectives.

• Control Environment
• Risk Assessment
• Control Activities
• Information and Communication
• Monitoring

These components establish the foundation for sound internal control within the organisation through directed leadership, shared values and emphasising accountability for control.

The primary objectives of internal control are to ensure:

• The accomplishment of established objectives and goals for operations or programmes.
• The integrity and reliability of information.
• Compliance with policies, plans, procedures, laws, regulations, and contracts.
• The economical and efficient use of resources, and
• The safeguarding of assets.

From these objectives, you can see that effective internal controls are in the interest of the project manager, too.

Some examples of control activity related questions the auditor may ask include:

Are formal signoffs performed for:

• project approval by sponsor/steering committee
• the business case
• the business requirements, functional requirements and technical requirements
• user acceptance testing
• project management documents e.g. the comprehensive project plan, risk management plan, communications plan, testing plan, security plan, and
• waivers from established policies or procedures.

Are security concerns addressed:

• What steps have been taken to ensure security is properly designed, approved, tested and implemented?
• What processes are in place to ensure the right people have the right access?
• How is separation of duties managed?

Risk related questions might include:

• Is there a Risk Management Plan?
• Is relevant and reliable information regarding project risk identified, compiled, and communicated in a timely manner to those who are positioned to act?
• Are project risks identified, the significance of risks estimated, the likelihood of their occurring assessed, and actions taken to mitigate them?
• Are controls in place to assure that management decisions regarding risk are properly carried out?
• Are risks reassessed periodically through the project?

The auditor will also probably ask to see selected signoff documents, so be prepared to comply.

Review of Documentation

The auditors normally request documentation prior to coming onsite as well as during the audit. The project manager should be prepared to field such requests – or delegate them to his staff. Documents should have a cover page, showing the document owner, revision number and history, and signoff/acceptance. Commonly requested documents include:

• Project Organisation Chart
• Comprehensive Project Plan
• Approved Project Charter
• Approved Business Case
• Communications Plan
• Responsibility Accountability Matrix
• Steering Committee Meeting Minutes
• Project Status Reports
• Project Status Meeting Minutes/Action Plans
• Comprehensive Test Plan
• Training Plan
• Risk Management Plan
• Quality Assurance Plan
• Project Quality Reports
• Issues Logs
• Risk Registers and Action Plans
• Project Phase Approval
• Post-implementation Audit Plan

Documents may also be requested on policies and procedures, such as:

• Project Management Framework
• Project Management Methodology
• Policies and Procedures Relating to Quality Assurance
• System Development Life Cycle (IT projects)

Onsite Work

An opening conference or kick off meeting is usually held to explain the audit objectives, process and timing. This allows the project manager to ask questions and enables all parties to meet face to face. If not already provided, the project manager should use this time to request a copy of the audit programme. This document lays out the specific areas that will be examined, and the standards and criteria to be used. Many internal audit functions will provide this to demonstrate transparency. It's also a good idea to set up a periodic meeting between the project manager and the lead auditor. This allows the project manager and lead to discuss what the auditors have observed, and for the project manager to present a different viewpoint, if necessary.

Control self-assessments are often part of the audit process. This approach uses questionnaires to elicit data about controls, risks, and processes. The questionnaires are completed by employees involved in the organisation's operations, rather than by the auditor. The responses are compiled by the auditors and used to review key business objectives, risks involved in achieving the objectives, and internal controls designed to manage those risks. They can also be used to collect information that the auditor then uses to determine higher risk areas on which the auditor will spend more time.

Field work performed by the audit team usually consists attendance of meetings, observations, examination and evaluation of documentation, and documenting the evidence in work papers. Findings, also called issues or audit points, must be documented with evidence that is clear, convincing, complete, accurate, objective and concise. Anecdotal interview accounts are compared with documentation and observations. Much of the fieldwork time is often spent at the project site(s), and is often two to three weeks long though this depends on the scope and complexity of the audit. Interviewing project stakeholders using structured techniques either in-person or by telephone is part of the field work. Usually, individual rather than group interviews are performed. This protects the confidentiality of the interviewee (at least in non-government audits) and reduces the pressure interviewees might feel in a group situation.

The project manager should feel free to ask how long interviews might take (the answer is likely to range from 30 to 60 minutes). If the project manager has an assistant who may be able coordinate the interviews, the auditors are sure to appreciate it. The project manager should let his staff know that they're to be open, honest, and cooperative in answering questions. It's best that deficiencies or difficulties, especially those that are cross-functional, be reported and the underlying causes identified in an open and honest environment. Being truthful is especially important, as auditors cross check information, and not being totally honest can put one's credibility in doubt. It's ok to say you don't know something, but also volunteer that you will find out – and then do so. This builds trust, and that's always important in relationships.

The Audit Report

Findings, along with recommendations, are the core of an audit report. A finding is a conclusion related to an auditor's examination which identifies problems and provides recommendations for corrective action in order to prevent their future recurrence. When the auditor has what he thinks is a finding, he'll generally discuss it with the project manager. There are several reasons for this: the auditor wants to make sure the conclusion is accurate, as well as document what the project manager's response is. In these conversations, the project manager should determine if the auditor has the complete picture – and if not, be ready to provide evidence supporting his position. After all, the auditors are there for a short period of time, while the project manager has (ideally) been with the project since initiation and, from an overall perspective, should understand the project best. There may be mitigating circumstances of which the auditor needs to be made aware, e.g. a variance to not follow certain procedures was requested by the project manager and approved by management.

Findings are often categorised by risk: high, medium, or low. If the project manager agrees with the audit finding, but not the level of risk, he should discuss the reasons with the auditor.

During the audit, it's important that the project manager keep the sponsor apprised of any findings and general progress. Just as you, the project manager, are concerned about the audit, so is the sponsor.

The format for an audit report varies, although there are some commonalities. The audit report will have a management summary in which an audit opinion or conclusion is made. The most difficult part of the report to write, and what the project sponsor and management are most likely to remember, is the audit opinion. Colors such as green/yellow/red are often used as visual aids to determine if a project is satisfactory, concern exists, or is unsatisfactory. As project manager, you should realise there are rarely audits with no findings, but lower risk findings are always better than higher risk. What you as a project manager want is a balanced report, an optimistic opinion, and ideally, a “green” or satisfactory rating.

Closing Processes

The closing conference allows participants to discuss the most important findings of the audit, and next steps. Management is asked to submit their response and an action plan to this version, which is called a draft. This document, combined with the response and action plan, is then issued as a final report to a carefully selected distribution list.

For each finding in the draft report, a responsible party is assigned so that they can create an action plan. The plan proposes time-bound corrective and preventive actions to address any weakness identified by the audit. The audit team then assesses the action plan and may be involved in verifying its subsequent implementation. Procedures for verifying the close-out of the action plan should be agreed between the project manager being audited and the audit team.

Depending on the policies of the organisation, the project manager may be asked to complete a customer satisfaction survey. This is regarded as a best practice in the audit world as it allows input into the process and provides assurances that the process is conducted fairly and professionally.

Case Study

Background

In 2000, the International Fund for Agricultural Development (IFAD), a United Nation organisation, embarked on a Strategic Change Programme (SCP). This programme included plans to upgrade and integrate its financial and human resource systems, based on documented business cases from its process-reengineering programme. The changes involved implementing an Enterprise Resource Planning (ERP) system and other information technology components. IFAD selected PeopleSoft as the ERP technology platform upon which to build and redesign its financial and human resources systems. In 2002, IFAD selected an internal consulting firm, ABCD, to serve as the implementation vendor to provide a variety of services in connection with the SCP programme and PeopleSoft implementation. The contract negotiated was fixed price. Included in the contract was a clause that outlined the parameters for a project audit, provided for its exercise at IFAD's discretion, and clearly stated that full cooperation was mandatory.

Profile

An aggressive timetable was set, and the organisation found implementation to be extremely challenging, requiring more effort than anticipated. Management committed additional resources to attempt to meet the deadlines. However, some milestones could not be met. A critical system for loans and grants that was originally planned to be in PeopleSoft was not implemented, because the software did not meet the necessary requirements. Interim solutions were discussed and an alternative long-term solution was sought.

These challenges caused IFAD to exercise its option in 2003 to retain a firm with no direct competitive position to ABCD and fully independent from technology vendors to conduct a series of quality assurance reviews of the ERP implementation. IFAD issued a request for bid and subsequently selected an independent firm to conduct the reviews. The objectives were to comprehensively review the programme, focusing on these areas: programme planning and monitoring, risk and issues management, testing, data migration, integration issues, communication, training and change management as well as contract performance.

Process Overview

The firm reviewed documentation onsite and using structured questionnaires, interviewed IFAD staff and ABCD resources identified as deeply involved in the implementation. Testing and training sessions were observed, and the audit team attended project related meetings. Standards used included IEEE Std 730, IEEE Guide for Software Quality Assurance Planning and IEEE Std 1058 and IEEE Standard for Software Project Management Plans. The report contained an executive summary of key findings and recommendations, a detailed list of issues, findings, and recommendations, a list of persons interviewed, and an initial list of documentation reviewed.

Subsequent in-process reviews were conducted at three month intervals. They analysed the degree to which the programme had taken actions in response to issues, findings, and recommendations listed in the report and presented new issues, findings, and recommendations found in subsequent reviews.

Results

The reviews enabled IFAD to recalibrate the programme to achieve its goals. As part of the recalibration, ABCD's engagement as implementation partner was terminated and an amicable withdrawal arranged. A new programme structure was established, and the remaining project work was re-planned with two phases. The implementation phase of the SCP with one planned exception relating to the loans and grant system was completed in 2005.

Summary

The process for many types of audits are actually quite similar whether they called a project audit, a gateway review, a quality assurance review or a post-implementation audit. The objectives of the audit are what make the audit unique. Audits should have a systemic approach and be transparent. They should have clearly documented procedures including a well-defined planning process, clear audit criteria, report approval and distribution. Management and implementation of the audit process should be transparent to all relevant stakeholders. The project manager who understands the audit process and prepares properly shouldn't be concerned if his project is selected for an audit. With proper planning, a project manager can continue to focus on the project and use the audit results to address any identified issues in time.