Introduction
There is no such thing as a perfectly executed project. Each has it own set of obstacles and each has its own areas in which quality suffers. Successful engagements are those that continually measure themselves against a high standard in real-time as the project is executed. The goal is to catch areas of low performance and improve them before the project’s close. A significant gap between the current state of the project and the desired state indicates improvements are warranted. However, improvements in project performance will result only when it is clearly understood what specific improvements are necessary and the impact those improvements will have on the project. In order to identify what specific improvements are necessary, research and investigation techniques must be used to assess the current state of the project and locate the problems.
Quality Assessment and Improvement Processes and Techniques must be followed to place rigor in this practice. Lack of formal rigor in assessing quality, directly impacts the level of success any subsequent improvements may have. For without employing rigor, items may be missed or not fully understood, and hence the improvements may be incomplete or miss the intended goal. This paper presents Quality Assessment and Improvement Processes and Techniques in accordance with the information format of A Guide to the Project Management Body of Knowledge (PMBOK® Guide): Description, Inputs, Techniques, and Outputs.
Assess Quality
Assess Quality is initiated with a Quality Assessment Request, typically received from one or more project stakeholders. An assessment plan is prepared to establish how the assessment will be performed. Once the assessment plan has been formalized, interviews, audits, and inspections are conducted to ascertain the level of quality, the adherence to quality standards, and the observance to best practices being employed. An assessment report is written to discuss the audit’s findings and recommended courses of action. Audit findings are compiled and formally reviewed. The review will take as input the recommended courses of action, and produce a formal list of corrective actions to be performed on the project. Each corrective action is written as a
Corrective Action Required (CAR) form and includes a description of the problem, possible causes of the problem, and stakeholders of the corrective action.
Initiate Quality Assessment
Description
The Initiate Quality Assessment process kicks off an assessment engagement. One or more stakeholders may request a quality assessment of a project. This request will commence planning activities that will result in written approach on how and when to proceed with the assessment. A Quality Team will produce the plan based on what is delineated in the request, combined with best practices for performing a quality assessment.
Inputs
Quality Assessment Request. This is a formal request to perform a quality assessment on a project. The request should contain the project name, stakeholders, an initial list on which quality items to assess, key project members to be involved in the assessment, a budget, and a target completion date.
Tools and Techniques
Best Practices and Planning Analysis. Best practices and planning analysis should be employed to produce a reasonable approach or plan of activities for conducting audits and gathering information.
Stakeholder Review. The plan should be reviewed and approved before conducting quality audits.
Quality Measurement Techniques. Objective measurement techniques for each quality item should be used to delineate and prescribe a measurement scale for each quality item. Example measurement scales are: A score of 1–10, a grade of “A” through “F,” a qualifier in the set of {poor, fair, average, good, best}, or a label such as “non-existent,” “exists,” “exists and is being followed.”
Outputs
Assessment Plan. The plan serves as the guide for conducting the assessment. The plan should identify methods by which information will be gathered, and the sources, individuals or organizations from which the information will be obtained. The plan should include a schedule, and should specifically identify the quality items to be assessed during the audit. The plan should also identify the measurement scale for each quality item.
Exhibit 1. Quality Assessment and Improvement Process Overview
Exhibit 2. Assess Quality Process Overview
Assess Quality Items
Description
The Assess Quality Items process obtains project information for each quality item and measure its quality against an established measurement scale. The Assessment Plan will dictate how each audit will be conducted. Possible quality items to be audited are: Documentation, Communication, Schedules, Project Plan, Cost Baseline, Project Tracking, Skill Levels, Project Processes, and Stakeholders.
Inputs
Assessment Plan. The plan serves as the guide for conducting the assessment. Reference the Outputs section under Initiate Quality Assessment for further information about Assessment Plan.
Quality Standards and Best Practices. Quality Standards and Best Practices refer to established and approved procedures and processes that have set to control methods for project work. Quality Standards are typically a part of the business Quality Policy. Best Practices may be less formally documented and will embody procedures and processes that are the result of successful past experiences and historically proven methods.
Audit Inputs. Audit Inputs refer to the collective set of information obtained during audits conducted by a Quality Team. Audit Inputs are collected and refer to information that reflect what activities are showing higher levels of quality, and which activities may benefit by introducing quality improvement strategies. To determine a quality measure, a metric should accompany each Audit Input, according to the measurement scale described in the Assessment Plan (refer to the Tools and Techniques and Outputs sections under Initiate Quality Assessment). The Audit Input is not complete until it includes both the information obtained and a corresponding measure.
Request for Further Assessment. After analyzing the results of an assessment, it may be determined that further assessment is required. It may be determined that additional participants be interviewed to obtain further information on a quality item. The request will contain specific instructions detailing the additional information and information sources.
Tools and Techniques
The following tools apply to this process.
Informal Meeting. The purpose of an informal meeting is to have an open discussion, without agenda, to gain insight into problems the participants perceive exist on the project. The result of an informal meeting should be a list of perceived problems with measure of quality, accompanied by recommendations for their resolution.
Formal Meeting. A formal meeting is a meeting that is driven by a formal agenda, and is conducted to foster discussion on specific, predetermined subjects. Although open discussion should be encouraged, the purpose of the meeting is to gain information on the agenda items only. The result of a formal meeting should be the information obtained for each agenda item with measure of quality, accompanied by recommendations for their resolution.
Interview. An interview is similar to the Informal and Formal Meeting descriptions above, yet it is a one-on-one meeting conducted between the Quality Team (or individual) and an audit participant.
Questionnaire. Where a formal meeting fosters discussion on a subject, a questionnaire asks specific questions without the need for discussion, to obtain information about known or potential problem areas. Opt for short questionnaires where the information obtained for each question will have the tendency to be complete and well thought out. Each questionnaire should be limited to ten or less specific questions, to reduce the time impact required to complete the questionnaire.
Survey. Surveys are used to analyze distributions of results, such as in the case of taking a poll, and they are typically given to a broad audience. Surveys will contain few questions, and maybe only one question. Multiple-choice answers are provided to support distribution analysis on the answers.
Checklist. A checklist is used to verify the existence of items or processes within a project, and does not involve interaction with participants. A checklist contains simple “yes/no” or “true/false” questions, for example, “Is a Project Plan in place for the project?” Each question should have a score associated with it as prescribed by the measurement scale in the Assessment Plan.
Inspection. An inspection is conducted to review project processes and deliverables, for clarity, content, and completeness. Usually this activity may be conducted without the need for direct interaction with key participants. Inspections may be conducted with checklists to itemize what is being inspected.
Analysis. An analysis is conducted to verify quality of design and solution. At a high level, architectural solution quality may be measured, and at a low level, mathematical solution quality may be ascertained. Results from the analysis may include alternative solutions, however the main objective of an analysis audit is to measure the quality of solutions proposed by the project.
The following techniques apply to this process.
Communicate. Each participant should know beforehand what is expected, when it is expected, and the tools that will be used. Notify each participant formally at least one week prior to the audit. Establish a formal mechanism for distributing the assessment results. Answer all questions truthfully and to the point.
Be helpful. The purpose of an assessment audit is ultimately to improve the project quality. Therefore the attitude of the Quality Team should be one of providing help to the project, not providing hindrance. Create an atmosphere of “We are here to help you be successful.” An effective quality auditor is not perceived as a spy or an internal affairs officer. Be part of the solution, not part of the problem.
Make the participants feel at ease. Information is seldom successfully gathered with a hammer. Be relaxed and courteous during all interactions with participants. They will feel uncomfortable, for no one likes being told what they are doing wrong. Be supportive.
Use time wisely. Do not waste the time of project members. All interactions should be short and to the point. Schedule all interactions effectively.
Be thorough. The assessment results are only as good as the information used to determine the results. If possible, do not stop with simple answers to questions. Ask about the details. Each response should spawn new questions.
Listen. Perhaps the most important technique is to utilize effective listening skills. An auditor gains nothing without listening. Hear what is being said and how it is being said. Include all participants in group discussions. Each participant will add additional information and aid in the complete understanding of the problem. A measure of how widespread the problem is will also be indicated.
Outputs
Audit Results. The conclusion of each quality item audit will result in a gathering of information and associated quality measures. The results should be clearly and formally documented, factual and without opinion. Graphs should be used where appropriate, such as in the case of surveys. Results should be individually documented by quality item.
Compile Assessment Results
Description
The Compile Assessment Results process gathers all of the results from the audit activities, combine results as necessary and make general observations. A macro view of the entire assessment process may be taken by analyzing the individual audit results to ascertain higher-level results. When combined through analysis, the results may indicate other problems, trends, and/or potential problems above and beyond what is individually indicated.
Inputs
Audit Results. Reference the Outputs section under Assess Quality Items for a detailed description.
Tools and Techniques
Comparative Analysis. Comparative analysis is the act of determining the relative consistency in the information obtained during the audits. The comparison will surface problems that are not evident within individual audit results, such as potential communication problems among project members (e.g., one person said there was a problem and another had no idea a problem even existed). This comparison may lead to understanding the scope of the problem and the intensity of the problem.
Combinatorial Analysis. A combinatorial analysis technique is the act of combining the different audit results of more-than-one, unrelated quality item and determining a new quality issue. The new quality issue may have not been detected unless a macro view of several individual quality assessments was made. For example, a quality audit of a scope document has determined that the document is meeting all associated quality standards. An assessment of the document distribution process indicates that several documents are returned, tagged as “incomplete.” By combining the results of these individual audits, it is determined that the quality standards, governing the development of the scope document, need to be improved to meet the customer/deliverable requirements.
Outputs
Compiled Results. Compiled results are a complete assessment of all individual audits, including any new quality items and recommendations that have been determined through the tools and techniques of the compilation process.
Analyze Assessment Results
Description
Prior to preparing an assessment report, the audit findings should be analyzed for total completeness. This process may be as simple as performing a final review of all compiled audits results. Or, this process may discover in the final review that audits are incomplete or inconclusive. In this case, a request for further assessment may be issued to the audit team to conduct additional audits in specific areas, with specific guidelines. Once the final analysis has completed, the assessment report will be prepared.
Inputs
Compiled Results. Reference the Outputs section under Compile Assessment Results for a detailed description.
Tools and Techniques
A formal review should be scheduled and the audit results should be distributed to team members in preparation for the review. The team should review the Assessment Plan to make sure that all audit requirements have been met. All graphs, charts, and diagrams should be reviewed for completeness and correctness. Text should be reviewed for clarity. The analysis should result with recommendations based on the audit findings. Recommendations may be made for each quality item, which may benefit from improvement activities. Recommendations may be broad in scope and/or may be specific in detail.
Outputs
Request For Further Assessment. Reference the Inputs section under Assess Quality Items for a detailed description.
Completed Assessments. This information represents the compiled and analyzed assessment results that are now ready for placement in an Assessment Report. Recommendations are included with the assessment results.
Prepare Assessment Report
Description
The Prepare Assessment Report process is responsible for preparing a formal document describing all of the audit results obtained and the recommendations made by the Quality Team.
Inputs
Completed Assessments. Reference the Outputs section under Analyze Assessment Results for a detailed description.
Tools and Techniques
An Assessment Report template or outline should be generated and reviewed by the Quality Team prior to writing the report. The template should be logically arranged such that a clear story of the assessment process is told. The report should reference audit requirements that have been met according to the Assessment Plan. The report should describe and include the tools used during the audit process, and the specific results obtained. The report should identify all members of the Quality Team, and all audit participants.
Outputs
Assessment Report. The complete and final documentation of the assessment process.
Review Assessment Report
Description
The Assessment Report is distributed to all stakeholders of the assessment process for review, to include the stakeholder(s) that issued the original Quality Assessment Request. The Quality Team conducts a presentation to the stakeholders. Recommendations are presented and discussed, and may be modified during the review. Approved recommendations are written as Corrective Action Required, which are formal task directives to plan, schedule and execute quality improvements.
Inputs
Assessment Report. The complete and final documentation of the assessment process.
Tools and Techniques
Formal Review. A formal meeting to review the material is a good forum for discussing the audit results with the stakeholders and making. Preliminary planning may be accomplished during the review to identify which corrective actions should be taken. Preliminary schedules may also be prepared. Corrective Action Required forms should be completed for each corrective action deemed necessary. The Assessment Report should be distributed prior to the meeting to allow ample time for individuals to read and understand the information in the report.
Exhibit 3. Improve Quality Process Overview
Presentation. A formal presentation of the findings and recommendations is an excellent technique to use to convey information to stakeholders. A presentation may be prepared using information in the Assessment Report. Slides should contain bulleted information that highlights the findings and recommendations.
Outputs
Corrective Action Required. This is a form that contains a description of the corrective action and planning information to be used when performing the corrective action activities. Information to be included on the form is as follows.
a. Corrective Action Number
b. Problem Description
c. Potential Cause of Problem
d. Corrective Action Description
e. Assigned to
f. Stakeholders
g. Corrective Action Plan Overview.
Improve Quality
The Corrective Action Required (CAR) forms resulting from the Assess Quality process formulate the inputs to the Improve Quality process. This process establishes and executes a quality improvement project to accomplish each CAR. In essence, the set of CARs form a Statement of Work for the improvement project. Detailed plans and activities are scheduled, and the project is executed against the plan and schedule. Measurements are established to measure actual quality improvements and the effectiveness of the improvement project. Once measurements indicate that overall improvement objectives have been met, the improvement project closes, usually with a plan for subsequent, periodic revisiting of the improvements as in the case of continued Quality Assurance.
Initiate Quality Improvements
Description
Initiation is the process of formally recognizing that a new project exists and in this case it defines the existence of a Quality Improvement Project. The assessment process has determined that corrective actions are required. These corrective actions are translated into a Project Charter, thus signifying the beginning of a project.
Inputs
Corrective Action Required. Reference the Outputs section under Review Assessment Report for a detailed description.
Tools and Techniques
Expert Judgment. Expert judgment will often be required to assess the corrective actions required, and may be provided by any group or individual with special knowledge or training in quality improvement initiatives.
Outputs
Project Charter. A project charter is a document that formally recognizes the existence of the Quality Improvement Project. The charter should describe the project, reference the corrective actions, identify a project manager, and target a high-level plan and schedule. Constraints should be included that identify factors that will limit the project’s options.
Plan for Quality Improvements
Description
This process produces the project plan for the Quality Improvement Project. This process may be iterated several times to go from a draft plan with none-specific, interim schedule dates to a detailed plan that reflects specific resources and explicit dates. A project plan is used to guide the project, document planning assumptions and decisions, facilitate communication among stakeholders, and define key management reviews.
Inputs
Project Charter. Reference the Outputs section under Initiate Quality Improvements for a detailed description.
Change Plan Order. While the project is under way, the project’s controlling process may determine that a change to the project plan is required. Based on continual inspections and measurements, the controlling process may order that new or modified approaches or methods be utilized in the project to meet the project’s objectives. This would then require replanning efforts to institute a modified project plan.
Tools and Techniques
Project Planning Methodology. A project planning methodology is a structured approach used to guide the project team during the development of the project plan. It may be as simple as following a project plan template or standard forms. Making use of project management software, such as Microsoft’s Project, is a good approach to creating a project plan.
Analogous Planning. Making use of knowledge gained from similar projects is a good benchmark for creating a project plan. Analogous projects may help identify necessary project resources, budget, and schedule.
Outputs
Project Plan. The project plan is a formal, approved document used to manage and control the Quality Improvement Project. The plan may change over time as more information becomes available about the project. A plan should include:
a. Description of the project
b. Corrective actions to be taken
c. Cost estimates
d. Schedule dates
e. Review dates
f. Major milestones
g. Key staff
h. Key risks
i. Open issues and pending decisions.
Execute Quality Improvements
Description
The Execute Quality Improvements process is the primary process for carrying out the project plan. The project team will institute quality improvements within the organization. Business processes may change or be introduced. Training may be employed by the project team to introduce new ways of doing business to the organization.
Inputs
Project Plan. Reference the Outputs section under Plan for Quality Improvements for a detailed description.
Change Execution Order. While the project is underway, the project’s controlling process may determine that a change to the project execution is required. This may be the case when it is determined that the plan is satisfactory, but the plan is not being executed properly or completely. Based on continual inspections and measurements, the controlling process may order that the project’s execution be modified so that the project will meet the project’s objectives.
Tools and Techniques
General Management Skills. General management skills such as leadership, communicating, and negotiating are essential to effective project plan execution. These skills include problem solving skills and the ability to effectively influence the organization.
Interpersonal Skills. It is very important that the project management team on a quality improvement project has the capability to present ideas and concepts clearly and without strong emotion. This project will be changing the way things are done. The ability to obtain support from organization staff is critical to reduce operational impact and increase the success of the overall improvement effort.
Training. Often, the introduction of quality improvements in an organization requires training the organization’s staff. Plan training sessions in conjunction with the deployment of quality improvements within the organization. Training may take the form of typical training courses, meetings, presentations, or distributing documentation.
Status Reviews. Holding status reviews is a good technique for communicating information within the project team and to stakeholders. Status reviews should be held at various management levels and should be schedule periodically.
Outputs
Improvements. Improvements are the actual corrective actions taken in the project. Improvements may take the form of:
a. New or modified business processes
b. Software product utilization
c. Standards and Procedures
d. Techniques to follow
e. Distributing information
f. Coordinating activities
g. Changes in staff.
Performance Reports. Usually the result of a status review, a performance report is a project report of activity, cost, and schedule. These periodic reports will identify which activities have been completed (or to what percentage they are complete). Cost and schedule variances are included to illustrate how the project is performing against the cost and schedule baselines. A performance report may also include action items. Action items are things-to-do or problems to be solved that have surfaced during the execution of the project.
Control Quality Improvements
Description
The Control Quality Improvements process will continually measure the success of the quality improvements that have been installed in the organization. This process will measure the effectiveness of each improvement and request changes to the plan or project’s execution if the measurement indicates that an improvement is ineffective.
Inputs
Performance Reports. Reference the Outputs section under Execute Quality Improvements for a detailed description.
Measurements. The controlling process will take measurements of each improvement. These measurements will influence decision-making that may be required if improvements are not effective.
Tools and Techniques
Several different tools and techniques may be utilized to measure the effectiveness of an improvement. These tools and techniques are similar to those described in the Assess Quality Items section of this document. Reference the Tools and Techniques paragraph under Assess Quality Items for more information. In fact, the same tool or technique that was used in the assessment may be reused to obtain information necessary to ascertain a measurement. For example, reusing the same questionnaire should produce results that indicate improvements to the quality item, if the improvement is effective.
Outputs
Change Execution Order. Reference the Inputs section under Execute Quality Improvements for a detailed description.
Performance Completion Status. This output indicates that the project plan is complete and that all of the improvements are in place and are considered effective.
Change Plan Order. Reference the Inputs section under Plan for Quality Improvements for a detailed description.
Close Quality Improvements
Description
Once the project objectives have been met, the project should perform administrative closure, record the project’s performance, and document lessons learned. This process involves generating, gathering, and disseminating information to stakeholders and management.
Inputs
Performance Completion Status. This input indicates that the project plan is complete and that all of the improvements are in place and are considered effective.
Tools and Techniques
The tools and techniques involve effective documentation and communication of the project’s results. The project team should document the effectiveness of the project and the lessons learned while performing the project. This documentation should identify the failures and project objectives that were not met, and why. This documentation should be historically maintained and used in subsequent quality improvement initiatives.
Outputs
None.
