Nearly half of business leaders around the world say their organizations are “deeply involved” in automation projects, according to a 2018 Deloitte survey. But automation success isn't automatic. The complex systems designed to replicate human tasks come with many possible risks, which can have severe consequences when the automation systems launch operations.
For instance, a computer algorithm failure in Britain's automated health services discovered in May prevented women from receiving a digital reminder to schedule their mammograms. As a result, 450,000 patients in England missed vital breast-cancer screenings and up to 270 patients might have died because of the oversight, according to an analysis. And project teams missed an opportunity to build a backup notification process into the system.
“Automation looks to increase speed and reduce human errors, but during deployment it might amplify mistakes,” says Carlos Carnelos, PMI-RMP, PMP, Latin America transformation leader for technology support services, PMI Global Executive Council member IBM, São Paulo, Brazil. “Automation projects require careful planning, tests and execution. Otherwise, by the time you realize you've made a mistake, it could be too late to prevent a business impact. You can cause a lot of damage, not just to the client's processes but to your relationship with stakeholders.”
As the pace of change accelerates, the pressure is on project managers to build and execute a precise and comprehensive risk management plan. Whether it's missing critical requirements, establishing lackluster testing procedures or failing to identify a system's long-term post-deployment needs, poor risk management can wreak havoc on automation projects.
ANTICIPATING THE WORST
There is no one-size-fits-all environment for automation projects. Project managers must have a firm grasp on the particulars of each project environment when they develop the risk register. For Mr. Carnelos, that means mapping detailed client processes, so teams can customize automation solutions based on how each client does business—down to preferences for how a system password is changed. A project manager must aim to have full visibility into all of the variables a human encounters when executing a task and all the activities connected to it, he says.
To ensure nothing is missed, Mr. Carnelos recommends adding shadow periods to the project plan. This means sitting with workers who perform tasks that will be automated to check whether their actions align with what was previously documented. Shadowing also provides an opportunity to ask front-line stakeholders to identify “what-if” scenarios that can be added to the risk register, Mr. Carnelos says.
“The closer you are with the specialists and the better the understanding of the piece of work you are trying to automate, the better the test phase and final results will be,” he says.
—Carlos Carnelos, PMI-RMP, PMP, IBM, São Paulo, Brazil
Going to extremes to observe and ask the right questions helps teams prevent risks that can arise during the requirements-gathering phase, says Manish Agarwal, vice president of telecom network consulting, APAC, for PMI Global Executive Council member Accenture Consulting, Gurgaon, India. Even spending time on the floor with people to capture every possible perspective of the task that's being automated can help teams identify and analyze every link to every task. Such gating criteria helps stitch together the end-to-end process, he says. “If you leave out something, it is a big risk that can derail a project.”
That's precisely what happened on a recently completed network optimization project that involved automation for one of Europe's biggest telecom operators. Mr. Agarwal and his team failed to fully integrate the operations and business support systems. The result: The client was delivering a service to its customers but not billing for it. The costs went beyond lost revenue for the company: Mr. Agarwal had to increase the project budget 20 percent to fix the problem.
Obsolete Value
Although automation projects center on computers and machines, project teams can't afford to overlook the human risk—namely how implementing a system that threatens to eliminate a team member's role could hurt their performance.
Manish Agarwal, vice president of telecom network consulting, APAC, for Accenture Consulting, Gurgaon, India, says some of his teams have reduced in size because of automation. It's natural for some team members, such as software engineers, to express apprehension about implementing systems that could make their roles obsolete. Mr. Agarwal says such scenarios can lead to a lack of interest or even foot-dragging among technical staff.
“This poses a very high risk that all the necessary information and details will not be captured to build the automation framework. And this results in significant delays,” he says.
Keeping technical team members motivated in the face of possible job loss is a delicate task for project managers. The primary solution: Explain how their performance will help build their skills and experience, Mr. Agarwal says. Those who embrace the automation project will also gain an advantage over others for future opportunities—whether it's a contract job for the organization's next project or a full-time position elsewhere.
“Have conversations with them to make it clear that the worker who makes himself redundant is also the one who gets the maximum opportunity for redeployment,” Mr. Agarwal says.
—Manish Agarwal, Accenture Consulting, Gurgaon, India
“We did not really capture the requirements correctly because we didn't ask the right questions,” he says.
PASSING THE TEST
A well-planned and comprehensive testing phase will help identify unknown risks—and ensure more effective deployments. In an ideal environment, teams can test solutions without damaging any systems or deliverables, then slowly scale up from there, Mr. Carnelos says.
The right testing location will minimize impact on operations. But when a suitable testing environment doesn't exist, it's best to “choose a less critical geographical location or a system with fewer users,” so that any problems don't reverberate, Mr. Carnelos says. It's also important to think about how timing can impact the value of testing. For instance, retail organizations should avoid major holidays, when testing errors could create massive shopping transaction breakdowns, he says.
Defining the most effective testing duration is also a challenge for project teams, Mr. Agarwal says. Although automation projects work on a model in which a proof of concept phase comes before testing, the algorithms may not be complete—even after a successful proof of concept phase. That can lead to longer, more complex and potentially more costly testing phases.
In some cases, automated processes may still run manually while the automation continues to cycle to collect enough data to become reliable. On one recent project, this process took up to four months to complete, Mr. Agarwal says. Such longer testing periods also mean added costs. He proactively factors these costs into a contingency budget and alerts project sponsors to the possibility of additional time and costs right from the start. A contingency of 13 to 15 percent of the total project cost typically covers extended and more complex testing phases, he says.
“When I launch a service, if there is not enough data to cover all scenarios in the real world, then the end-user experience may not be that great,” he says. “You can lose revenue by losing customers who become frustrated or agitated.”
To avoid this, Mr. Agarwal recommends a beta phase in which operations use the system until it is 95 percent completed, then teams can gradually scale the automation to fully interface with end users. Having all key stakeholders involved during the testing phase also helps reveal all known and unknown risks, says Yang Song (Aaron), PMP, senior project manager, Honeywell Process Solutions, Tianjin City, China.
For instance, if a project manager is automating a paper factory, it isn't enough to invite only the end user to the testing phase. The project manager should also involve a representative from the manufacturer of the machinery to ensure the machines are running as designed. “We may be the automation control system experts, but they know their system, so it's important to get them involved,” he says.
—Yang Song (Aaron), PMP, Honeywell Process Solutions, Tianjin City, China
AVOIDING AFTERSHOCKS
Project teams must balance the need for comprehensive testing phases with the reality of potential post-deployment problems. Whether it's a technical glitch that needs to be resolved immediately or a client that decides to change the system, project managers have to equip organizations with the proper post-project data and documentation to help mitigate those risks and ensure the automation project delivers long-term benefits.
Kumar Rao, PMP, senior program manager, PMI Global Executive Council member Microsoft, Seattle, Washington, USA, says teams should build testing and monitoring capabilities into completed systems so that organizations can continue to gather data that helps identify risks.
“A lot of people don't do it or they do it as an afterthought—and that is really risky,” he says. “The moment you start to produce bad data, you have lost your stakeholders’ trust, and it is expensive to regain. We build data quality checks into the system so testing is done every time the process runs.”
—Kumar Rao, PMP, Microsoft, Seattle, Washington, USA
Teams must document automation steps and create proper maintenance procedures as part of the project handoff. This leaves breadcrumbs for the operators when problems arise or when organizations decide to tweak the system. For example, in one of Mr. Carnelos’ projects to automate the database for a multinational retail client, which he finished in 2017, the client wanted some changes made after deployment. It was only then that Mr. Carnelos realized he and his team had failed to update maintenance processes and procedures for the client—procedures to follow and whom to contact in case the organization had problems or wanted a change. Mr. Carnelos had to spend extra time adding processes that should have been there in the first place.
“Having a good checklist of all the aspects of the solution that you must work on and foresee is important,” he says. “You have to cross-check all the critical steps and ensure you aren't forgetting anything.” PM
