Project Management Institute

Crowd control

img

CROWDSOURCING PROJECTS CAN TAP INTO A HUGE TALENT POOL AND CUT COSTS—BUT ALSO RAISE QUALITY AND SECURITY RISKS.

img

Alora C. Chistiakoff, BrainMatch, Austin, Texas, USA

PHOTO BY ROBERT HOUSER

There's wisdom in crowds, and an increasingly connected world allows companies to take advantage of it.

Crowdsourcing first seemed to be the cure-all for businesses working with limited budgets and constrained resources. But several misfires point to a need for organizations to adapt their project and portfolio management processes to address the risks crowdsourcing raises.

“Project managers shouldn't ignore the potential of crowdsourcing as a solution to the supply of products and services in their projects,” says Marcelo Yamada, IT project coordinator at construction company Promon Engenharia in São Paulo, Brazil.

Insider Knowledge

While crowdsourcing can be a useful way to expand the capacity of in-house project teams on an as-needed basis, even the savviest crowd won't have knowledge on par with employees when it comes to company strategy, competitive positioning or internal processes, says Matt Johnston, uTest, South-borough, Massachusetts, USA. “It's best to hand off clearly defined pieces of a project that don't require a great deal of internal domain expertise beforehand.”

Of course, crowdsourcing isn't as simple as presenting a project to the masses and waiting for the work to come flowing in. Google, for example, relies on crowdsourcing for its Places online phone book and social network project. But the company soon discovered that when information isn't closely monitored, problems arise. In September, the Internet giant admitted that some unscrupulous business owners were listing their competitors as “permanently closed” in an effort to poach customers.

Google's experience serves as a cautionary example for businesses that choose to explore a crowdsourced approach to projects. Risk management must be a top priority if organizations are to reap the benefits of the crowd while avoiding sabotage and unintentional errors.

DIVIDE AND CONQUER

Crowdsourcing creates discomfort within enterprise organizations, particularly for corporate legal teams, says Alora C. Chistiakoff, Austin, Texas, USA-based founding team member of BrainMatch, an organization that works with companies to provide students with project-based internships. Organizations that open their IT infrastructure up to people outside their control face risks, including security breaches and, in extreme cases, the need to pursue legal action.

A PROJECT MANAGER'S “WORST NIGHTMARE”?

In the post “Does Crowdsourcing Work in a Project Environment?” on the Voices on Project Management blog, Geoff Mattie says that crowdsourcing “seems like a project manager's worst nightmare.”

The requirements and quality management alone must be a huge undertaking:

  • How do you ensure a team of people who aren't getting paid remain focused enough to see your project through to completion?
  • How do you ensure no one is trying to game the system?
  • How do you reward those contributing more than others?

Laurent Stanevich replies:

I think you're exactly right on many fronts—adding compensation to the mix opens up a whole new level of complexity. It's hard enough just getting a crowd to row in the same general direction, but the open source movement has actually managed to evolve a pretty extensive set of approaches that they use to coordinate development. You still see projects die on the vine or totally blow deadlines all the time, but there are definitely some that work pretty efficiently.

Nevertheless, that's in sort of an economic vacuum. As soon as you start to have money flowing through it, or even vaguer issues like recognition and creative credit, it all gets a lot more complicated.

Read more at pmi.org/voices.

What's My Motivation?

You and your colleagues may register a palpable sense of urgency surrounding projects. But don't assume that feeling automatically transfers to crowdsourcing participants.

There are ways to motivate those in the crowd, though—even when participants don't receive monetary compensation.

“Of course, money talks. But the transparency of crowdsourcing motivates people to build strong reputations based on performance,” says Matt Johnston, uTest, Southborough, Massachusetts, USA. “This is often the only way they can earn future work and the respect of their peers.”

Organizations may want to consider vetting their crowdsourcing companies based on the effectiveness of their performance ratings and recognition levels for top-performing community members. That allows them to choose and incentivize the right professionals in a community.

“It's far better to partner from the get-go with crowdsourcing vendors that take a proactive approach to driving high-quality work than to be in the position of having to reactively motivate volunteers who are performing below expectations,” Mr. Johnston says. “If a crowdsourcing company can't walk you through its process of vetting, rating and matching its crowd with each project, then keep asking questions or shop around.”

For the past year, Tim Causer, PhD, research associate at University College London in London, England, has helped oversee hundreds of unpaid volunteers for a crowdsourcing effort to transcribe the manuscripts of philosopher Jeremy Bentham. The project has more than 1,400 registered volunteers helping, but a group of about 12 “super transcribers” do the bulk of the work, Dr. Causer says.

The team developed a points system and leaderboard to rank those involved in the crowdsourcing effort. But when it conducted a survey to discover what led people to take on complex transcription tasks without pay, the team found competition and recognition were low motivational factors. Instead, volunteers—including the most prolific one—said the work was stimulating and were mostly motivated by an interest in history, philosophy and Mr. Bentham in particular, as well as contributing to the “greater good” by making transcripts available to the public.

What motives one crowdsourcer may not motivate another—and it's up to project leaders to figure out what works.

“Any successful crowdsourcing project must have a number of motivational strategies,” Dr. Causer says.

img

“Try suing a developer in Ukraine for copyright infringement, and see how far you get,” she says. “There are plenty of legal departments that do not consider this an acceptable risk.”

By limiting crowdsourced team members to isolated tasks within a project, you can help protect your organization from risk.

“A project manager can parse out stand-alone deliverables within a project and use crowdsourcing to complete them independently while retaining the integration effort in-house,” says Ms. Chistiakoff, who is also an engagement manager at SysIQ Inc., an e-commerce implementation firm, where she focuses on large-scale engagement and program management. “That way, no one crowdsourced team member has the full picture.”

In Action

  • One crowdsourcing project tapped users of the online game Foldit to solve the structure of a protein virus. More than 600 players took part, solving in 18 months a problem that scientists had wrestled with for more than a decade.
  • NASA (U.S. National Aeronautics and Space Administration) created a competition to help with its software code-writing projects. Researchers post their needs online, then choose the best solution from those submitted.
img

She notes that the intelligence community has historically used a similar approach, breaking apart the pieces of a puzzle to make sure that no one without authorization ever understands the overall picture.

That's the tack Deniz İren, PMP, researcher and project manager at METU (Middle East Technical University) in Ankara, Turkey, took on a recent crowdsourced project to digitize documents. To keep the sensitive data confidential, the project team used image-processing techniques to develop a program that scrambled the words.

Sectors that deal with sensitive information are often less suited for crowdsourcing. “While it's not a hard and fast rule, it's more difficult to crowdsource work in heavily regulated industries,” says Matt Johnston, chief marketing officer at uTest, a Southborough, Massachusetts, USA-based company that provides crowdsourced software-testing services.

For example, if a defense contractor were to use the technique, it would have to address the security clearance of every single person brought onto the project team.

That still leaves vast opportunities to leverage crowdsourcing for project tasks such as graphic design, animation, content production and application testing, Mr. Johnston says.

img

While it's not a hard and fast rule, it's more difficult to crowdsource work in heavily regulated industries.

— Matt Johnston, uTest, Southborough, Massachusetts, USA

TRUSTING STATISTICS

Organizations crowdsourcing a project must ensure that quality control practices are planned according to the nature of the work and conducted throughout the process, Mr. İren says. If the work package consists of a large number of “microtasks,” such as translating sentences or mapping craters on the surface of Mars, checking the quality of all outputs is generally not feasible.

Instead, he suggests project teams periodically conduct quality control checks by statistical sampling. One microtask should be performed multiple times to see if a consensus exists. This may sound inefficient and costly, Mr. İren says, but it can offer an effective strategy for quality assurance and sabotage prevention.

The technique proved its worth on a crowdsourced digitization project.

“In crowdsourcing, we pay for the deliverable, not for the time spent,” Mr. İren says. “Thus it eliminates many risks and costs. And the pay rate is significantly lower than the pay rate of full- or part-time staff. Even if it causes redundant work, it still is cheaper.”

Documents were scanned as images, and the crowd would transcribe the text. The team set the redundancy threshold at two tasks, meaning that when a picture of a word was transcribed the same by two people, the system recognized it as correct.

A project team member controlled the quality of the work, comparing the actual texts with the scans of selected documents.

“The team member carried the role of ‘crowd-master,’ and constantly monitored the work on the crowdsourcing engine,” Mr. İren says.

Thanks to crowdsourcing, the project team digitized over 45,000 pages, with more than 90 percent accuracy.

NEEDED: A NEW SKILL SET

Crowdsourcing can create challenges for integration—one of the most critical functions of project-based organizations.

“If you have a piece of software that has been divided out among several teams, each delivering different pieces, then who makes sure that they all work together?” asks Ms. Chistiakoff.

That role often falls to the technical lead on a project. But juggling teams that are not allowed to know about each other's work changes the nature of the job, she says. And not everyone with experience as a technical lead will be able to do this type of parsing and reassembling of compartmentalized information and assets.

“It will be a few years before the IT industry and project managers really understand the full extent of the new management disciplines that will be needed in order to really maximize crowdsourcing's power,” Ms. Chistiakoff predicts.

Crowdsourcing's potential to offer project managers an inexpensive, decentralized supplier for products and services means dealing with a large quantity of occasional suppliers of small work packages, contracted with minimal bureaucracy and low loyalty—an entirely new landscape for most project professionals.

“Relying on this type of delivery requires the development of new management processes—both project management and knowledge management,” says Mr. Yamada, who also teaches knowledge management at Senac de São Paulo University.

Project professionals are already figuring out some of those so they don't get lost in the crowd. PM

Pluses and Minuses

img
  • 1. Don't have to pay staff or contractors
  • 2. Larger workforce means projects can reach completion faster
  • 3. Inexpensive, decentralized supplier of products and services
img
  • 1. Opens companies up to potential data security breaches
  • 2. Quality control is more of a challenge
  • 3. Integrating divided project tasks is much harder
This material has been reproduced with the permission of the copyright owner. Unauthorized reproduction of this material is strictly prohibited. For permission to reproduce this material, please contact PMI.

PM NETWORK JANUARY 2012 WWW.PMI.ORG
JANUARY 2012 PM NETWORK

Advertisement

Advertisement

Related Content

Advertisement