Configurations of project-specific incentive and appraisal systems and their impact on project success in a US-German comparison
Thomas G. Lechler
Projects are complex, unique endeavors, with substantial performance demanded of those who execute them. This performance is likely only to be achieved if the participating individuals are highly motivated. From a theoretical perspective this effect could be explained by Porter-Lawler’s valence theory (Lawler, 1968, pp. 462 – 468). Following this proposition, individuals will expect adequate rewards commensurate with their project performance. As a consequence if rewards are not offered or not awarded as promised, the motivation will be compromised. This theoretical relationship between the motivation, expectancy, and valence (i.e., perceived attractiveness) of rewards in the context of project work suggests that the project performance of the participating individuals depends on the offered rewards.
Although the question of using rewards to control motivation is elaborately discussed in the general management literature and seems to be quite important for successful project execution, it is not yet discussed from the perspective of project management. Little is known about how a project-related incentive system is related to project performance. Despite the fact that many organizations operate with an increasing amount of project work, incentives and performance appraisals are typically based on routine tasks, not on the generally more demanding work required by a project. In ordinary matrix organization settings functional managers are responsible for allocating rewards to project team members. Mohrman, Resnick West & Lawler (1989) characterized this situation as: “Individuals tended to be matrixed into projects, but their supervisors remained in their functional bases” (p. 12). The consequence seems to be that project team members are more likely to emphasize on line-related routine tasks than project tasks. Also, project managers are generally more familiar with the project-related performance of the team members and are in a better position to accurately evaluate the participating individuals. Thus, it is questionable whether project performance is adequately rewarded. The central question of this paper is: How do project specific incentive systems (PIAS) affect project success?
This paper empirically addresses the impact of PIAS on project success. Analyzing two large project samples consisting of more than 600 projects that were collected in the U.S. and in Germany the paper attempts to address three goals. The first goal is to identify variables that are suitable for measuring the presence of a PIAS. The second goal is to describe the gestalt of an existing PIAS and the third is to determine the effects that a PIAS has on project success.
In the first section of this paper the choice of motivational foundation of a PIAS is described and a concept for a PIAS is defined. In the second section hypotheses are derived from the field of performance measurement and motivational theories. The third section consists of the statistical analysis of the survey data samples, describes the current state of PIAS implemented on the project level and explores its impact on project success. In the fourth section the conclusions are discussed and questions for further research are raised.
Functions and Elements of a PIAS
In the conceptual development of a PIAS model we follow Porter and Lawler’s process-oriented extension of expectancy theory, rather than a content-focused theory (e.g., Maslow’s needs hierarchy, Herzberg’s two-factor theory [Herzberg, 1966, 1987]) or equity theory (Adams, 1963, pp. 422 – 436). Porter and Lawler’s theory is “widely supported by a diverse array of reviewers” (Miner, 2002, p. 203) and is one of the most recent and advanced theories. The adoption of a process view improves our understanding of how motivation is created. Porter and Lawler extend Vroom’s (1964) expectancy theory by adding a dynamic feedback loop between motivation, performance, and satisfaction. According to their model, motivation results from a multiplication of effort-to-performance expectancy with the added products of all relevant outcome’s performance-to-outcome-expectancies multiplied by their valences (Miner, 2002).
Motivation = Effort-to-performance expectancy multiplied by the sum of all operating factors (performance-to-outcome expectancies multiplied by the valences).
Valence theory integrates the reward with the appraisal function. The reward function allocates material and immaterial rewards on the basis of performance appraisals. Thus, the effectiveness of the reward system depends on the quality of inputs received from the appraisal process as well as from the attractiveness of the potential rewards.
Following the basic ideas of Porter and Lawler we define a PIAS with three basic elements:
- Incentive type. This component describes the type of incentive choices: the types of rewards and the use of individual and group rewards.
- Process configuration. The configuration of a PIAS represents the process of appraising and awarding rewards and describes the effectiveness of the process of creating motivation, such as perceived fairness of rewards and line of sight.
- Power structure. The structure includes organizational aspects, such as the degree of project manager authority in handing out rewards.
These three elements also demonstrate that a PIAS is a coherent system, not just a number of independent rewards. The following hypothesis addresses the development of a measurement scale for a project-specific incentive system:
Hypothesis 1: The three components—process configuration, incentive type, and power structure of PIAS—can be empirically confirmed.
An incentive system attempts to control the motivation of employees to ensure the achievement of a company’s, or a project’s, goals. This goal-oriented motivation is accomplished by awarding extrinsic and intrinsic rewards for achievement of a certain performance level. This study focuses on extrinsic rewards that are empirically linked with higher team performance (Tosi, Katz, & Gomez-Mejia, 1997). The members of a project team are expected to be highly motivated if rewards and incentives are offered resulting in an increased likelihood that the project will be successfully completed (Banker, Lee, Potter, & Srinivasan, 1996; Tosi et al., 1997). This effect could only be reached if the appraisal processes are perceived as fair. From a structural point it seems to be obvious that the project manager should have the authority and the power to give incentives. The performance hypotheses propose that the three components of a PIAS significantly contribute to project success.
Hypothesis 2a: The incentive type component impacts different criteria of project success.
Hypothesis 2b: The process configuration component impacts different criteria of project success.
Hypothesis 2c: The power structure component impacts different criteria of project success.
The model defined in this study does not address intrinsic rewards, though they undoubtedly play an important role in generating motivation (Herzberg, 1987). The design of this study did not permit a consideration of this reward type. Also, intrinsic rewards could be regarded as inherent in project work, which is characterized by a high degree of innovativeness and challenge. Project work is more interesting and challenging than routine work and includes a latent potential for intrinsic motivation.
Data and Sample
The units of analysis in this study are projects executed within larger organizations. Two different samples were used to assess the impact of organizational structure on project success. One sample of project data was collected in Germany in 1997 and the second was collected between 2001 and 2004 in the U.S. In both cases a survey was conducted using similar questionnaires designed to measure the impact of a project’s managerial variables on project success. The questionnaire includes 199 single items and some quantitative measures of project-specific characteristics. Out of these, 67 items were directly taken from Pinto’s (1986) questionnaire, with permission of the author. The remaining items were developed for the purpose of this study. Each item was assessed on 7-point rating scales with a range from “strongly agree” to “strongly disagree.” The questionnaire was written in German and Pinto’s adopted items were translated from English to German. The German questionnaire was pre-tested and modified after in-depth interviews and responses by a group of experienced project managers. In order to collect data on U.S. projects the German questionnaire was then translated into English. To demonstrate consistency and accuracy in translations the translated documents were back-translated using two experts who discussed the translations and together corrected any inconsistencies.
In Germany, the questionnaires were sent out to the members of the German Project Management Society (GPM, Gesellschaft für Projektmanagement). Each respondent was asked to complete two questionnaires, one for a completed successful project and the other for a failed project. This concept of pair-wise comparison was first introduced by Rothwell et al. (1974) and has the advantage of reducing the personal bias of the key informants. In Germany a response rate of 43% was achieved, resulting in a total sample size of 448 projects (257 successful and 191 unsuccessful). The respondents had different roles in the projects: 46% were project managers, 27% were core team members (with technical or business-related responsibilities) and 27% were external consultants responsible for specific project tasks with intimate knowledge of the project. The projects were implemented in different industries including automobiles, machine tools, software, pharmaceutical, and construction. More than 50% of the projects were new product development or software development efforts. The sample provides a fairly representative cross-sectional distribution of projects carried out in the German industry.
The U.S. sample was gathered between 2001 and 2004 with the assistance of project team members and/or project managers. They were asked to select, within their organizations, a single successful or failed project that was recently completed or that was close to completion with a budget of at least $500,000 and duration of at least six months. These individuals were each handed three identical questionnaires that they were asked to distribute to the project manager, a team member and the senior manager responsible for the funding of the project. The questionnaires were answered independently by the different participants. Prior to final data collection in the U.S. the instrument was tested in a pilot of 20 projects. The U.S. sample size is 181 projects (126 successful and 55 unsuccessful projects). In total, 181 project managers, 81 senior managers, and 214 project team members from a variety of U.S. companies in manufacturing, software, pharmaceutical and telecom industries answered the questionnaires.
To avoid mono-method bias we aggregated the data gathered from the 420 stakeholders (project managers, team members, and senior managers) in the U.S. sample by taking the average of the project scores. To test this aggregation at the project level we calculated the within-unit agreement rWG(j), and the eta-squares (George, 1990). All constructs were tested with Cronbach alpha and factor analyses for composite validity.
The literature review did not reveal a measurement instrument for the PIAS. Therefore a total set of 14 items was developed to measure the three PIAS components. These items were rated on a 7-point scale ranging from 1 (strongly disagree) to 7 (strongly agree). In the U.S. sample a project’s score on each of these scales was the average of each scale’s items aggregated to the project level. Aggregation tests revealed that the average rwg(j) values for the incentive type, the process configuration and the power structure scales were .95, .91 and .84, above the generally acceptable level of .70 (George, 1990), thus demonstrating within-unit agreement.
Project Success Variables
Pinto and Mantel (1990) identified three distinct aspects of project performance: (1) the implementation process; (2) the perceived value of the project; and (3) client satisfaction with the delivered project outcome. Shenhar, Dvir, and Levy (1997) suggested four criteria to assess project success: (1) meeting design goals, (2) benefits to customers, (3) commercial success, and (45) future potential.
In this study we follow Lipovetsky, Tishler, Dvir, and Shenhar (1997) and Shenhar, Tishler, Dvir, Lipovetsky, and Lechler (2002) as well as Pinto and Mantel (1990) to measure different aspects of project success, reflected in four different scales: efficiency, effectiveness, customer satisfaction and business results (see Appendix). These success items were developed by Pinto and Slevin (1988) and modified and supplemented by Lechler (1997). All the success items were rated by senior managers, project leaders and project team members in the U.S. sample and by project leaders in the German sample using 7-point rating scales ranging from 1 (strongly disagree) to 7 (strongly agree). Cronbach’s alpha values for the four success scales ranged from .71 to .87 indicating sufficient measurement quality (see Table 1).
|Scale||Measures||Reliability (α)||Reliability (α)|
|Business results|| ||0.71||0.73|
|Customer satisfaction|| ||0.76||0.74|
Table 1. Measurements of project success
Components of PIAS
The 14 survey items were assigned with factor analyses to the three conceptual PIAS components. The items achieve satisfying factor loadings (see Table 2). Contrary to the author’s expectations, the item that measured the degree of goal-orientation in the assessment of individual performance (item 11) did not achieve satisfying statistical results in both samples and had to be rejected. This item might have to be regarded as an independent characteristic of an incentive system, though further research is necessary to confirm this.
|1||Good performance was rewarded with promotions and further career prospects.||.847||.906|
|2||Good performance was rewarded with financial rewards (rise in salary, bonus…).||.834||.916|
|3||Good performance was rewarded with immaterial rewards (appreciations…).||.745||.769|
|1||Incentives of team members depended on their individual project performance.||.758||.660|
|2||Performance rewards could be considered fair and equitable.||.841||.900|
|3||Performance rewards were generally considered as attractive.||.879||.891|
|4||All given promises, regarding rewards, were met without exceptions.||.757||.776|
|5||The link between performance and rewards was clear to all team members.||.823||.813|
|1||The project manager had significant influence in assessing the performance of team members.||.915||.899|
|2||The project manager had significant influence in deciding on the kinds of rewards.||.915||.899|
Table 2. Item statistics of factor analysis
|Scale / items||% Variance||Reliability (α)||% Variance||Reliability (α)|
|Incentive type||65.60%||.73 (N=439)||75.00%||.833 (N=175)|
|Process configuration||66.07%||.87 (N=438)||66.02%||.866 (N=153)|
|Power structure||83.80%||.80 (N=440)||80.77%||.758 (N=178)|
|Tables 2 and 3 summarize the results of the factor analyses for the German and US set of data, respectively. Each factor explained a high percentage of variance and the items achieved high factor loadings indicating sufficient validity. The reliability of the three PIAS scales were tested with Cronbach’s alpha and the values in both samples ranged from .74 to .84 indicating good to excellent measurement quality (see Table 2). For the following analyses the constructed variables are called: INCENT (incentive type), CONFIG (process configuration), STRUCT (power structure). The empirical results support the first hypothesis (Hypothesis 1) that proposes the existence of the three suggested PIAS components. A second factor analysis with these three constructs (and a Cronbach’s alpha value of 0.9) suggests a possible aggregation of these variables into a variable incentive system which will be called: SYSTEM.|
Table 3. Measurement scale statistics
Comparison of PIAS Implementation Levels: U.S. vs. Germany
The three confirmed PIAS components (INCENT, CONFIG, and STRUCT) and the aggregated system variable are used in this step to examine the state of incentive systems implemented on the project level. Tables 4 and 5 contain the mean values of the four variables for the U.S. and the German data sets.
Table 4. Means for incentive system variables - German data
Table 5. Means for incentive system variables – U.S. data
A comparison shows that the mean values of the four model variables are higher in the U.S. sample than in the German sample. The system variable has a mean of 3.35 in the German sample and a mean of 4.45 in the U.S. set. A Mest shows that the mean differences between the two samples of the variables STRUCT and SYSTEM are significant at the 1% level.
Impact of PIAS on Project Success
In this step the set of hypotheses 2a to 2c that propose relationships between the components of PIAS and project success is tested with correlation analyses employing Pearson correlation coefficients. The correlations between the component variables and SYSTEM and the four criteria of project success are shown in Tables 6 and 7.
|Customer satisfaction||Business results||Effectiveness||Efficiency|
|** Correlation is significant at the 0.01 level (2-tailed). |
* Correlation is significant at the 0.05 level (2-tailed).
Table 6. Impact of PIAS components on project success – German data
|Customer satisfaction||Business results||Effectiveness||Efficiency|
|** Correlation is significant at the 0.01 level (2-tailed). |
* Correlation is significant at the 0.05 level (2-tailed).
Table 7. Impact of PIAS components on project success – U.S. data
With regard to the proposed success hypotheses support is found for:
- Hypothesis 2a is supported by both sets of data.
- Hypothesis 2b is supported by both sets of data.
- Hypothesis 2c could only be supported for the U.S. sample, while the correlations between the variable STRUCT and the four success variables in the German sample were, with one exception, not statistically significant.
By using two separate samples collected in two different countries, the revealed measurement concepts of a PIAS show that the components of incentive systems apply in Western industry cultures. The presence of a project-specific incentive system can be measured on the project level with the variables incentive types offered, its process configuration and its power structure. The components also give some direction in the configuration of project-specific incentive and appraisal systems.
The means analysis of the three-model variables represents the use of PIAS in more than 600 projects. Overall, the quite low means reflect a limited use of project-specific incentive and appraisal systems. The critique of Mohrman and Mohrman (1992) still remains that the authority over incentives and the appraisal process lies in the hands of the functional managers as demonstrated by the consistently higher means of the incentives over the incentive authority of a project manager, e.g., incentives are offered but the project managers do not have in the same way the authority to award them. Interesting are also the differences between the two samples. Particularly, the significant lower mean value of the power structure variable suggests that in Germany more project managers are assigned to projects with less personnel authority than in U.S. companies. The recruiting policies seem to differ between U.S. and German companies in assigning managers from different hierarchical levels to projects. Also, project performance-related incentives are offered more often by U.S. companies than by their German counterparts.
The major question, if a PIAS matters for the successful project execution, is supported by both samples. The results of the correlation analyses are quite convincing for the single components as well as for the entire system and confirm Porter and Lawler’s components of their valence theory. Each of the identified three PIAS components is significantly correlated with all four project success measures, meaning that the omission of one component would have consequences for beneficial effects of the whole system on project success. Following the magnitude of the correlations the most important component are the existence of incentives. It could also be seen as a precondition for the appraisal and rewarding processes and the organizational structure of these processes.
These variables on project or company success is in line with previous research conducted in the field of performance measurement and management (Mohrman, Resnick-West, & Lawler, 1989). In their article on team performance management, Mohrman and Mohrman (1992) used a similar model to verify their predictions on the impact of performance management on company effectiveness.
Also interesting are the differences between the two samples. The magnitude of the correlations differs between the U.S. and the German sample. Overall the correlations between the PIAS components and the four success variables are lower in the German sample. In three cases the power structure variable does not show any significant relationship. A possible explanation for this low impact is indicated by the descriptive analysis of the data.
The significant and quite high correlations between the PIAS variables and the project success variables are surprising because this study does not address the individual motivation of the project team members. As the motivational theories suggest, an incentive system influences directly the motivation of the individuals. Thus the effect of the incentive system on the project success is mediated by the individual motivation and performance of the project team members. This indirectly sheds light on the importance of the team member motivation on project success and requires for more incentive and appraisal power of the project managers.
Implications and Outlook
This paper focuses on the direct impact of rewards and the appraisal process on project success. The justification for this narrowed focus is provided by the hypothesis that project work is intrinsically motivating due to its inherently high degree of challenge and innovation.
This study suggests several promising questions to be addressed by future research. It should address the question how individual motivation is mediating the effect of a PIAS. Also contextual influences such as the type of project organization should be considered to better understand the conditions under which a PIAS is most effective. The measurement model developed in this study should be empirically verified and extended by others. Another interesting area is the cultural influence on the impact of PIAS. The question is how effective a PIAS is in different cultural settings and if the characteristics of a PIAS change between projects or if they change between different organizations.
The data of more than 600 projects support the three goals of this paper: (1) develop a measurement scale for project-specific incentive and appraisal systems, (2) examine the use of PIAS in practice, and (3) empirically verify the impact of incentive systems on project success. The presence of a PIAS can be evaluated by the three components incentive types offered, its process configuration and its power structure.
The relative low mean values of the four analyzed PIAS variables indicate that the use of incentive systems is far from widespread. The comparison of the two samples shows that U.S. project managers seem to have more authority in determining the types of incentives handed out than in Germany.
The three components of a PIAS identified in this paper contribute significantly to the dimensions of project success. In other words: it does make a difference for project success whether a project-specific incentive and appraisal system is implemented or not. Managers now have some very valuable empirical evidence supporting the assumption that incentive systems positively influence project success. This is a valuable insight that finally allows management to focus on the relevant areas when implementing or assessing an incentive system and ultimately increasing their chances of making their system a more effective one.
Adams, J. S. (1963). Toward an understanding of inequity. Journal of Abnormal and Social Psychology, 67, 422 – 436.
Banker, R. D., Lee, S.-Y., Potter, G., & Srinivasan, D. (1996). Contextual analysis of performance impacts of outcome-based incentive compensation. Academy of Management Journal, 39(4), 920 – 948.
George, J. (1990). Personality, affect, and behavior in groups. Journal of Applied Psychology, 75, 107 – 116.
Herzberg, F. (1966). Work and the nature of man. Cleveland, OH: World Publishing Company.
Herzberg, F. (1987). One more time: How do you motivate employees? Harvard Business Review, 65(5), 109 – 120.
Lawler, E. E. (1968). A correlational-causal analysis of the relationship between expectancy attitudes and job performance. Journal of Applied Psychology, 52, 462 – 468.
Lechler, T. (1997). Erfolgsfaktoren des Projektmanagements [Success Factors of Project Management]. Frankfurt: Peter Lang Verlag.
Lipovetsky, S., Tishler, A., Dvir, D., & Shenhar, A. (1997). The relative importance of project success dimensions.
R&D Management, 27(1) 97 – 106.
Miner, J. B. (2002). Organizational behavior: Foundations, theories and analyses. New York: Oxford University Press.
Mohrman, A. M., Jr., & Mohrman S. A. (1992). The performance management of teams. In W. J. Bruns (Ed.), Performance measurement evaluation and incentives (pp. 217 – 241), Boston: Harvard Business School Press.
Mohrman, A. M., Jr., Resnick West, S. M., & Lawler, E. E. (1989). Designing performance appraisal systems: Aligning appraisals and organizational realities. San Francisco: Jossey Bass.
Pinto, J. K. (1986). Project implementation: A determination of its critical success factors, moderators and their relative importance across the project life cycle. Unpublished dissertation, University of Pittsburgh, Pittsburgh, PA.
Pinto, J. K., & Mantel, S. J., Jr. (1990). The causes of project failure. IEEE Transactions on Engineering Management, 37(4), 269 – 276.
Pinto, J. K., & Slevin, D. (1988). Project success: Definitions and measurement techniques. Project Management Journal, 19(1), 67 – 72.
Rothwell, R., Freeman, C., Horsley, A., Jervis, V., Robertson, A., & Townsend, J. (1974). SAPPHO updated—Project SAPPHO Phase II. Research Policy, 3, 258 – 291.
Shenhar, A. J., Dvir, D., & Levy, O. (1997). Mapping the dimensions of project success. Project Management Journal, 28(2), 5 – 13.
Shenhar, A. J., Tishler, A., Dvir, D., Lipovetsky, S., & Lechler, T. (2002). Refining the search for project success factors: A multivariate typological approach. R&D Management, 32(2), 111 – 126.
Tosi, H. L., Katz, J. P., & Gomez-Mejia, L. R. (1997). Disaggregating the agency contract: The effects of monitoring, incentive alignment, and term in office on agent decision making. Academy of Management Journal, 40(3), 584 -602.
Vroom, V. H. (1964). Work and motivation. New York: Wiley.
Deci, E., Koestner, R., & Ryan, R. (1999). A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychological Bulletin, 125(6), 627 – 668.
Drucker, P. F. (1954). The practice of management. New York: Harper.
Dvir, D., & Lechler, T. (2003). Plans are nothing, changing plans is everything: The impact of changes on project success. Research Policy, 33(1), 1 – 15.
Lechler, T., & Dvir, D. (2005). The impact of project organization structures on project success. Working paper.
Lewin, K., Dembo, T., Festinger, L., & Sears, P. (1944). Level of aspiration. In J. Mc-Vicker Hunt (Ed.), Personality and the behavior disorders (vol. 1, pp. 333 – 378). New York: Ronald Press.
Locke, E. A., & Latham, G. P. (1990). A theory of goal setting and task performance. Englewood Cliffs, NJ: Prentice Hall.
McGregor, D. (1957). An uneasy look at performance appraisal. Harvard Business Review, 35(3), 89 – 94.
Raghu, T. S., Sen, P. K., & Rao, H. R. (2003). Relative performance of incentive mechanisms: Computational modeling and simulation of delegated investment decisions. Management Science, 49(2), 160 – 178.
©2006 Project Management Institute