Quantifying qualitative models
Ed Mechler, PMP, CCP, Principle Consultant, Project Controls Incorporated
There is a great desire to apply quantitative methods, mathematical rigor, to our qualitative project management models to describe or predict, as physical science does. The first part of this paper will review the characteristics of physical and social conditions and how they relate to mathematical models. It will then develop a technique to help in modeling social conditions. The last part of the paper will apply the concepts and techniques to the PMI Organizational Project Management Maturity Model (OPM3) Program. The goal of the paper is to help the project manager in applying quantitative methods to qualitative conditions.
Physical vs. Social
First, what are we trying to do? We want to take a social condition, i.e. structure, elements, processes, practices, and apply a logical or mathematical model to enhance our understanding, description or prediction: just as the physical sciences apply logic or mathematics to the physical conditions. As an example, we want to predict that applying the appropriate set of PMBOK® techniques to a particular project will increase our assurance of completion. We want to emulate applying the correct algebraic equation to the voltage and current we consume to determine the power and the cost, once we know the unit cost.
If we agree with the above statement of intent, then we need to investigate the concept of physical and social conditions, or as we generally say, physical and social models. First, we must always keep in mind the difference between physical and social conditions and physical and social models. The physical and social conditions are the actual structure, elements, and processes that we observe. As an example for the physical conditions, if we hold a book at arm length and release it, it will fall because of gravity. The activity of loading a new operating system on a personal computer would be an example of a social condition. According to Cleland a model is an abstraction of reality, an abstraction of the physical or social conditions. If we wanted to model the physical condition of the book falling, we could use, v = -gt, for velocity if the initial velocity is zero, and F = ma for the force hitting the floor. To model the loading of the operating system we could use the network diagram to depict the activity relationship to other activities, its duration and its criticality. The reason for the distinction between conditions and models is two fold. When we apply a model for understanding it is a partial understanding, but we use words, like model and the actual conditions, interchangeably and this leads to inferences that may not be true. The second reason is that the distinction sets the stage for describing the differences between physical and social conditions and models.
Applying logical or mathematical models to physical or social conditions for better prediction or description first requires a general understanding of the physical or social conditions. Using Simon's definitions of natural (physical), and artificial (social) the differences become apparent. According to Simon, natural phenomena .... are subservience to natural law where as artificial phenomena (1) are synthesized, (2) may imitate appearances in nature, (3) are characterized in terms of functions, goals, adaptation, and (4) are discussed in terms of imperatives as well as descriptives. The key is, the physical conditions are directed by natural law and to describe or predict we need to understand the natural laws, just as the book falling at arm length. The social conditions are built by humans with goals in mind, that change, implying more then just a set of laws to understand. At this time, or maybe never, depending on our philosophical view, there isn't any defined or clearly discernable set of social laws. The difference between physical and social conditions requires a different approach to applying models.
To apply models to social conditions for understanding, we need to examine two aspects of social conditions, interaction with physical conditions and systems. Simon's definition of social conditions that “may imitate appearances in nature” gives us a means of understanding the social conditions more fully. The physical conditions, until the twentieth century with the advent of relativity and quantum theory, were absolute, leading to understanding through the physical laws alone. The social conditions may include physical conditions as well as social. The dropping of the book can be viewed as only physical but the loading of the operation system can be viewed as both a physical and social condition; actual loading is the physical while the timing of the loading, social. Even PMBOK® makes a distinction between the project and product; the project is social and the product could be physical or social. This interaction leads to our thinking the social conditions can be viewed as the physical conditions and our use of physical terms for social conditions. The interaction also implies a complexity that surpasses the physical; the physical could be only part of the conditions requiring modeling. It might require a new way of looking at the conditions.
For the physical conditions we have science. Science works from observation to hypothesizing, to testing, to theory. The main requirement of the process is repeatability. Science is analytical because the physical laws that govern are fixed and we need to understand them. This is why it is repeatable or has to be for scientific purposes. It is the Atomistic View, break down to smallest component and/or the most important component for testing. Within scientific literature there is considerable discussion about systems, an alternative view, but this deals with a set of physical laws interacting, as with a solar system.
Figure 1. System Graphic View
For a social condition, say coordinating activities on the critical path, we may have observed previous projects and hypotheses. But we cannot test or have repeatability. Once the condition, a set of activities or a project is complete, we do not repeat the process. We do not have the same conditions as the physical, as Simon's artificial phenomena characteristics point out. We need to change our scientific view for at least part of the social conditions. General Systems Theory (GST) appears to have a number of answers for the social conditions, only social or social and physical. According to Laszlo, each system has a specific structure made up of certain maintained relationships among its parts, and manifests irreducible characteristics of its own. The parts of a system along with their relationships form a new entity. A system is dependent upon the components and the summation of the components. A system takes inputs, transforms them, and gives an output. It has goals and must interface to the environment; see Figure 1. Examples of systems for social conditions are companies, departments, government agencies, human beings, processes, etc. Isn't the system a better way of describing social conditions? Isn't a project a systematic approach to implement a new system within an old system? We need to look at the interaction within systems and between systems for social conditions to develop models.
In conclusion, to model social conditions for better understanding we need to separate them from the physical conditions and use a systems view to model. Applying physical condition characteristics alone may lead us to implications that may be misleading for physical laws do not govern social operations. For social modeling, as Cleland further describes, we need to make assertions, which express the relationship of various elements of the system ... which can be used in lieu of the social conditions themselves.
Now that we have defined the structure of the physical and social conditions and the circumstances for applying models, a couple of question may be on our minds. First, should we even try to apply mathematical models to social conditions? The answer is yes. By just trying to apply mathematical models, we increase our understanding of the social condition. We will have increased the level of modeling just by trying. The second question is, do we have to start from scratch developing models with all the models that exit today? There exists algebraic equations, cost-benefits analysis, decision trees, earned value analysis, etc. Have we analyzed them out of use? The answer is no, if we use them with reserve.
We have applied various mathematical models to numerous applications, sometimes appropriately, other times misleadingly. All we need to do is watch the news. Why are some reports, example eating eggs, really bad and then a few years later not as bad? Are we applying the models appropriately or has more data become apparent? It should be our hypothesis that we are applying the mathematical models inappropriately more so then new data is affecting the outcome. If the hypothesis is true, we need a way of classifying all the existing models so that we can better map the existing models to our present social application.
A process that has worked before is, to classify the models as a brief statement concerning the data the model works on and the results obtained. As an example, regression works on two sets of data, or n sets, to determine a relationship and the degree of the relationship. Another example is critical path analysis that works on a set of activities with relationships to define the longest path. With descriptions like these we could determine if the model fits the description of the application. What we need is a book with these statements, maybe more definitive ones, to help us in quantifying our qualitative structures. But until we find one, we can use this approach to further our applying mathematics to social conditions.
Table 1. OPM3 Engineering Model Template
Let Π : Organizations success criteria.
Let Ccv: Contingency Variables.
Let C1...n: Clusters of 59 elements from the Delphi procedure, e1...n, supporting Π.
s1.n steps leading to the maturity of the element, sn
defines the maturity of the cluster.
and s1...n will be modified and enhanced by the Design Cells.
|CCV must be applied to and/ or s1...n for each organizational condition.|
So far we have examined applying models to conditions, we need to also look at collecting data, or metrics. Again we need to apply restraint because our mind set is focused, applying as physical science does. To assign numbers to a system under observation, the observed system must be isomorphic (same structures in relationships and operations) to the numerical system. There are four levels of measurement to consider. There is the Classificatory Scale, example car license plate with county designation; the Ranking Scale, Classificatory Scale with ranking, example military grades; the Interval Scale, Ranking Scale with the same difference between classes, example temperature measurements; and the Ratio Scale, Interval Scale with a true zero and independent of unit of measurement, example is weight or mass. The Ratio Scale is isomorphic to arithmetic and the Interval Scale the differences are isomorphic to arithmetic. Being isomorphic to arithmetic gives us all the operations of arithmetic. The physical conditions can use a Ratio Scale for measurement but the social conditions usually measure in the Classificatory or Ranking Scales with occasionally in the Interval Scale and rarely in the Ratio Scale? We need to decide the scale the data fits before applying the appropriate mathematical operations. If not, we need to be careful what operations we apply and the results interpretation.
Organizational Project Management Maturity (OPM3)
As an example of the concepts professed above, let's examine the PMI Program OPM3. We will not examine the details of the program because other papers do; see references. We will examine OPM3 from the Engineering Team (ENG) point of view, to develop a “grounded theory” that explains the “cause and effect” linkages between the different capability clusters that constitute a mature organization. In other words; to quantify the qualitative conditions or to develop the engineering model from the social model.
OPM3 purpose is to enhance an organization's ability to implement organization strategy through the successful, consistent, and predictable delivery of projects. It is not the PMBOK®, focusing upon implementing a project. OPM3 is an R&D program/project by nature; leading edge of relatively new organizational processes being integrated. The stakeholders will be skilled professionals from numerous disciplines; they will be volunteers on a virtual team. The initial process was for the Model Review Team to collect key elements from existing models and the Survey/Test Team to collect key elements from organizations. The Synthesis Team was to assimilate the resulting elements and populate the components of the social model. The Engineering Team would quantify the model while the Assessment Team would develop the process to measurement progression and the Global Outreach Team would develop organizations for participation. Due to insufficient elements the process was augmented with a Delphi process, the results of which were utilized by a set of Design Cells, within the Synthesis Team, to design the social model. The process also changed from a linear or waterfall to a concurrent prototype approach. With this brief description, see references for details, the OPM3 program is a pure social condition as described above.
The ENG Team, of course, had more information then the brief description above but what did we have to apply an engineering model? We did not have a set of data to statistically analyze to determine the elements and processes of OPM3. We had a very abstract theory. It was proposed: Organizational Success is related to Strategic Management Processes and Multi-Project Management Process constrained by Business Conditions.
Table 2. Models Reviewed
|Regression Models||Management Science and Industrial Engineering|
|Organizational Development Models||Engineering Models|
|Genetic Algorithms and Programming||Social Networks|
|Adaptive Non-Equilibrium Systems||Non-Linear Dynamic Systems|
|Organizational Consultant||Richard M. Burton and Borge Obel, Strategic Organizational Diagnosis and Design, 1998. Kluwer Academic Publishers.|
We had other models and organizational experiences that supported parts of the theory. We had from other models a hazy structure.
With the above information, we first developed an abstract list of proposed components and relationships as shown in Table 1. We then divided the standard models and leading edge models into subgroups to determine fit to the OPM3 social model, Table 2. None of the standard and leading edge models fit without reservations. Finally, we briefly reviewed commercial modeling systems. The following is a brief description of each but if you are interested you need to investigate more:
Ithink—System mapping language to rigorously link the map of a process or strategy to the dynamic patterns of behavior.
Vite—Simulation information flow to organize for fast-paced, high-quality product development projects.
Metis—Visual modeling tool to use complex enterprise knowledge to answer critical questions and business problems.
Organizational Consultant—Rule-based knowledge representing theories of organizational design.
The problem with the commercial models was the learning curve required, we did not have the time. Then a member of the ENG Team who had experience with Metis contacted a friend and we had five experienced members who would develop the visual model for us.
At the submission of this paper the instructions and forms for the inputs to Metis have been developed and the Design Cells are designing the structure and relationships. A preliminary output of Design Cell 2 is shown in Figure 2. It shows the objects of the model, elements, constructs, roles, etc., and the relationships, lines. The results of the Design Cells will be reviewed by the OPM3 teams and integrated together. The Metis model of OPM3 will help team members visualize and develop the components and relationships. We will then try to quantify the model by adding weights, computations or other mathematical models.
Although we would like the rigor of mathematics as the physical conditions apply, we need to be cautious when working with social conditions. We need to understand the difference between physical and social conditions and models; apply a new view, GST, to model social conditions; keep trying to apply mathematical models but with reservation in interpreting results; and conscious of the scale and appropriate operations. With the above information the project manager will have gained insight in applying quantitative methods to qualitative conditions.
NOTE: Due to paper submission in April and presentation in November the presentation will contain additional information, OPM3 is an in progress program. The additional information can be found at http://www.pmi.org/opm3/engpaper
Cleland, David I., and King, William R. 1983. Systems Analysis and Project Management. New York: McGraw-Hill Book Company.
Cook-Davis, Terry, Bredillet, Christophe, and Schlichter, John. 2001. Beyond the PMBOK Guide: Creating a standard for organizational project management maturity. PMI Seminars & Symposium 2001, Research Topics Track.
Kalantjakos, Nikitas. 2001. Assessing Organizational Project Management Maturity. PMI Seminars & Symposium 2001, PM Advanced Track.
Laszlo, Ervin. 1972. The Systems View of the World. New York: George Braziller, Inc.
PMI Standards Committee. 1996. A Guide to the Project Management Body of Knowledge. Newtown Square, PA: Project Management Institute.
Figure 2. Metis Panel 1
PMI Standards Committee. 2000. A Guide to the Project Management Body of Knowledge. Newtown Square, PA: Project Management Institute.
Schlichter, John. 2001. PMI‘s Organizational Project Management Maturity Model: Emerging Standards: PMI Seminars & Symposium 2001, PM Advanced Track.
Siegel, Sidney. 1956. Nonparametric Statistics. New York: McGraw-Hill Book Company.
Simon, Herbert A. 1984. The Sciences of the Artificial. Cambridge, MA. MIT Press.
Proceedings of the Project Management Institute Annual Seminars & Symposium
November 1–10, 2001 • Nashville, Tenn., USA