A program planning model for a statewide research and development system
University of Wisconsin
Andre L. Delbecq, Ph.D.
University of Wisconsin
Marc Hansen, M.D.
University of Wisconsin
Ed. Note: Project Management is increasingly finding applications in new fields; in this case the development of health care delivery systems. Of particular interest is the systematic process for finding problems and creating project definition that was developed and implemented for this program.
R&D is a concept usually associated with improving the technical quality of products or services. For example, it is usually associated with aircraft “specifications” in the airline industry. Yet, the behavioral features of an individual aircraft is a small portion of an airline system, and perhaps its least vexing. Attracting customers, its logistics of flight arrangements so as to minimize underutilized equipment, the systems which facilitate an effortless flow of passengers and baggage to and from the aircraft; these concerns are as important as the technical specifications of its aircraft used. Thus, a successful transportation industry is as much dependent on the technical excellence of its total system, as it is on the core technology of the airplane. Private industry has spent millions of dollars researching and developing systems to deliver products and services efficiently and at low cost, thus increasing the availability to an ever enlarging public.
Health care is an also large “industry.” Wisconsin citizens paid $550 million in health insurance premiums alone in 1970.1 Yet it is an industry which allocates only a fraction of its resources to development of improved methods or for delivery of health services. As a result there is little direction to guide the allocation of our health delivery resources.
The value of research in the bio-medical sciences has been amply demonstrated in the last 40 years. The prevention or cure of many major illnesses can be traced directly to the success of such research. By contrast only during the last few years has the importance of research into the system of delivering health care received attention. It can properly be asked whether such research into delivery systems requires or deserves attention at the state level, and if it does, what steps are to be taken by the state to assure that such research benefits its citizens.
The problems in the organization and delivery of health care in a state are indeed complex and recapitulate the issues now under debate at the national level. A wide variety of ways of providing and paying for care now exist and new methods are vigorously proposed. This diversity has even increased by recent developments such as prepaid health plans and the use of new types of health workers. Federal actions mandated planning activity at the state level.
Adding to the complexity, factors such as state licensure of the health profession and institutional regulation cause each state to face special or even unique problems in the organization of care. As a result organized research at the state level is necessary if we are to evaluate and improve relative effectiveness of the alternate forms of health delivery.
The purpose of the study reported here was to design the specifications for a health delivery research and development system for a state. How can innovations that would improve the delivery of health care be more rapidly implemented so that delays do not limit their acceptability or potential impact? How might new delivery systems be better tested before they are widely implemented? By what means might health delivery problems with the highest priority be selected for a state or region?
The implications of this paper extend beyond the health field for it demonstrates the utility of a systematic process for finding problems and creating definitions of projects to meet needs. Many organizations wait for requests for proposals to come to them, then squeeze out a proposal that force fits their organization’s capabilities to the requirements of a previously defined project. The process proposed here offers a means by which organizations might better develop their best capabilities and in the process become more initiatory and less reactive.
The development of an effective research and development function for health delivery is a complex problem. It was one of several proposals generated by the Governor’s Health Planning & Policy Task Force making use of the “Program Planning Model” of Delbecq & Van de Ven.2 The PPM is a normative regimen.
There are several distinct planning phases in PPM. In the first phase (PROBLEM EXPLORATION) the principal barriers limiting the effectiveness of the system under study are identified by users of the system. In the second (SOLUTION EXPLORATION), experts recommend essential components which they feel will incorporate the technical quality of a problem(s). In Phase three (SOLUTION ADAPTATION) the general outline of a solution is reviewed and modified by resource controllers and administrators in order to increase acceptability of solution recommendations. In phase four (SOLUTION IMPLEMENTATION) the proposed solution is implemented and evaluated in a small, creditable demonstration before full scale system change is approved.
In broad conceptual outline the strategy is very similar to several other problem solving approaches. However, sophisticated group processes are also suggested to formulate specific phases.3, 4 Second, PPM recognizes that no group of people is qualified to monopolize the activities of identifying problems and identifying solution ingredients, and implementing and evaluating a demonstrative solution.5 Hence the PPM format draws upon different resource groups in different phases and avoids dominance of its process by an elite staff group insulated from users, providers, resource controllers and technical opinion leaders.
We can now review how PPM concepts were used in the development of the Health Delivery R&D system in Wisconsin, keeping in mind the processes’ and the solution’s applicability in non-health areas.
Phase I: Problem Identification
The first phase of study was to identify the strengths and weaknesses of existing R&D concerned with health delivery. Since the results should have applicability beyond Wisconsin, a national sample of qualified individuals were involved. Forty-nine persons were nominated and asked to participate:
- Administrators and providers with veto power relative to implementing any changes recommended.
- Resource controllers within both the federal government and philanthropic foundations who are principal sources of funding for demonstration projects.
- Managers of research and development programs who are aware of the operational pitfalls of R & D concerned with adding systems.
As these people were located throughout the country they could not easily meet as a group. So a “Delphi” process (a method developed by RAND Corporation6 for technological forecasting but which had since been used for other purposes including problem identification) was used. The Delphi involved a four-stage questionnaire process extending over a three-month period. The first questionnaire asked the participants to list the strengths and weaknesses of existing R&D systems by responding to the following questions:
1a. List the factors that are operating when UNIMPORTANT problems are selected for research and development.
1b. List the factors that are operating when IMPORTANT problems are selected for research and development.
2a. List the factors that are operating when POOR (inadequate or inappropriate) solutions to important problems are developed.
2b. List the factors that are operating when GOOD solutions to important problems are developed.
3a. List the factors that are operating when INADEQUATE evaluations of research and development programs are conducted.
3b. List the factors that are operating when GOOD evaluations of research and development are conducted.
The second questionnaire summarized the results of the first and requested a preliminary vote on the relative importance of items identified as well as reactions and arguments for and against these items.
The third questionnaire summarized the prior results to date and requested a final vote and comments.
Note that in each phase participants were asked to consider three areas: Problem Selection, Solution Development and Program Evaluation. A fourth area, Transfer of Technology, had been studied the year before through a grant to the University of Wisconsin from the National Academy of Sciences7 and was not repeated.
The results of this effort were analyzed by a study group of the Wisconsin Governor’s Health Planning and Policy Task Force. This analysis resulted in 1) a weighted list of objectives growing out of the Delphi analysis, 2) a brief statement of why the objective is desirable and 3) a list of preliminary recommendations as to how each objective might be achieved. For instance, the most important objective identified was to increase in researchers an understanding and sensitivity to the problems of real world implementation of the results of their studies. This objective is desirable because the results of research should have import on the world as soon as possible. Preliminary recommendations included finding a way to involve providers, consumers, and financial interests in the identification of problems, development of solutions and evaluation of results.
A similar approach to problem identification and analysis could be taken in non-health issues. The key elements here are to involve the right people (which in this phase are the users of the products of the system being considered) and to use group process techniques that will maintain a problem focus. The result should be a prioritized list of strengths and weaknesses influencing the effectiveness of the system under study.
Phase II: Solution Exploration
The second phase of the study was to identify knowledge resources and to involve multidisciplinary and multi-agency expertise in the design of innovative solutions to these problems. Experts in areas of technological innovation, management of research and development, and program evaluation were involved. They included organization theorists, economists, administrators, and researchers. All had first-hand knowledge, either theoretical or practical, of the issues under study. Through interviews and surveys they identified the critical features which must be incorporated into a successful program to solve the problems.
Their task was structured in this way so as to focus their attention on our problem and its key elements. This is important because theoreticians would rather talk about their theory and organizational representatives about how they are doing things back home. This assignment helped bring out the key elements of importance to us.
The results of these consultations included definition of the characteristic selected as essential, a literature reference supporting this point, and the implications this characteristic had for our deliberations and the consequent action we took in response to those implications.
For instance several characteristics were identified in response to the problem of researcher sensitivity to problems of real world implementation. One was the need for a formal link to a mission oriented section of state government. The importance in studies conducted by the National Academy of Sciences8 and by independent investigators such as Filley and Delbecq.9 The reason for this characteristic was to insure an interest in the work by an organization capable of influencing implementation. The recommendation was incorporated into the final system by having funds from the mission oriented organization support methodology development in the areas of problem identification, problem evaluation and technology transfer.
Note that in this approach one can benefit from the opinions and advice of leading experts without getting locked into their pet solutions to a problem. In a sense the experts establish guidelines for solutions that staff, knowledgable about the unique solution they form, can then use to create a solution that meets with gendition and their unique needs.
Phase III: Solution Adaptation
The solution ingredients were transformed into a preliminary version of a program to meet these guidelines and a discussion paper was prepared. It was circulated among all Delphi participants to insure that it took into account both initial problems which had been identified and theoretical precepts for successful solution upon which it was based.
The paper was then distributed to all members of the task force as a discussion paper. Forced field analyses elicited potential strengths and weaknesses. The paper was modified again.
Finally the paper was distributed for comments to 108 Health Service researchers throughout the state for their reactions.
At the different stages depth was added to the initially sketchy working paper. By the end of phase III a substansive paper, broadly endorsed was submitted to and approved by the task force.
An important point was to prepare the discussion paper early enough so that the staff would not be locked into a solution before advice on its improvement was sought. Such early imput insures that the project really benefits from that advice, and the process is not one of completion.
The result of this effort was a proposal for the creation of a five-part system for research and development concerned with health delivery.
1. An advisory board to monitor and coordinate these activities.
2. A development and demonstration activity concerned with facilitating more rapid and rational transfer of already proven innovations.
3. A technical assistance and outreach service concerned with facilitating more rapid and rational transfer of already proven innovations.
4. A methodology development service concerned with developing better methods to evaluate programs, identify problems and implement results.
5. Basic research and development.
The specific operating system finally developed is described in the following section.
A. Monitoring and Coordination (M & C)
A committee of the Health Policy Council has been established to review the programs, policies and priorities of the development and demonstration, technical assistance and outreach, and methodology development services.* This committee will find ways to avoid duplication of efforts, act as a forum for resolving issues, channel funds to the three activities described below. The committee will not interfere with any research and development efforts organized and supported independently in the state.
The Committee has the following responsibility:
1. To identify which demonstration and technical assistance projects will be studied for feasibility.
2. To select from these potential projects the ones that will be submitted by the Council for funding.
3. Review and approve proposal for funding before submission.
4. To receive the funds for the conduct of the project and to subcontract these funds at prespecified levels to collaborating organizations.
5. To review project progress and approve continued operation of the project at prespecified intervals.
6. To review project of the D & D and TAO Program effort.
The membership on this committee are composed of leading providers, planners, policy makers, researchers, administrators of research and heads of delivery organizations. They understand the broad role of innovation in the health system and are in a position to influence its effectiveness.
B. Development and Demonstration (D & D)
The principal role of the D & D function is to develop and test the alternative approaches to Wisconsin’s most pressing health delivery problems, and coordinate appropriate demonstration activities.
Priority areas will be mandated by the R&D committee of the health council. The D & D steps would involve a feasibility study* to determine whether practical delivery systems can be developed based on existing knowledge. If the feasibility study indicates that further work is desirable, a proposal involving budget and potential sources of funding would be developed. Once funded a project group would develop and test the experimental delivery system. In some cases, several alternative systems could be tested.
When a new health delivery program requiring approval of the major health policy units in the state is proposed, a second D & D activity would be to promote and review the evaluation of such purposes.
The development and demonstration component provides a means for improving the health delivery system in several ways.
• The Health Policy Council gives a strong voice to providers, planners and consumers in establishing priorities concerning health delivery which by subcoordinating with existing health organizations for design and development facilitates the utilization of existing D & D skills.
• It provides for demonstrations of new delivery systems in typical settings so as to exercise the probability of effective technological transfer.
The D & D unit also reduces inappropriate implementation by serving as a screening mechanism for all new delivery programs and the evolutions requiring state approval.
The development and demonstration activity is designed to draw upon the full resources of the state. It will not be tied to or duplicate any specific research, education or delivery organization. It will not suplicate, but strengthen existing efforts. The D & D will not conduct research or demonstrations itself. Rather it will be coordinating function to define what needs to be done, find the people and institutions most qualified to do the work and provide opportunities or incentives to undertake the job. One vehicle for accomplishing this will be subcontracting. Potential subcontractors include: delivery organizations, planning organizations, provider organizations, professional organizations, research organizations, state agencies, third party payers and universities. Another vehicle will be leaves of absence for experts from one or more of the resource organizations to join a D & D project group.
C. Technical Assistance and Outreach (TAO)
The Technical Assistance and Outreach Service is to reduce the delay in implementation of innovations that could benefit citizens of the state and reduce the cost of such technology transfer.
Its specific responsibility is to (1) encourage and assist planners and users in the rational implementation of already characterized desirable innovations and (2) advise regulatory bodies concerning innovations tht should be encouraged through appropriate incentives.
The health council will identify and set priorities on innovative programs that should be implemented on a broad scope. The Technical Assistance Service will provide information and advice to components to the health delivery systems as they seek to incorporate these new delivery systems.
Technical Assistance and Outreach will have to undertake a variety of activities to fulfill the missions of technological transfer including:
- education of providers
- development of incentives in collaboration with resource controllers and regulators,
- technical assistance to providers during their initial efforts in utilizing a new system.
The TAO will have to develop ready access to both knowledge experts in areas related to innovations and plans of policy makers and will facilitate technology transfer.
D. Methodology Development (MDS)
Development and demonstration relies heavily upon its ability to evaluate program cost and effectiveness. Technical assistance and outreach relies heavily upon its ability to speed transfer of new innovations to practical use. The health council relies upon its ability to identify important problems for investigation. While tools for performing these tasks exist, the surveys conducted by the Task Force also indicated that in each of these three areas improved methods are badly needed. MDS will encourage the development and improvement of these methods.
The surveys also indicated that communication is one of the primary areas where the R&D system can be improved. MDS will translate the results of research into a language and format understandable to those who wish to use these products to develop practical solutions to problems.
MDS will not develop the tools for evaluation, problem identification and transfer of technology. Rather, it will encourage qualified investigators throughout the state to do so. Unlike the development and demonstration unit, MDS will not be assigned specific issues to attack. Investigators will select issues having greatest potential. The function of MDS then is to create incentives for qualified investigators to concentrate on important methodological problems.
MDS will not be tied directly to any single research institution. It will build upon the state’s strengths so it can assist those having the greatest potential for improving Wisconsin’s R&D system. In this supportive role, it will not limit the creativity or independence of investigators. Rather, it will assist them to improve the effectiveness of their work within the broad focus described above.
E. Basic Research and Development
The strength of our present health delivery research system lies in two areas:
(1) Its freedom to select and solve the problems it wants and to report the findings regardless of their implications for current policy.
(2) Its ability to develop new knowledge that can be translated into action to solve important problems.
Government control, while sharpening the focus on problems, also shortens and narrows perspective. A commitment to a program of action commits one to the assumptions on which it is based. There is a reluctance to conduct investigations that may prove these assumptions wrong. Pressure from the government for timeliness and relevance causes resistance to the exploration of questions related to knowledge development that might take a long time to answer or lead to a considerable review and revision of the original assumptions.23 (NAS study of IRP)
Changes in the health delivery R&D system must retain the freedom of inquiry that is its principal strength today. None of the recommendations should be viewed as attempts to limit the freedom of inquiry that has led to the success in health delivery research to date. Rather, the recommendations are designed to supplement and strengthen this.
Implementation of Recommended System
The Governor’s Health Policy Council accepted these recommendations and gave them high priority. The R&D Committee is in operation, and the Council has authorized the establishment of an independent non-profit corporation, whose membership is the Health Policy Council. The corporation will do none of the R&D itself. Rather it will influence policy formulation by providing independent analyses of critical issues. It will reality test proposed policy recommendations by conducting feasibility studies to determine if the timing is right to field test a recommendation. It will see that innovative solutions to health problems are properly field tested.
In the future the corporation will take on the two other functions: technical assistance and methodology development.
There are several unique characteristics to this study. First it rigorously followed the tenets of a systems approach to social program planning. Second it used a delphic process to involve leaders throughout the country in this study. Third it proposes a system that improves the continuity of research and development through to implementation. It provides a practical means for tying R & S to state level policy making in a manner that does not hinder creativity but improves responsiveness to state needs. Finally it provides a means for drawing upon the full resources of the state including educational, governmental, consumer and provider. For these reasons we feel the resulting system will lead to a significant improvement in R & D delivery.
1. The Wisconsin Governor’s Health Planning and Policy Task Force. Research and Information Supplement. Vol. 1, January 1973. Department of Administration, 1 West Wilson, Madison, Wisconsin.
2. Delbecq, A., Van de Ven, A., and Gustafson, D., Group Techniques for Program Planning—A Guide to Nominal Group and Delphi. Chicago: Scot Foresman, Inc., 1974.
3. Van de Ven, A. and Delbecq, A., “Nominal Versus Interacting Group Processes for Committee Decision Making Effectiveness,” Academy of Management Journal Vol. 14:2, June 1971.
4. Turoff, M., “Delphi and Its Potential Impact on Information Systems,” AFIPS Conference Proceedings, Vol. 39. Montvale, New Jersey: AFIPS Press, pp. 317-326, 1971.
5. Utterback, J., “The Process of Technological Innovation Within the Firm,” Academy of Management Journal. Vol. 14:1, March 1971.
6. Dalkey, N„ DELPHI. Santa Monica, California. The RAND Corporation, (3704), October 1967.
7. Committee on the Interplay of Engineering with Biology and Medicine, Study of Engineering in Medicine and Health Care: A Final Report to NIH, June 30, 1973, Washington, D.C.: National Academy of Engineering.
8. National Academy of Science, “Policy and Program Research in a University Setting—a Case Study,” Washington, D.C.: US Printing, 197-.
9. Filley, A. and Delbecq, A., Program and Project Management in a Matrix Organization: A Case Study. University of Wisconsin-Madison: Bureau of Business Research and Services, Monograph 9, Ch. 3, 1974.
*The role of the proposed R&D system is to concentrate on Wisconsin’s most critical problems. However, neighboring states frequently face similar problems. They also possess complementary resources and opportunities for testing innovations. The M & C board should begin exploration of the possibility for developing a coordinated program between states.
*Clearly the activity must not duplicate studies related to similar issues in other states or at the national level. Since many issues will have national importance, national results should be modified as needed to be appropriate for Wisconsin. Attempts should be made to collaborate with ongoing national and regional studies.