Project Management Institute

Learning from the East African experience

PROFIDE--a model for improved program management in the security and development environment

Abstract

The paper draws on detailed thematic analyses of the lessons learned data from 53 projects and programs in Ethiopia (24), Kenya (5), Uganda (4), Somalia (2), Sudan (2), Rwanda (2), and 13 other African countries and regions. The lessons learned analysis has identified six key themes, which the paper argues are fundamental to improved program management performance. The six themes—Policy, Governance, Stakeholders, Definition, Capacity, and Process—are brought together and presented in a new empirical model called PROFIDE: Program Factors in an International Development Environment. The PROFIDE model brings into greater focus the important interplay between policy, governance, and stakeholder engagement as being keys to creating the right environment within which the program can be defined, developed, and delivered. Only when this environment exists, can program definition be effectively addressed, capacity issues understood, and process-related requirements discussed.

Keywords: security; development; project; program; improvement

Introduction

The field of project and program management (PPM) has played an important role in supporting the delivery of international assistance in developing and post-conflict countries (Ika et al., 2009); however, such donor-led interventions often take place under complex and uncertain conditions and thus present special challenges to any PPM approach. Previous research has described the nature of some of these challenges (Ika et al., 2011; Ika et al., 2010). Recent developments, such as the launch of PMDPro in 2009–2010 clearly go some way in helping at the level of the ‘project.’

This paper maintains that there is considerable scope for applying more effective management at the level of the ‘program.’ Amidst increased attention by researchers, policymakers, and practitioners on the field of program and project management, is a rising wave of large, multi-faceted international development programs that support the achievement of the United Nations Millennium Development Goals (MDGs) aimed at ending poverty by 2015. Depending on the nature of challenges impacting on each country's prospects for growth, this array of international development programs and projects can extend from the construction of dams, roads, and rail networks, to institutional reform across all basic services and government departments.

Across the high number of developing countries targeted in these exercises, and as evidenced in its recent 2010 MDG progress report, Ethiopia stands as one that is well on the road to achieving its MDG targets (United Nations, 2010)1. In parallel to this progress, a number of strategic shifts in the wider Horn of Africa region, including the declaration of independence in South Sudan, progress in combating Somali-based terrorist groups, and regime change in North Africa have all led to the acceleration of external investment in Ethiopia—a country viewed as having great geo-strategic reach to exploit regional and global trade, a friend of both the Arabic north and the developing Christian south, a relatively peaceful society, and the rapid development of human capacity and skillsets across the country's population.

Ethiopia is undertaking the most ambitious reform process that any country in the developing world has ever attempted (Asefa, 2003; Erasmus, Lodwyck, & Powell, 2005). The government of Ethiopia has embarked on a long-term strategy of “state transformation,” characterized by bold attempts to implement multiple reforms in parallel; the massive scale-up of institutional development efforts across tiers of government; and the deliberate expansion of the scope of public sector capacity-building initiatives. With the support of International Development Agency (IDA) credits and grants, the Government of Ethiopia has made important strides in a number of core sectors. Important results include a near twofold increase in the number of children in primary school between 2001 and 2010, a reduction in child mortality rates from 204 in 1990 to 109 in 2010; increased rural access to safe water to 65.8% in 2010, from only 19% in 1990, all the while building local government capacity for service delivery and increased accountability (World Bank Group, 2011). Much, if not all, of this development activity is in the form of projects or programs. For example, the World Bank has provided lending for over 130 projects in Ethiopia, involving infrastructure, protection of basic services, food security, and education; as of February 2011, the portfolio had 25 active projects, with a net commitment value of over US$3.6 billion.

A fundamental premise of this paper is that, in order for the sustainable benefits to flow, such commitments must be made within an environment that has the right balance of determining factors. The research aim is to understand the nature of those factors and provide a view on how this ‘rebalancing’ might be done. This approach has led to the development of a new model: PROFIDE (Program Factors in an International Development Environment).

This paper is organised into four sections. The first section overviews developments in the policy and practice of security and development work over the past decade and reinforces the importance of PPM for addressing these outstanding challenges. The second section introduces the field of PPM, tracking its general development and defining its key terms of reference. The third section outlines the methodology used in the study of 43 World Bank–led projects and programs in a number of sub-Saharan African countries, with a particular emphasis on Ethiopia and the Horn region. The fourth section will uncover the lessons learned from the wider security and development PPM (SDPPM) implementation experience, with the final section describing the PROFIDE model and offering recommendations aimed at more systematic and effective SDPPM in the future.

Literature Review

Project and Program Defined

Since its emergence as a distinct management discipline in the 1950s, the development of normative frameworks and guidance documents for project management has been led by the United States and, to a lesser extent, the United Kingdom, with contributions from others (including Australia and most recently, Japan). Drawing on this body of literature, the British Standard for Project Management, BS6079 (2000) defines a ‘project’ as:

A unique set of controlled and coordinated activities with definite starting and finishing points, undertaken by an individual or organisation to meet specific objectives within defined time, cost, and performance parameters.

and project management as:

The planning, monitoring, and control of all aspects of a project and the motivation of those involved in it to achieve the project objectives on time, and to the specified cost, quality, and performance.

Project Management Institute (PMI) adopts similar definitions for a project (“a temporary endeavour undertaken to create a unique product, service or result”) and project management as “the application of knowledge, skills, tools and techniques to project activities to meet project requirements” (PMI, 2004).

In the United Kingdom, BS6079 (2000) defines a program as “a group of related projects,” whereas Office of Government (OGC) guidance (MSP, 2007) defines a program as:

A temporary, flexible organisation created to coordinate and oversee the implementation of a set of related projects and activities in order to deliver outcomes and benefits related to the organisation's strategic objectives.

This compares well with the PMI definition:

A group of projects managed in a coordinated way to obtain benefits not available from managing them individually.

The need for, and key characteristics of, the program approach are reflected in these definitions through the consistent reference made to the coordination of project level activity. However, these definitions, although consistent, are in some respects incomplete because they do to not entirely reflect the uncertain and dynamic nature of a program environment that yields benefits over an extended period. It is this combination of coordination, integration, uncertainty management, and progressive benefits realisation that demands a program management approach and shapes the roles associated with this approach.

Definitions adopted within SDPPM are generally believed to be consistent with international standards, and this paper is not suggesting otherwise. However, and as the sections below suggest, there does appear to be evidence pointing at a tendency to treat a ‘program’ as a large ‘project’ and to use the terms interchangeably. For example, the term ‘IDPs’ is taken to mean ‘International Development Projects or Programs’ (Ika et al., 2009, p. 1). The strategic intent of program management requires a longer term perspective, capable of dealing with emergence and ambiguity. This is in contrast to the relatively short term and ‘tactical’ nature of project management, with an inevitable focus on the delivery of outputs and monitoring and the evaluation of specific deliverables.

The Growth of Security and Development Projects and Programs

International and bilateral donor organisations work in a range of transitional societies, all of which become characterised by varying degrees of complexity and uncertainty. This work can be described as taking the form of mainstream development assistance in relatively benign conditions, or increasingly, development and security-related assistance in countries emerging from internal or cross-border conflict.

The mutually reinforcing relationship between security and development has been discussed extensively in both policymaking and academic circles over the past decade (Brzoska, 2003; DFID, 2005; Tschirgi & Mancini, 2008; Hout, 2010; Denney, 2011). This ‘security-development nexus’ debate has led to further research about how these types of ‘combined’ donor-sponsored2 interventions could be more effectively ‘harmonised’ (Picciotta, 2007) in order to provide more inclusive and comprehensive support that addresses not only the symptoms but also the causes of conflict and insecurities (Collier & Hoeffler, 1998).

The coming together of these two areas also led to a number of related policy debates, such as human security, access to justice programs, and security sector reform. The enlarged ‘sub-policy’ agenda resulted in a proliferation of donor-funded program and project activity, which some organisations collectively described as ‘comprehensive’ or ‘integrated’ approaches. For example, at the outset of the international intervention in Sierra Leone in 1999, parallel program activities ranged from disarmament, demobilisation, and reintegration of former combatants. Similar SDPPM ‘landscapes’ can be seen in most of the countries that currently play host to a UN-integrated mission.

Since 2005, more thinking has emerged on the way in which SDPPM supports stabilisation interventions. Dennys argues that the limitations of Euro-centric and Westphalian approaches to state building have led to a ‘project-based approach’ to stabilisation becoming ‘a new form of state building’ (Dennys, 2011). In these environments, projects—often in the forms of Quick Impact Projects—appear to take the lead in international interventions. The ‘trial and error’ approach to project implementation in stabilisation operations then forms, at least in principle, the basis of a larger, more multi-faceted program.

Despite the growing importance of projects and programs in security and development, there is a paucity of published literature in this area (Ika et al., 2009; Thermistocleous & Wearne, 2000). Even after nearly 20 years of publications and international conferences on the topic of international coordination, critical reviews of in-country programs still remark consistently on the lack of coordination across a broad range and a high number of programs (Rees, 2010; OECD, 2010). Others have also commented on the way in which these ‘siloed’ approaches appear to undermine the very principles that support the delivery of development aid and security sector assistance (Peake, 2010).

Security and Development Project and Program Management (SDPPM)

In the review of security and justice sector reform (SJSR) programming3 in Africa (DFID, 2007), it was reported that:

A large number of the activities are stand-alone projects and many of those do not appear to be related to broader programs. Stand-alone projects are less likely to reflect all the attributes of a coherent approach of having the impact of programs.

The report calls for the adoption of a “coherent strategic approach” while claiming that lacking from “most” SJSR programming in Africa was “rigorous political analysis, regularly updated and supplemented by a risk mitigation strategy.” The general causes of these shortcomings were described under the headings of ‘coherence,’ ‘effectiveness,’ and ‘impact’, and among the recommendations of the review was the need to strengthen program management, in particular, the area of governance involving both donor and national and state and non-state actors.

Meanwhile, Ball et al. (2007) describe the need to move from ‘project’ to ‘program’ in response to the emergent and long-term nature of security and justice-related developments. Their report makes clear the need to ensure a ‘fit’ between short-term (even urgent) actions and long-term development objectives and stresses the opportunity for donors to “rethink how to conduct their support activities with regard to timelines, funding mechanisms, the expertise required of national donor personnel, and program management” (2007, ii). These views are supported by Robinson who calls for short-term stabilisation imperatives to be balanced with longer-term thinking vis-à-vis more macro-strategic frameworks, such as national development and security strategies (Robinson, 2011).

The demands placed on national personnel and the link between sustainable development and project management competence are explored by Oyalowo and Adeoyo (2010). The authors argue for the need to “harness the development potential of project management as a structured profession” in Nigeria and perhaps in developing countries more generally. This is entirely in line with principles that call for ‘local ownership’ and the need for ‘national buy-in,’ which authors, such as Laurie Nathan, emphasise is the importance of providing support to broader security sector reform programs (Nathan, 2007). Nathan's view is corroborated by the work on “Conflict Sensitive Planning,” which called for the engagement of local actors to inform planning and ensure that unintended consequences of project (and even more so, program) activities were limited by exposing issues through consultation exercises (Saferworld, 2009).

The importance of effective governance is alluded to in the analysis of Ika et al. (2009) in their review of World Bank projects. The authors found that two key actors are involved in international development project management: the task manager (TTL) at the headquarters of the multi-lateral agency and the national project coordinator (NPC) as head of the local project management unit. In other words, the title ‘project manager’ does not exist at the World Bank. Their findings infer that it is the NPC who is considered to be the ‘real’ project manager, because TTLs are not involved in day-to-day project management.

Recent progress was made in 2009 by the UK Government's Department for International Development (DFID) that developed a more updated approach to the use of LogFrames; the new model employed more horizontal ‘depth’ connecting programs with projects, as well as additional information such as ‘assumptions,’ ‘risks,’ and ‘changes in budget,’ which enhance the Framework's capacity to support changing environments and conditions. This has also evolved the LogFrame to become a model that now serves both the ‘planner’ and the ‘evaluator’ and, as such, could provide a useful framework to support more effective SDPPM.

However, the LogFrame requires both ‘output’ and ‘outcome’ (and even ‘goal’) to be defined in the context of the project. This is sensible, but in doing so, the LogFrame critically assumes that the essential linkage between ‘project’ and ‘program’ is maintained through effective governance and continuity of ownership ‘outcome’ at the program level. A pre-occupation at the level of the project—the deliverables—means that, in reality this may be a flawed assumption.

Integral to the LogFrame approach is a requirement for monitoring and evaluation (M&E). These two processes are inevitably grouped together as one—M&E—and treated as a singular set of activities. In practice, ‘monitoring’ concerns the progress of activities and delivery of outputs on a continuous basis, whereas ‘evaluation’ is more closely related to the achievement of outcomes and goals at a point—or series of points—in time. Moreover, monitoring typically falls within the purview of the implementing organisation, whereas evaluations are best undertaken by impartial, objective, and often independently commissioned evaluators or evaluation teams.

In practice, M&E is most commonly undertaken at the level of the activity or project, where clearly defined, short-term measures are available. This reinforces a ‘project-centric’ approach and undermines, discourages, and may even prevent due consideration of strategic intent, program outcomes, and sustainable benefits.

Finally, SDPPM inevitably involves introducing change into political, geographic, economic, and social environments that are, by their very nature, complex. In these arenas, projects are often introduced as a lead form of engagement, which then ‘morph’ over time and develop into broader country programs. Research underway at Cranfield University (Maylor, 2011) on the subject of ‘complexity’ has identified three main complexity groups: structural (relating to size, scale, and number of inter-dependencies), socio-political (associated with stakeholder or people issues in organisations), and emergence complexities (associated with changes). Underpinning these groups is a common factor—a dynamic denominator—that requires a management response able to recognise that which is (and must be) fixed and that which is possibly (or probably) going to change. This in turn requires competence—appropriate behaviours and motives, and not just knowledge and skills—at the level of the individual, the team, and the wider organisations involved in the change.

Research Methodology

Approach

An investigation that examines projects and programs supporting the delivery of security and development assistance could be tackled in a number of different ways; however, due to the lack of consistency across bilateral and multilateral donor project management approaches, World Bank–sponsored project and program data have been used for a number of reasons. First, the World Bank serves as one of the principal multilateral donor organisations that subscribes to the ‘security-development nexus’4 and that supports projects in both areas. Second, the World Bank takes a reasonably regular and current approach to updating reporting on all projects and programs it funds and offers a consistent framework of data across which comparisons can be made. Last, due to both the scale and the reach of World Bank activities, the data draw on both projects and programs and offers some thoughts on how different dynamics impact at both levels of engagement.

In order to make some broad and more general observations on the donor-sponsored SDPPM, data from a number of sub-Saharan African countries are explored (see below for further details of the geographic spread). Further analysis looks more closely at SDPPM in Ethiopia and the wider Horn region. Ethiopia and the wider region serve as an instructive case study due to the fairly balanced approach to work in both the security and development spheres. Based on the volatility of its surrounding region and specific border frontiers, and the stability that the country could bring to the Horn of Africa region, Ethiopia is currently considered a high-priority area for a number of donor countries and thus serves as a case study of current interest.

Last, through the Cranfield University Masters in Security Sector Management program, a part-time executive post-graduate export variant, which is run out of Addis Ababa, the authors had access to 25 senior civilian and uniformed professionals. The group provided a good cross-section of individuals from civil society through to the security-related Ministries such as Defence and the Federal Police. A survey was administered to the group, which solicited additional data on their PPM experience as both implementers and beneficiaries of SDPPM, together with further case studies for analysis. This ‘locally informed’ data were used to augment the World Bank data.

The research therefore adopted a qualitative approach with data collection comprising four key elements:

  • Desk study review of 20 World Bank Project Completion Reports5 during the period between 1983 and 1995, comprising 11 Ethiopian projects6 and 9 projects drawn at random from the Horn region of nations;
  • Further desk study reviews of all 23 World Bank ‘Implementation Completion and Results Reports’ for African projects reported in 2010;
  • Meta-analysis of an additional 10 Ethiopian project case studies7 submitted as part of the assessment of the SSM MSc PPM module; and
  • Survey of senior civil servants and serving military officers studying the SSM MSc and completing the PPM module.

No attempt was made to differentiate between ‘large’ and ‘small,’ nor between ‘project’ and ‘program.’ Instead, the approach adopted was shaped by three key considerations: geographic focus, longevity (1983–1995) and currency (2010) of data; a complete list of the projects included in the analysis is presented in Annex 1. In summary, the 53 projects and programs analysed comprised:

  • Ethiopia (24)
  • Kenya (5)
  • Uganda (4)
  • 2 each from Gambia, Somalia, Sudan, and Rwanda
  • 1 each from Chad, Senegal, Tanzania, Democratic Republic of the Congo, Ghana, Ivory Coast, Cameroon, Burundi, Malawi, Niger, Nile Basin, and Central Africa Republic.

Lessons learned data were extracted verbatim and consolidated to facilitate thematic analyses of the issues described (and opportunities identified) to be undertaken. Independent analysis was undertaken concurrently by both authors and their results compared. The approach adopted was based on the three-stage ‘thematic synthesis’ process as described by Thomas and Harden (2007). Results of the desk study reviews are presented in summary form in Annex 2.

Limitations

Thomas and Harden take the view that “the quality of qualitative research should be assessed to avoid drawing unreliable conclusions” (2007: 6) and assessed the ‘quality’ of their own (qualitative) studies with regards “to the degree to which they represented the views of their participants” (2007: 12). Similarly, but in the context of mixed-methods research, Brannan (2005) proposes quality-related concepts such as ‘credibility’ (cf. internal validity), ‘fittingness’ (cf. external validity), and ‘auditability’ (cf. reliability).

Maylor and Blackmon (2005) define validity as “the extent to which the underlying truth of a situation has been captured and is free from particular influences” (2005: 363). Bryman and Bell (2003) identified three specific attributes of validity: causality, generalisation, and replication. ‘Causality’ refers to the extent to which there is confidence in the underlying research conclusions and that there is a justified link between the data collected and the arguments presented; ‘generalisability’ relates to the relevance of the research to, and its application and extension in, domains other than the one of immediate relevance to the research; last, ‘replication’ relates to the repeatability of the research within other domains, that is to say, the potential to conduct similar research in other sectors but with limited adaptation.

To the extent that the data used have been extracted from published project reports and subjected to independent analyses by both authors, the results are believed to be reliable. This assumes, of course, that the project reports themselves provide an accurate account of what happened, although there is no reason to doubt that; however, the sample of 53 projects represents a relatively small proportion (on the order of 10%) of the totality of World Bank commitments in the region over the time period considered. As such, care must be taken not to overstate the findings of the research.

Data Analysis and Discussion

Across all 53 projects, a total of 201 discrete lessons have been identified. These lessons fall into six broad categories or themes. The themes (together with the proportion of lessons associated with each theme) are:

  • Policy (7%): matters relating to government policy affecting PPM implementation;
  • Governance (19%): including structures boundaries, roles and responsibilities, key forum and decision-making bodies;
  • Stakeholders (19%): engagement, management, communication, collaboration, stakeholder engagement and management;
  • Definition (22%): including scope definition, changes in scope, defined vs. emergent scope, alignment to strategy;
  • Capacity (19%): including human resource availability, training needs, people development, use of external consultants; and
  • Process (14%): elements relating to specific processes such as life cycle, scheduling, risk management, benefits management.

Policy

Policy becomes the basis for strategy, which in turn informs programs and projects. Although the data underscored the importance of having wider political support for sustainable project outcomes, it indicated that most borrowers and beneficiary countries were not well-equipped to undertake far-reaching policy reforms. Many of the projects examined also emphasised the significant inter-dependencies between sub-sectoral project efforts, which further supported the need for policy-backed processes. For example, in many African countries, there is a direct linkage between transportation, infrastructure developments, and improvements in the agricultural sector. Projects and programs must have a strong link back to strategic level policy goals in order to harmonise across key strategic growth areas in a way that maximises the benefits realised from finite resources.

Good political support for projects and programs was also found to increase the likelihood of regulatory institutions being strengthened in order to support sustainable structural change. The lessons drawn from a number of World Bank project reports suggest that structural change could not be achieved without more general economic policy and political stability that, together, create the enabling environment for growth. Last, some documented lessons learned also recommended that future lending by the World Bank should be contingent on government agreement to major sub-sectoral reforms.

Governance

Earlier sections of the literature review raised issues surrounding the ‘loose’ governance arrangements underpinning SDPPM. Based on the data supporting this paper, it is reasonable to conclude that the achievement of project objectives is highly dependent on the strength and commitment of the project ‘owner.’ However, the data also suggest that there are many ‘disablers’ that limit the extent to which the project ‘owner’ is incentivised towards this required level of commitment.

The data produced by the World Bank reports and practitioner surveys indicated that project governance between government agencies is often not subjected to the same degree of rigour as projects between commercial organisations. Similar processes, practices, and general approaches are required in order to eliminate waste and enhance sustainable project performance. This is particularly the case for multi-faceted projects that involved overseas contractors, whose experience and knowledge gaps can lead to misunderstanding and projects becoming prone to problems. Equally, projects spread over large geographical areas, where local procurement is required or where there is a lack of project management know-how and competence, require a more hands-on approach from both the owner and the donor.

The data analysis also suggests that project performance will be enhanced based on the existence of a more inclusive ‘platform’ constructed between the implementer/borrower and the lender/donor. The data indicated that some projects did not provide project implementation units with a significant ‘voice,’ which often led to the implementers being excluded from the decision-making. The tendency to bypass these groups and leave the decision-making in the hands of government ministries, which are generally ‘detached’ from the project implementation, proved to have a negative effect on the morale of the implementers, undermining ‘buy-in’ and commitment, and thereby threatening delivery. Data also indicated that the exclusion of authorities more closely involved in the implementation of the project resulted in lost opportunities in terms of the delivery of wider societal benefits beyond the obvious infrastructure. Last, a tighter interface between the highest levels of project authority and the implementation team can often result in remedial action being identified and acted on at the earliest stages of a project's life.

The role of the resident donor mission must be clearly identified and adequately staffed. Due to the changes in governments and ministerial personalities that often take place in developing and post-conflict countries, there is an important role, which should be led by the resident donor mission in facilitating communication supporting the project and its future development. This is particularly the case when a number of different donor agencies are implementing concurrent projects and when clarity is required on their relative importance, inter-relationships, and mutually reinforcing elements. One World Bank report argued that Task Team Leaders (TTLs) can change as regularly as five times during the life of a project, which often led to slowdowns in the project timelines. Although projects involving multi-year field-based TTLs yielded more effective project performance, the uncertainties that surround the requirement for external TTL expertise would be allayed by a more robust governance structure.

The data also indicate that good practice has been observed in the implementation of projects that involved a steering committee. Based on past practice, steering committees appear to have played an important role in providing incentives for coordination, in enabling good policy dialogue between governments and donors, and in providing forums to review project activity. However, implementation and review reports also suggest that steering committees should not be assimilated into project implementation units, because there is an important distinction to be drawn between the purpose and responsibilities of each mechanism.

Two other areas of good practice also lent well to the importance of effective ‘broader’ governance arrangements overseeing project activity. A number of project reports and surveys indicated that the use of non-governmental organisations positively reinforced the socialisation of project objectives at the grass roots level of society. This enabled a ‘bottom-up’ approach to come together with a ‘top-down, more conventional project approach, which is important, for example, in societies where rural areas are often significantly disconnected from capital centres and, it follows, the state institutions responsible for authorising the project work. In addition, a regional approach to SDPPM implementation has also proven helpful in facilitating the exchange of good practice across a number of like-minded countries that faced similar strategic environmental challenges.

Stakeholders

Stakeholder engagement probably represents the most important element in effective SDPPM, particularly in complex and uncertain areas. The data collected indicate that even highly complex programs can succeed if there is political will, participation, prudent institutional and technical design, and strong partnerships. This was particularly the case for decentralised approaches to SDPPM, where community participation became the key to success. The importance of stakeholder engagement is also closely linked to effective communications among actors, which supports more timely and effective decision-making. A number of SDPPM experiences studied indicated that detailed consultations with the public, by way of seminars and workshops, proved to be effective.

A lack of stakeholder engagement can cause serious setbacks to SDPPM. For example, experience indicates that improved stakeholder management in projects being implemented within farming communities would have prevented time-consuming and costly legal proceedings. The data also underscore the importance of ‘multi-level’ stakeholder engagement. More specifically, despite what may appear to be a good level of government ‘ownership’ and ‘buy-in,’ resistance can often be felt from implementation stakeholders whose organisations may have been affected due to project-related government decisions. One project experience led to the divesting of the enterprise involved; this was something that was not communicated effectively to the enterprise prior to the project launch. Some projects experienced greater degrees of ‘multi-level’ stakeholder engagement when close dialogue and frequent field missions were encouraged with government representatives. In some cases, the provision of budgetary resources to participating agencies enhanced ‘ownership’; similar arrangements and inducements should also be created to ensure that government ministries remain engaged from design through to implementation. Effective stakeholder engagement also led to significant levels of parallel community development and enhanced community cohesion. For example, sensitive community organisation efforts led to the formation of cohesive and effective housing cooperatives whose self-help efforts were responsible for the efficient construction of low-income dwellings. The data also indicated that involvement of local stakeholders can also lead to a greater command of the local development process and the improved delivery of basic social services. It appeared that, due to the lack of capacity for local communities to co-fund projects, innovative ideas were often required to develop a positive sense of ownership including the use of ‘in-kind’ contributions and cost-sharing in the form of formal or informal local credit.

The research findings also indicated that parallel interventions that integrate pro-poor and livelihood enhancement opportunities played an important role in achieving the levels of community cohesion necessary to advancing project activity. However, it was also important to note that environmental projects such as tree planting and soil and water conservation efforts tend not to expose desired benefits for a long time. Immediate wins are important in order for the local community to link such gains to the (ideally) well-promoted project objectives.

Stakeholder engagement can also enable a ‘bridge’ to the more traditional and non-state structures in societies. The research data indicated that pastoralist groups were often left out of stakeholder engagement strategies. Other data suggested that community militia groups could make a useful contribution to the security of rural programs and projects, particularly if these areas are prone to insurgent attacks. Other project experiences underscored the important role played by local elders and tribal leaders, whose leadership across communities in conflict management and resolution was paramount. The latter areas are important as, notwithstanding the benefits it can bring, a participatory approach to SDPPM can sometimes create expectations that are difficult to meet.

Despite the above arguments supporting effective stakeholder engagement, project authorities and donor organisations must accept that local communities will have a steep learning curve in understanding project objectives and broader implications for community development and environmental benefits. Simplicity should underpin the process of preparing a local community's strength of ownership. A clear understanding of the objectives, scope, and merit of each project should be communicated in line with a well-designed stakeholder engagement strategy. Participation of all intended beneficiaries is fundamental, and failure to do so could result in a project or program being rejected by the beneficiaries at an early stage.

Definition

A clear, well-understood, realistic and feasible project definition should be produced at the earliest stages of the project, and the work required to develop it should not be underestimated.

The scope of projects researched in this paper appeared to be generally problematic. Geographically, the data indicated that projects conducted across more modest boundaries could yield important lessons in the way such projects could continue more productively during the life of a broader program. Due to the large number of pastoralist and nomadic communities in the Horn of Africa region, smaller versus wide-scale project coverage was thought to be more effective at allowing instruments to be developed (perhaps through pilot projects) to influence the behaviour of these groups. More generally, it was felt that interventions into pastoralist economies should be cautious and expected output should be modest.

Recognising that the rebuilding of institutions takes a long time and that key actions must be addressed early—and that trade-offs must often be made between robust preparation processes and responding to more immediate fiscal, economic, and social needs—the data suggested that more effective SDPPM should involve a focus on fewer strategic actions. The lower number of strategic actions should be implemented through appropriate sequencing, which directly addresses the constraints on sustainable and equitable agricultural growth or other related goals. In this context, the data also indicated that tempering ambitions around the pace and scope of reform that could be achieved annually was perhaps more effective than pursuing ambitious multi-year targets. Such an approach could be promoted by the use of ‘phased’ PPM, which allows for a ‘design phase,’ a ‘feasibility study,’ and an ‘implementation phase.’ In most cases, projects that did not employ a feasibility study worked with unrealistic budgets and schedules.

Arguably, a ‘phased’ approach may also enable more flexibility to be exercised over time periods. For example, one positive lesson observed in a number of World Bank project completion and implementation reports was the benefit of flexibility being exercised around time extensions and the modification of the scope of the project. This was particularly important for reaching project objectives. The data also indicated that more flexibility and innovation were required to influence the design of capacity-building projects, particularly in countries where the absorptive and institutional capacities were not strong.

Capacity

Project activities should not commence until technical and implementation capacity of the participating organisations has reached the required levels. The research data indicated that institutional strengthening prior to, or in the early stages of, the project is important.

In this context, realistic estimates of the capacities of the project implementing partners are critical. In post-conflict environments, this capacity is often weak. High staff turnover rates and staff rotations often undermine the efficiency of the system supporting the project. As a result of this high turnover, early training efforts are often wasted, particularly where positive lessons were learned about up-front training of procurement knowledge and general IDA procedures.

In many cases, a lack of parallel civil service reform program does not lead to consistency of staff support, and the success of most projects often becomes based on a small number of highly overworked employees. Management capacity in areas such as business planning, financial management, purchasing and inventory management are often lacking. In addition, the more people-related management skills including leadership, stakeholder management, and teamwork are important and are often weak.

Capacity-building is a long-term activity and donors need to be realistic about the results that can be achieved in the project life cycle. Appropriate levels of investment and realistic results-based frameworks should be used to achieve sustainable capacity. The data suggested that assumptions were often made about the degree of knowledge surrounding the implementation of complex policies as well as the existence of information systems supporting sound policy development. This indicates that effective management and operating systems should be built into the broader program design of security and development projects.

Last, the data confirmed that project delays were often incurred due to weak contractor management capacity combined with the ineffective supervision of contractors by local staff. It also appeared that many projects had been strengthened as a result of the management support and technical assistance provided by the donor in areas where capacity was weak. This accounted for areas such as the knowledge of good partners, procurement processes, and systems for monitoring performance. In addition, providing the borrower with time for ‘re-programming,’ and allowing the borrower to incorporate lessons learned during a project's first phase—which was often reported to be missing—also proved beneficial where institutional capacity was weak.

Process

The data surveyed indicated that fundamental elements of general PPM, and PPM best practice, are not always employed in the more uncertain, more complex, and arguably less sophisticated project environments associated with the international security and development assistance environment.

Monitoring and evaluation represented one area of recurring problems. It was often the case where monitoring and evaluation amounted to the reports provided by internal engineers supporting the project or on annual reports drafted by the funding organisation. In uncertain and complex environments, neither will annual reports nor independent assessments support effective short-term decision-making. This annual and internal approach to monitoring and evaluation also meant that no critical variance of planned and actual project performance could be observed and, therefore, acted on if necessary. Project performance systems are required, which are aligned to the capacity of the ‘borrower’ and based on well-defined project ‘milestones,’ which subsequently, support more regular and timely project scrutiny.

The data also indicate that, although risk management was sometimes carried out in the preparatory stages of the project, this analysis often did not encompass the interests of all groups. For example, one project had to be scaled down because the bids exceeded initial estimates due to the rising political tensions and the impact this rise in tensions had on price increases. Lessons from other projects reinforced the fact that, even where project risks were assessed, these risks were not reflected in the project documents and finances. More robust analyses on the interdependencies and inter-relationships between project and program objectives will enhance an organisation's ability to minimise ethnic conflict, be proactive—rather than reactive—towards mitigating threats and maximising opportunities, and support conflict prevention more generally. In this context, the social and political dimensions of special interest groups emerged as being important in preparatory stages.

The final issue relating to ‘process’ concerned the lack of any written guidance on PPM and the lack of knowledge of the accounting and auditing procedures of international development organisations. There were a number of projects, which exposed not only a lack of general PPM knowledge, but also a lack of PPM discipline instilled in both the organisation and management. Moreover, this was against a backdrop of overwhelming support for PPM found in the survey. Simplified procedures and standards are also necessary to support the ‘softer side’ of SDPPM; the beneficiary cannot simply rely on contractor ‘goodwill’ to make this happen. Basic PPM frameworks and operating principles are also important due to the nature of SDPPM involving a large number of concurrent projects and programs where information management and supply chain integration could increase the effectiveness of risk management and support more effective coordination.

Moving Forward: Improving SDPPM

Recognising that SDPPM is Different

The data analysis presented above reflects the additional and dominant dynamics and environmental conditions that shape SDPPM. Although much current thinking on PPM is transferable to these environments, additional guidance is required to address the significant policy, governance, and capacity gaps that were found to exist. In addition, the fact that security and development programs often work in parallel to a high number of other interventions supporting national growth and regional stability—and often in countries governed by traditional, grass-roots systems as well as more formal state processes—more innovative thinking is required for promoting sustainable institutional and society change.

There are, therefore, significant differences between conventional PPM and SDPPM, which limit the transferability of traditional frameworks. First, mainstream PPM often features a commercial organisation or government department that acts unilaterally in shaping policy pertinent to its business strategy. In developing and post-conflict societies, the policy may be that of the donor, meaning that the country strategy is ‘institutionalised’ and subsequently not ‘owned.’ In areas of weak state capacity, supply-driven projects are often not supported by wider policy reforms. These approaches also tend to engage the services of a number of international contractors, which introduces issues regarding familiarisation and socialisation with the local context.

In conventional practice, scope definition is informed by clear policy, shaped by a realistic understanding of resource constraints, a degree of certainty surrounding longer-term investment, and clarity in the understanding of both the ‘project’ and the ‘program.’ In SDPPM, initiation appears to be driven by the donor on behalf of the owner. This often develops from a short-term and tactical orientation, to a longer ‘emergent’ transformation process with a blurred sense of project and program. A supply-driven, top-down approach is most prevalent in larger projects and programs. However, to be successful, these larger initiatives must be accompanied by feasibility studies aimed at identifying specific objectives before the main investment is made. A demand-driven, bottom-up approach (c.f. Community Driven Development) works best where the scope of the project is limited to a single community or manageable area; for example, falling under the auspices of a single administration. This allows for more realistic annual planning, reflection, lessons learned, and if necessary, a degree of support from the donor organisation, particularly in areas such as financial management, performance management, and procurement.

Conventional PPM involves governance arrangements, which are well defined, relatively stable, and predominantly internally focused. This is in contrast to SDPPM, in which governance arrangements are often very unstable and externally focused. Some SDPPM projects have more of the look and feel of a ‘program’ in terms of their scope, duration, and overarching aims but there is no evidence that governance is aligned accordingly. Although the more contemporary LogFrame approach goes some way in clarifying the link between project level and program level, project outputs are often achieved in a way that is isolated from overall program-level progress. An effective governance framework able to establish and sustain this connection is vital, particularly as the research data confirmed that there were important inter-dependencies and interrelationships across project and program objectives that were often ignored or neglected.

Stakeholders involved in conventional PPM often share a common language, are relatively local and therefore easily physically engaged, and characterised by low cultural diversity. SDPPM is often challenged by a lack of common language, a high level of cultural diversity, and geographically and economically dispersed stakeholders. Security and development initiatives that are most successful are those that engage early and effectively with beneficiaries, especially when these include local communities with strong traditions, customs, and practices. Such stakeholder management, combined with innovative ways of promoting community buy-in (such as the use of informal credits and ‘in-kind’ support) is also important for achieving broader community cohesion, a sense of ‘self-help’ and project sustainability; dynamics that are critical for national growth and conflict prevention. There is, therefore, strong evidence supporting the need for greater focus on stakeholder engagement in general, and in local, rural communities in particular. Stakeholder analysis is not systematically done, which results in less than optimum communications planning, resulting in blocking actions to the detriment of the project. Ironically, ‘blockers’ often tend to be those who the project is intended to help. In contrast, effective stakeholder engagement can help construct a ‘platform’ that links together all the relevant groups and individuals, across both the formal and the informal systems.

Another issue related to language lies in the very use of the terms ‘project’ and ‘program’ and the need for clarification and standardisation of terms and associated roles. The project manager position is the key; yet, in the context of the World Bank, we have identified potential confusion in the context of TTL and NPC roles. The ‘generic’ program approach makes clear the vital part played by the beneficiary organisation through the roles of ‘Sponsor’ (owning the business case) and ‘Business Change Managers’ (representing the business in the program and responsible for realising the program benefits). These roles and responsibilities are often not so well-defined in SDPPM. For example, the mere fact that the term ‘donor sponsor’ is often used in the context of international development assistance indicates a degree of uncertainty as to who the actual sponsor is, and what the role entails. The term also exposes the limitations of generic thinking and the assumptions this field makes about the key role of indigenous ‘sponsors.’ In post-conflict environments in which interim international administrations or transitional authorities are sometimes used to provide a functioning government system, the roles of ‘sponsor’ and ‘business change manager’ become even more unclear.

Other key roles include those of the program manager (with day to day responsibility for program delivery) and the program office (a vital support function implementing program level process). The role of program manager in SDPPM is very elusive and unclear, and although there is evidence of project coordination and project implementation units, the presence of a more ‘strategic’ support function that looks across related projects is missing. Inadequate levels of human capacity and thus the abilities to ‘own,’ ‘drive,’ and support projects and programs in beneficiary countries, are other complicating factors. Paradoxically, Program Office is one area where local capacity could play a much more active and productive role. Finally, although conventional PPM is characterised by processes, which are often standardised and with a high level of maturity, SDPPM processes are typically based on emergent activity and often shaped by low levels of education, training, and maturity.

Improving Performance: An Empirical Model

The research has identified six key factors, which collectively represent the sum total of 201 lessons arising from 53 projects and programs in Ethiopia and the wider Horn region. These factors—Policy, Governance, Stakeholders, Definition, Capacity, and Process—are believed to provide funding agencies and sponsor states with an opportunity to more openly and explicitly discuss and assess the environment within which security and development initiatives are nested, before, or certainly as part of, project and program initiation.

These factors have been brought together in a new model: PROFIDE (Program Factors in an International Development Environment) illustrated below.

The PROFIDE model

Figure 1: The PROFIDE model.

Application of the PROFIDE model—with each factor being underpinned by a set of related questions8—provides key parties with the means to assess the context within which a particular commitment is being considered, and in so doing, provides key parties with a better understanding of the PPM environment. These factors are keys, not only in respect of the need for their absolute consideration, but also, and crucially, in respect of the optimum sequence with which they are best addressed. Although iterative, based on the findings of this research, the optimum sequence for addressing these factors is given below:

  1. Policy—the policy of the sponsoring state, not that of the funding agency;
  2. Stakeholders—all key actors: agency, state, and local community;
  3. Governance—appropriate structures, roles, and responsibilities;
  4. Definition—vision, mandate, program/project definition;
  5. Capacity—locally available, skill, and competence gaps;
  6. Process—appropriate, fit-for-purpose.

A ‘virtuous loop’ of ‘policy—stakeholder engagement—governance’ allows the program vision and mandate to be developed. These ultimately shape the program definition and once this is better understood, issues of capacity and process can be more fully addressed. Clearly, this must be an iterative process, but all the evidence suggests that crucial to success is the consideration of state policy (shaped by government) and stakeholder engagement (including local beneficiaries), supported by fit-for-purpose governance (as local, as complete, and as stable as possible). Without these key factors being addressed up-front, program definition is likely to be flawed, a focus on ‘process’ pre-mature and capacity-related issues poorly understood.

Conclusions and Further Work

Adoption of PPM in the complex environment of international development and security interventions will inevitably be fraught with risk, uncertainty, and perpetual challenge. Good work has, and continues, to be done, in applying PPM principles to addressing the needs of the sector. However, this paper argues that the experience of recent delivery provides us with clear evidence of the opportunities to build on the work done to date and improve the effectiveness of the delivery of overseas assistance and so increase the benefits that are intended to flow from delivery.

The evidence suggests that although there will always be a need for short-term, output-based, tactical interventions driven by immediate needs, more attention and effort should be given to providing continuity of donor and sponsor structures and governance in respect of longer term programs shaped by transformational policy goals. The link between ‘policy’ and ‘governance’ must be made more explicit and transparent. Owners must begin to truly, ‘own.’

The PROFIDE model emphasises the importance of creating the right environment within which the program is to be delivered. An environment shaped by state policy, with appropriate engagement of stakeholders and governance arrangement that are fit for this purpose. These three factors define the ‘primary’ loop of the program environment. Only when this environment is understood, can program definition be effectively addressed and capacity and process-related needs discussed to inform strategic, rather than tactical decision-making.

In this way, the PROFIDE model highlights the need for greater attention to be paid to factors that shape the ‘causality of change,’ particularly in the context of longer term, transformational programs. The urge to act and drive to spend, although clearly sometimes justified, can often be as a result of a pre-occupation with short-term issues rather than longer term needs. The evidence suggests that the key to re-addressing this balance is more effective stakeholder management aimed at engaging those most affected by change and best placed to affect its successful outcome. This engagement must, in turn, be used to shape and reinforce governance structures as a means to effective, longer term decision-making and action taking. In this way, the ‘central causality of change’ must be made to happen because it sets the scene and establishes the context for other critical success factors. Governance and stakeholder engagement combine to enable a common understanding of program definition. This, in turn, gives rise to a more informed understanding of key enablers: capacity and process.

This paper was intended not to criticise, but rather to provoke a dialogue aimed at building consensus around the nature of the opportunities that exist in shaping, supporting, and successfully delivering and sustaining beneficial change in the security and development sector. The PROFIDE model is presented in that spirit. Further work to test the ideas put forward would add weight to this argument. In this context, the following two areas are proposed for further research:

  1. Engage with the donor community and ‘trial’ the PROFIDE model in a number of projects and programs; and
  2. Explore the application of LogFrame in SDPPM with a view to improving understanding of its application in programs, particularly with regards to appropriate levels of governance extending from the proposed outcomes to the range of individual projects.

References

Asefa, S. (2003). Developing democratic institutions in Ethiopia: The challenge of building enabling institutions for economic growth and development, Northeast African Studies, 10(1), 67–113.

APM (2007). APM body of knowledge (5th edition). High Wycombe: Association for Project Management.

Ball, N., Scheye, E., & van de Goor, L. (2007). From project to program: Effective programming for security and justice. The Netherlands Institute of International Relations, Clingendael.

British Standards (2000). BS6079 — Project Management Vocabulary. London: British Standards Institute.

Brannen, J. (2005). Mixed methods research: A discussion paper. London: ESRC National Centre for Research Methods.

Brzoska, M. (2003). Development donors and security sector. DCAF Occasional Paper No 4, Geneva: DCAF Publications.

Bryman, A., & Bell, E. (2003). Nature of research. Business Research Methods. Oxford: Oxford University Press.

Collier, P., & Hoeffler, A. (1998). On economic causes of Civil War. Oxford Economic Papers 50, 563–573.

Ball, N., Biesheuvel, P. Hamilton-Baillie, T., & Olonisakin, F. (2007). Security and justice reform programming in Africa. London: DFID.

DFID (2005). Fighting to build a safer world: A strategy for security and development. London: DFID Publications.

Egginton, B (2010). Project management education, training, working and learning: A longitudinal study. PMI® Global Congress, Research Conference, Washington, DC, July 2010. Sylva, NC: Project Management Institute.

Andrews, D., Erasmus, L., & Powell, J. (2005). Ethiopia: Scaling Up. Finance and Development (An IMF Publication), 42(5), retrieved from http://www.imf.org/external/pubs/ft/fandd/2005/09/andrews.htm

Funaki, Y. (2009). The UN and security sector reform in Timor-Leste: A widening credibility gap. New York: Center for International Cooperation.

Guba, E.G., & Lincoln, Y. (1994). Competing paradigms in qualitative research: Handbook of qualitative research. Thousand Oaks: SAGE.

Hout, W. (2010). Between development and security: The European Union, governance and fragile states. Third World Quarterly, 31(1), 141–157.

Ika, L.A., Diallo, A., & Thuillier, D. (2009). The most critical success factors for World Bank projects: The task team leaders perspective. PMI, Berlin, Germany.

Ika, L.A., Diallo, A., & Thuillier, D. (2011). The empirical relationship between success factors and dimensions: the perspectives of World Bank project supervisors and managers. International Journal of Managing Projects in Business, 4(4), 711–719.

Ika, L.A., Diallo, A., & Thuillier (2010). Project management in the international development industry: the project coordinator's perspective. International Journal of Managing Projects in Business, 3(1), 61–93.

Maylor, H., & Blackmon, K (2005). Researching business and management. London and New York: Palgrave Macmillan.

Ministry of Finance and Economic Development, Federal Democratic Republic of Ethiopia (September, 2010). Ethiopia: Trends and Prospects for meeting the MDGs in 2015.

Nathan, L., (2007). No ownership, no commitment: A guide to local ownership of security sector reform. Birmingham: University of Birmingham.

Office of Government Commerce (2007). Managing successful programmes. London.

Organisation for Economic Cooperation and Development (2010). Security sector reform: What have we learned? Results and Trends from the Publication and Dissemination of the Security System Reform Handbook, OECD Publication., retrieved from http://www.oecd.org/dataoecd/63/44/44391867.pdf

Oyalowo, B.A., & Adeoye, A.B. (2010). A case for project management education as a catalyst for sustainable development in developing countries. Newtown Square, PA: Project Management Institute.

Peake, G.(2010). A Lot of Talk But Not a Lot of Action: The Difficulty of Implementing SSR in Timor-Leste in Security Sector Reform in Challenging Environments, eds Hans Born and Albrecht Schnabel (Münster: LIT, 2009), 213–238, retrieved from http://www.dcaf.ch/yb2009.

Picciotto, R., Alao, C., Ikpe, Kimani, M., & Slade, R. (2005). Striking a new balance: Donor policy coherence and development cooperation in difficult partnerships. Background paper for OECD/DAC Learning and Advisory Process on Difficult Partnerships, Paris: OECD.

Project Management Institute (2004). A guide to the project management body of knowledge (PMBOK® Guide)—Third edition. Newtown Square, PA: Author.

Rees, E. (2004). Under pressure FALINTIL: Three decades of defense force development in Timor Leste 1975–2004. Geneva: Geneva Centre for the Democratic Control of Armed Forces.

Robinson, C.(2011). Developing a framework for post-conflict army reconstruction, PhD Thesis, Cranfield University.

Saferworld (2010). Conflict Sensitivity Assessment for European Union Programmes in Sri Lanka., Brussels: European Commission, retrieved from http://www.saferworld.org.uk/downloads/pubdocs/Conflict-Sensitivity%20Assessment%20Sri%20Lanka.pdf

Senge, P. (1990). The fifth discipline: The art and practice of the learning organisation. London: Nicholas Brealey.

Senge, P. (1994). The fifth discipline field book. London: Nicholas Brealey.

Swiss, L. (2011). Security sector reform and development assistance: Explaining the diffusion of policy priorities among donor agencies. Springer Science and Business Media, retrieved from http://www.ingentaconnect.com/content/klu/quas/2011/00000034/00000002/00009194

Thermistocleous, G., & Wearne, S.H. (2000). Project management topic coverage in journals. International Journal of Project Management, 8(1), 7–11.

Thomas, J., & Harden, A. (2007). Methods for the thematic synthesis of qualitative research in systematic reviews. London: ESRC National Centre for Research Methods.

Tschirgi, N., Lund, M., & Mancini, F. (2009). Searching for critical connections. New York: Lynne Reiner Publishers.

Thomas, J., & Mullaly, M. (2008). Researching the value of project management. Newtown Square, PA: Project Management Institute.

Annex 1: Project List

1. Third Education Project; Ethiopia; Date: 1983/06/27; No: 4571
2. 5th and 6th Highways Projects; Ethiopia; Date: 1984/06/28; No: 5168
3. Technical Assistance Project; Ethiopia; Date: 1994/11/10; No: 13704
4. Urban Development Project; Ethiopia; Date: 1992/06/12; No: 10721
5. Petroleum Exploration and Geothermal Project; Ethiopia; Date: 1992/12/21; No.: 11454
6. Port Engineering and Construction Project; Ethiopia; Date: 1991/04/30; No.: 9540
7. Second Coffee Processing and Marketing Project; Ethiopia; Date: 1993/06/07; No.: 11949
8. Sixth Telecommunications Project; Ethiopia; Date: 1993/09/17; No.: 12330
9. Rangelands Development Project; Ethiopia; Date: 1991/08/30; No.: 9830
10. Second Road Sector Project; Ethiopia; Date: 1995/03/17; No.: 14065
11. Drought Recovery Project; Ethiopia; Date: 1990/05/21; No.: 8639
12. Agricultural Sector Adjustment Project; Kenya; Date: 1990/06/22; No.: 8797
13. Second and Third Telecommunications Project; Kenya; Date: 1991/12/31; No.: 10250
14. Integrated Rural Health and Family Planning Project; Kenya; Date: 1992/08/31; No.: 11079
15. Economic Recovery Program; Uganda; Date: 1992/03/12; No.: 10415
16. Second Power Project; Uganda; Date: 1995/12/26; No.: 15194
17. Second Agricultural Sector Adjustment Program Project; Somalia; Date: 1993/10/29; No: 12435
18. Second Education Project; Somalia; Date: 1983/06/27; No.: 4573
19. Public Electricity and Water Corporation Project; Sudan; Date: 1990/09/19; No.: 9014
20. Fourth Power Project; Sudan; Date: 1994/12/06; No.: 13767
21. Agricultural Research & Training Project (2nd Phase); Uganda; No.: ICR951
22. Local Development Program Support Project; Chad; No.: ICR1686
23. HIV/AIDS Prevention and Control Project; Senegal; No.: ICR1763
24. Second Productive Safety Nets (APL) Project; Ethiopia; No.: ICR1676
25. Agricultural Input Supply Project; Kenya; No.: ICR1648
26. Western Kenya Integrated Ecosystem Management Project; Kenya; No.: ICR1533
27. African Regional Capacity Network for HIV/AIDS Project; Tanzania; No.: ICR1452
28. Emergency Multisector Reconstruction project; Democratic Republic of Congo; No.: ICR1506
29. 2nd Economic Management and Governance Project, Central African Republic; No.: ICR1682
30. Human Resources Development Project; Rwanda; No.: ICR1455
31. Africa Region – Nile Basin Initiative, Shared Vision Program Project; No.: ICR1493
32. Protection of Basic Services Project; Ethiopia; No.: ICR1569
33. Public Sector Reform and Growth Grant Project; Gambia; No.: ICR1530
34. Trade and Investment Gateway Project; Ghana; No.: ICR1460
35. 2nd Economic Governance and Recovery Grant Project; Cote d'Ivoire; No.: ICR1483
36. Second Road Sector Development Program Project; Ethiopia; No.: ICR1221
37. Gateway Project; Gambia; No.: ICR1399
38. Water Sector Project; Niger; No.: ICR1231
39. Urban Infrastructure and City Management Project; Rwanda; No.: ICR1375
40. Community Development Program Support Project; Cameroon; No.: ICR976
41. Financial Management, Transparency and Accountability Project; Malawi; No.: ICR1458
42. Food Crisis Response Development Policy Grant Project; Burundi; No.: ICR1379
43. Fifth, Sixth and Seventh Poverty Reduction Support Credit Project; Uganda; No.: ICR 1121
44. EDCSC Future HQ Project
45. Hara Meda Airfield Rehabilitation Project
46. Berahle – Dallol Road Project
47. Western Gode Irrigation Project
48. Tendaho Sugar Project
49. Addis Ababa Housing Program (2 reports)
50. Conflict Prevention and Resolution Project
51. Burbax Rural Development Project
52. Woreda Integrated Rural Development Project
53. Pastoralist Community Development Project (Phase 1)

ANNEX 2: Presentation of Data from World Bank Project Completion Reports, World Bank Project Implementation Reports, Cranfield University Case Studies, and Ethiopian SD Practitioner Survey

A. World Bank Lessons Learned: Project Completion Reports (PCR)

The following table contains details of the projects reviewed (including project location, report date, and report number) and key lessons learned9.

1. Third Education Project; Ethiopia; Date: 1983/06/27; No: 4571

1.1 In the case of innovative project items or institutions still in the process of conceptual planning, it would be prudent to allow sufficient time between follow-on projects for the application of lessons learned to subsequent project components. In addition, there should be close cooperation in the Bank between RMEA and Headquarters when dealing with similar components in different projects.

1.2 Technical assistance in the guise of more intensive supervision would improve and accelerate project implementation though the Borrower's implementation capability may not improve commensurately. However, issues between Government and Bank staff might have been foreseen or identified earlier, so that remedial action could have been initiated in order to minimize delay.

1.3 In the case of innovative components, evaluation should be built into the project.

2. 5th and 6th Highways Projects; Ethiopia; Date: 1984/06/28; No: 5168

2.1 The Fifth Highway project had to be scaled down, because the bids exceeded the estimates due largely to political tensions already apparent at the time of bidding, leading to sharply increased prices and wages and successor authorities were no longer exempt from duties and taxes which increased their financial burdens.

2.2 A quantitative and meaningful re-evaluation of the economic returns on the investment in each road, plus those of the related agricultural development schemes is difficult to carry out because of lack of road-specific reliable agricultural data. However, the re-evaluation carried out by the Government and IDA's indicates that agricultural investments were sound and justified.

2.3 One key lesson may be learned from the implementation of these two projects in view of the reevaluation of their benefits. In cases where almost 100 percent of the economic benefits of road projects (rural roads) are to be derived from the agricultural sector, either the construction of the roads should be a component of the agricultural project, or a monitoring procedure for the collection of specific data and statistics during project implementation should be agreed upon in advance between the Agriculture and Transportation Divisions of the Regional Projects Department.

3. Technical Assistance Project; Ethiopia; Date: 1994/11/10; No: 13704

3.1 Decentralization in the project design was an innovation. It helped to address problems at the agency level where problems existed and also engaged those directly affected to participate in devising and applying the solutions.

3.2 Implementation of recommendations from studies and consultants’ reports are important for project results, and even more so for the sustainability of project achievements. When implementation of some recommendations may extend beyond the project life, there is need for establishing continuity with future lending operations to ensure that actions recommended under a project will be carried out.

3.3 IDA accounting and auditing procedures need to be clear to all participating organizations from the outset, particularly in cases where the project design is decentralized.

3.4 The Resident Mission was very important for identifying the need for the project and in facilitation of communications; however, the time when the mission's role became unclear, 1990–1993, with respect to leadership for supervision activities coincided with a significant slow down in implementation. Despite other factors, it is clear that in a project of this nature, involving other parties, e.g., UNIDO, UNDP, OPS, and where Government organizations change frequently the Resident Mission must be closely involved, and their role appropriately defined and staffed.

4. Urban Development Project; Ethiopia; Date: 1992/06/12; No: 10721

4.1 Community participation in project planning, design, and management was a key ingredient to the overall success of the Project. Sensitive community organization efforts mounted by the project unit resulted in the formation of cohesive and effective housing cooperatives whose self-help efforts were responsible for the efficient construction of low-income dwellings for owner occupation.

4.2 A central aspect of the effective community organization which characterized the Project was the process of group formation based on the workplace, i.e. housing cooperatives were formed on the basis of groups working for the same employer. This facilitated the release of workers from their jobs, thus improving the coordination of mutual self-help construction activity.

4.3 The establishment of special project implementation units outside of agencies normally responsible for municipal services and shelter development should be avoided if any long term institutional development is to take place. A follow-up urban project in Addis Ababa was implemented directly by the city administration while the project unit set up under this project was given the responsibility to implement a project directed at selected market towns: and

5. Petroleum Exploration Promotion and Geothermal Reconnaissance Project; Ethiopia; Date: 1992/12/21; No.: 11454

5.1 The enormous value during implementation of good project preparation with PPF assistance,

5.2 Implementation agency and project team should consider both public and private promotional meetings and having a proper balance of the two.

5.3 A great deal of the Project Manager's time and effort was spent in keeping track of MME technical people who were sent to universities to obtain Master's degrees in geology and geophysics. Much of the Project Manager's time was spent in picking and choosing candidates as well as keeping track of their progress or lack of same.

5.4 Postponement of closing date was helpful in the project's attainment of most of its objectives. Bank management should always approach the subject of extension with an open mind addressing the needs of the project and the host country.

5.5 When the petroleum unit was set up by MME its reporting channel was supposed to have been established to be through the General Manager of the Institute. However, from the very beginning, most decisions were made by high Ministry officials, leaving little significant role to be played by the Institute in implementation of the project. The petroleum unit coordinated project activities and advised Ministry officials but was increasingly left out of most decisions made with regards to project activities. This resulted in partial breakdown of reporting channels and reduction in overall morale.

5.6 The Bank kept changing project supervisors in the early days of the project and the result was confusion and breakdown in communications. In the last days of the project, the Bank supervisor was corresponding mainly with the project resident advisor.

6. Port Engineering and Construction Project; Ethiopia; Date: 1991/04/30; No.: 9540

6.1 The success of the project was achieved mainly due to the full commitment of the borrower, i.e. of the agency (HTA) responsible for project implementation.

6.2 Lack of familiarity with the Bank's procedures on the part of MTA created some delays in the processing of the project. A procurement workshop in Ethiopia (carried out by IDA staff) was found to be useful.

6.3 Timely monitoring of the works resulted in no critical variance between planned and actual project performance.

6.4 Project risks were correctly identified and realistically calibrated, though not stated in the project documents in financial terms.

6.5 No actions or decisions that affected the normal functioning of the implementation process were taken either by the Bank or MTA.

6.6 Each project was separately initiated and its conceptual foundation was clearly laid down in the basic documents prepared and submitted to the Bank for its consideration.

6.7 All the concerned parties discussed and clearly understood the objective, scope and merit of each project so that they can participate more effectively in the study process.

7. Second Coffee Processing and Marketing Project; Ethiopia; Date: 1993/06/07; No.: 11949

7.1 Given the unpredictability of Government economic policies at the time of project design, the Bank was correct to be cautious in defining project scope (e.g., reducing by half the number of new stations proposed by Government). However, this was contradicted by the overly optimistic expectation that such a project could be completed in three years.

7.2 Whenever a project is implemented by more than one agency, adequate coordinating mechanisms should be put in place together with incentives for the agencies to cooperate (and overcome potentially divergent interests).

7.3 Up-front training in IDA procurement procedures (e.g., starting with a project launch workshop) should be given to staff in the implementing agency if they do not have prior familiarity with such procedures. Alternatively, some technical assistance in procurement should be provided. Lack of procurement knowledge inevitably leads to slippage in project implementation.

7.4 Providing equipment to reduce pollution is insufficient if not accompanied with enforcement or incentives for the owners to operate it.

8. Sixth Telecommunications Project; Ethiopia; Date: 1993/09/17; No.: 12330

8.1 Where institutional capacity is lacking, institutional strengthening either prior to or in the early stages of the project should be an important requirement and should itself be a monitorable activity.

8.2 IDA correctly identified the areas requiring institutional strengthening (e.g., computization, tariff policy development). However, the project did not incorporate specific monitorable actions that would be required as part of the covenants. Consequently, non-implementation of institutional strengthening measures resulted in non-compliance with project covenants. For example, there were explicit linkages between the state of computerization of the Management Information System of ETA and its ability to meet covenants.

8.3 There was good project implementation and continuous monitoring of project components was undertaken with suitable modifications made in the project design, e.g., where traffic was higher than initial projections, digital circuits were introduced sooner than programmed.

8.4 Technically, ETA is one of the best run telecommunications authorities in Sub-Saharan Africa. ETA's weaknesses lie in its institutional capacity in the peripheral functions (i.e., business planning, financial management, purchasing and inventory, etc.) where not enough management attention has been focused in the past.

9. Rangelands Development Project; Ethiopia; Date: 1991/08/30; No.: 9830

9.1 Expectations of the possible rate of change in the attitudes of traditional peoples were overly ambitious. The project failed to establish a system of controlled range use, which required the cooperation of pastoralists. This was a key element in the development of the rangeland, without which the project's achievements in infrastructure would not yield the expected results.

9.2 Instead of wide-scale coverage, the project could have focused on a smaller area and developed instruments for influencing pastoralists’ behaviour through pilots, studies, and use of participatory planning and design techniques.

9.3 Participation of intended beneficiaries in defining project concept is fundamental, and that participation may result in the project concept being rejected by the intended beneficiaries, a possibility that should be recognized by project designers.

9.4 A clear lesson is that interventions into pastoralist economies should be cautious and expectations of increases in output modest.

10. Second Road Sector Project; Ethiopia; Date: 1995/03/17; No.: 14065

10.1 Contractual obligations to ensure environmental performance of contractors can be strengthened. Although provisions for environmental mitigation were made, further action can help ensure contractors do not renege on their contractual obligations, such as increasing retention amounts to ensure environmental compliance.

10.2 The urban interface is a complex component of roads projects, which must be approached more systematically. Urban areas cities, towns, even villages have more complex, contextual elements that must be planned for and incorporated than standard rural sections of roadway. There is room for these complexities to be more systematically taken into account at the national level.

10.3 Training and capacity building are not as effective as they could be, as long as civil service reform is not on the agenda. Inability to retain staff results in high staff turnover; as a result, investments in human capital do not have much longevity, since junior staff tends to move on to other opportunities elsewhere after being trained.

10.4 Mobilization delays continue to be a source of difficulties for both on-time delivery and final project costs. The donor community needs to maintain focus and collective purpose to avoid such delays as much as possible. These delays seem to stem from contractors’ weak management capacity combined with ineffective supervision by selected firms.

10.5 Performance monitoring systems must be used to monitor contract performance and progress via benchmarks, rather than relying exclusively on supervising engineers’ assessments. Similarly, project costs should be monitored enabling more independent assessments of cost estimates.

10.6 To effectively address constraints to accessibility and unmet transport needs, the GOE and its donor partners will need to look beyond the provision and maintenance of road infrastructure, toward a more holistic approach to the sector, including services. To ensure that transport plays its full role in alleviating poverty and contributing to the economic life of the country, the availability and affordability of transport services will need to be approached in a consistent policy framework. In the process, there is a need to pay more attention to strengthening regulatory institutions.

11. Drought Recovery Project; Ethiopia; Date: 1990/05/21; No.: 8639

11.1 In emergency projects of this nature, it is correct to keep the project simple by limiting it to easily manageable key components, with emphasis on action, and giving other institutional aspects only secondary consideration;

11.2 To accelerate the arrival of key inputs such as trucks which were crucial to the transport of relief materials, the Borrower should avail itself of the Bank's sample bidding documents and specifications when these are relevant, and the Bank should provide technical assistance to help the Government through the procurement process when the implementing agency lacks this experience.

12. Agricultural Sector Adjustment Project; Kenya; Date: 1990/06/22; No.: 8797

12.1 Thorough efforts are needed right from identification stage to build commitment at both the political and technical levels of the implementing agencies. Ways to achieve this may include providing budgetary resources to the participating agencies to give an “ownership stake” in the program.

12.2 Policy decisions on key reform actions should be secured early in the reform program. These “up-front” decisions will help reduce the risks of unforeseen changes and/or weak implementation, and may result in quicker disbursements.

12.3 Focus on a few strategic actions, with appropriate sequencing, which directly address constraints to sustained and equitable agricultural growth or other relevant goals.

12.4 Formulate clear, unambiguous, and monitorable action plans and secure the necessary policy decisions “up-front,’ early in the process.

12.5 Allow a long enough implementation period in which to achieve the more complex reforms in the program.

12.6 Establish the necessary institutional arrangements and inducements to the participating Ministries in order to ensure their “ownership” of the reform program, from design through implementation.

12.7 The ASAO Steering Committee (SC), comprising PSs of the Ministries concerned, and chaired by the PS of Finance, proved to be an effective instrument, which should be continued in any follow-up operation.

12.8 The linkages between related investment activities could have been stronger and more explicit. For the proposed follow-up operation, these linkages should be specified and closely monitored during implementation, focusing on the most important policy and investment interventions.

12.9 The ASAO served as an instrument to articulate Government's policy reform program, assist in its implementation and mobilize quick-disbursing assistance from seven donors. It allowed a wider policy dialogue between Government and donors than would have been possible with any other instrument, and justified fully the sizeable IDA staff.

13. Second and Third Telecommunications Project; Kenya; Date: 1991/12/31; No.: 10250

13.1 The main problem areas were quality of service, overstaffing, unsettled arrears from public sector, very large and costly unprogrammed investments and poor financial management evident in deteriorating financial performance.

13.2 It is essential that discipline be instilled in organisation and management, development planning, and finances

13.3 Adequate monitoring must be introduced in the sector development and operation, not just at the project and operational levels, but also to overall sector policy, structure and performance monitoring.

13.4 There is need for an in-depth study of the findings and issues discussed in this report, with a view to full scale restructuring of the sector, reinforcing managerial discipline and better monitoring of its investment programming and operations.

13.5 It is suggested that such study for Posts and Telecommunications be initiated by Government and supported by the Parastatal Reform Project, which is currently proposed for Kenya.

14. Integrated Rural Health and Family Planning Project; Kenya; Date: 1992/08/31; No.: 11079

14.1 Involvement of five Government Ministries and nine Non-Government Organizations in an inter-agency information and education program on population and family planning enabled many target audiences to be reached.

14.2 A forum in which the implementing agencies reviewed project implementation on a quarterly basis was found useful by the agencies. The managers of the various projects supported by the council attended the meetings.

14.3 The Bank, at the inception of the project, provided assistance in the elaboration of the agencies’ project activities on population and family planning.

14.4 The Bank further reviewed the project's implementation process and suggested ways and means of enhancing it, e.g., advising on revising annual work plans

14.5 During the evolution and implementation of the Project, the Borrower learned many lessons in the area of project design, preparation, implementation, monitoring, evaluation and re-planning. These lessons proved useful during re-programming where the Borrower played a very active role.

15. Economic Recovery Program; Uganda; Date: 1992/03/12; No.: 10415

15.1 The experience of the first two years of the ERP showed how difficult it was to gauge Government commitment to the program. Dialogue with the Government must, from the outset, emphasize the fact that reform is a continuing process.

15.2 Dialogue must go beyond the narrow circle of top politicians and officials to involve a broader spectrum of decision-makers and implementers. The greater the number of people who understand and support the policies, the greater the prospects of sustaining reforms.

15.3 Government officials must be given every opportunity to participate in the design of the program so that sooner or later the Government can “own” the program; focus on a few actions which are critical to macroeconomic stability; pay more attention to implementation capacity, especially in the sequencing of structural measures; set realistic performance targets.

15.4 The disbursement of ERC I funding taught three lessons. First, much more effort is needed to make the Government manager responsible for the allocation of funds fully conversant with IDA procurement and disbursement procedures. Second, tied aid is, by its nature, not quick-disbursing; mixing this type of aid with untied aid leads to the overestimation of the amount of import support that can be utilized in a given period. Third, donors who ask IDA to administer their funds must accept that the pace of disbursement of those funds would be determined by IDA procedures and local banks’ ability to comply with the required procedures.

15.5 It is easy to overestimate the Government's capacity to implement and monitor the required actions. In reality, the program depended, for its success, on just a handful of grossly-overworked officials. Moreover, the elementary information systems necessary for sound policy formulation and monitoring did not exist.

15.6 Rebuilding institutional capacity takes a long time; the key actions in this area must therefore commence very early.

15.7 Perhaps, the most important lesson is that results, in the form of an early supply response, hold the key to the sustainability of structural adjustment programs. The quick rebound of economic activity was a major selling point for the reforms. The Government was thus in a stronger position to persevere with the fight against inflation and implement other reforms.

16. Second Power Project; Uganda; Date: 1995/12/26; No.: 15194

16.1 Project implementation could have been expedited had project scope and cost estimates been prepared more accurately. However, given that the project was the first Bank operation in Uganda's energy sector in more than 20 years and that the country had hardly recovered from an emergency situation, the task of obtaining fully accurate data was difficult.

16.2 The Bank's flexibility to modify the scope of the project based on new information of investment priorities and costs was important for achieving project objectives.

16.3 In spite of the constrained economic and security situation and the fact that Uganda had not undertaken major construction in several years prior to project appraisal, its implementation capacity was judged to be strong enough to undertake the project in a relatively tight timetable. In retrospect, it is evident that the project timetable, particularly with regard to start-up was too optimistic.

16.4 The project has shown that measures to improve institutional and financial performance have to be initiated up-front and have a more comprehensive approach than what was included in the project.

16.5. The project should have addressed the weaknesses in power sector management, including the lack of autonomy and commercial orientation of national operations.

17. Second Agricultural Sector Adjustment Program Project; Somalia; Date: 1993/10/29; No: 12435

17.1 The lack of wider support for commitments undertaken by the Minister of Finance and the Governor of the Central Bank was a major factor behind the poor implementation record. Efforts were later made by the Finance Minister and the Bank to open consultations with representatives of other economic ministries on the content of the economic program.

17.2 Structural Change is not sufficient without improved economic policy and political stability. The structural adjustments, while carried out in accordance with the program, could not benefit the economy as a whole without reform in general economic policy and political stability. In other words the enabling environment for growth following the structural changes must be present.

17.3 A chronic problem during implementation was the weakness of the institutions responsible for the implementation of reform. The lesson is that the Bank should be cautious about assuming that government institutions are able to implement complex policies. Strengthening of institutional capacity to implement policy reforms should be a long-term goal of programs in such countries.

18. Second Education Project; Somalia; Date: 1983/06/27; No.: 4573

18.1 Any project component requires a quick feasibility study taking into consideration the risks and benefits involved so as to avoid placing an unnecessary burden upon Member States. Studies should address costs involved, readiness to develop the proposal, and the capacity of the institutions responsible for the execution.

18.2 Components, which are new and where no experience exists in the country, need to be prepared in detail by the local institutions concerned. The project preparation document should include development and conceptualization activities of such components. A short-cut Preparation/Appraisal approach should be discouraged in such cases.

18.3 Components of different projects, which are intended to be for the same or similar target groups, should be evaluated before preparation to see if they can be integrated for multi-purpose use. This might entail minor modifications of the existing components and the proposed new ones.

18.4 The Project Implementation Unit should be required to have a liaison officer to look into the nature of the project components and ensure that new demands and requirements are feasible within the limits of the project and can be integrated into the project (i.e., requirements management).

18.5 Just as there is a chapter on management procedures and standards to implement the physical aspects of a project, an effort should be made to outline working procedures and standards to implement ‘softer’ aspects.

18.6 It would be desirable to make realistic estimates of the PIU's workload and its capacities when several projects are being implemented at the same time.

18.7 An information system should be developed capable of generating data indicating the changing needs of the project and help in monitoring the implementation.

18.8 Frequent changes and rotations in staff is a major problem that severely undermines the efficiency of the system. The Government should attempt to retain staff in post for a minimum of two years.

19. Public Electricity and Water Corporation Project; Sudan; Date: 1990/09/19; No.: 9014

19.1 During project preparation, implementation problems (e.g., transportation) should have been identified and adequate provision made to deal with them.

19.2 Ensure that detailed descriptions of all project components and/or sub-components are discussed with the donors and their sources of financing clearly identified.

19.3 IDA should also have pursued more actively efforts to link improved financial and managerial performance so as to be able to retain talented staff.

19.4 In the design of the project, sufficient emphasis was not placed and resources allocated for institution building activities; and

19.5 Future projects should endeavour to implement more effective management and other operating systems.

20. Fourth Power Project; Sudan; Date: 1994/12/06; No.: 13767

20.1 The Project has shown that sustained institutional and financial improvements are difficult to obtain without major changes in the way the power sub-sector is operated.

20.2 The project should have addressed National Electricity Corporation's institutional weaknesses in a more comprehensive manner, including increased autonomy.

20.3 Future lending to the Sudan power sub-sector should be contingent on the Government agreeing to major power sub-sector reforms including the commercialization of NEC and the encouragement of private sector participation in the financing and management of power facilities, as discussed in the Bank's Power Sub-Sector Policy Paper, approved by the Board in October 1992

B. World Bank Lessons Learned: Implementation Completion Reports

As for PCR above, the following table contains details of the Implementation Completion Reports for all projects10 reported to have completed in 2010. Details include project title, location, and report number together with key lessons learned.11

21. Agricultural Research & Training Project (2nd Phase); Uganda; No.: ICR951

21.1 It is important to improve management capacity at all levels (particularly at the zonal and district levels), to deliver results through a jointly agreed results framework between key stakeholders (including private sector and farmers’ organizations).

21.2 It is difficult to ensure that two institutions charged with complementary tasks will coordinate activities in the absence of formalized incentive structures. If a Government chooses to use partner funds, it should always be ready to step in, in the event that such funding becomes unavailable.

22. Local Development Program Support Project; Chad; No.: ICR1686

22.1 Participatory approaches create expectations that can be difficult to meet. To the extent that projects based on participatory community-driven development (CDD) approaches succeed in mobilizing rural communities to articulate their development needs, they have a tendency to generate expectations that may be difficult to meet if the capacity of local communities to contribute co-financing is limited.

22.2 Capacity building is a long term process. A consistent message heard from many stakeholders was the importance of continuing the capacity building activities that were launched during the first phase of the project. Results frameworks and Monitoring and Evaluation (M&E) plans must be realistic given capacity constraints.

22.3 The results framework and the associated M&E plan for the project turned out to be extremely ambitious given the limited capacity available for their implementation. Information collection and reporting is essential but must be realistic, practical, and sustainable.

23. HIV/AIDS Prevention and Control Project; Senegal; No.: ICR1763

23.1 Political leadership is vital for program achievement, and program achievement in turn provides a justification for continued political commitment.

23.2 The local banks’ flexibility and willingness to support project initiatives and innovations is important in promoting this political/technical linkage. Inconsistent bank priorities and rigid attitudes limit the potential contribution to project success.

24. Second Productive Safety Nets (APL) Project; Ethiopia; No.: ICR1676

24.1 The use of existing Government channels and structures for implementation was consistent with longer-term public sector management objectives, but reliance solely on existing civil service staff may be insufficient to address capacity constraints. This may result in the need for contracted staff.

24.2 Some weaknesses identified include delays in reporting, shortage of qualified accountants and auditors, limited focus on internal audit, and delayed staffing of the external audit function. Means to strengthen the multi-donor framework developed in First Adaptable Program Loan (APL I) were identified. Coordinated semi-annual project reviews were placed as dated covenants of the project and the donor coordination unit was reinforced.

25. Agricultural Input Supply Project; Kenya; No.: ICR1648

25.1 Vouchers can be used effectively in a post-conflict setting if the necessary organizational structures for retailing and redemption of coupons are intact. Use of the private sector has proven effective, despite the constraints.

25.2 Without strong mechanisms of social accountability, selection of beneficiaries under such schemes can be subject to fraud and corruption. Constraints in the flow of funds to districts and to agro dealers can undermine confidence that the vouchers will be redeemed or that product will be available in a timely manner, and can reduce effectiveness of the programs.

26. Western Kenya Integrated Ecosystem Management Project; Kenya; No.: ICR1533

26.1 Environmental interventions such as tree planting and soil and water conservation normally take a long time for desired benefits to be generated, integrating pro-poor, livelihood enhancement options that produce immediate economic gains into community-driven activities are essential to sustain communities’ motivation and cohesion.

26.2 Participating communities must be helped to understand linkages with broader environmental benefits. At the initial stage, the project had to deal with resistance to project interventions due to community and cultural beliefs and practices. The project revealed that changing local mindsets required patient and tireless efforts, as communities had a steep learning curve. However, once the positive results of the new technologies became apparent, communities became quite cooperative and eager to expand their activities.

26.3 A holistic approach is required (e.g., in this project, at the entire catchment level) to achieve economic and environmental sustainability.

26.4 The project performance was significantly affected by unavailability of anticipated co-financing as well as low institutional capacity for project implementation, which required close supervision and an adaptive management approach at all stages of project implementation.

26.5 In addition, modifying Project design (e.g., reducing scope and areas of intervention) should be considered early on to ensure that the adjusted Project can achieve tangible results.

27. African Regional Capacity Building Network for HIV/AIDS Prevention, Treatment and Care Project; Tanzania; No.: ICR1452

27.2 Respond to changing technical landscape. At project design, the Bank was the leading actor in the field and national systems were not mature. Working with regional institutions of learning worked. This allowed the flexibility of updating course content with cutting edge information, kept the costs low, and strengthened these institutions.

27.3 The regional approach was a high win. This allowed the exchange of best practices across different countries, established relations between practitioners and, most importantly, responded appropriately in helping alleviate the regional epidemic. The Bank's internal structures discouraged the much needed restructuring.

28. Emergency Multisector Rehabilitation and Reconstruction project; Democratic Republic of Congo; No.: ICR1506

28.1 A distinction needs to be made between emergency operations caused by natural disasters and those in a post-conflict context. In the first case, institutional capacity and policy tend not be major issues. In the second case, because of the destruction caused by conflicts, the process starts off with much lower levels of capacity and knowledge. This argues for the need to include institutional reforms, capacity building and sector strategy formulation as was done in this project in order to start laying the foundations for the future.

28.2 Strong commitment from management in terms of resources and continuity within the task team is a key determination of successful implementation. The use of Non-Governmental Organizations proved to be an effective and efficient way of implementation the social activities at the grass root level. This should serve as a lesson and good precedent.

28.3 Even in emergency situations, an effort should be made to address sustainability issues at the project or program design stage. If not done, the investments made could be jeopardized over the long term. If this is not discussed during project preparation and appraisal, it should be included.

29. Second Economic Management and Governance Reform Grant Project; Central African Republic; No.: ICR1682

29.1 After a long period of internal conflict, international support is likely to be needed for some time, both in terms of security forces and financial resources. Demobilizing non-government armed forces takes time and requires resources. The population typically suffers from a major deficiency in basic needs.

29.2 Focused reforms are needed to restore the government's basic financial functions and increase their transparency. The reforms and outcome indicators need to be realistic, consistent with the government's capacity, and supported by Technical Assistance (TA) as needed.

29.3 A Development Policy Operation (DPO) is an effective instrument for addressing key issues in restoring a functional government and contributes to setting the stage for deeper reforms. TA should be available to help implement the reform program and build up capacity in key areas.

29.4 A close dialogue with the government should be maintained, through frequent supervision missions and field presence. A series of DPOs allows the government to take on the parts of a program outlined in the Poverty Reduction Strategy Paper (PRSP) and the Country Partnership Strategy (CPS) that are appropriate in time and manageable. This approach may be better than laying out a lengthy, strict, programmatic series of credits, which may be incompatible with changing circumstances and could lead to an inappropriate sense of non-achievement.

29.5 The documentation of each DPO should indicate clearly ultimate objectives and achievements in prior operations.

30. Human Resources Development Project; Rwanda; No.: ICR1455

31. At the time of project preparation, the education sector in Rwanda faced numerous challenges, which were complex and diverse. The post-conflict nature of the country meant there was significant political and social pressure to address a multitude of issues through the HRDP project. However, one of the lessons learned from the former education project was that project objectives need to take account of local implementation capacity.

32. Project activities should not commence until the technical and implementation capacity had reached the required level. In most cases, post-conflict States have weak or dysfunctional institutions and policies. In the case of civil works activities, there may be very few local contractors within the private sector who could provide the required services given the risks of working in a post-conflict environment.

33. The risks for international contractors may be too high to warrant their involvement. Project designs which involve community driven development processes need to nurture and support the huge potential that exists at the community level.

34. Bank task teams should weigh the impact of adding new activities towards the end of the project period.

31. Africa Region – Nile Basin Initiative, Shared Vision Program Project; No.: ICR1493

31. Even highly complex regional programs can succeed if there is political will, participation, prudent institutional and technical design, and strong partnerships.

32. The location of the core task team is critical for effectively managing a regional operation.

33. Designing and managing an ambitious and complex regional program requires highly experienced technical, communication, legal, social and political skills, both on the parts of the clients and the bank.

32. Protection of Basic Services Project; Ethiopia; No.: ICR1569

32.1 Efforts must be made to ensure government ownership. Government of Ethiopia (GOE) demonstrated strong ownership for most PBS activities, contributing to the overall successes observed in the development outcomes.

32.2 There is a trade off between a robust preparation process (including fully addressing implementation readiness conditions) and the need to be adequately responsive to the immediate fiscal, economic and social needs of the situation.

32.3 It is necessary to prioritize critical and pressing country needs to ensure that there is an adequate risk assessment with appropriate mitigating actions, as well as a strong implementation support structure.

33. Public Sector Reform and Growth Grant Project; Gambia; No.: ICR1530

33.1 The participation of several donors in a multi-year program for budget support can increase the level of resources and focus them on a common objective but this arrangement is more vulnerable to annual changes in donor priorities and requires timely communication and coordination.

33.2 The sequential donor financing of programmatic budget support is inherently more risky than co-financing of a single annual operation because planned resource allocations may change by the time the donor's annual program for a given country is approved.

33.3 Measurable performance indicators aligned with the capacity of the Borrower are necessary but not sufficient elements of timely, effective Monitoring and Evaluation (M&E). It needs to be accompanied by a clear delineation of reporting responsibilities.

34. Trade and Investment Gateway Project; Ghana; No.: ICR1460

34.1 Timely restructuring of a project can lead to a successful implementation of a project. When a project becomes very difficult to implement it's usually a sign that there is a need to revisit the fundamentals of the design and its concepts to enable retrofitting or adjustments of key features of the project for it to work.

35. 2nd Economic Governance and Recovery Grant Project; Cote d'Ivoire; No.: ICR1483

35.2 The importance of careful risk analysis, which takes into account the political and social dimensions, identifying and weighing the relative importance of special interests. This is especially important in the case of fragile transitional coalitions.

35.3 Given the difficulty of getting a consensus on difficult issues, it is essential to limit the focus of reform to key functions of the Government and to sectors vital to economic recovery and growth and on policy measures which ‘open the door’ to a deepening of reform.

35.3 Once reform has found its footing, it is important not to slow momentum but rather deepen the reform, rather than broadening the scope (which risks over-loading decision makers’ focus and span of control while stretching high-level human resources).

36. Second Road Sector Development Program Project; Ethiopia; No.: ICR1221

36.1 Training and capacity building are not as effective as they could be, as long as civil service reform is not on the agenda. Inability to retain staff results in high staff turnover; as a result, investments in human capital do not have much longevity, since junior staff tends to move on to other opportunities elsewhere after being trained.

36.2 To effectively address constraints to accessibility and unmet transport needs, the government of Ethiopia (GOE) and its donor partners will need to look beyond the provision and maintenance of road infrastructure, toward a more holistic approach to the sector, including services.

37. Gateway Project; Gambia; No.: ICR1399

37.1 It is critical to examine the support for the program in a country prior to incorporating such activities in a project.

37.2 Despite the indications of strong commitment by GoG during project preparation robust analysis should have been undertaken on the social impact on its stakeholders. Resistance during implementation emerged from other stakeholders including management of the enterprises to be divested.

37.3 The Gateway Project had two Project Coordinating Units (PCUs), which did not work and reverted to one PCU. While a PCU can often be inserted into an institution, project coordination should not distract the agency from its other functions.

37.4 Field based task teams (including Task Team Leaders [TTLs]) are an effective way for the Bank to provide continuous support to the Borrower.

37.5 There were five TTLs over the life of the project. This high rotation resulted in a slowdown in the project implementation pace. It was when the TTL was field-based and available over a few years, that the project consistently achieved delivery against Project Development Objective (PDO) Key Performance Indicators (KPIs).

38. Water Sector Project; Niger; No.: ICR1231

38.1 Private sector participation is a feasible delivery option even in a poor country like Niger, which was able to attract international competition for urban water services and at the same time was able to promote local private operators for rural facilities.

38.2 Project Coordination Units (PCUs) should not be assimilated to Project Implementation Units (PIUs). PCUs do not act as substitutes to Government departments or agencies and do not hamper capacity building; they are simply a very effective tool for coordinating multi-agency, multi-donor programs.

39. Urban Infrastructure and City Management Project; Rwanda; No.: ICR1375

39.1 The city contract approach can be an efficient tool to facilitate improvements in service delivery and municipal and urban management, and to accompany decentralization, even within a limited timeframe.

39.2 Land management is highly sensitive and the needs for slum upgrading immense. A focus on the needs of the affected populations, with clear demonstration of the benefits of a market-based land valuation in determining compensation led to a changed approach to upgrading and resettlement.

39.3 With the majority of the urban population living in slums, recognition of their needs and rights is an important element in sound urban land management.

40. Community Development Program Support Project; Cameroon; No.: ICR976

40.1 In communities with limited technical capacities and weak control mechanisms, financial decentralization should follow a phased approach. The first phase of the project successfully tested a mechanism for channelling resources to communes and communities. This success gave sufficient confidence to the Government, which enacted a decision to transfer areas of competence and associated resources for nine ministries to the communes.

40.2 One of the greatest achievements of the project was to create a new environment for local development planning, financing and management, leading to a greater command of the local development process by communities, and to an improved delivery of basic social services. Simplicity in the process of preparing local and communal development plans helps strengthen ownership and improve implementation.

40.3 Systematic use of simplified guides and procedural manuals is critical for quality insurance and greater fiduciary compliance at local level. Sectoral ministries have a critical role to play in ensuring quality control of infrastructure realized through the local development process.

40.4 Using in-kind contributions of beneficiaries provides a greater flexibility in financing local development; however, this mode of financing is also source of quality issues and implementation difficulties.

41. Financial Management, Transparency and Accountability Project; Malawi; No.: ICR1458

41.1 The project design should be simple and designed for easy implementation and take into account the absorptive capacity of the country and the prevailing political context. For efficient use of available resources in a country, project design should take account of grant financing from donors, which however, requires more flexible design of capacity building activities.

41.2 Changes in political economy are not predictable, but could be better adapted to if project design allowed greater flexibility in implementation. Constantly evaluate and adapt to change in circumstances to better manoeuvre public management reform programs.

41.3 Candidness in project supervision reporting is essential, as is early management attention to emerging problems.

41.4 A project should not proceed with Board presentation unless a full technical appraisal and feasibility study of its components have been conducted.

5. Inter-ministerial and donor coordination is crucial to the success of project implementation.

42. Food Crisis Response Development Policy Grant Project; Burundi; No.: ICR1379

42.1 The respective obligations of the Government and the World Food Program (WFP) should have been more clearly defined. Ideally, a Memorandum of Understanding (MOU) specifying what should be done might have provided the legal framework and the incentive for a more expeditious treatment of school feeding activities and quick adoption of practical measures to handle unforeseen operational problems.

43. Fifth, Sixth, and Seventh Poverty Reduction Support Credits Project; Uganda; No.: ICR 1121

43.1 The PRSCs are an excellent vehicle for dialogue, not only between the Government and Development Partners, but also between the Ministries. This is a well-known benefit of PRSCs, but it bears repeating. The ministries reported widely that the PRSC series created a valuable forum to discuss cross-cutting issues such as public service reform and decentralization.

43.2 Pragmatic use of specific, measurable, attributable, and relevant, time bound (SMART) criteria is essential. For example, if an indicator is attributable, but not time-bound, then it cannot be useful to measure results of a program. Conversely, an indicator may be time bound, but too general to be attributable to the activity whose results are being measured. The indicators for these projects were driven by the Government's results framework and Bank's desire to foster ownership. However, the Bank should have recognized that these were not suitable for the Bank's goal of measuring results. One solution might be that gradations should be better specified and nuanced, and actions and targets better calibrated, so that it will be clear that the policy agenda is moving forward while also moving at a rate where there is sustained government ownership and an actual ingraining of the policy reform.

43.3 The Bank may also need to better temper its ambitions about the pace and scope of reform that can realistically be achieved each year, recognizing that many reforms will be incremental, rather than substantial.

C. Cranfield University Case Studies

The following table contains details of the case study reports produced as part of Cranfield University Masters in Security Sector Management. Details include project title and key lessons learned. All projects were in Ethiopia.

44. EDCSC Future HQ Project

44.1 Project planning and effective risk identification and management are essential disciplines for the Customer to follow, especially at the early stages of publically funded projects.

44.2 Sponsors, and particularly government organisations, should maintain the disciplines associated with the early phases of the project life cycle such as project definition, project set up and launch, and not simply rely on the ‘letter of the contract’ or Contractor good-will to make the project happen

44.3 The transfer of risk to Contractors is an illusion that a contract provides. In reality, government sponsors ultimately bear the consequences of risk as was evidenced here in respect of both delay to the construction of the facility and consequential costs.

44.4 Contracts that involve third parties from overseas, either as prime contractors or sub-contractors are particularly prone to misunderstanding and misinterpretation. Time should be taken to ensure that all parties are totally familiar with the project constraints before entering into any formal agreement.

44.5 A phased approach to delivery (e.g., design contract prior to the main contract), helps ensure clarity and reduce risk.

45. Hara Meda Airfield Rehabilitation Project

45.1 The feasibility study allowed detailed understanding of location, geology and geography that informed decisions on construction materials, processes and procedures.

45.2 The project team was led by the Ministry of Work and Urban Development and comprised a mixed skill set from both uniformed and civilian organisations.

45.3 Project governance included a monthly project steering committee, led by the minister, and with regular site visits.

45.4 Scheduling (including interdependency management), reporting and control and monitoring processes were established and effective.

45.5 Improved stakeholder management involving the local farming community would have prevented time consuming and costly legal proceedings.

45.6 Risk management is an essential discipline if the adverse impacts of unforeseen events are to be minimised.

46. Berahle – Dallol Road Project

46.1 Contracts between government agencies are often not subjected to the same degree of commercial rigour as those between commercial organisations. If waste is to be avoided and effective and sustainable project performance is to be achieved, agreements between government agencies must be subject to the same processes and procedures as those contracted with commercial organisations.

46.2 Greater focus on ‘front-end’ activities is essential and the temptation to slide into tacit understandings without clear understanding of the risks and issues is essential.

46.3 Early engagement with, and active participation from local communities as beneficiaries of such projects, is essential if problems later on are to be avoided.

47. Western Gode Irrigation Project

47.1 Early, careful consideration of the local community is essential in gaining buy-in to projects in rural and unstable regions. Keys to this process are community elders and clan leaders.

47.2 Time must be taken to identify key local stakeholders and communications planning implemented to ensure that the purpose, benefits and contribution of the project to local communities are well understood.

47.3 Local community can be an important source of labour, and local militia can have a role, together with the army, in protecting isolated projects from insurgency attack.

47.4 Project managers leading projects in environmentally and politically challenged projects must have the appropriate level of competence in respect of leadership, team work and stakeholder management.

48. Tendaho Sugar Project

48.1 Vague and defective agreements inevitably lead to issues during implementation. More time should be spent hammering out the detail of intra-agency agreements and those between government and foreign investors.

48.2 Contractual commitments should only be made in full view of the assumptions made and risks associated with those assumptions. Contract signature does not automatically translate into timely and effective mobilisation.

48.3 Appropriate management capacity and competence is required to ensure that all resources, including human resources, are effectively managed and utilised. Both the project management and that of the delegated government bodies must carefully consider skills and experience before key roles are assigned.

48.4 Regular control, monitoring and evaluation of project status are important, especially when corrective actions are required by individuals and agencies beyond the project management team. Lack of information at the right time, to the right people means that corrective measures will not be implemented or perhaps even defined.

48.5 The ‘people side of projects’ must not be neglected nor ignored. Delivery of the hardware itself is insufficient if the project is to deliver operational benefits.

48.6 Development projects should make more explicit the practical participation and cooperation of local community and its employees as part of the project plan and implementation activities. The internal and external environment must be cultivated and developed to make this a natural part of project delivery

49. Addis Ababa Housing Program (two reports)

49.1 A high level of engagement and participation by all stakeholders are keys not only to gaining buy-in and agreement, but also in leveraging that support in implementing the program.

49.2 An element of cost-sharing in the shape of formal or informal credit and saving associations may have helped in creating a positive sense of ownership, and would have reduced confusion and uncertainty about the objectives and associated benefits.

49.3 In public programs led by public office - in this instance the Addis Ababa City administration – there is an opportunity to deliver benefits beyond the obvious infrastructure. However, this requires a high level of cooperation and collaboration between city and government agencies and offices, which again requires effective management and leadership.

49.4 Programs of this magnitude involve different contractors operating concurrently and require efficient procurement processes and supply chains. Program management resources, structures, and information management procedures are essential for effective integration of designers, purchasers, suppliers, and other service providers.

49.5 The capacity of the customer/sponsor organisation to program manage must be assessed and taken into consideration in determining the pace of the program.

50. Conflict Prevention and Resolution Project

50.1 A greater focus on pro-active rather than reactive action is required. Prevention in inter-ethnic conflict is a new paradigm that requires local security forces and officials to introduce new ways of information gathering to deter and reduce ethnic group conflict.

50.2 New training methods aimed at a greater ‘community policing role’ are required in preventing inter-ethnic conflict. Police recruitment must be more inclusive and draw from the full diversity of ethnic groups.

50.3 Project scoping must include consideration of the agreed implementation strategy. The needs of both donor and beneficiaries should be considered as well as the expertise required for implementation.

50.4 Planning should include the analysis of the inter-linkages between clusters of objectives and related tasks. The planning process also needs to take into consideration the analysis of the capacity and intentions of key stakeholders and their wider institutional context.

50.5 Active participation of all stakeholders is a function of the local setting. Actors that may be involved are local communities (as problem owners), donor organisations, implementing organisations, NGOs and local government officials. The engagement and contribution of local elders and communities to conflict prevention and resolution are paramount.

51. Burbax Rural Development Project

51.1 Building the capacity of local people as beneficiaries must be given due consideration. Cross cutting issues (e.g., harmful customs and practice, HIV/AIDS) have to be identified and assessed if they are to be adequately dealt with.

51.2 Effective communications among all actors (owner, funding agency, local political bodies, line offices, etc.) is required if timely decisions and necessary actions are to be taken.

51.3 The achievement of the project objectives is highly dependent on the strength and commitment of the project owner (e.g., ORDA).

52. Woreda Integrated Rural Development Project

52.1 The project took a ‘top down,’ supply-driven approach in so far as the high level project objectives were defined at the top, and subsequently used to identify local needs from the bottom.

52.2 Detailed consultation with the public, including seminars and workshops to communicate the overarching aims of the project to those (beneficiaries) most likely to be affected.

52.3 Both pre-feasibility and feasibility studies were conducted before the commencement of the main delivery phase. However, early risk assessment was limited to general threats. As a result, for example, the impact of increases in costs of raw materials (local and imported) has resulted in unforeseen project overspend.

52.4 Although annual reporting and monitoring processes were in place, these were inadequate to support control and decision making in the short term.

53. Pastoralist Community Development Project (Phase 1)

53.1 Investment in infrastructure projects must be accompanied by investment in training to allow local communities to operate and maintain the following hand over to those local communities. Community capacity building must be an integral part of such projects.

53.2 The lack of feasibility study resulted in unrealistic schedules and budgets.

53.3 Projects spread across large geographical areas especially where local procurement is required or where there is a lack of project management know-how and competence require a more hands-on approach from owner and donor.

D. MSc Delegate/Practitioner Survey

Profile of relevant experience

There was a broad range of experience across the sample. Some 60% of the delegates—all acting in the capacity of Task Team Leader—had been involved in less than five projects, ranging in value from US$50,000 to US$5 million, with most project durations being less than 12 months. An additional 20% of delegates, acting as a Head of Mission or a National Project Coordinator, had been involved in 10 to 20 projects, with values not less than US$1 million and typically longer in duration (1 to 5 years).

Success and Success Factors

Some 80% of responses stated that ‘Most’ projects had been successful, with 10% claiming ‘All’ and the same proportion stating that only a ‘Minority’ had been successful. The most important role (50%) was reported to be ‘Government/ Sponsor,’ with NPS the second most important role (20%). Other roles referred to were Project Director and Technical Experts. The most important Success Factor (80%) was that the ‘Project was linked to Strategy.’ The second most important Factor was the presence of a ‘Robust Business Case.’

Relevance of Project Management to Security Sector

90% of delegates responded that project management was either ‘Extremely’ or ‘Very’ relevant to the Security Sector, with the balance stating that it was ‘Relevant’.

Improvement Actions

There were a number of comments that suggested that the “practice of project management was not frequently used” and that “decisions are not project or program based,” with “most systems relying on formal bureaucracy.” In addition, three key themes emerged from the comments received in respect of improvement actions.

First, there was recognition of the importance of effective stakeholder engagement and the need to “involve people from the start,” especially with regards to demonstrating project benefits. Second, there was a need to establish “good organisational structure” and related to that, the need to increase capacity in terms of the project management knowledge and skill. Finally, the need for effective and consistent performance monitoring and evaluation coupled with the provision of supporting systems.

____________________

1 The 2010 Progress Report on the achievement of Ethiopia's MDGs confirms that Ethiopia has been assessed as being ‘on track’ to achieve five of its eight MDGs and ‘likely to be on track’ to achieve the other three goals. The report indicates that no goal ranks under ‘off track,’ (See Ethiopia: Trends and Prospects for meeting the MDGs in 2015, Report published by the Ministry of Finance and Economic Development, Federal Democratic Republic of Ethiopia, September 2010.

2 The use of the terms ‘donor’ and ‘sponsor’ and the paradox of ‘donor-sponsored’ projects and programs is explored later.

3 The use of the term ‘programming’ appears to be commonplace in SSR literature. This is generally taken to mean the allocation of budget to projects, usually over a given time period. However, and in line with the definitions given earlier, the terms ‘program’ and ‘program management’ are far reaching in their meaning. There does appear, therefore, to be the potential for confusion when referring to ‘programming’ in the context of ‘program management,’ a point we refer back to later.

4 Indeed, the World Bank's commissioning of the study entitled ‘Voices of the Poor’ in 1999 served as the first form of empirical research that underscored the importance of the relationship between security and development.

5 There is a total of 1,255 Project Completion Reports contained on the World Bank project database; of these, 385 relate to African countries and 11 were Ethiopian projects. The PCR report was replaced by the Implementation Completion and Results Report (ICR) in c.1995.

6 No distinction is drawn between ‘program’ and ‘project’ in the publishing of Project Completion Reports and Implementation Completion Reports.

7 The reports analysed covered a range of small, medium, and large projects, ranging from 0.5 year to 5 years in duration, and from US$0.5million7 to US$427 million in value. Sectors covered included roads, housing, community development, rural development, irrigation, sugar, and conflict prevention and resolution.

8 Further research is underway by the authors in testing the application of PROFIDE using such an approach.

9 It is important to note that the lessons learned narrative is a precise of the PCR content with specific focus on project and program management and process, rather than financial, economic or technical content.

10 These projects have been numbered to follow from Section 4.2 for ease of reference in the analysis and discussion.

11 Here again, the lessons learned narrative is a precise of the ICR content with specific focus on project and program management and process, rather than financial, economic, or technical content.

©2012 Project Management Institute

Advertisement

Advertisement

Advertisement