Project management in academia
friend or foe? An exploratory study of the social sciences and humanities
JR Turner, visiting professor, The Centre for Project Management, University of Limerick (Supervisor)
Disclaimer: The research for this paper was conducted as a project in partial completion of the requirements for the Masters in Project Management at the University of Limerick. The author is solely responsible for the content of the paper and the views expressed therein.
This paper will look at the application of project management in the academic sector in Ireland, focusing on the “soft” disciplines of the social sciences and humanities. The findings confirmed that project management is increasingly being imposed on the academic sector by broader concerns for public sector accountability. With the increased level and number of funding opportunities for research during the past 10 years, funding bodies are increasingly using programme management to provide a coordinated approach to administering funds and, in turn, awarding funding to applications with management as an inbuilt part of the project. This has resulted in an increased administrative burden on academics and forced institutions to adapt their structures in response to requirements, thus increasing the levels of bureaucracy involved in conducting research.
Limited research has been conducted in the area of academic research project management. This research is restricted to Ireland and will look at the strong culture of the academic profession that is focused on retaining autonomy. It also looks at the impact of increased accountability on the day-to-day activities of the academic researcher and considers how they are adapting to these changes. Finally, it considers how project management can be applied to enhance the research process so as to provide a useful support without impacting on creativity and innovation.
The academic sector in Ireland consists of higher education institutions, such as universities, third-level colleges, institutes of technology as well as research institutes. Deem describes academics as “… expert knowledge workers engaged in teaching and research as part of the knowledge economy and knowledge society” (2007, p. 67). Drucker (1966) first coined the term “knowledge worker” to distinguish between the manual worker who works with their hands providing new goods or services and the knowledge worker who works with their mind to produce new ideas, knowledge, and information. Academics are knowledge workers that engage in teaching and research activities, the latter being dependant on their ability to be innovative and creative. They also form part of the policy-making community, advising governments and policy makers.
Career progression within the sector is based on a researcher's ability to produce high quality research, which is determined by peer review and a researcher's ability to conduct innovative and creative research. Much of the literature on what motivates the “knowledge worker” agrees that it is his or her ability to self-organize and develop, as well as take ownership of his or her own research rather than any overarching desire to obtain more resources (Marginson 2008; Considine 2006). It is also proposed that motivation can be hampered by bureaucracy, which is viewed as eliminating “an individual as a thinking and valuing person” (Marginson 2008; Hayek, 1952).
Against this background, it should be noted that significant changes have taken place in the University sector since the latter half of the last century. Up to the 1960s, universities were relatively autonomous organizations that operated in an environment of consensus decision making in the absence of hierarchy, whereby decision making was considered inefficient and almost impossible (Livingstone, 1974). The difficulties of imposing change in older universities is well documented, and for the most part attributed to the constraints imposed by centuries’ old governance arrangements and authority structures, where cultures and norms are deeply rooted and are averse to contemporary management structures (Deem 2007; Kellaway 2006; Salmi 2009).
Trust forms an integral part of academic relationships, setting it apart from most sectors. It is suggested that were that trust to erode, that the academic profession would erode (Power 1997; Schimank 2005). Marginson (2008) notes that the impact of New Public Management1 (NPM) on the profession suggesting that some of its management techniques tend to be incompatible with academic freedom. This originates from the desire for public sector accountability, which is being driven by a desire for best practice, which is imposing change within the academic sector at both national and international level.
In the UK2, it is thought that the election of a new conservative government in 1979, which focused on public service efficiencies as well as the recommendations of the Jarrett Report (1985), mobilized change in the workings of UK universities. This led to an expansion in the number of universities, through the government's commitment to expand the number of students, and resulted in an increase in the workloads of many academics. Not only was there a “shift from autonomy and consensus decision making towards regulated autonomy” (Hoggett, 1996) and “soft bureaucracies” (Courpasson, 2000), there was also a shift from collegiality and self-governance to “institutionalizeddistrust” (Reed, 2007).
Very little has been published on the impact of accountability mechanisms in the Irish case, however numerous reports reference the impact of legislation and government policies on the sector (Higher Education Authority 2007; Barrett 2004; Connell 2005). From 1986 to 1992, government policy sought to increase the participation of lower socio-economic groups in third-level education through targeted grants. In 1996, the government abolished third-level fees, increasing the reliance of the sector on government grants by eliminating their main source of other income (Barrett 2004). In 1997, the Irish Universities Act was enacted and sought to clarify this new arrangement, allowing universities to generate other sources of income without impacting on their government grant, as well as formalizing the management power given to university Presidents, a shift from previous flat structures of the Irish system that involved consensus decision making and collegiality. Barrett (2004 p.6) criticizes these changes, arguing that academics chose “the traditional collegiate system” of the University over a career in the more bureaucratized private sector or civil service, and suggests that new managerialism caused by the Act has led to the “widespread demoralization of academic staff”.
The Higher Education Authority (HEA) noted in their Report on Quality in the sector that the environment in which it operates has become increasingly more complex and demanding as it tries to impose accountability and regulatory responsibilities that are in line with international best practices (Dunne et al, 2005). Literature in the area suggests that the role of the academic may be under pressure as a result of poor resources and structures to facilitate the process and notes that universities are more reliant on the intellectual abilities and commitment of its academic staff than any other industry (Pienaar, 2009 p.249). He suggests that the increase in accountability requirements and subsequent internal process changes have added to the levels of stress experienced by academic staff. Winefield et al (2008) supports this argument and refers to the increased pressure to obtain external funding whilst making “their scholarship more relevant to society (which in some cases can actually undermine basic research), to publish in the world's leading journals to enhance the ranking of the university and at the same time to maintain excellence in teaching”. They suggest that the increase in accountability requirements is undermining the autonomy previously experienced by academics. With limited time and increased workloads, academics need help to enable them to meet the increased demands for accountability.
Having identified the constraints that academics are working under, the aim of this research is to demonstrate that project management principles can be applied to the “soft” disciplines of social sciences and humanities, so as to provide a useful support without impacting on innovation and creativity. In order to do this, we need to consider the following:
- What sort of help do academics need to better achieve their objectives?
- What sort of help can project management give?
- How can we judge how well project management is contributing?
What sort of help do academics need to better achieve their objectives?
Fox notes that Institutions do not do research, individuals do, but institutional conditions do affect productivity (in Taylor 2006, p.3) whilst Kenny (2003) suggests that:
“For innovation to occur, the culture, structure, and processes of an organization need to be supportive of multi-skilled teams, which operate largely autonomously, yet their activities are linked to the organizational priorities and outcomes”.
Bain (1999) argues that the context in which innovation occurs needs to be addressed if the university wishes to exploit any long-term benefits from the outputs of the research project. In other words, researchers require institutions to offer conditions that will not impact negatively on their ability to engage creatively with their discipline, whilst organizations require systems that will ensure that this is achieved whilst meeting organizational objectives and ensuring accountability.
Ernø-Jjølhede (2000) describes the research environment as one involving a constant balancing act of paradoxes:
- A researcher's desire for autonomy over his or her work and consensus decision making vs. the need for control mechanisms to meet budget and time constraints;
- The conflict between both cooperation (joint project goals) and competition (getting published, promotion opportunities, grants etc) on the project;
- The requirement of predictability of project output vs. unpredictability of project outcome and follow-up opportunities, which he attributes to output being different to that initially envisaged yet considered to be qualitatively superior or more useful for the projects intended purpose;
- The difficulty of interpreting progress on research projects vs. the need to report to the client with certainty and the need to act as if there is certainty when making management decisions;
- The lack of knowledge asymmetry between the project manager and the individual researcher, where often it is the latter who is better placed to make decisions regarding his or her work;
- The need to take risks to be innovative vs. the need to reduce risks to ensure that a project is delivered on time and within budget.
It is these paradoxes that effective project management should seek to address. However, the application of project management to the academic environment is further complicated by the general acceptance that academics are “trained as critical thinkers and can apply this to anyone trying to manage them” (Deem 2004). Whilst there is a need to ensure accountability both to the university and the sponsor, the research project life cycle is not always straight forward, which makes implementing traditional management practices problematic.
What sort of help can project management give?
“No good book was ever written on command, nor can good teaching occur under duress. And yet, conceding this, the fact remains that left entirely to their own devices academic communities are no less prone than other professional organizations to slip unconsciously into complacent habits, inward-looking standards of quality, and self-serving canons of behavior. To counter these tendencies, there will always be a need to engage the outside world in a lively, continuing debate over the university's social responsibilities.”
(Bok, 1990, p.111)
The requirement for increased governance, through accountability and transparency, has been driven by a public sector agenda that is concerned with best practices as a means to ensure stakeholder satisfaction. Crawford and Helm (2009) argue that project management is invaluable in meeting government objectives as a means for ensuring public sector accountability. Their study of project implementation across four public sector organizations3 was reviewed against the following themes (ibid, 2009):
- Accountability and transparency;
- Control and compliance;
- Risk management;
- Consistency and delivery;
- Ensuring value for money; and
- Stakeholder engagement.
They suggest that project management provides a tool through which accountability can be implemented to meet stakeholder expectations, a view supported by Thomas and Mullaly (2007, p. 81) who proposed that “investing in a particular form of project management provides a specific type of benefit in a specific context”.
Management in academic projects is two-fold, firstly there is the research management of the project and secondly there is the project management of the research project. Such projects can vary in size and budget. The type of project also dictates the reliability of the project plan and the ability to effectively meet project deadlines. Academic researchers are obliged to comply with the clients’ guidelines if they wish to avail of funding, which is aimed at increasing both the quality of research output, as well as ensuring a coordinated approach toward research globally. Project management acts as a tool to facilitate this process.
Types of projects engaged in by academics in the social science and humanities.
Various types of research projects are conducted in academia, from one-of-a-kind focused on creating new knowledge, to projects focused on applying knowledge in new ways. Within the social sciences and humanities, there are various types of projects that are undertaken, for example, top-down commissioned research with defined milestones and specific goals, or bottom-up research where the goals are clearly defined but the methods proposed to achieve them may not have been done before. For the most part these would be categorized as Type 4 Air projects where goals can be loosely defined, as are the methods of achieving them (Payne and Cochrane, 2003).
Crawford and Pollack (2004) found in their review of the literature that the distinction between hard and soft aspects of projects could influence project success. They recommend the identification of a framework for identifying and addressing each aspect of the project to enhance appreciation for the complexities involved in a project and “legitimizes questioning the standard application of the more readily available hard approaches to project management.” (ibid, 2004). Turner and Cochrane (1993) classify projects according to their Goals and Methods Matrix and propose the use of different planning and control approaches for different types of projects. They suggest that using the wrong planning and management method will increase the likelihood of failure on the project.
Aligning project goals to the organization.
The benefits of a common approach to project management within an organization are widely recognized. Payne and Turner (1999) argue that this should be limited to the strategic level where each project has a strategic plan that is in line with organizational norms. Yet like other sectors, organizations that engage in research must plan their long-term goals and set short-term goals and projects around these (Kenny, 2003). This will facilitate the accountability process, as universities are being held accountable for their activities and legislation requires them to illustrate that quality assurance procedures are in place to ensure that governments are getting value for money.
The concept of agile project management in academic research.
Much of the traditional project management literature focuses on structured planning and control methodologies. In traditional projects, the lifecycle of the project follows a predefined course that enables a project to be fully planned from the outset. In a research environment, projects do not or perhaps more accurately, often cannot, follow a predefined set of processes. Bates (2000) argues that successful projects in academic environments requires
“… a much looser project management approach that specifies responsibilities and completion date but does not attempt to quantify and qualify every activity at micro level”.
Agile Project Management (APM) has been pioneered by the software industry in response to the challenges posed by traditional methods, specifically to the unstable project environment of the industry that relies on individual creativity and skills of employees, not unlike the academic environment. Although still an emerging topic, APM provides a set of descriptive guidelines as opposed to A Guide to Project Management's Body of Knowledge's (PMBOK® Guide) more prescriptive methodologies. Alleman (2003) highlights communication as the underlying value of APM, based on tumultuous environment of agile projects.
How can we judge how well project management is contributing?
The value of project management on research projects can be measured by identifying those elements that make the project successful. Turner (2009, p.47) proposes that there are two aspects that define project success: success criteria and success factors. He describes success criteria as the dependant variables, which decide the successful outcome of a project, and success factors as the independent variables that influence the achievement of the success criteria.
A study conducted by Waterbridge found that when project stakeholders interpreted the project in the same way, it was more likely to be successful, whereas if they interpret it differently, then failure was imminent (Turner, 2009, p.49). Much research has been conducted on identifying critical success factors considered as being appropriate for measuring project success (Pinto and Slevin, 1988; Cooke-Davies, 2002; Muller and Turner, 2005).
Another method of judging the success of project management in academic research projects is through benchmarking business processes and performance against widely accepted best practices across the sector. Research on maturity models offers the most practical solution to assessing the performance of academic organizations in the area of research project management. It is widely agreed that they all originate from the principles of the Software Engineering Institute's (SEI) Software Capability Maturity Model (Pennypacker and Grant, 2003; Rosenstock, Johnston and Anderson, 2000).
The purpose of this model is to provide a benchmark for software organizations to assess their organization's maturity and to develop frameworks and policies that will enable them to advance through the levels. Figure 1 outlines the key process areas as identified by SEI that organizations are required to achieve before advancing through the levels of maturity. They argue that an organization cannot skip levels, as each level lays the foundation for the next.
Figure 1: The Key Processes by Maturity Level (SEI)
The unstable environment of the software sector is similar to the academic research sector, in that, project goals tend to be loosely defined, however they must operate within a structured reporting environment promoted by a society that is driven by broader concerns for accountability.
Given the exploratory nature of this topic and the lack of existing available data, a case study approach offered a distinct advantage over other research methodologies. Three organizations that engage in academic research activities were asked to participate. Two universities and one research institute agreed to engage in the process. The aim was to interview one senior administrator, and three academics that engaged in research, but were at different stages in their career within each organization. Participants were found using the snowball effect4 and each were invited to participate via email, where a high level overview of the research topic was outlined to give the participants an opportunity to prepare for the semi-structured interview. Interviews were semi-structured and centered on specific areas to allow for easy comparison of findings5. Table 1 provides an overview of the participants of this research.
Table 1: The Participants
|Carrer Level||University Alpha||University Beta||Research Institute Gama||Funding Bodies|
|Senior Lecturer/Senior Researcher||1||2||1|
University Alpha is one of Ireland's largest Universities, whilst University Beta is significantly smaller in terms of both student and staff population. Both Universities engage in teaching and research activities. Research Institute Gama is a “think tank” engaged in research, whose remit is to contribute to public policy whilst remaining independent of government. Unlike the two universities, it is focused solely on research activities and must competitively tender for funding in the open market to support its activities.
The majority of funding available to the academic sector comes from public sector sources. Two funding bodies, one focused solely on the provision of funding for research in social sciences and humanities (Organization A), and the other, has responsibility for environmental issues in Ireland and engages in the provision of funding across all disciplines of research (Organization B). Whilst a member from each organization was interviewed, the majority of the material used in the case studies has been taken from published material provided by the participants and available on their websites.
To ensure accuracy of content, each participant had an opportunity to review his or her individual case study6.
Challenges and constraints
The time frame given to produce this study was insufficient to capture the widespread use, or lack of, project management processes in the projects engaged in by the soft disciplines such as the social sciences and humanities. Other challenges and constraints were also identified:
- With the limited availability of literature in the area of research project management, quantitative analysis was not considered feasible due to a lack of available “known” information on this particular area.
- The time period for conducting this study conflicted with peak holiday season within the university sector, which restricted access to some participants. This mainly affected University Alpha, where a junior member of staff willing to participate could not be identified and was therefore excluded from the process.
- During the course of the interviews, it proved challenging to identify the success in implementation and acceptance of University procedures by academic staff. Senior administrators interviewed as part of this research generally explained their documented policies for research project administration. Within the responses provided by the senior administrators, the level of other information provided seemed to vary according to the administrator's perception of the level of internal flexibility regarding his or her own project management policies.
- Due to the close-knit nature of the higher education sector in Ireland, as well as the requests from the participants, the names of each organization has been anonymised. It was also considered necessary to anonymize the names of the offices within the universities that are involved in the research process7.
- In the case of the Institute Gama, it was agreed to anonymize gender due to the size of the organization, the number of people employed at the levels included, as well as the limited number of people involved in the particular type(s) of projects referred to.
This will be split into two sections for analysis: the academic organizations and the funding bodies.
Analysis of the Application of Project Management within Academic Organizations
The Implications of Accountability on the Profession.
Accountability is driven by national legislation, codes of practice and project sponsor requirements across the three organizations interviewed8. University Alpha seems to be more proactive in this area, and appears to be driven by broader concerns for international best practice, continuously adapting their policies in line with same.
The general consensus amongst senior researchers across all three organizations related to the increased levels of bureaucracy involved in academic research, though University Alpha was perceived9 to be the most bureaucratic. This is perhaps explained by the size of the organization and the level of research funding it administers on an annual basis10. This may explain its approach for organizing research support services to facilitate a coordinated approach to research project management across the university. Research Institute Gama, the smallest organization examined, appears to be the least bureaucratic of all the organizations. Support activities are located within the same building and are easily accessible, whilst researchers retain overall control of the management of their project. Thus perceived bureaucracy appears inversely related to organizational size, which is, perhaps, not surprising.
The Impact of Accountability.
During the course of the interviews it became apparent that all three organizations had faced similar challenges during the past number of years. In the university sector, the abolition of fees and the introduction of the 1996 Universities Act led to reduced financial autonomy, increased pressure to seek external funding opportunities and changes to internal policies such as quality and governance. Research Institute Gama has also been affected by similar issues, such as changes to its funding structure moving from an almost fully government funded organization to one whereby funding was reduced to 30% with the balance being sought from external sources. The introduction of the Single European Market and subsequent procurement guidelines has impacted on the awarding of traditional sources of external funding, requiring the Institute, like other organizations, to adapt to the changing environment.
Each organization referred to the positive impact that the European process had in terms of providing significant funding opportunities for academic research. This has also been seen in terms of domestic funding opportunities where the number of opportunities available has increased significantly during the past 10 years. In line with this, an increase in the level of complexity involved in financial reporting has impacted on internal operations as universities and research organizations attempt to adapt their operations to meet client requirements and draw down funding.
Changes to the Academic Environment.
During the course of the interviews, participants were asked to outline any changes that have impacted on the sector during the past number of years, and how they perceived these changes. The senior administrator from University Alpha viewed the European process as having the most positive impact on the sector, and the country in general. Professor A suggested that there was a high level of bureaucracy involved in proposing, negotiating, and managing projects, not only external but also internal to the organization. The Senior Administrator refuted this, suggesting that internal policies were driven by external requirements and international best practice. Professor A also suggested that some of the changes had impacted on the manner in which research was carried out11. The success of European proposals is to a large degree based on cross-national data collection that forms the basis of research.
From University Beta, Dr. C suggested that the management of research has been significantly professionalized during the past number of years. She viewed the internal changes within the university as being driven by external requirements, and questioned how these were contributing to the research process.
Research Institute Gama cited the changes to its funding structure and the impact of full economic costing12 on the manner in which the organization was managed. Professor H suggested that it had enabled the organization to better manage its activities and discover which areas were viable. From a research perspective, Professor H was hesitant about the contribution of the public procurement process on the quality of research being conducted at policy level, suggesting that the development of the research questions can lack the technical expertise of those who have the skills to deliver the project.
Research Support Services.
University Alpha dedicates three offices to the research project life cycle. One office covers research supports services, which facilitates the tendering stage of a project up to the signing of a contract. They alert researchers to funding possibilities, assist with the preparation of the tender application and offer feedback on applications up to the contract award stage. All applications must go through this office. When a contract has been awarded, this office submits the contract to the pre-award office for signing. Finally the post-award office looks after the compliance of the project with the financial requirements of the client in consultation with the researcher.
University Beta dedicates two offices to the research project life cycle. This includes one office that deals with the pre-award stage of the project, and a second office that looks after the post-award administrative reporting of the project, including signature of the contract.
Research Institute Gama dedicates individuals within a loosely defined framework to support the research process and does not dedicate a particular office to research support.
The significant difference across the three institutions is size, and support appears to be reflective of this.
Types of Projects
All three organizations participated in similar type projects:
- Top-down commissioned research whereby the client sets the research question or agenda and the academic offers the solution;
- Bottom-up commissioned research involving the researcher, who is given an opportunity to influence the research agenda in consultation with the funding body;
- Consultancy research, which can be conducted in an individual capacity, except in the case of Research Institute Gama, where consultancy is conducted on behalf of the organization;
- Survey-based research, which requires ethical approval, is expensive, and contributes to original research; and
- Collaborative/individual projects using internal sources of funding.
Management of the Project.
University Alpha is the only organization to provide training courses on research project management. Across all three organizations, the lead researchers13 on projects explained that their planning takes place during the development of the research proposal and is based on “experience”.
For the most part, communication on projects is carried out via face-to-face meetings, emails, and telephone conversations. On project monitoring, only one researcher claimed to use a project management software tool that allowed him or her to communicate centrally on all aspects of the project and monitor progress. Other projects were monitored by client reporting requirements, steering committees, Gantt charts, and involvement.
In general, the client and/or the project influenced the type of project management to be used, and principle investigators tailored their management style to the project. There was no reference to project management being conducted for self-funded projects using internal resources. However, University Alpha does offer an internal scheme that provides funding for proposals that have the potential to be awarded external funding if developed further.
Critical success factors.
During the course of the interviews, participants were asked to identify the factors that they believe make a project successful. Ranking has been decided by citation as outlined in Table 2 and has been limited to the top five factors mentioned during the course of the interviews.
Table 2: Critical Success Factors Ranked by Citation14
|Critical Success Factor||Cited||Ranking|
|Research that leads to publications||8||1st|
|The learning capacity of the project||4||3rd|
|Trust and collegiality||4||3rd|
|Choosing the right project||3||4th|
|A balanced project team||3||4th|
|Impact on policy||3||5th|
All three organizations offer a high-level framework for project management within their overall structures. This appears to be driven by client requirements, and in the case of University Alpha some procedures have been implemented in line with best international practices. There is no standard method for managing the delivery of research projects within and across organizations, though face-to-face meetings, emails, and telephone conversations were found to be the main tools used.
In conclusion, all three organizations are facing a common threat in terms of reduced financial autonomy in an increasingly competitive market. In response to these threats, all three organizations appear to be responding differently. University Alpha offers three research support offices devoted to research project management. The impression received from the interviews was that University Alpha is the most bureaucratic as its procedures are driven by client requirements and its proactive approach to implementing international best practices. In University Beta, there was a perception that the university had become more bureaucratic in recent times, however the researchers interviewed perceived these to have been driven by client requirements and they acknowledged some flexibility when necessary. On the other hand, Research Institute Gama appeared to be the least bureaucratic of the three. They are also the smallest of the three organizations interviewed, with a relatively flat management structure in comparison with a strong influence from academic researchers in the overall governance of the Institute. Finally, in terms of managing projects, none of the organizations had a set of guidelines or procedures on how to manage and/or deliver the project and individual researchers developed their own style, which was tailored to suit the project; although University Alpha offered training courses, which are generic and not tailored to different types of projects.
Analysis of Research Priorities within Funding Bodies
Organization A and Organization B fund research within the disciplines of the social sciences and humanities. Organization A is focused on providing funding for research in third-level institutions and selected research institutes, whilst Organization B provides funding for projects involving both “soft” and “hard” disciplines in both the private and higher education sectors.
Both organizations have faced common challenges, such as the drive for public sector accountability and value for public money, as well as changes imposed by the European process. This in turn has impacted on the structuring and reporting requirements of their research schemes. Organization B has responded rigorously in terms of implementing what has been described as “often overly bureaucratic” reporting requirements in an attempt to ensure public sector accountability that is in line with best practice. On the other hand, Organization A has taken a less stringent approach with the academic review board monitoring financial and technical reports simultaneously, with a further financial check from the executive board. Projects funded by Organization A are subject to audit, whilst Organization B requires six monthly audits of expenditure.
Accountability and Transparency.
Organization B's values are associated with ensuring accountability to its stakeholders and its tendering and reporting procedures serve to ensure that its values are maintained.
Organization A also provide guidelines on submitting tender proposals, however schemes revolve around bottom-up proposals, which are evaluated by international experts who consider both the feasibility of the proposal and the time and cost allocated to producing same. Reporting is monitored by a review board, which includes a member of the council and experts from other institutions in Ireland.
At first glance, it would appear that Organization A has less onerous reporting requirements, which is perhaps explained by its governance structure which consists mainly of practicing academics who understand the need for a degree of autonomy. This is reflected by its reporting requirements that focus on the quality of output, which is assessed against the financial report as being feasible and providing value for money in academia. However, it is at the proposal stage that academics are expected to submit a well thought out plan on how they propose to carry out their research project. This can be considered as constraining by the academic whose profession is, for the most part, focused on answering the unknown, and often will not know if his or her chosen methodology is feasible until he or she has started the project.
Organization B provides detailed guidelines on submitting tender proposals that require planning based on individual work packages with associated costs and resources, the use of Gantt and pert charts, as well as risk and quality plans if successful. Reporting involves six monthly technical and financial reports, whilst further reporting occurs during steering committee meetings.
Organization A also requires six monthly technical and financial reporting, although its planning and reporting is not as structured as the requirements of Organization B. Its focus appears to be on the quality of the proposal, and that the project outputs are aligned with the original proposal. The award of funding is based on a peer review evaluation process that also considers the feasibility of the proposal, including the planning process.
Research output priorities.
Organization A provides funding for schemes that are focused on funding proposals that offer a career path for young researchers, such as post graduate and postdoctoral schemes. The research priorities of Organization A appear to be aligned with Drucker's (1999) view that knowledge worker productivity is primarily a matter of quality and not quantity. To that end, it appears to measure quality on the number of publications in peer reviewed journals as a result of the project.
Organization B on the other hand funds applied research that has the potential to impact at policy level, though they also have targets for publication output. There is a body of literature that is focused on the “clash of cultures” between the academic community and the policy-making community (Schimack, 2005, p. 372). The literature suggests that research outputs that can be applied in a policy making environment is at odds with research that is suitable for publishing in academic journals, which often leads to the promotion of academics (Minstrom, 2008; Caplan, 1979; Reimers and McGinn, 1997).
Both funding organizations are operating in an environment where they must ensure value for public money and report to the Auditor and Comptroller General, while at the same time ensuring that their operations are in line with best practices within the sector. Whilst both funding bodies are facing the same challenges, they appear to be responding in different ways. Organization A is actively involved in European research networks, such as the Humanities in the European Research Area (HERA) and New Opportunities for Research Funding Agency Co-operation in Europe (NORFACE), where the sharing of information enables it to influence the European agenda and contribute to the development of best practices within the sector. Its reporting requirements are considered to be less structured than those employed by Organization B, where assessment is considered against the quality of output and feasibility of expenditure against progress. Organization B, on the other hand, stipulates detailed project plans broken down into work packages with cost and resource allocations and detailed planning of activities using Gantt and Pert charts.
Table 3 analyzes the level of project income received by each research organization from the funding bodies interviewed, which may help explain the bureaucracy perceived at internal level. Overall it was possible to identify that at least 70% of income generated by projects from University Alpha, and at least 60% in University Beta came from funding bodies known to fund “hard disciplines”.15
Table 3: Analysis of Project Funding
|% of Project Income||% of Funders Budget|
|Research Institute Gama|
It could be argued that what is perceived as bureaucratic reporting requirements from Organization B is grounded in its attempt to provide a standard reporting template across all disciplines and sectors. Such an approach may not be practical as it fails to take into consideration the idiosyncrasies of each sector and disciplinary area. The same argument could also be applied to University Alpha and University Beta, whose internal research support structures are standard to the organization, and not to the disciplinary areas16.
Summary and discussion
The abolition of fees and the introduction of a block grant administered by the HEA has reduced the financial autonomy of the third-level sector in Ireland, forcing them to actively compete for funding from alternative sources17. In tandem with these changes, the general research environment has undergone significant changes. The European process has dedicated considerable resources into the promotion of a knowledge economy as a means of promoting future economic development. The benefits of pursuing a knowledge economy have also been cited by the Irish government, who responded during the economic boom by increasing its investment in the area18.
Research project management in the field of social sciences and humanities is driven by increased requirements for public sector accountability. Whilst no standard approach has been adopted by the sector, it is increasingly apparent that different funding bodies are requiring different levels of reporting, with the European Commission being the main instigator of change in the sector. In response to these requirements, the higher education sector (and the academic research sector in general) has had to respond by adapting its internal structures to meet the accountability requirements of project sponsors. The degree to which each organization is responding to the demands for increased accountability varies. For example, University Alpha have been proactive in implementing change within its organizational structures, which are seen as being in line with international best practice; on the other hand, University Beta was viewed as responding to the demands of funding bodies, as was Research Institute Gama. When the interviewees were asked about their perception of the level of reporting within their organization, University Alpha, the largest of the three interviewed, was seen to be the most bureaucratic in terms of the number of internal requirements during the project lifecycle. University Beta was described as being somewhat bureaucratic, however due to its relatively small size there was scope for some flexibility, and the levels of bureaucracy was perceived as a direct response to external requirements.19 Research Institute Gama was considered the least bureaucratic, with reporting being aligned to the requirements of funding bodies; however this is also the smallest organization involving research support services at both the departmental and organizational level. All three organizations had adapted their internal reporting structures to provide a high level framework for managing projects within the existing structures of their organization. Researchers interviewed across all the organizations cited using different project management styles for different projects. When this was probed further, it was found that funding bodies are increasingly using project management principles to develop a framework within their funding schemes that subsequently imposes specific project management tools on researchers to facilitate how progress is monitored whilst ensuring accountability for public sector expenditure.
Can Project Management make a useful Contribution to helping Academic Researchers in the Social Sciences and Humanities to better achieve their Objectives?
The benefits of project management in the academic sector, and in particular in the area of the social sciences and humanities can be summarized as follows:
- Project management can be used as a business planning tool that aligns individual projects with the organization's research strategy. At an organizational level, all three participants have detailed research strategies and projects are ideally chosen within these areas. This sits well with the literature, for example, Payne and Turner (1999) advocate limiting the project management approach to the strategic level and ensuring that projects are in line with institutional norms, whilst Kenny (2000) proposes business planning that aligns project goals with those of the university. Turner (2009) proposes a business planning process tool to facilitate organizations in choosing project opportunities that are in line with norms. This reflects the approach being used within the sector to promote its research strategy.
- It can assist academic researchers by providing focus and encouraging detailed planning on research proposals, forcing researchers to conduct feasibility studies and develop realistic project plans.
- It serves to increase both the quality and quantity of research output, which is integral to academic career progression and to the development of their discipline.
- The findings of this research illustrated how the European Commission, government policies, funding bodies and academic networks are using programme management as a means of providing a co-ordinated approach to knowledge building in society. Their programmes promote a culture of accountability and quality of output as part of their awards process and post-award reporting requirements.
Is a more systemized approach to project management in academia required?
Given the broad spectrum of projects that can be carried out in academia, a standard approach to project management would be difficult and ill advised. The findings of this study indicated that the style of project management is tailored to the project, which supports the findings of other studies (Payne and Turner, 1999; Thomas and Mullaly, 2001; Crawford and Helm, 2009).
This research found no consistent guidelines for project management across funding bodies. They appear to be adapting the structures of their programmes to encourage academics to plan their work in a particular way. It would appear that some funding bodies are promoting standard reporting templates that span across disciplinary areas and research sectors, which may explain why they have been accused of being overly bureaucratic, as they do not take into consideration the idiosyncrasies of different professions.
During interviews, participants were asked how they plan projects. Academics cited “experience” as having taught them how to write a research plan and proposal. It is perhaps a tactical decision on the part of the university administration and the funding body to provide a project management framework but not to specify “how” the project should be run. The strong culture of autonomy over individual research and the risk of negatively impacting on the quality of research output may explain the lack of guidelines for the actual implementation of the project in academia. Organizations that rely on creativity and innovation are acutely aware that academics conduct research and that the project environment should serve to facilitate this process. However, all research organizations may benefit from devising training courses whereby senior researchers can share their research project management knowledge to their less experienced colleagues.
These arguments highlight the importance of acknowledging the differences across disciplines and the culture of academic research. Currently, project management in this sector appears to focus on the strategic level. Where funding bodies award funding across disciplines, as in the case of the Organization B, its standard reporting requirements can be perceived as overly bureaucratic as they may not sit well with the softer disciplines. However, as the university sector tries to adapt its management structures to meet the compliance requirements of all funding bodies, there may be an argument for all funding bodies to develop consistent models of reporting that are appropriate to both “hard” and “soft” disciplines, with input from the academic sector.
What are the Appropriate Success Criteria on Academic Research Projects within the “soft disciplines” of Social Sciences and Humanities? How can they be enhanced by Project Management?
The following critical success criteria, ranked accordingly, were identified by the participants interviewed as contributing to the success of a project, which are similar to those identified in by Muller and Turner (2005).
- Research that leads to publications;
- Good communication;
- The learning capacity of the project;
- Trust and collegiality;
- Choosing the right project;
- A balanced project team;
It is worth noting that the most commonly listed criteria for success, publication output, support the literature on knowledge worker productivity and motivation factors (Drucker, 1999; Minstrom, 2008).
Table 3 outlines the components of a model that can be used to measure project success over the short term to the long term. Such a model may be useful to the academic sector as a means of monitoring project success.
Table 3: Shenhar & Dvir (2007): Components of Process Success
|EFFICIENCY||IMPACT ON TEAM||IMPACT ON CUSTOMER||BUSINESS SUCCESS||PREPARATION FOR THE FUTURE|
| || || || || |
How Mature are Academic Organizations in the Area of Project Management?
The SEI propose a Capability Maturity Model (CMM) as a framework for organizations within the software industry to follow, in order to develop its maturity as an organization and increase its ability to predict the success of project outcomes. Figure 2 attempts to emulate SEI's model in a format that can be applied to academic organizations.
It is questionable if academic organizations can, or even should, achieve a level 520 on the CMM. The literature review highlighted the issues associated with imposing rigid controls on the academic sector. Whilst it is acknowledged that the purpose of CMM is not to impose rigid controls on an organizations operating environment, its application in an academic research environment may prove challenging. For research to be innovative and creative, it needs to be carried out within an environment that does not impose any unnecessary controls that might negatively impact on a researcher's ability to carry out high quality research. The development of a model for the sector would need to balance this requirement whilst complying with the need for regulation in terms of meeting external requirements and monitoring productivity.
Figure 2: Proposed Capability Maturity Levels for Academic Organizations
Figure 3 provides an analysis of the responses received from each participant in respect of his or her perception of the level of project management within his or her organizations. What is significant is that the funding bodies have a higher level of maturity in terms of their procedures for setting the research agenda, awarding contracts, and the procedures they use to monitor same. Organization B has been awarded a higher rating as it imposes a more rigid approach to programme/project management in terms of its reporting requirements. Organization A has been awarded a slightly lower rating as its reporting is considered less rigid, and is based on peer review that is perceived as acceptable21.
Figure 3: Perception of project maturity
Across the organizations interviewed, the strategic framework for project management within the organization is placed a level 3, and is based on the documented procedures for projects within each organization as expressed by the senior administrators interviewed. This supports Turner's argument that business planning should be limited to the strategic level, as each organization offers a strategic framework for managing projects though they do not impose specific management processes for delivering the project. Finally, at the tactical or operational level, projects engaged in by academics are located at level 2, as different approaches are used for different types of projects.
Allen, D.K. (2008). Organizational climate and strategic change in higher education: Organizational insecurity, Higher Education, 46(1), 61–92.
Barrett, D. (2004). OECD Review of Higher Education in Ireland http://www.tcd.ie/Economics/TEP/2004_papersZTEPNo5SB24.pdf [Accessed 12 August, 2009]
Bates, A.W. (2000). Managing Technological Change, in Bates, A.W. (eds.) Strategies for College and University Leaders, San Francisco: Joss Bass.
Bain, J. D. (1999). Introduction, Higher Education Research & Development, Special issue: Learning Centered Evaluation of Innovation in Higher Education, 18(2), 165–172.
Bok, D. (1990). Universities and the Future of America, Durham, NC: Duke University Press, p. 111
Caplan, Nathan (1979). The Two Communities Theory and Knowledge Utilization. American Behavioral Science, 22, pp. 457–70
Connell, H. (2005). (eds) University Research Management: Meeting the Institutional Challenge, Paris: OECD.
Conraths, B., & Schmidt, H. (2005). The funding of University-Based Research and Innovation in Europe. An exploratory study, EUA Publications.
Considine, M. (2006). Theorizing the University as a Cultural System: Distinctions, Identities, Emergencies', Educational Theory, 56(3), 255–270.
Cooke-Davies, T. (2002). The Real Success Factors on Projects, International Journal of Project Management, 20(3), 185–90.
Courpasson, D. (2000). Managerial Strategies of Domination: Power in Soft Bureaucracies, Organizational Studies, 21(1), 141–162.
Crawford, L., & Helm, J. (2009). Government and Governance: The Value of Project Management in the Public Sector, Project Management Journal, 40(1), 73–87.
Deem, R. (2007). The Knowledge Worker and the Divided University, in Deem, R., Hillyard, S. & Reed, M. (eds.) Knowledge, Higher Education, and the New Managerialism: The Changing Management of UK Universities, Great Britain: Oxford University Press.
Deem, R. (2004). The Knowledge Worker, the Manager-Academic and the Contemporary UK University: New and Old Forms of Public Management?, Financial Accountability & Management, 20(20), 107-128.
Directorate for Education, Education Committee (2004). Review of National Policies for Education: Review of Higher Education in Ireland Examiners' Paris: OECD.
Drucker, P.F. (1966). The Effective Executive, New York: Harper Business.
Environmental Protection Agency. (2009), Research Programmes, http://www.epa.ie/researchandeducation/research/
Erno-Kjolhede, E. (2000). Project Management Theory and the Management of Research Projects. MPP Working Paper, No.3, Department of Management, Copenhagen Business School
Fegan, D. & Slevin, J. (2004) Research Infrastructural Deficits http://www.ria.ie/publications/pdfs/research.pdf [Accessed 2 September 2009]
Forfas. (2004/ Building Ireland's Knowledge Economy: The Irish Action Plan for Promoting Investment in R&D to 2010, Dublin: Forfas.
Henkel, M. (2000). Academic Identities: Policy Change in Higher Education, Philadelphia: Jessica Kingsley Publishers.
High level Reference Panel, chaired by Mr. John Dunne. (2005). Review of Quality Assurance Procedures in Irish Universities Dublin: Higher Education Authority (reference document) www.iuqb.ie/GetAttachment.aspx?id=ea1a157a-e1ef-4f5d-9c8f [Accessed 2 September 2009]
Higher Education Authority (2007). Governance of Irish Universities: A Governance Code of Legislation, Principles, Best Practice and Guidelines, Dublin: Irish Universities Association.
Hoggett, P. (1996). New Modes of Control in the Public Service, Public Administration, 74(1), 9–32.
Houston, D. & Studman, C.J. (2001). Quality Management and the University: A Deafening Clash of Metaphors?, Assessment and Evaluation in Higher Education, 26(5), 476–487.
International Assessment Committee. (2004). The Programme for Research in Third Level Institutions [PRTLI] Impact Assessment, Dublin: Higher Education Authority.
Irish Universities Act 1997 No. 24 of 1997, Dublin, Irish Stationary Office.
Irish Universities Quality Board (IUQB) (2007). Quality Review of the Irish Universities Quality Review Board – Self Evaluation Report. http://www.hea.ie/files/files/file/archive/policy/IUQB%20Review/IUQB_Self-Evaluation_Report_and_ESG_Compliance_Statement.pdf
Jarrett Committee (1985). Report of the Steering Committee for Efficiency Studies in Universities, London: Committee of Vice Chancellors and Principles.
Kellaway, L. (2006). Why Academics Make an Unfit Subject for Management. Financial Times, February 27.
Kenny, J. (2003). Effective Project Management for Strategic Innovation and Change in an Organizational Context, Project Management Journal, 34(1), 43–53.
Kenny, J. Where Academia meets Management: A Model for the Effective Development of Quality Learning Materials using New Technologies, http://www.ascilite.org.au/conferences/melbourne01/pdf/papers/kennyj.pdf.
Kernzer, H. (2003). Project Management: A Systems Approach to Planning, Scheduling and Controlling, 8th Ed., New Jersey: Wiley.
Livingstone, H. (1974). The University: An Organizational Analysis, Glasgow: Blackie.
Marginson, S. (2008). Academic creativity under new public management: Foundations for an investigation, Educational Theory, 58(3), 269–287.
McDermott, J. (2008). Funding Deficit Threatens our Universities’ World Standing. The Irish Times, August 7th. http://www.irishtimes.com/newspaper/opinion/2008/0807/1217984184925.html
Mintrom, M. (2008). Managing the Research Function of the University: Pressures and Dilemmas, Journal of Higher Education Management, 30(3), 231–244.
Moodie, G.C. & Eustace, R. (1974). Higher Education and State: Universities and Colleges Montreal: McGill-Queen's University Press.
Muller, R. & Turner, J.R. (2007). Matching the Project Manager's Leadership Style to Project Type, International Journal of Productivity and Performance Management, Vol.25, 21–32.
NORFACE ERA-NET (2007). Report on Programme Development and Management Germany: NORFACE ERA-NET
Obermaier, A. (2009, October 1). REMC Briefing Presentation, European Commission, Research Directorate-General, Science, Economy and Society Directorate, Governance and Ethics Unit [presented at final project meeting]
Paulk, M.C., Curtis, B., Chrissis, M.B., & Weber, C.V. (2005). The Capability Maturity Model for Software http://homepages.inf.ed.ac.uk/dts/pm/Papers/cmm.pdf [Accessed 15 August 2009]
Payne, J.H., & Turner, J.R. (1999). Company-wide Project Management: The Planning and Control of Programmes of Projects of Different Type, International Journal of Project Management, 17(1), 55-59.
Pennypacker, J. S., & Grant, K. P. (2003). Project Management Maturity: An Industry Benchmark, Project Management Journal, 34 (1), 4–11.
Pienaar, C. (2009). Is the Academic Work Role still Sustainable?, Tydskrif vir Geesteswetenskappe, 49(2), 229-265.
PMI, (2004) A Guide to the Project Management Body of Knowledge (PMBOK® Guide), 3rd edition. Newtown Square, PA,: Project Management Institute
Power, M. (1997). The Audit Society: Rituals of Verification, Oxford: Oxford University Press.
Reed, M. (2007). New Managerialism and Public Services Reform: From Regulated Autonomy to Institutionalized Distrust, in Deem, R., Hillyard, S., & Reed, M. (eds.) Knowledge, Higher Education and the New Managerialism: The Changing Management of UK Universities, Great Britain: Oxford University Press.
Reimers, F., & McGinn, N.F. (1997). Informed Dialogue: Using Research to Shape Education Policy Around the World Westport, CT: Praeger.
Rosenstock, C., Johnston, R., & Anderson, L. (2000). Maturity model implementation and use: a case study, Proceedings of the 31st Annual Project Management Institute 2000 Seminars and Symposium, Project Management Institute, Newtown Square, PA, Paris, Vol. 2000
Salmi, J. (2009). The Growing Accountability Agenda: Progress or Mixed Blessing? Higher Education Management and Policy, 21(1), 1–21.
Schimank, U. (2005). New Public Management and the Academic Profession: Reflections on the German Situation, Minerva, 43(4) 361–376.
Taylor, J. (2006). Managing the Unmanageable: The Management of Research in Research-intensive Universities, Journal of the Programme on Institutional Management in Higher Education: Higher Education Management and Policy, 18(2), 1–25.
The Irish Research Council for the Social Sciences and Humanities, Research Programmes, www.irchss.ie
Thomas, J. & Mullaly, M.E. (2007). Understanding the Value of Project Management: First Steps on an International Investigation in Search of Value, Project Management Journal, 38(3), 74–89.
Turner, J.R. (2009). The Handbook of Project Based Management: Leading Strategic Change in Organisations, Third ed., USA: McGraw-Hill.
Turner, J.R. & Cochrane, R.A. (1993). The Goals and Methods Matrix: Coping with Projects with ill-defined Goals and/or Methods of Achieving Them, International Journal of Project Management, 11(2), 93–102.
Winefield, A.H., Boyd, C., Saebel, J., & Pignata, S. (2008). Job Stress in University Staff: An Australian Research Study, Sydney: Australian Academic Press.
Yin, R.K. (2003/ Case Study Research: Design and Methods, Third, Fifth ed., Applied Social Research Methods Series, Sage Publications.
Table 3: A Comparative Overview of Project Management Processes within Academic Organizations
|University Alpha||University Beta||Institute Gama|
|Context||Large University (approx. 20,000)||Small University (approx. 10,000)||Non-profit Research Institute (approx. 100)|
|Research Support Services||Three separate offices covering: |
1. Pre-award (identification of opportunities, tender/negotiation)
2. Contract signature
3. Post-award (financial monitoring)
|Two offices covering |
1. Pre-award (tender/negotiation)
2. Post-award (financial monitoring)
|Three areas involved: |
|Types of Projects|| || || |
|Internal Procedures|| || || |
|Perceptions of change|| || || |
|Accountability|| || || |
|Management of the project|| || || |
|Critical Success Factors|| || || |
Table 4: Comparative Analyses of Research Priorities for Funding Bodies
|Organization A||Organization B|
|Mission||“to promote excellence and high-level output in third-level research in the humanities and social sciences, law and business studies through an interlinked suite of research funding programmes”||“Today's environmental research is tomorrow's environmental protection.”|
|Accountability/Transparency|| || |
|Project Management|| || |
|Challenges|| || |
|Changes|| || |
|Research agenda|| || |
|Research output priorities|| || |
Table 5: Case Study Tactics (adapted from Yin, 2003, p. 43)
|Tests||Case Study Tactic||Phase of Research in which Tactic occurs|
|Construct validity||Use multiple sources of evidence||Internet search|
|Published Annual Reports|
|Semi Structured Interviews (across three research organisations and two funding bodies)|
|Establish chain of evidence||Recording of interviews22|
|Detailed note taking|
|Have key informants review draft case study report||Draft case studies are sent to each interviewee for review to ensure accuracy of content.|
|Internal validity||Do pattern-matching||Each semi-structured interview followed a checklist of six areas to allow for comparison across individuals and organizations.|
|Do explanation building|
|Data analysis – based on observations|
|Address rival explanations||and/or facts|
|Use logic models|
|External reliability||Use replication logic in multiple-case studies||Research design – interviews will be broken into areas, which can be analyzed using tabular format for cross-organizational analysis.|
|Reliability||Use case study protocol||Data collection|
|Develop case study database||Data collection - notes and recordings of interviews securely held|
1 Since the 1980s there has been a drive to improve efficiency in the public sector, a new wave of thinking that has come to be known as New Public Management.
2 Whilst it is not proposed that Irish universities can be directly compared to the United Kingdom (UK) situation, the changes that have occurred internationally reflect the same challenges faced in the Irish case. The main distinction is that in the UK, universities must undergo a Research Assessment Exercise (RAE), which assesses the quality of research and teaching in universities as a means of awarding funding.
3 Their study focused on the public sector in Australia.
4 This is a term that is used in qualitative research to describe the referral of participants by those who have already participated in the research methodology.
5 See Appendix I and II for comparison of information across organisations by topic covered.
6 See Appendix III for more information on how each case study was prepared in accordance with the best practices.
7 In Ireland, there is no standard organizational structure of administration services within the university sector. It was agreed that naming the offices involved in research project management would allow for easy identification of the University. I have anonymised the names of the office(s) involved however a description of their role in the process is briefly outlined.
8 The Single European Act 1986, Irish Universities Act 1996, Code of Practice of State Bodies 2009, Governance of Irish Universities, 2007.
9 This comment was expressed by both a current and former member of research staff within the university, who had experience in running projects in other organizations; however, this would not be unusual for a large organization.
10 University Alpha administered approx €114 million euro in research income compared to University Beta who administered approximately €13 million worth, in 2008.
11 This supports Conrath and Smidt's (2005) exploratory study, which found that the increased diversification of funding opportunities was “slowly creating a cultural change” within the university sector as it learns to become more competitive and entrepreneurial in its approach to research.
12 Full Economic Costing is a funding model that was promoted by the European Commission as a condition of awarding funding to non-public bodies. The Commission (DG Research) is currently seeking to expand this policy to the university sector. In June 2009, it agreed to defer the original deadline for implementation by February 2010 due to the difficulties encountered by the sector in meeting this deadline.
13 More commonly referred to as the Principal Investigator or P.I.
14 These are broadly aligned to the Research Output Priorities cited by the funding bodies interviews in Appendix III.
15 These figures are based on an analysis of annual reports for each organization.
16 Research Institute Gama only engages in projects that fall into the disciplines of the social sciences and humanities.
17 For further discussion on the impact of the abolition of fees see Fegan et al, (2004) and McDermott, J. (2008).
18 See Forfás (2004)
19 Such as those imposed under the 1997 Universities Act and the increasing requirements from funding bodies
20 Implementing change in academic organizations can prove time consuming as academics are trained as critical thinkers, thus at level 5 it may not be feasible to continuously improve on existing processes at management level.
21 This comment was made by Professor A from University Alpha.
22 Not all interviews were recorded, some participants requested not to be recorded, and it was perceived that others were less forthcoming with information when the recorded was used, which was impacting on the quality of the interviews.
© 2010 Project Management Institute. All rights reserved.