Innovation Resilience Behavior and Critical Incidents
Validating the Innovation Resilience Behavior-Scale with Qualitative Data
Peter R. A. Oeij, TNO, Netherlands Organisation for Applied Scientific Research, Leiden, The Netherlands
Steven Dhondt, KU Leuven, Catholic University of Leuven, Leuven, Belgium
Jeff B. R. Gaspersz, Nyenrode Business University, Breukelen, The Netherlands
Tinka van Vuuren, Open University of the Netherlands, School of Management, Faculty Management, Science & Technology, Heerlen, The Netherlands
Project teams carrying out innovation projects are investigated during critical incidents. Earlier, a Team Innovation Resilience Behavior (IRB)-scale was successfully applied to quantitative survey data (Oeij, 2017). Team IRB is the team's capacity to effectively deal with possible incidents and ensure the project's continuation. This study uses qualitative data to validate the Team IRB concept. Methodologically, it is concluded that the concept of Team IRB allows for its application to both quantitative and qualitative data. The content conclusion is that teams that score highly for Team IRB are better in recovering from critical incidents than they are in preventing critical incidents.
KEYWORDS: project management; innovation, team; innovation resilience behavior (IRB)
According to Mulder (2016), 65% to 70% of megaprojects and information and communication technology (ICT) projects seem to fail, and Castellion and Markham (2013) assert that at least 35% to 40% of product innovations fail. These figures are substantial, especially when compared to the functioning of high-reliability organizations (HROs) whose teams seldom seem to fail. Thus, the question emerges: Can innovation teams learn from HRO teams?
“(…) under very trying conditions all the time and yet manage to have fewer than their fair share of accidents. HROs include power grid dispatching centers, air traffic control systems, nuclear aircraft carriers, nuclear power generating plants, hospital emergency departments, wildland firefighting crews, aircraft operations, and accident investigation teams. (…) They face an “excess” of unexpected events because their technologies are complex, their constituencies are varied, and the people who run these systems have an incomplete understanding of the systems and what they face. (…). HROs (…) act mindfully (…) organize themselves (…) to notice the unexpected (…) halt its development (…) focus on containing it (…) focus on resilience and swift restoration of system functioning.” (Weick & Sutcliffe, 2007, pp. 17–18, italics in original)
Weick and Sutcliffe (2007) studied HROs and developed instruments to audit team behavior, with the purpose of supporting organizations to assess their state of the art with reference to handling ‘the unexpected’ and, subsequently, seeing how teams could improve their performance. Their audits have been converted into items for questionnaires, which have been applied mainly to the fields of hospitals and education (Sutcliffe, Vogus, & Dane, 2016; Vogus & Sutcliffe, 2007; Ray, Baker, & Plowman, 2011; Vogus & Iacobucci, 2016). The thinking behind HROs and how they operate has hardly been applied to innovation management (Vogus & Welbourne, 2003). HRO principles, described below, were assumed to be beneficial to innovation teams. Other than HRO teams, where the sense of urgency is driven by preventing casualties and disasters, the sense of urgency for innovation teams to introduce HRO principles is far less obvious. While Weick, Sutcliffe, and Obstfeld (1999) suggested that for non-HROs the sense of urgency is to invest in becoming a learning organization to improve competitiveness, the specific win for innovation teams is to solve occurring problems faster and better and, hence, improve their chances of innovation success. This assumption was investigated among team leaders and team members of innovation teams in 11 Dutch organizations that performed innovation projects by teams, where innovation projects were project-based assignments aimed at realizing a renewal or improvement of a product, service, or process. The results showed that teams characterized by HRO principles—called IRB teams—reported more positive project results than teams less characterized as IRB teams (Oeij, 2017). Those IRB teams also had more favorable team conditions, in terms of psychological safety, team learning behavior, complexity leadership (to deal with ambiguities), and team voice, which we referred to as ‘mindful infrastructure.’ Mindful infrastructure is the antecedent of IRB, and IRB mediates the relationship between mindful infrastructure and project results (Oeij, 2017). The focus in this article is, however, on IRB.
The purpose of this article is first to validate the IRB scale we used for the quantitative survey (Oeij, 2017) in a qualitative study of innovation teams based on in-depth, qualitative interview data. Second, we investigate how innovation teams deal with critical incidents. Critical incidents are events that can cause a project to significantly deviate from the original plan and even fail. The aim is to contribute to the literature on innovation management and innovation projects and determine whether or not the IRB concept of crisis management and safety literature can be applied to their domain. The main question to address is: Does the IRB scale enable accurate descriptions of the behavior of innovation teams when dealing with critical incidents in their projects? To answer this question, we carried out a number of steps that form the design of this article. We start by explaining the origins of innovation resilience behavior and its relation to the literature of team resilience. Subsequently, we will discuss the method and data we used to answer the question and present the results; in the final section we present points of discussion and draw conclusions.
Innovation Resilience Behavior and Team Resilience
Innovation resilience behavior in teams—or Team IRB—is derived from theories applied in crisis management and safety science. In order to investigate its applicability in another context, namely innovation management, one can apply the concepts of team resilience developed more generally and with a generic applicability. By comparing both approaches we can assess the degree of their agreement. First, we describe Team IRB, followed by how it relates to team resilience.
Team innovation resilience behavior (Team IRB) is the capacity of a team to withstand and overcome critical incidents (i.e., stressors that threaten the innovation project, team cohesiveness, and performance) in a manner that enables sustained activity toward the goals of the innovation project by critical recoveries (i.e., handling and bouncing back from challenges) that safeguard team cohesiveness and performance (Oeij, 2017). This concept was derived from Weick and Sutcliffe (2007), who defined a set of HRO principles that enabled teams in high-reliability organizations (HROs) to anticipate ‘unexpected’ problems, contain them, and recover quickly. The unexpected problems that such teams face are small events that can have large consequences if they remain unnoticed. The five HRO principles are: (1) preoccupation with failure (a focus on weak signals of lapses), (2) reluctance to simplify (seeking validated information), (3) sensitivity to operations (connecting operations and the relationships between events to the big picture), (4) commitment to resilience (keeping errors small and improvising to keep the system working), and (5) deference to expertise (and, if necessary, authorizing experts, regardless of rank, to decide) (Weick & Sutcliffe, 2007, pp. 9–16).
These HRO principles overlap with a generic construct of team resilience. In a review study of team resilience Alliger, Cerasoli, Tannenbaum, and Vessey (2015) demonstrated that resilient teams exhibit three behavioral strategies for dealing with pressures, stressors, and difficult circumstances: minimizing, managing, and mending actions. Their insights into team resilience stem from 25 years of research with all types of teams, but mainly with firefighting and oil exploration teams, surgical and other medical teams, emergency response teams, and law enforcement and military teams. On the basis of their study, a Team Resilience construct can be formulated.
First, Alliger et al. (2015) describe that resilient teams perform minimizing actions before the arrival of a problem, which involves four types of behavior. Resilient teams (1) anticipate challenges and plan for contingencies; (2) assess and understand the team's current readiness by monitoring; (3) vigilantly identify the early warning signs of potential problems; and (4) prepare to handle difficult situations by documenting what-if situations and standard operating procedures. Second, such teams perform managing actions when difficult situations occur. The five behaviors of resilient teams are: (1) assessing challenges quickly, honestly, and accurately; (2) addressing chronic stressors (such as a noisy work environment, ambiguous team roles, lingering personality conflicts), even though it is tempting to ignore them, because they realize that these stressors can affect team cohesiveness and effectiveness; (3) providing backup and assistance to one another and recognizing each other's needs; (4) consciously maintaining basic processes under stress and being able to face emergencies; and (5) seeking guidance and support when needed. Third, resilient teams perform mending activities after a stressful event, which involve recovering from stress, learning from experience, and adapting as necessary. The four behaviors associated with mending activities in resilient teams: (1) regaining situational awareness as quickly as possible to know what needs to be done; (2) debriefing by reviewing their actions and reflecting on what went wrong and what went right; (3) ensuring they address concerns and risk points that became evident during the encounter with the challenge; and (4) expressing appreciation to build bonds and team norms (Alliger et al., 2015).
The minimizing, managing, and mending behaviors of resilient teams set out by Alliger et al. largely overlap with the five team behaviors of Weick and Sutcliffe (2007) discussed earlier, which we used as the basis for innovation resilience behavior. The two approaches are compared in Table 1.
Through additional detailing of minimizing, managing, and mending behaviors, Alliger et al. even describe as many as 40 types of behaviors in which resilient teams engage (2015, p. 181). The overview discussed above is helpful for analyzing what innovations teams do when they have to deal with critical incidents. This article describes the exploratory research into whether teams showing that innovation resilience behaviors are more successful in dealing effectively with critical incidents. The main purpose, however, is to investigate whether or not the Team IRB scale can be converted and applied to qualitative data using the Team Resilience construct. Therefore the main question is:
Can the Team IRB scale be adequately applied to qualitative data?
To investigate this question in practice we address two sub-questions to innovation teams, namely;
1. Which innovation teams show innovation resilience behavior?
2. How do innovation teams deal with critical incidents in terms of innovation resilience behavior?
Methodology and Data
The research is part of a multiple-case study into the team dynamics of innovation teams carrying out innovation projects (Oeij, 2017). Quantitative data are gathered via a survey among team leaders and team members of 11 organizations based in the Netherlands. In total, 368 respondents were approached and of 232 individuals who completed the relevant items, data were used in this article; 107 of these 368 respondents belong to one of 18 innovation teams in the same 11 companies. In addition to the survey data, from those 18 teams, qualitative data were also gathered via in-depth interviews. The quantitative data measure the Team IRB scale, and the qualitative data measure the Team Resilience construct. The organizations in the study carried out project-based innovations in teams, and come from private and non-private sectors and from industrial and service sectors. An innovation is a renewal or improvement of a product, service, or a process.
|Team Resilience (Alliger et al., 2015)||Innovation Resilience Behavior (based on Weick & Sutcliffe, 2007)|
|Minimizing (Before a critical incident) |
Anticipating challenges and planning contingencies
Understanding current readiness
Identifying early warning signs
Preparing to handle stressors
|Being preoccupied with failure/tracking small failures /being alert for weak signals |
Being reluctant to simplify/resisting oversimplification
Being sensitive to operations/making continuous adjustments
|Managing (During a critical incident) |
Assessing challenges quickly and accurately
Addressing chronic stressors/providing backup and assistance
Maintaining processes under stress
|Being sensitive to operations/making continuous adjustments |
Being committed to resilience/maintaining a stable state
Deferring to expertise/value expertise higher than rank
|Mending (After a critical incident) |
Regaining situation awareness
Conducting team debrief
Addressing concerns and risk points
|Being committed to resilience/regaining a stable state|
Table 1: Overlap of the Team Resilience construct and innovation resilience behaviors (Team IRB scale).
Quantitative Data: Team IRB and Survey Data
One of the measures used in the survey is the Team IRB scale. The innovation resilience behavior of teams was measured using a short version (containing 18 items) of Weick and Sutcliffe's five Audits of Resilient Performance scale (2007, pp. 94–102), which consists of 48 items. The scale was made context-specific for teams, and respondents were asked about the extent to which the five types of team behavior (preoccupation with failure, reluctance to simplify, sensitivity to operations, commitment to resilience, and deference to expertise) were present in their project teams; they answered on a 7-point scale, ranging from ‘not at all’ (1) to ‘to a very great extent’ (7).
Based on the scores for team innovation resilience behavior, 12 of the 18 teams1 can be classified as teams that showed innovation resilience behavior (high score for IRB), and six could not (low score for IRB), as shown in Table 2.
The mean value (of all five scales together) for Team IRB is 4.82 (SD = 0.75). On the basis of the qualitative interviews, however, one of the teams below that score (Team06, M = 4.81) was regarded as more IRB-minded than the survey data told us. This team encountered no critical incidents but was nonetheless very alert to detecting possible incidents; therefore, the threshold for considering that a team showed IRB was set at 4.8. There are 12 out of 18 teams for which IRB was observed to be present (a score of M = 4.81 or higher).
Qualitative Data: Critical Incidents
From the qualitative in-depth interviews we will use in-depth, face-to-face interview data of how teams dealt with critical incidents. Critical incidents are occurrences or conditions that interrupt the normal procedure of an innovation project (Flanagan, 1954). Critical incidents can stop or delay a project, or can speed up the innovation process. In these cases, we studied delays as critical incidents and we understood critical recoveries—if these happened—to be occurrences and conditions that allowed the project team to get the innovation project back on track. We tried to assess whether innovation resilience behavior took place. A critical recovery can be regarded as an example of innovation resilience behavior. While IRB is an action of the team, a critical recovery can be caused by a condition external to the team, such as a decision by higher management, customer behavior, or a market development. The inventory of critical incidents and recoveries is based on a first interview with the team leader, a subsequent interview with the team members, and a third interview with the team leader and team members together. When a critical recovery is caused externally, it is not considered Team IRB. The end result presented represents a consensus of the whole team. This was done by first letting team members give the researcher feedback on the selected critical incidents and recoveries by their team leader in a separate group interview, and second, by having a follow-up interview with the team leader and team members together to discuss the researcher's findings from these former separate interviews with the whole team (those who took part in the interviews). For each team, the assessment of the critical incident(s) in their project was visualized by drawing the project's progress timeline and its subsequent actions on a sheet of paper and to indicate whether or not and when critical incidents and critical recoveries occurred. As indicated above, this was first done with the team leader, discussed with the team second, and, third, in the follow-up interviews it was checked for consensus with the team leader and team members together. What happened before, during, and after those critical events was discussed during the interviews.
It is instructive to mention that incidents, in the sense of critical incidents and recoveries, are perceived differently by the different respondents in the teams. More experienced teams differ from less experienced teams in how they assess events, including setbacks. In the interviews, the question was whether the team had experienced a critical incident, namely an occurrence or event that threatened the continuation of the innovation project and made it necessary to deviate from the original plan. Experienced teams that dealt with some setbacks perhaps viewed such events as insignificant ‘bumps in the road’ that one encounters when driving, whereas less experienced teams saw them as significant issues that hindered the innovation project. Some experienced teams were confident that the bumps ahead might be difficult but could still be handled effectively and were hesitant to even call those incidents ‘critical.’ These teams were confident about developments, whereas inexperienced teams were unsure about the possible effects.
In this analysis, we applied the distinction of team resilience to the qualitative interview data made by Alliger et al. (2015): minimizing, managing, and mending team resilience. We studied all 18 cases and evaluated the presence of team resilience behaviors; in other words, for every phase, before, during, and after (minimizing, managing, and mending) a critical incident, Alliger et al. (2015) distinguished four main groupings of behavior (see Table 1), which were used as an analysis grid, named the Team Resilience construct. We analyzed the recorded interviews per case and assessed the likeliness of the presence of the distinguished team resilience behaviors according to the Team Resilience construct in each of the 18 teams before, during, and after a critical event. A subjective expert judgment was made (by the first researcher) on the basis of the oral interview data. This researcher has (1) visited the company to get acquainted with its innovation projects and innovation management policy; (2) spent time with the team and its department to make observations and select a proper project for the study; (3) studied the innovation project documents, including the project plans and project management approaches applied by the company; (4) interviewed the team leader (twice), the team members as a group (twice), their accountable project manager, and analyzed about 99 hours of audio-recorded interviews; (5) carried out the quantitative survey analysis; and (6) wrote 18 case reports on each project team's innovation process during the project (Oeij, 2017).
Table 2: Team innovation resilience behavior (subscales and overall score) for the 18 teams and ‘other teams’ (team members in other teams).
Associating Quantitative and Qualitative Data
In the next step we will relate the findings assessed with the Team Resilience construct to Team IRB. For each team, we counted the number of team resilience behaviors according to the Team Resilience construct, which will be compared with the average score of Team IRB of every team by applying a t-test for the independence of both samples. In fact, in this article we confront the qualitative data with the quantitative data and try to draw conclusions from this exercise.
Results: Distinguishing Team IRB Across Teams
Table 3 presents the 18 teams and the presence of critical incidents and critical recoveries in key terms as described by the team. The teams are ranked according to their score for Team IRB (high to low).
Table 3: IRB cases and the presence of critical incidents and critical recoveries (source: Oeij, 2017).
Table 3 suggests that the critical incidents seem to be grouped into three clusters: (1) technical issues; (2) decision-making issues (including dissent, conflict, and vacuum); and (3) clustered incidents (a combination of events adding up to an incident).
The critical recoveries cluster in another way: there is (1) an active side in which we see [a] team initiative (adjustment of plan and outcome, monitoring, team building, clustered measures—7 times), [b] management initiative (new project leader, new steering group, Kanban team (meant here as an emergent problem-solving team), management support—4 times), and [c] project management tools (8D-solving team), risk management methods—4 times); and there is (2) a passive side where we see limited resilience and limited management commitment or reactive responses to market demands. The passive actions seem to dominate the low IRB cases (5 out of 6), whereas the active actions reside with the high IRB cases (11 out of 12).
Whether or not critical incidents lead to failure, is not entirely clear, because teams that did not show IRB did not consistently discontinue their projects, and there were also successful results for such low-scoring IRB-teams, as in the case of Team05. The teams that showed no IRB, however, seem to have encountered issues such as critical incidents during certain periods of the project that were not dealt with in a way that was effective for keeping on track toward the intended innovation goal. Team10 and Team13, for example, were well under way with their projects at the time the research was done, but had had previous periods in their projects with very limited or no progress, and demonstrated a team process that lacked team resilience. The project undertaken by Team06 was a project without any critical incidents, and thus it needed no critical recoveries; it went smoothly because of anticipatory risk monitoring, was a routinely run project, and as such, it was a deviant case in the sample, illustrating that IRB is irrelevant for routine processes in innovation projects. A comparable inference can be made for the case of Team08, which was an IRB team. Although this project did not encounter critical incidents and recoveries either, the team nonetheless acted with resilience by anticipating possible critical situations and holding intensive communication with its stakeholders. We do not know if the presence of Team IRB prevented critical incidents from emerging here, but it could be observed that the team largely consisted of very experienced team members and the team leader was also very experienced.
Table 3, therefore, does not reflect a black and white situation; the cases are nuanced. To understand the team dynamics better, we need to illustrate what the teams do. For this purpose, we will describe the critical incident and recovery and the team innovation resilience behaviors of four of the twelve teams showing IRB as examples.2 We use the division of critical incidents, namely technical issues, decision-making issues, and clustered events; we also provide an example of an IRB team without critical incidents.
Examples from Practice
1. Technical issues as critical incidents
The first example, with the highest score for Team IRB, is Team15, a team forming part of the R&D department of a manufacturer of components for electronic devices.
The project encountered several technical drawbacks, which were assessed by the experienced project leader as ‘small spikes on the road,’ such as one normally encounters. Nonetheless, the technical problems were not easy to tackle. Initially the team wanted to start with the toughest issues, based on the notion of ‘structural similarity,’ assuming that the smaller problems would then be solved more easily. One issue was that a supplier could not fix a technical problem for which they had IP rights. This caused months of delay. At a later stage, there were several technical issues in production and problems with finding solutions.
The project leader, who is experienced, mentioned matter-of-factly that in cases of trouble one needs to stay calm, look ahead at what needs to be done, and be reassured or confident that resources such as 8D-teams (see explanation below) will be made available if they are really needed. He saw his role as closely monitoring the process and talking to stakeholders at the appropriate times to see what options were available for solutions. In the case of the tough issues, it was decided during a business review meeting (an Exception Review) not to follow the structural similarity approach, but to reverse this approach: to start with easy problems and move ahead. What helped was that there was a client who could be supplied with components that could be produced on the basis of this ‘small solution.’ Later, when the other technical problems in production emerged, the organization applied its 8D-team method. Within 8D there is always a thorough method of analysis, a root cause analysis. Consequently, decisions are validated. Apart from that, team members learn systematically to avoid oversimplification and jumping to conclusions. The 8D-method can be seen as a valid procedure applied when such issues pop up: in fact as a successful aspect of project management. It is no surprise that the project leader explained that he did not see the issues as ‘critical incidents’ but rather as regular issues that appear in a project like this. Some project deviations are thus ‘normal.’
This team showed a number of innovation resilience behaviors. The team members were research-driven, which implies that they built decisions on validated knowledge. Technical issues were researched with rigor. Team members communicated and discussed findings in a critical but constructive way, meaning that they listened to each other and respected one another's expertise. The team leader acted calmly, without panicking; he structured the tasks and steps to be taken; he gave clear information to other departments (e.g., managing the expectations of the production department); he asked higher management for help when the need was urgent; he applied project management tools that support the analysis and management of risks; and he managed the team by walking around, meaning that he communicated intensively, built bridges between stakeholders, and kept the team together (team cohesion). The organization applied project management tools in innovation projects. One example is the 8D-team method, which means that a team is quickly brought together to solve any issue that arises. The main purpose of this tool is that the innovation process continues in order to meet the initial planning, while the 8D-team works on its special task.
Other teams in this category are Team09, Team16, Team17, and Team18.
2. Decision-making issues as critical incidents
Team01 is an R&D team in the dairy industry and the project was a co-innovation with another company.
The project had two main phases: research and development and application to business. The critical incidents in the first phase were a number of events: it took a great deal of time to mobilize the right stakeholders; there was a great struggle to agree on the IP rights with the partner, and the partner redefined the scope several times, which required more work and caused delays. In the second phase, it emerged that the business side of the partner had not approved the innovation, which was critical for the application of the innovation (an ingredient) in end products for customers. There was no convincing business case (perhaps there was too much of a technology push). The project team and project manager could not do much about this incident except to wait, as this was an internal matter on the partner's side. This matter was not resolved during the period of the study.
The role of a project leader is to monitor the process of an innovation project very closely. The way he or she does this is highly relational and personal. Bringing people together and keeping them in contact with each other (brokerage) is one of his or her central activities. When there was a conflict about IP rights, the project leader decided to have an external expert mediate in the matter as a process intervention. In the start-up phase he organized a session to develop possible routes to the end-product, so-called ‘conceptual approaches.’ Despite the fact that the partner redefined the scope a number of times, the progress to develop the ingredient went quite well. The project leader monitored the project very closely by stakeholder management, interventions, good timing and, when necessary, ‘threatening to quit’ to put functional pressure on others. His own project management approach was his guide.
The innovation resilience behavior of this team was centered on the project leader. The team members worked at different locations, and the team of the partner organization worked on their own premises. The project leader closely monitored the innovation process, the interests of the stakeholders, and the opportunities for interventions and actions. He was keen on relationships and managed the project by walking around and carefully planning his actions at the appropriate times. He decided on crucial actions, such as having a conflict mediated by an external expert or organizing sessions with the team to realize breakthroughs. To assist his work, the project leader developed a personal project management approach as a tool and stated that rhythm was important, which meant that meetings should be planned even if nothing is happening to gauge what is going on. In addition to all this, the team members were resilient in the way they strived as researchers to find valid data on which to base their decisions.
Another team in this category is Team04.
3. Combinations of clustered events as critical incidents
Team07 was a team within the R&D department of a food and cosmetics product producer and was responsible for deploying products. This deployment involved preparing the product in order to get it on the shelf on time for consumers for a specific market segment in a specific region or country.
There was a cluster of incidents that, taken together, were not very critical because they were managed quite well. In the collaboration between stakeholders there were some critical events: the local management of the brand (in another country) was very worried about the new product and demanded a large number of tests in order to be convinced and assured that it would be evaluated positively by consumers; the business development team resided in a different country but was to relocate to yet another country, and the composition of this team was changed to include quite inexperienced people (which led to a delay in artwork). Then there was a test location in another country again that had limited capacity, also causing delays. All these issues demanded that the project team closely monitor relations and manage expectations across different countries (four to five countries were involved).
The project team worked very hard. One example was that the testing was done extremely thoroughly, and took twice as much time as usual. Another example was that the communication with all the partners was meticulously prepared by the project leader and executed with a great deal of attention to detail and with validated information, especially for test results. The team worked very carefully and precisely and was well prepared. Much was done regarding risk management. The project was well scoped and managed by the project leader. There was some delay (caused by others) but the deadlines were not very rigid (customers do not notice a relaunch, so a delay is less sensitive in that regard).
The innovation resilience behaviors of this team were demonstrated in how they anticipated possible discussions and disagreements by performing thorough research and acquiring valid information. The team worked in a highly transparent and intensive way, and discussed strategies and actions toward stakeholders. The work was carefully scoped, documented, and planned. The team documented what they did, why they did it, what the results were, and so on, to make it explicit who was doing exactly what during the process. The team leader organized the process in a detailed way and was open to the opinions of team members. If she was in doubt, she consulted higher management promptly; after the project was finished she celebrated the results with the team. The organization applied project management tools to guide this process, especially for risk management and to create alternatives for what-if situations.
Other teams in this category are Team12 and Team14.
4. An absence of critical incidents
Team08 was a project team of an education center in the field of management, consultancy, and change; its innovation was a massive open online course (MOOC). This was the only case of a team with IRB in which there was no critical incident.
The project had no critical incidents. There were some setbacks, such as the fact that the work of the facilitators (these were former students who played an active role in the MOOC) proved harder than foreseen, causing some of them to terminate their participation; another point was that the interaction inside the MOOC was less than had been hoped for (the intention had been to create a great deal of online interaction and many learning situations). Setting up a MOOC was framed as a learning experience for the organization: even if it failed, it could not fail because the learning would still win. Therefore it was not easy to identify any critical incidents.
Apart from the absence of critical recoveries (as they were not needed) the team solved issues based on experience, intensive consultation (with facilitators), and some IT solutions (with the IT supplier) to simplify the MOOC structure and stimulate interaction. The limited interaction within groups was partly solved by enlarging/clustering the groups participating in the MOOC and by reducing the number of groups.
Despite the absence of critical incidents and recoveries, the team was mindful and alert about things that could go wrong and had prepared for what to do in such instances. It had thought out well and in advance how to develop this MOOC and communicated intensively with the stakeholders involved and with the MOOC participants (students).
Another team in this category is Team06.
From these cases we learn that: (1) decision-making issues are related to the issue of having sufficient power and influence to be able to steer the project. In cases with (2) technical issues, a great deal of weight was put on having validated data and knowledge about whether to move on or not. In cases with (3), clustered incidents and those without critical incidents, coordination and leadership played significant roles in keeping the project on track.
Linking the Team Resilience Construct and the Team IRB Scale
The next exercise is to cluster the innovation resilience behaviors of the twelve high IRB and eight low IRB cases according to the grid of Alliger et al. (see Table 1). Table 4 (with parts A and B) show the results of the analysis in which we applied the minimizing, managing, and mending team resilience behaviors to the 18 cases studied. We evaluated the presence of team resilience behaviors (i.e., a subjective expert judgment was made by the first researcher on the basis of the oral interview data) and discussed with the other three researchers (i.e., the three other authors). As discussed, we applied the Team Resilience construct based on Alliger et al. (2015) as an analysis grid. The team resilience behaviors of Alliger et al. make distinctions between what a team does before, during, and after a challenge or critical incident, which allows for a dynamic view of the course of a project. The researcher who analyzed the interviews assessed the likely presence of the team resilience behavior with the help of the grid and the results were, to a certain degree, discussed with the other three researchers (see Table 4a and 4b).
An ‘X’ implies an interpretation by the researcher (first author) that team resilience behavior was manifest or present or very likely to be present; an empty cell (blank space) means that the resilience behavior was not assessable, or was absent or latent. The central finding is that teams with high IRB are especially competent in managing and mending resilience behaviors and less good at minimizing resilience behaviors (boxes 33, 34, and 25, respectively, marked with an ‘X’ in Table 4a). This is a finding that corroborates Weick and Sutcliffe's (2007) statement about the big difference between HROs and other organizations: HROs are more alert to weak signals. The low IRB-scoring teams have much lower counts of team resilience behaviors, except for Team11. This team was highly alert in monitoring its project, but had difficulties getting top management support for their plan.
Table 4a: Innovation resilience behaviors of the 12 teams with a high IRB score.
Table 4b: Innovation resilience behaviors of the six teams with a low IRB score.
The following remaining observations can be made from Table 4. First, looking at the number of boxes marked with an ‘X,’ most teams with high IRB scores apply more than half of the team resilience behaviors identified by Alliger et al. Team15, Team07, Team16, and Team06 scored particularly well. In contrast, it can be observed that Team04 has fewer than half of its boxes marked with an ‘X.’ This perhaps is mainly due to the fact that this project was not carried out by a team in the sense of a group of people working on the same goal with interdependent tasks. The team members worked more or less separately on their tasks, and were only in direct contact with the project leader.
Second, most high-scoring IRB teams displayed a kind of personal or distributed leadership that could handle complex situations or enable them to switch leadership styles when a situation required it. The individual leadership of the project leader was a leverage factor in teams such as Team15, Team01, Team12, and Team06, whereas distributed leadership among team members played a role in Team07, Team14, and Team16. Leadership is part of the mindful infrastructure, hence it is conditional to IRB (Oeij, 2017).
Third, in five high-scoring IRB teams we found no distinct team resilience behaviors in the minimizing phase of preventing critical incidents from occurring. The case of Team18 is clear, as this was a project created to recover delivered products that did not function reliably. The Team04 case signifies a transition from exploration to exploitation with a new team leader and the replacement of team members to make this transition work. In the same vein, Team17 consisted of a team that had completely replaced the original team that had started the project, and it gradually gained grip and direction after critical incidents emerged. Team14 was an ongoing generation-innovation, namely the release of a new version of an existing product, in which the team was confronted with a take-over by another company that introduced new tools and procedures and team-based working. The product had to be adjusted for international markets, which did not allow for proactive minimizing actions. Team09 did not show minimizing behavior, because the first phase of the project was characterized by failures and a very limited grip on the project by the team and its leader.
Fourth, most high-scoring IRB teams performed mending behaviors, which means that these teams learned from what had happened during critical incidents and adapted to the changed situation when necessary. If we look again at the teams for which minimizing behaviors were absent (Team09, Team17, Team14, Team18, and Team04), it is interesting to observe that they all performed mending behaviors at a later stage.
The teams with low IRB scores (see Table 4b) differ within their group in that some teams do have team resilience activities in every phase (Team05, Team10, and Team 11) and other mainly have a few team resilience activities in the mending phase (Team02, Team03, and Team13).
In the next step, we relate these findings of team resilience to Team IRB by making an interpretation and adding the survey results to this interpretation. Team resilience as operationalized by Alliger et al. (2015) shows considerable overlap with innovation resilience behavior (IRB) (see Table 1) if we focus on the 12 high-scoring IRB teams. It comes as no surprise that these 12 teams performed well on a number of dimensions of team innovation resilience behaviors, which therefore support the central conclusion. As indicators of ‘managing and mending actions,’ that is, actions for dealing with critical incidents, all 12 high-scoring IRB teams (see Table 2) scored rather highly on sensitivity to operations (M = 4.97, SD = 0.89, range 1–7), scoring 4.9 or above, on commitment to resilience (being able to change course if needed) (M = 4.74, SD = 0.78, range 1–7), scoring 4.7 or above, and on deference to expertise (expertise is valued higher than rank) (M = 4.88, SD = 0.96, range 1–7), scoring 4.6 or above. On their ability to anticipate possible critical incidents, these 12 teams had favorable scores on ‘minimizing actions’ as well; they scored 4.7 or higher on preoccupation with failure (being alert for weak signals) (M = 4.80, SD = 0.92, range 1–7), and 4.5 or higher on reluctance to simplify (preferring validated data) (M = 4.75, SD = 0.87, range 1–7). The average score for team innovation resilience behavior was M = 4.82 (SD = 0.75, range 1–7) and all 12 teams had a value of 4.81 or above. High-scoring IRB teams again scored better on managing and mending behaviors than on minimizing behavior. On average, the teams with low IRB scores perform, not surprisingly, worse on Team IRB (see Table 2).
As a step to check the robustness of the qualitative analysis, an independent sample t-test was conducted to compare the two groups of teams with, respectively, a Team IRB mean below or above the median score of Team IRB. The group of teams with a low Team IRB score had a significantly lower mean TR count score (M = 4.89, SD = 2.98, N = 9), than the group of teams with a high Team IRB score (whose mean TR count score was 8.00, SD = 2.06, N = 9; t(16) = 2.58, p < 0.05, two-tailed). The magnitude of the differences in the means (mean difference = 3.11, 95% CI: –5.67 to 0.55) was quite large (eta squared = 0.29). Hence, both methods–quantitative Team IRB score and qualitative Team Resilience counts—provide a similar outcome, which can be regarded as significant support for the construct validity for the Team IRB scale.
The contribution of this study is that the Team IRB-scale concept, derived from crisis management and safety science, can be applied to qualitative data in the realm of innovation management. The research supports the notion that the IRB concept, based on HRO principles, can thus be applied to both the quantitative and qualitative data of team behavior in innovation projects. Just as is the case with the original Audits of Resilience Performance scale of Weick and Sutcliffe (2007), applied to the realm of crisis management and safety science, the items of the Team IRB can be used as survey items or questions that can be used for in-depth interviews. The Team Resilience construct, based on Alliger et al. (2015), was useful in applying to the qualitative data as it overlaps with the Team IRB scale. Although this construct is not branch or sector-specific, it has more affinity with the realm of HROs than non-HROs, because the review of Alliger et al. mainly studied the literature of crisis management and safety science, which is why it is relevant and could also be applied to innovation management issues. Most teams show an association between their IRB score and their team resilience counts, but not all. Team06, for example, does not have a self-reported high-IRB score, but has the maximum number of team resilience counts, which is in line with our decision to regard this team as high-scoring IRB based on our in-depth interviews. Perhaps this team is rather modest in its self-assessment. We point this out because, despite the fact that there are outliers like Team06, there seems to be a pattern to the data.
Some critical remarks should be made about our study. One critical reflective remark is that the grid of Alliger et al. (2015) was applied after the data had been collected. An expert judgment was made on how this grid of team resilience behaviors made a fit with the data, which is a subjective activity and possibly leads to errors of judgment. Another critical observation is that the cases include factors that vary across teams and could not be held constant. For example, some projects were not fully innovation projects in the sense that something new or something improved was being developed. Despite the fact that some kind of renewal took place in all cases, Team12 and Team16 were largely carrying out implementation projects, and Team18's project was, to a considerable degree, a recovery project. Because the study does not focus on innovative or creative behavior as such, it could be argued that the findings do not solely apply to innovation teams, but perhaps are applicable to a broader range of project teams, including organizational change and implementation teams. A final critical remark is that the analysis was carried out by one person (the first author) and that no inter-relater-reliability tests could be carried out due to the vastness of audio-recorded data and additional sources. We tried to compensate for this situation by showing that the researcher (1) studied many sources and (2) sought the feedback of interviewees at crucial stages in the study—notably concerning the assessments of critical incidents and what teams did to deal with those; (3) had intensive discussions among all four authors to minimize the risks of method bias, fallacy of incomplete evidence (‘anecdotalism,’ cherry picking), and stimulate countervailing evidence (Silverman, 2013) and, thus enhance the validity of the findings.
This article addressed the question of whether the quantitative Team-IRB scale, developed for surveying larger populations, could be applied to qualitative data by using the Team Resilience construct, the grid of team resilience of Alliger et al. (2015), as an operationalization of the scale. To test this, we then analyzed how innovation teams deal with critical incidents and what innovation resilience behavior looks like by analyzing what the teams in the study did in practice, by using the Team Resilience construct. Finally we applied a t-test to statistically confirm that the lower and higher scoring IRB teams significantly differed in their Team IRB score and Team Resilience construct counts. From the results, we conclude that the Team IRB scale can also be used for qualitative data. There is much agreement on the findings of the survey data and the findings of the in-depth interview data. The study results are in corroboration that the Team IRB scale and the Team Resilience construct show much resemblance. The difference between the two is that the Team IRB scale stresses team behavior related to innovation projects, whereas the Team Resilience construct is more generic. Furthermore, the IRB scale can be applied quickly in a (short) survey, whereas the Team Resilience construct application requires a more labor-intensive approach. The Team Resilience construct can, however, better distinguish the minimizing, managing, and mending phases, both empirically and theoretically. In future research, the theoretical contribution could be extended to innovation management models that also have phases such as ideation, prototyping, and testing, and implementation and evaluation.
The methodological conclusion here is that the construct validity of the Team IRB concept allows its application to both quantitative and qualitative data. The content conclusion stresses the differences between high and low-scoring IRB teams in their dealing with critical incidents, for which the first group seems to outperform the second one. The 12 high-scoring IRB teams (out of the total of 18) that had a relatively high score for Team IRB varied somewhat in their actions. The variation in Team IRB relates particularly to the degree to which teams showed more ‘managing and mending’ team resilience behavior than ‘minimizing’ behavior, which would prevent or curb the escalation of critical incidents.
Apart from variations in IRB there are also commonalities. All 12 high-scoring IRB teams showed IRB in relation to ‘managing actions’ and were sensitive to what went on in their operations and were able to change course if needed. Some of the teams that did not show IRB were not resilient in this regard; they either lacked the power if, for example, there were external powers that could not be influenced (Team11), or they lacked a consensus to set out on a new course (Team02, Team03) (see Table 3). In most cases, the role of leadership, either that of the project leader or as distributed among team members, was important. Effective leaders proved to be able to switch styles or apply particular styles when required. Leadership must be dynamic to a certain extent in order to meet these situational requirements. Most teams also performed ‘mending actions’; they learned from critical incidents and adapted to the new situation. Some teams were self-reliant, with autonomous team members who performed tasks without being asked. Resilience can therefore be regarded as a form of professionalism, in which team members understand what needs to be done and have an intrinsic drive to carry out their tasks.
This study indicates that the concept of HRO principles used in safety management and crisis management studies are not only applicable to innovation management contexts. The applicability of HRO principles might even be broader because, as stated earlier in our study, there were teams that may not have had all the characteristics of innovation teams. Other types of project teams encountering critical incidents or high uncertainty and risks may benefit from these findings. Further research into the applicability of Team IRB could incorporate teams performing several types of projects—ranging from manufacturing and building ‘things’ to change management and implementation ‘processes.’ Team Innovation Resilience Behavior appears to be a promising concept.
An earlier version of this contribution was presented at EURAM 2016 (Paris, France, June 2016), where it was awarded ‘Best Student Paper’ by the Project Organising General Track. The authors would like to thank the participants for providing valuable feedback on a paper presented earlier, notably Alexander Kock who was a co-reviewer during the session; Ernest de Vroome of TNO provided methodological advice; Samuel Quain (Koforidua Polytechnic, Ghana); and Sarah Frith (Proof-reading-service.com) who proofread major parts of the text. We also thank Hans Georg Gemünden, the Editor-in-Chief of Project Management Journal®, for his support.
Alliger, G. M., Cerasoli, C. P., Tannenbaum, S.I., & Vessey, W.B. (2015). Team resilience: How teams flourish under pressure. Organizational Dynamics, 44(3), 176–184.
Castellion, G., & Markham, S. K. (2013). Perspective: New product failure rates: Influence of argumentum ad populum and self-interest. Journal of Product Innovation Management, 30(5), 976–979.
Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327–358.
Mulder, N. (2016). Value-based project management. How to bring focus into complex projects. Elst, Belgium: Mulder Projectmanagement.
Oeij, P. R. A. (2017). The resilient innovation team. A study of teams coping with critical incidents during innovation projects. PhD dissertation. Open University of The Netherlands. Retrieved from http://publications.tno.nl/publication/34622536/QA3j9S/oeij-2017-resilient.pdf
Ray, J. L., Baker, L. T., & Plowman, D. A. (2011). Organizational mindfulness in business schools. Academy of Management Learning and Education, 10(2), 188–203.
Silverman, D. (2013). Doing qualitative research: A practical handbook (4th ed., 1st ed. 2000). Los Angeles, CA: Sage.
Sutcliffe, K. M., Vogus, T. J., & Dane, E. (2016). Mindfulness in organizations: A cross-level review. Annual Review of Organizational Psychology and Organizational Behavior, 3, 55–81.
Vogus, T., & Iacobucci, D. (2016). Creating highly reliable health care: How reliability-enhancing work practices affect patient safety in hospitals. ILR Review, 69(4), 911–938.
Vogus, T. J., & Sutcliffe, K. M. (2007). The safety organizing scale: Development and validation of a behavioral measure of safety culture in hospital nursing units. Medical Care, 45(1), 46–54.
Vogus, T. J., & Welbourne, T. M. (2003). Structuring for high reliability: HR practices and mindful processes in reliability-seeking organizations. Journal of Organizational Behavior, 24(7), 877–903.
Weick, K. E. & Sutcliffe, K. M. (2007). Managing the unexpected: Resilient performance in an age of uncertainty (2nd ed.). San Francisco, CA: Jossey-Bass.
Weick, K. E., Sutcliffe, K. M., & Obstfeld, D. (1999). Organizing for high reliability: Processes of collective mindfulness. In: R. S. Sutton and B. M. Staw (Eds.), Research in organizational behavior, Volume 1 (pp. 81–123). Stanford, CA: Jai Press.
Peter R. A. Oeij, PhD, holds master of arts degrees in history and sociology from Erasmus University, Rotterdam, the Netherlands; a master of science degree in psychology; and he received a PhD in management science (both from Open University, Heerlen, the Netherlands). Dr. Oeij was affiliated with IVA Tilburg Institute for Social Research, Tilburg University, the Netherlands; the Open University of The Netherlands (as a PhD candidate); and is a senior researcher/consultant for TNO, The Netherlands Organisation for Applied Scientific Research, in Leiden, the Netherlands. He can be contacted at email@example.com
Steven Dhondt, PhD, is a visiting professor at the KU Leuven (Belgium; Chair, Social Innovation) and a senior research scientist at TNO, The Netherlands Organisation for Applied Scientific Research, Leiden, the Netherlands. Professor Dhondt received degrees from the KU Leuven (in political sciences and sociology), TIAS (in information management), and the University of Leiden (a PhD in networking between organizations). He can be contacted at firstname.lastname@example.org
Jeff B. R. Gaspersz, PhD, is a full Professor of Innovation at Nyenrode Business University in the Netherlands and director/owner of an innovation consulting company. Professor Gaspersz previously lectured at the Erasmus University Rotterdam and worked for KPMG as manager of the KPMG Trendwatch Center and as Director of the KPMG Center for Innovation in the Netherlands. He can be contacted at email@example.com
Tinka Van Vuuren, PhD, holds a chair in Strategic Human Resource Management (i.e., Vitality Management) at the Open University of The Netherlands. Since 2006, Professor Van Vuuren has been working as an organizational psychologist for APG/Loyalis, a Netherlands-based pension fund and insurance company and advises and conducts research for public and private organizations. She can be contacted at tinka. firstname.lastname@example.org
Appendix: Legend to the Tables: A typification of the organizational embedding of the project teams
|Team01||R&D department in agribusiness|
|Team02||Consultation firm in engineering|
|Team03||Consultation firm in engineering|
|Team04||Consultation firm in IT/ICT|
|Team05||R&D department in food and cosmetics|
|Team06||R&D department in food and cosmetics|
|Team07||R&D department in food and cosmetics|
|Team08||Training firm for organizational change professionals|
|Team09||IT department of education organization|
|Team10||Governmental organization in construction / engineering|
|Team11||Governmental organization in construction / engineering|
|Team12||Change team in a municipality|
|Team13||Manufacturer of medical equipment|
|Team14||Manufacturer of medical equipment|
|Team15||R&D department in manufacturing|
|Team16||R&D department in manufacturing|
|Team17||Manufacturer of transport equipment and materials handling|
|Team18||Manufacturer of transport equipment and materials handling|
Project Management Journal, Vol. 48, No. 5, 49–63
© 2017 by the Project Management Institute
Published online at www.pmi.org/PMJ
1For reasons of privacy, the cases are anonymous. At the end of this article, a short description of each case (project and team) is provided in the Appendix.
2 All 12 remaining examples are described in Oeij, 2017; in that source—the overall multi-case study—all 18 cases are studied from different angles as well.