Benefits Realisation Management and its influence

Abstract
Business strategies, which imply organisational change, usually require the development of projects, e.g. IT projects. However, organisations fail in
implementing their strategies even though they employ project, programme and portfolio management techniques. Benefits Realisation Management
(BRM) is a set of processes structured to close the gap between strategy planning and execution by ensuring the implementation of the most valuable
initiatives.However, there is no empirical evidence of its effectiveness. This paper presents the results of a survey to practitioners in Brazil, United Kingdom
and United States evaluating the impact of BRM practices on project success rate. Our results show BRM practices being positive predictors to project
success on the creation of strategic value for the business. Therefore, these results suggest that BRM practices can be effective to support the successful
execution of business strategies.
© 2014 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-SA license
(http://creativecommons.org/licenses/by-nc-sa/4.0/).
Keywords: Project management; Benefits realisation; Strategy implementation; Strategy execution; Project success; Project governance

  1. Introduction
    Industry reports, e.g. The Economist (2009), German Project
    Management Association (2010) and Price Waterhouse Coopers
    (2007), suggest that practitioners recognise projects as a structured
    way to implement business changes, an opinion also shared by
    academics e.g. Buttrick (1997), Kerzner (2009) and Turner (2009).
    Project success is a vital component of business success (Price
    Waterhouse Coopers, 2007) and the global economy. Although
    projects in an organisational portfolio can address different
    objectives (Gray and Larson, 2006; Jenner, 2010; Kendall and
    Rollins, 2003; Levine, 2005), they are mainly undertaken to
    support the execution of business strategies (Buttrick, 1997).
    Therefore, organisations need to ensure the success of their
    projects in order to succeed in executing their strategy and in
    turning their vision into reality.
    In order to be successful, project management teams need
    to define clearly how to evaluate whether each project is
    successful. However, there is no consensus on the definition
    of project success (Prabhakar, 2008; Yu et al., 2005). A recent
    analysis of articles published from 1986 to 2004 in the
    International Journal of Project Management and the Project
    Management Journal has found 30 articles discussing project
    success, but with no consensual definition (Ika, 2009). In
    parallel, surveys performed in the last twenty years have
    found between 60% and 80% of all organisations failing in
    executing their strategies by not delivering the expected
    outcomes of their changing process (Kaplan and Norton,
    2008).
    This paper, analyses success by two different approaches:
    Project management performance, also called efficiency,
    which evaluates success mostly based on budget, schedule
    ⁎ Corresponding author. Tel.: +44 7428 225343 (mobile).
    E-mail addresses: [email protected] (C.E.M. Serra),
    [email protected] (M. Kunc).
    www.elsevier.com/locate/ijproman
    http://dx.doi.org/10.1016/j.ijproman.2014.03.011
    0263-7863/00/© 2014 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-SA license
    (http://creativecommons.org/licenses/by-nc-sa/4.0/).
    Available online at www.sciencedirect.com
    ScienceDirect
    International Journal of Project Management 33 (2015) 53–66
    and requirements goals; and project success, which evaluates how
    well projects deliver the benefits required by business strategies in
    order to meet wider business objectives and to create value
    (Cooke-Davies, 2002; Serrador, 2013). Despite the clear role
    projects have in implementing business strategies, organisations
    are still evaluating projects only by their efficiency and not by the
    benefits delivered and a large group of organisations claims that
    project benefits are very hard to measure (Zwikael and Smyrk,
    2012), especially benefits realised during product operation, often
    long after project end (Yu et al., 2005).
    Recently, some scholars (Bradley, 2010; Jenner, 2010; Melton
    et al., 2008) have suggested that Benefits RealisationManagement
    (BRM) makes the value and the strategic relevance of each project
    clear, enabling an increased effectiveness of project governance.
    More than just governance, ‘strategic governance’ leads organisations
    to work towards the delivery of planned benefits
    (Gardiner, 2005). Organisations with mature processes of benefit
    realisation – and therefore stronger governance – have their
    management boards prioritising and supporting mostly those
    projects which can deliver the most relevant benefits. By
    increasing the effectiveness of project governance, Benefits
    Realisation Management can arguably reduce project failure rates
    from a strategic perspective. However, these practices are not
    widely employed yet, or employed as a subset of other project
    management processes, and there is scant evidence about its
    impact on project success (Cooke-Davies, 2002). Thus, this paper
    intends to evaluate the use of Benefits Realisation Management
    among the project management communities of three countries:
    United Kingdom, United States and Brazil in order to understand
    its impact on project success rates and evaluate the impact of
    projects on the creation of organisational value (Bryde, 2005; Yu
    et al., 2005; Zwikael and Smyrk, 2012).
  2. Theoretical background
    After organisations set their visions and create their strategy, the
    management team creates individual projects or programmes,
    which are groups of projects managed together (Thiry, 2002), to
    deliver the business strategy. However, organisations do not have
    infinite resources to invest (Amason, 2011) so they choose those
    projects that deliver the most valuable results for the implementation
    of the business strategy (Amason, 2011; Gray and Larson,
    2006) in the most effective and efficient way (Gray and Larson,
    2006). Then, organizations use project portfolio management
    methods, such as financial and non-financial appraisal and
    evaluation models, to select and prioritise the best set of projects
    (Jenner, 2010).
    Once the correct projects are selected, project success can be
    assessed in two steps usually called appraisal and evaluation. The
    appraisal occurs before the beginning of each project in order to
    support the approval of the business case, while the evaluation
    occurs at project closure in order to identify project success or
    failure (Jenner, 2010; Zwikael and Smyrk, 2011). The appraisal
    measures the relevance of each project and defines expectations,
    which are inputs for the definition of success criteria. Since
    projects are investments which usually aim to maximize return,
    an important part of this step is the financial appraisal (Jenner,
    2010; Levine, 2005) or feasibility studies (Yu et al., 2005). Later,
    the evaluation analyses the actual achievements against those
    success criteria previously defined in order to identify whether
    projects were successful (Jenner, 2010; Zwikael and Smyrk,
    2011).
    While there are several different models to measure project
    success, many authors, such as Baccarini (1999) and Pinto and
    Mantel (1990), agree on two approaches to its assessment:
    project management performance and delivery of benefits to
    the business, clients and stakeholders. In the past, project
    success was evaluated mostly based on criteria associated to
    the “triple constraint”: cost, schedule and scope (Ika, 2009;
    Shenhar and Patanakul, 2012; Zwikael and Smyrk, 2011), which
    are strongly related to the evaluation of project management
    performance, usually assessed using Key Performance Indicators
    – KPIs – designed to measure the adherence to budgets,
    schedules and technical specifications (Bryde, 2005). However,
    a complete evaluation of success requires a value related
    component (Kerzner, 2011), replacing this evaluation method
    for another focused on the project contribution to the business
    strategy (Patanakul and Shenhar, 2012) including the creation
    of shareholder value (Ika, 2009; Levine, 2005).
    Ika (2009) splits the benefit related component of the
    assessment into ‘Project/Product Success’ – satisfaction of end
    user and benefits to stakeholders and project staff – and
    ‘Strategic Project Management’ – business success, achievement
    of client’s strategic objectives. More recently, Camilleri
    (2011) divides benefit between ‘project success’ – outcomes
    and benefits – and ‘Project Corporate Success’ – the achievement
    of strategic objectives. Zwikael and Smyrk (2011) also separates it
    into ‘Ownership Success’ – benefits less dis-benefits and costs –
    and ‘Investment Success’ — financial return to the organisation.
    Although these authors have suggested different ways to
    assess the delivery of benefits and the consequent creation of
    strategic value to the business, this paper suggests that the
    delivery of benefits to stakeholders has to be related to
    business strategies and to the achievement of wider business
    objectives, especially by the financial perspective, considering
    ‘project success’ as a more comprehensive approach
    (Cooke-Davies, 2002).
    Although there are several criteria available to evaluate project
    success, the judgment of success or failure can be taken based on a
    more situational or subjective basis (Ika, 2009; McLeod et al.,
    2012). Different perspectives using the same criteria can evaluate
    the same project as a success and as a failure. On the other hand, a
    set of criteria can be suitable to some perspectives but unsuitable
    for others. For example, project management success, ownership
    success and investment success are assessed by different
    perspectives and criteria (Zwikael and Smyrk, 2011). Nevertheless,
    project managers are responsible for the alignment of
    expectations among stakeholders in order to define project success
    (Kerzner, 2011). Interestingly, these same project managers are
    usually kept apart of the rationale for project selection and
    prioritisation, so they may not understand the relevance of their
    projects in order to deliver the expected benefit to the business
    (Melton et al., 2008). Thus, a question remains unanswered for
    them: what value do businesses need?
    54 C.E.M. Serra, M. Kunc / International Journal of Project Management 33 (2015) 53–66
    2.1. What value do businesses need?
    Good business strategies are those that deliver stakeholder
    value, which is the organisation’s long-term cash generation
    capability or the ability to provide value public services, in case
    of public sector organisations (Johnson and Scholes, 2002).
    These business strategies set targets of future value, which are
    met by achieving strategic objectives. Since these objectives are
    measurable, the difference between the current situation and the
    target future situation sets the value gap, which is fulfilled by a
    portfolio of initiatives defined by the organisation in their
    strategic plan (Kaplan and Norton, 2008). As Fig. 1 illustrates,
    strategic initiatives usually fill the value gap by enabling new
    capabilities – or promoting changes – through the outputs
    delivered by a set of projects.
    Projects are organisational entities which employ resources
    organised on a new and unique way, for a specific time-frame, to
    enable positive and clearly defined changes in the business (Turner
    andMüller, 2003). These positive changes aim the achievement of
    organisational objectives and these strategic improvements in the
    business are called ‘benefits’. Benefits, which can be seen as
    improvements, are increments in the business value from not only
    a shareholders’ perspective but also customers’, suppliers’, or even
    societal perspectives (Zwikael and Smyrk, 2011). Benefits are
    usually achieved using programme and project management
    techniques. Therefore, the creation of value for business, by the
    successful execution of business strategy, strongly depends on
    programmes and projects delivering the expected benefits.
    Based on the benefit mapping techniques suggested by Thorp
    (2007), Ward and Daniel (2006), and Bradley (2010) and
    practitioners’ guides (Chittenden and Bon, 2006; Jenner, 2012;
    and OGC, 2007), a conceptual example of benefits realisation,
    starting from projects and reaching the achievement of business
    objectives, is presented on Fig. 2. Conceptually, the process starts
    on project outputs enabling business changes or directly delivering
    intermediate benefits. Business changes create outcomes, which
    prepare operations to realise benefits. Alternatively, business
    changes can also deliver intermediate benefits, regardless whether
    they are enabled by project outputs or not. They can also cause side
    effects, which are the negative outcomes from change, such as
    requirement of additional skills or cost increases. These side effects
    and consequences can also realise further intermediate benefits.
    Intermediate benefits contribute to the achievement of end benefits
    (Bradley, 2010) and end benefits directly contribute to the
    achievement of one or more strategic objectives of the organisation.
    Usually, end benefits are results of changing processes
    composed by sets of projects that are managed together as a
    programme (Bradley, 2010), which coordinates work in a synergic
    way to generate more benefits than projects could do individually
    (Thiry, 2002).
    Therefore, from a strategic perspective, successful projects
    deliver the expected benefits, then creating strategic value to
    the business. Careful management of each project ensures the
    delivery of outputs, enables outcomes, and then supports the
    realisation of the right benefits. Although benefits are not the
    only criteria to evaluate project success, they are a measurement
    of how valuable a project is. This is the realm of Benefits
    Management Realisation.
  3. Research methodology
    This research aims to test the relationship between BRM
    practices and perceptions of project success. Then, in order to
    elucidate a phenomenon by testing the relationship between
    variables, we performed a survey study using questionnaires
    and data analysis using analytical survey tools (Blaxter et al.,
    1996).
    3.1. Sampling procedures
    The sample was selected by stratified random sampling
    procedures over a population composed by Project Management
    Fig. 1. Filling the value gap.
    C.E.M. Serra, M. Kunc / International Journal of Project Management 33 (2015) 53–66 55
    Practitioners who have worked in the area in the last two years, in
    at least one project that is now concluded and in one of the three
    countries under analysis: Brazil, United Kingdom (UK) and
    United States of America (USA). We selected practitioners from
    USA because this is the largest community, the UK to provide a
    European perspective, while Brazil was included given its status
    of emerging market. The sample was stratified, because the
    assessment of independent strata of the population enables
    inter-group analysis (Field, 2009), which could confirm a regular
    pattern or lead to divergent results (Teddlie and Tashakkori, 2009).
    Project team members were specifically targeted since they are
    participants in their projects. Even though they may not have a
    complete overview; project management teams know the relevance
    and priority of project outcomes (Gray and Larson, 2006) and to
    have experienced the dynamics of project management, including
    roles, techniques and practices.
    In order to analyse experiences and to avoid loss of details or
    veracity, the data structure was defined as cross sectional, referring
    to one specific event occurred in no more than two years. That
    decision was made because the memory of respondents is
    compromised depending on how long the events under analysis
    has occurred (Foddy, 1993; Iarossi, 2006). When focusing on a
    single event some experiments show that in three years up to
    around 50% is irretrievable, but around a period of two years only
    10% to 30% of the details are irretrievable (Iarossi, 2006).
    3.2. Questionnaire design
    The quantitative questionnaire was composed by closed
    questions requiring respondents to identify perceptions of project
    success and BRM practices identified from the literature (see
    Tables 1 and 2) plus controlling variables. In order to identify
    respondents’ perceptions on project success and on how much
    BRMpractices had been applied in their previous experience, most
    questions were closed and subjectively responded by rating scales,
    Likert Scales. Likert scales are suitable to evaluate people’s
    Fig. 2. Chain of benefits.
    Table 1
    Questions and references. Project success criteria.
    Please rate how much you agree with the following statements from three different perspectives (project team, project sponsor, and project customer)
    Dimension Item Name Sources
    Project success PS The project was successful *
    Project management
    performance
    PSB The project has satisfactorily met the budget goals Zwikael and Smyrk (2011), Camilleri (2011),
    PSS The project has satisfactorily met the schedule goals Ika (2009), Shenhar and Patanakul (2012)
    PSR The project has satisfactorily delivered the required outputs (i.e. fulfilled its requisites)
    Creation of value
    for the business
    PSE Project’s outputs have supported the business to produce the expected outcomes
    PSU Undesired outcomes were managed and avoided
    PSI The project has provided the expected return on investment
    PSC The project’s outcomes adhered to the outcomes planned in the business case
  • General perception of success. No specific criteria or reference.
    56 C.E.M. Serra, M. Kunc / International Journal of Project Management 33 (2015) 53–66
    subjective states, such as opinions and perceptions (Iarossi, 2006)
    by rating how much the respondent agree to a declarative statement
    by using five categories from “strongly agree” to “strongly
    disagree” (Peterson, 2000). The same strategy has been previously
    employed on similar research about project success (Scott-Young
    and Samson, 2008). The questionnaire had no “opt out” question,
    because letting the respondent opt out, such as responding “do not
    know” increases the number of people not answering the question
    (Iarossi, 2006).
    Since project success is better understood when assessed by
    different perspectives (McLeod et al., 2012), our survey has an
    approach similar to the one suggested by Zwikael and Smyrk
    (2012), which divides project accountabilities among project
    management, project funder and project owner. However, in
    order to make it easier for respondents to associate each
    perspective to common roles, the questionnaire asked respondents
    to state their perception of success from the perspectives:
    Project team, project sponsor and project customer.
    In order to obtain qualified support to data gathering and
    validation from recognised professional bodies, the questionnaires
    were submitted to the Project Management Association
    (APM) and to the Project Management Institute (PMI). APM is
    the largest association of project management practitioners
    based on the UK (Association for Project Management, 2013)
    and it is a member of the International Project Management
    Association (IPMA), which is a European project management
    federation (International Project Management Association,
    2013). PMI is the largest institute based in the USA related to
    the field of project management (Project Management Institute,
    2013). Both organisations develop and support research on
    project management subjects, have developed their own project
    management bodies of knowledge and provide professional
    services to members and non-members, such as training,
    professional qualification, peer-reviewed and non-peerreviewed
    publishing, and networking. Both institutions have
    reviewed the questionnaires, and then advertised the survey
    using their institutional websites.
    Table 2
    Questions and references —BRM practices.
    Please rate how much you agree that, during the project’s execution …
    Item Practice Sources
    BRM1 Expected outcomes (changes provided by project outputs) were clearly defined Zwikael and Smyrk (2011), Bradley (2010), Melton et al. (2008) OGC
    (2007), Chittenden and Bon (2006), Buttrick (1997).
    BRM2 The value created to the organisation by project outcomes was clearly measurable Zwikael and Smyrk (2011), Bradley (2010), Jenner (2010), Melton et al.
    (2008), OGC (2007), Hubbard (2007), Chittenden and Bon (2006), Levine
    (2005), British Standards Institute (2000).
    BRM3 The strategic objectives that project outcomes were expected to support the
    achievement of were clearly defined
    Bradley (2010), Melton et al. (2008), OGC (2007), Kendall and Rollins
    (2003).
    BRM4 A business case was approved at the beginning of the project, describing all
    outputs, outcomes and benefits that were expected from the project
    Bradley (2010), Jenner (2010), Chittenden and Bon (2006), Buttrick (1997).
    BRM5 Project outputs and outcomes were frequently reviewed to ensure their
    alignment with expectations
    Amason (2011), Bradley (2010), OGC (2007), Chittenden and Bon (2006),
    Levine (2005), Thiry (2002), Buttrick (1997).
    BRM6 Stakeholders were aware of the results of project reviews and their needs
    were frequently assessed with a view to make changes
    Bradley (2010), OGC (2007), Chittenden and Bon (2006), Kendall and
    Rollins (2003).
    BRM7 Actual project outcomes adhered to the expected outcomes planned in the business
    case
    Bradley (2010), OGC (2007), Chittenden and Bon (2006), Levine (2005),
    Buttrick (1997).
    BRM8 Activities aiming to ensure the integration of project outputs to the regular
    business routine (training, support, monitoring, and outcomes evaluation)
    were executed as part of the project’s scope
    OGC (2007), Chittenden and Bon (2006).
    BRM9 After project closure, the organisation kept monitoring project outcomes in
    order to ensure the achievement of all benefits expected in the business case
    OGC (2007), Chittenden and Bon (2006).
    BRM10 From the first delivery to the project’s closure, the organisation performed a
    pre-planned, regular process to ensure the integration of project outputs into
    the regular business routine (including outcomes evaluation)
    Bradley (2010), OGC (2007), Chittenden and Bon (2006).
    BRM11 A project benefits management strategy is applied throughout the company Breese (2012), Jenner (2010), OGC (2007), Thorp (2007), Chittenden and
    BRM12 A project benefits management strategy was applied for the project under analysis Bon (2006).
    Table 3
    Controlling variables.
    Controlling variables
    Variable % Cases
    Region of project execution
    USA 23.00 76
    United Kingdom 19.00 63
    Brazil 47.10 156
    Others 10.90 36
    Role of the respondent
    Project governance role 15.40 51
    Project sponsorship role 0.91 3
    Project management role 77.04 255
    Other role 6.65 22
    Total 100.00 331
    Sponsor and customer are the same person
    Yes 31.10 103
    No 68.90 228
    C.E.M. Serra, M. Kunc / International Journal of Project Management 33 (2015) 53–66 57
    3.3. Respondents
    Nine hundred invitations were sent to project management
    practitioners through the social network LinkedIn (300 per
    country) at the beginning of 2012. In addition, the survey was
    advertised at electronic social networks and the websites of
    organisations specialised in project management. Until July
    2012, 331 responses were received, as presented in Table 3.
    The final response rate was 32%, similar to the response rate of
    31% considered as acceptable by Ritson et al. (2012) on their
    e-mail survey about successful programmes. Although invitations
    were sent only to the three selected countries, 36
    responses were received from other countries and employed
    on the general analysis, but they were not considered on
    comparisons between countries.
    3.4. Limitations
    While our survey shows that there is separation among these
    three roles (see Table 3), the results do not show significant
    differences in their opinions. The high number of cases with the
    same person playing at the same time the roles of Sponsor and
    Customer combined to a large number of respondents being
    part of project teams may have influenced the results, taking us
    to a narrower view, mainly from the eyes of project team
    members, and to a partial fusion of the Sponsor and Customer
    perspectives.
    Additional relevant constraints of this research are the lack
    of previous research about this subject, for an exception, see
    Zwikael and Smyrk (2012), Bryde (2005) and Cooke-Davies
    (2002), with most of the material published being non-refereed.
    Therefore, it limits the options on practical examples and
    sources for triangulation. Finally, due to inherited limitations of
    the approach employed, since questionnaires were selected as
    the data gathering method, practical and subjective aspects
    could have been missed.
    3.5. Data analysis
    After all the data had been collected, multivariate analysis was
    employed to identify the causal relationship between several
    independent variables and one dependent variable (Tabachnick and
    Fidell, 2007) using multiple regressions (Field, 2009; Pallant,
    2010; Tabachnick and Fidell, 2007) performed with the software
    package IBM SPSS 21.
    The 24 perceptions of project success on eight dimensions and
    from three different perspectives were split into the three groups
    presented in Table 1: Project success, project management success,
    and success on the creation of value to the business. Then, the
    variables in each of the three groups were grouped and combined
    using principal components analysis (Field, 2009), the technique
    also applied by Scott-Young and Samson (2008) to avoid high
    bivariate correlations and also to reduce the number of dependent
    variables to a more manageable and representative group of
    variables, called factors (Field, 2009). The results available in
    Appendix A present itemloadings greater than 0.50 for each factor,
    confirming then the structure and validity of the perceptual scales.
    The scales have high reliability, with Cronbach’s alphas between
    0.818 and 0.940, all well above the recommended limit of 0.70
    (Field, 2009) and above the range from 0.79 to 0.88 considered as
    acceptable by Scott-Young and Samson (2008) and close to 0.831
    considered as good by Ritson et al. (2012).
    Pearson’s r bivariate correlations were performed for all
    variables measured at project level (n = 331). The correlations
    vary from 0.139 to 0.697 (significant at the 0.01 level,
    two-tailed), being all below 0.8, which could be considered
    very high (Field, 2009), except by one exception, BRM11 and
    BRM12, with 0.807 correlation (significant at the 0.01 level,
    two-tailed). These results presented in Table 4 suggest an
    association between all the BRM practices and all the
    perceptions of success as well as between the overall perception
    success and the seven dimensions of success.
    After having confirmed the relationship between independent
    and dependent variables, the variance of perceptions between
    countries was assessed by the Kruskal–Wallis test, a one-way
    analysis of variance suitable to identify differences between groups
    of non-parametric datasets (Field, 2009) that is adequate to our
    non-normally distributed set of variables. The descriptive statistics
    for each country and the variances between countries on the
    perceptions of project success presented in Table 5 and on BRM
    practices in Table 6 suggest some regional or cultural misalignment,
    aspect which has already been found by other authors when
    comparing project management patterns across different countries
    (Müller and Turner, 2004; Müller et al., 2008; Zwikael et al.,
    2005).
    Six out of nine perceptions of success vary between the three
    countries, where Brazil has higher scores in overall project
    success, schedule goals, expected outcomes and adherence to
    business cases. The USA has the highest score on the
    consolidation of all dimensions and the UK has the highest
    score on return on the investment. In parallel, only three out of
    twelve BRM practices vary, where Brazil has higher scores on
    these three: The value created is clearly measurable, strategic
    objectives are clearly defined and actual outcomes adhere to the
    business case. Although the BRM practices follow a much
    more regular pattern between countries than the perceptions of
    success, both groups presented variances. Due to these
    variances, all the next sets of analysis will be stratified by
    country in order to enable the identification of regular patterns
    or further differences. The reasons for variations on success
    rates and BRM practices will not be analysed in more depth,
    since this is not an objective of our research.

Leave a Reply

Your email address will not be published. Required fields are marked *