Assessing innovation and technology management capabilities

To extract best value from investments in technology and innovation, managers need to address a wide range of aspects. Tools to assist this process can also provide insights into performance trends and identify issues commonly faced across the sector.

The emerging ISO standards for innovation management, provide an opportunity for developing these types of tools. In this study, researchers developed an Innovation Technology Management (ITM) process framework based on a set of 87 question. The survey covers three themes and ten areas of significance to technology-intensive organisations and was successfully applied to the oil and gas sector.

Assessing Innovation Technology Management performance

There are a number of innovation assessment approaches that are based on self-assessment, some of which are briefly described below as an illustration of the range available.

IMP3rove was established by the European Commission with the aim to improve innovation management performance of SMEs in Europe with sustainable impact. It covers innovation strategy, organisation, culture and innovation processes, including enabling factors. The online self-assessment tool supports and drives the process by generating reports on SME performance, developing insights into strengths and weaknesses, and monitoring progress and performance.
Innovate! New developed by CAU in Germany and other academic partners, is a benchmarking methodology based on an innovation excellence model, which has three central areas leadership & governance, innovation performance, process and projects in the context of six areas: strategy, organisation, launch, development, front end and culture.
The Innovation Management Maturity Assessment offered by CIMS, North Carolina State University, is a tool to assess a company’s level of innovation management proficiency. The assessment looks at self-assessed strengths and weaknesses across the organisation. By department, or throughout the entire enterprise, individuals rate their organisation’s performance across five core competences (strategy, organisation & culture, processes, tools & techniques, metrics) and five management dimensions of innovation (ideas, market, portfolio, platform and project management).

ITM tools and toolkits

Understanding which tools to use, in what combination, requires an understanding of the particular organisational context, and the effectiveness of management practices in order to focus improvement efforts. So, the process of developing ITM tools and toolkits requires active collaboration with industry and working with organisations on real management problems, due to the deeply context-specific situations they present.

The technology management assessment procedure (TMAP) was developed to assess technology management processes in manufacturing firms, it is based on Gregory’s (1995) process model for technology management, which consists of five generic process areas.

Figure 1 - Five-process technology management framework, showing example activities — *Figure 1 – Five-process technology management framework, showing example activities*

The technology management framework consists of five generic process areas:

Identification of technologies which are (or may be) of importance to the business. Example processes include scanning, monitoring, benchmarking and data collection.
Selection of technologies that should be supported by the organisation. For instance, forecasting, portfolio analysis and scenario analysis are associated with selection of technology.
Acquisition and assimilation of selected technologies. Example processes include technology transfer, research and development, corporate mergers and acquisitions.
Exploitation of technologies to generate profit, or other benefits. Example processes include licensing, new product development, incremental developments, process improvements, and supply chain management.
Protection of knowledge and expertise embedded in products and manufacturing systems. These processes include patenting, risk assessment, security management and staff retention.

The aims of the TMAP assessment were to 1) provide a framework for linking technology with business needs, identify critical technology management issues in the firm; 2) map and evaluate important technology management processes in the area of interest; 3) identify strengths and weaknesses, and hence areas of best practice for transfer, and areas for improvement; and 4) lead to recommendations for action plans which have cross-functional support and can be clearly justified.

Innovation and technology management assessment (ITM)

The paper presents a robust approach for assessing the innovation and technology management (ITM) capabilities within technology-based organisations so that focused improvements can be made, and appropriate interventions or solutions can be developed.

Technology management

Phaal et al.’s (2004) technology management framework (Figure below), is an advancement on Gregory’s (1995) because makes it clear that technology management decisions are often made in business areas such as strategy formulation, product development and in operational process, rather than discrete technology management processes.

Innovation Management

In addition, Goffin and Mitchell, identified the main elements of innovation management succinctly as follows:

Innovation strategy: which, in addition to other things, should capture the role of technology and the opportunities it can open up.
The development funnel: made up of ideation, prioritisation and implementation – a process model reflecting Wheelwright & Clarke’s innovation funnel.
People and organisation: which relates to creation of a culture of innovation. Looking back into literature compiled by Adams et al. (2006), one would identify organisational culture, management commitment and leadership as being part of people and organisation.

Thus, the inextricable nature of technology and innovation justifies the need for a framework that explicitly links and combines elements of Phaal et al.’s (2004) technology management framework and Goffin and Mitchell’s (2004) innovation management models.

ITM Framework has three themes

In summary, the innovation and technology management framework can be described as covering three themes and ten areas of significance to technology-based organisations:

Strategy: investigating the innovation and technology strategy and importantly, how it links and supports the organisation’s overall business strategy

Business strategy: is it sufficiently defined to enable ITM strategy (and other functional strategies) to be developed?
Innovation and Technology strategy: is there a clear statement in terms of competitive positioning and strategic direction in terms of what is required from the innovation and technology system?

ITM System: examining the fundamental technology and innovation management processes of technology identification, selection, acquisition, exploitation and protection;

Identification: how do we identify product or process technologies that are or may be of importance to the business?
Selection: how do we know which of these technologies are most appropriate to deliver business and ITM strategies?
Acquisition: how do we successfully acquire, develop and assimilate selected technologies?
Exploitation: how do we generate benefits (e.g. licensing, new products, process improvements etc.) from the technologies?
Protection: what strategies and processes are in place to ensure that our technology knowledge and capabilities, embedded in our products, processes and systems, are appropriately protected and denied to competitors?

People and Organisation: focusing on the organisation’s leadership and management style, organisational culture and the management of competences, and their appropriateness for desired ITM and, ultimately, business performance.

Leadership: how do we set and communicate clear objectives, manage interfaces and structures within the organisation and empower our personnel?
Organisational culture: does ‘the way we do things’ help or hinder the achievement of our business and ITM objectives?
Competences: do we have the skills and capabilities to deliver projects and manage our technology and innovation activities?

Development of ITM assessment

The 97 questions that made up the first version of the ITM were compiled by rationalisation of a total of over 400 questions/statements compiled from the various assessments, followed by selection of those considered most appropriate.

Following initial tests with organisations, the ITM assessment was revised to improve its usefulness, feasibility and usability and the resulting 87 questions spread across the ten areas of the ITM framework and correlated to a five-scale maturity model.

Maturity model

The maturity model is based on Bessant et al.’s (2005) levels of organisation absorptive capacity. This gives an indication of the knowledge or capability level of the organisation with respect to the issue being explored by the respective question.

Table 1 - ITM Assessment maturity levels — *Table 1 – ITM Assessment maturity levels*

Application: ITM Assessment survey of oil and gas sector

ITM assessment was applied to a selected group of oil and gas exploration and service companies, to provide an industry-wide assessment of a sector. It was found to give a preliminary indication of performance trends and issues which are commonly faced across the energy sector.

Figure 3 - Roadmapping-based workshop format for TIM Assessment deployment — *Roadmapping-based workshop format for TIM Assessment deployment*

Key trends

An arithmetic mean of the responses revealed some key trends in the oil and gas industry.

The highest variation in the average maturity levels is in the ITM System – Protection category with average scores ranging from 1.2 (Company 2) to 5.0 (Company 7). At the sector-level, the average maturity level is similar for nine of the ten ITM categories. The one notable exception is the People & Organisation – Competences category with the arithmetic mean average of 2.6 for the 15 organisations that completed the assessment. The lower maturity score for the competences area is perhaps unsurprising given the recent decline of the industry with rock bottom oil prices triggering significant staffing cuts worldwide starting in 2015.

Table 2 - Summary of selected findings in oil & gas survey — Summary of selected findings in oil & gas survey

The analysis of responses to date has revealed some key trends in the oil and gas industry and these average maturity levels are summarised for each assessment category. It also offers the potential for benchmarking against peers, higher scoring average maturity levels reveal promising trends while lower average levels reflect issues that may need attention.

Table 4 - Selected findings in oil & gas survey - Higher scoring questions — Table 4 – Selected findings in oil & gas survey – Higher scoring questions

Positive feedback from the questionnaire shows a focus on project execution, particularly with respect to new product development, the emphasis on supply chain performance and awareness of the value of intellectual property.

Existing successful products form the backbone of all their engineering development. Building on these allows the industry to keep their existing customer base happy with upgrades and satisfies new customers coming in, in terms of incremental improvement in products. Moreover, it appears there are well defined processes and procedures in place to protect company’s intellectual property as most companies seek to maintain and dominate a market.

The data shows that the surveyed organisations have very good customer relationships. Field personnel not only talk to customers about their needs, but also use their knowledge to predict what the customer will need. Combining these insights with good business intelligence and analysis provides these companies with a solid basis for creating products with superior value.

Key learning points

There are several key observations from this work:

In complex areas such as innovation and technology management, time efficient and tangible approaches to support managers in formulating their own action plans are welcomed by industry
Easy to access survey interfaces can enable companies to select where they wish to go into greater depth although they do not replace workshop discussions and face to face interactions
Companies are interested in comparing themselves to others in the same industry as a driver for performance improvement
Feedback from industry is a crucial input for the development and refinement of management tools such as the ITM survey
Building up a wider view of survey outputs could guide future research into building effective toolkits for industry

Paper submitted to: R&D Management Conference 2018 “R&Designing Innovation: Transformational Challenges for Organizations and Society” Imoh Ilevbare, Clare Farrukh, John Saiz, Rob Phaal and Nii Ahele Nuno

Download the paper from ifm.eng.cam.ac.uk

Summary of other innovation and technology management-related assessments:

Speeding New Products to Market (Gardiner et al. 1998),

Technology Management Assessment Procedure (Farrukh et al. 2000),

Innovation Management (Pentathlon Framework) assessment (Goffin & Mitchell, 2010)

Design Audit tool (Moultrie et al. 2007) which highlighted multiple facets of technology and innovation management.