Measurement and Assessment of Business Intelligence and Big Data Development in Organizations

Maturity Models

Business Intelligence and Big Data (BI&BD) projects are highly complex undertakings both of organizational and technological nature. Such projects are considered to be implemented based on proven research theories and methodologies. Many issues related to the measurement and effective implementation of BI&BD systems in organizations can be explained on the basis of the theory of maturity models, critical success factors (CSFs), as well as the theory of value creation.

The literature describes the term “maturity” as an ideal, perfect, complete state (of individuals, organizations, society) to which one should strive (Lahramnn et al., 2011). Achieving this state is usually associated with going through various stages of development (Fraser, Moultrie, & Gregory, 2002). The maturity model is a structured set of elements describing certain aspects of organizations maturity, which aids the improvement of its functioning as well as passing through the various stages of its evolution. In other words, it is a set of various tools and practices that enable assessing an organization’s competencies in management as well as improving key factors that lead to the achievement of objectives pursued (Looy, 2014).

The Object Management Group defines the maturity model as an evolutionary process of implementing key practices in one or several areas of enterprise operation. The maturity model enables a holistic view of an organization and a comprehensive assessment in terms of meeting key expectations set by various requirements (e.g., legal acts, firm’s internal arrangements) and its stakeholders (customers, contractors, employees, society, etc.).

Maturity models are regarded to have three basic functions (Becker, Knackstedt, Sc Poppelbub, 2009; de Bruin et al., 2013; Maier, Moultrie, Sc Clarkson, 2009; Poppelbub Sc Roglinger, 2011): (1) descriptive - as descriptive models, being a diagnostic tool, used to determine the current state, e.g., for internal and external reporting; (2) prescriptive - as normative, prescriptive models, used to improve an organization or some its areas by indicating the path to achieve specific organizational states; and (3) comparative - as models enabling internal and/or external benchmarking.

An important category of maturity models includes models related to the assessment of information systems. The first proposals regarding the maturity models of information systems appeared in the 1970s (Gibson Sc Nolan, 1974). It is estimated that today there are over 100 different variants (Mettler Sc Rosner, 2009). Many of the existing models refer to the assessment of the information systems design process (de Bruin et al., 2013). The first and one of the best known maturity models was the C.F. Gibson and R. Nolan model, in which the development of electronic data processing was presented in four phases (Gibson Sc Nolan, 1974).

In the early 1990s, the Capability Maturity Model (CMM) was developed by the Software Engineering Institute in the United States. This model is still popular today and is used to improve software development processes. It consists of five levels of maturity referred to as Initial, Repeatable, Defined, Managed, and Optimizing.

The Initial level means that processes are carried out ad hoc and in a chaotic manner. Software is usually created without formal procedures. Organizations carrying out projects at the first level are usually characterized by a lack of reliable, proven production, and product maintenance technologies.

The Repeatable level is characterized by the fact that software development procedures have been largely defined. Techniques are used to replicate successful projects based on information recorded during previous projects. At this level, team members are required to work more effectively, acquire new skills, learn from their mistakes, as well as develop the ability to document experience gained.

At the Defined level, there is a coherent set of definitions and standards relating to both the implementation of a project itself and the organization implementing such a project. Knowledge about how to conduct and organize work on the project becomes the way the firm operates. Standards and procedures are modified as the technological or organizational conditions change. Organizations are starting to modify the way projects are carried out based on their own experience so that they reflect specific characteristics of organizations and products they create.

The Managed level means the use of detailed metrics regarding the process itself and product quality. Although the process is managed and measurable, risky places are identified that should be subject to more detailed control. The issue of risk management is a crucial element at this level.

The goal of the Optimizing level is optimization and further improvement of the process, increasing its effectiveness (improving results) and efficiency (reducing costs). The organization is already mature enough to identify the strengths and weaknesses of the process. This information is used for further analysis aimed at choosing more effective technologies or organizational solutions.

A standard CMM structure, in the form of division into areas, goals, and assigned practices, as well as a five-grade maturity scale, constituted the basis for its development and increase in its popularity (Kasse, 2008). The model gave the rise to the development of a number of other models in the future.

Currently, the Capability Management Model Integration (CMMI) is one of the best known maturity models in the area of software development management. CMMI is the successor of the CMM. According to the Software Engineering Institute (SEI, 2008), CMMI aids the users to “integrate traditionally separate organizational functions, set process improvement goals and priorities, provide guidance for quality processes, and provide a point of reference for appraising current processes.”

Another well-known model is the process maturity framework (PMF), which was described in the ITIL library. PMF is used to assess the maturity of service management — both individual processes and their overall outcome (Lloyd, 2002). The assessment is carried out within five closely related fields on a five-grade maturity scale.

The model to study the maturity of IT service management processes is IT Service Management Maturity Model (ITSM Maturity Model). The structure of the model covers six domain areas to which from three to eight detailed processes have been assigned (www.itsmsolutions.com). In turn, model control objectives for information and related technology were developed by ISACA and IT Governance Institute to collect good practices in the field of IT process management. The assessment is performed within the identified five domains and 37 processes (version COBIT 5-0) on a six-grade maturity scale (ww.isaca.org).

An important characteristic (attributes) of maturity models, irrespective of their purpose, is the concept of maturity, its dimensions and levels, principles of maturity development, and methods of its assessment (Lahrmann et al., 2011) (Table 5-1).

Various research methods can be used to design maturity models, e.g., Delphi studies, case studies, and focus groups (Becker, Knackstedt, 6C Poppelbuss 2009). Quantitative methods are rarely used. The selection of the appropriate method is driven by the area of application, system stakeholders, and target recipients (Mettler & Rosner, 2009).

Table 5.1 Attributes of Maturity Models

Source: Elaborated on: (Lahrmann et al., 2011).