Thesis: Managing Product Usability

How companies deal with usability in the development of electronic consumer products

Complete manuscript available as download from the TU Delft Library repository.

Animated summary

Short summary

Why is the usability of mobile phones so poor? Why am I not able to do something as simple as hooking up my DVD recorder? Why do they make these products so hard to use? And are they doing it on purpose?

The usability of electronic consumer products – portable music players, washing machines, mobile phones – is under pressure. This is attributed to these products being equipped with more and more functions, becoming smaller, and being used in networks and in a large variety of usage contexts. To further complicate things development of these products takes place under enormous time pressure, at low budgets, and by globally distributed teams.

In the past years many methodologies and methods for user-centred product development have been developed, but consumers are still confronted with scores of unusable electronic consumer products. So what is going on? The problem seems to lie in product development practice. However, current literature on usability in product development practice does not study the product development process as a whole, contains few case studies, and only limited research is available on the electronic consumer products sector specifically.

To increase the insight into usability in the development of electronic consumer products three case studies were conducted. In each of the case studies an integrated approach was taken: the focus was on the product development process as a whole (as opposed to design only), and not just on the usability specialist and interaction designer, but on six roles that were considered to have most influence on usability: the product manager, marketing specialist, industrial designer, interaction designer, usability specialist and development engineer. Interviews were conducted with 69 product developers across 10 product development groups.

The results provide a description of how usability is dealt with in product development of electronic consumer products, an overview of mechanisms of barriers and enablers for usability, and two causal models of usability in product development. Finally, 25 recommendations for industry were written that suggest how to organize product development if the goal is to make usable electronic consumer products.

Comprehensive summary

Problem statement
Research goals
Conceptual framework
Three case studies
Case Study I: Four adjacent sectors
Case Study II: Five development groups of electronic consumer products
Case Study III: The origins of usability issues in three development projects
Conclusions
Recommendations for industry

Problem statement

Even though there is a large amount of methods for user-centred design, the usability of electronic consumer products (e.g., portable music players, washing machines and mobile phones) is under pressure. Usability is the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use. That the usability of electronic consumer products is under pressure is attributed to an increase in the functionality they offer, a decrease in size, their usage in networks, and the large variety of contexts of use. Furthermore, the development of these products has become increasingly challenging due to their increasing complexity, pressure on time to market, globally distributed product development teams, and the commoditization of electronic consumer products.

Research goal

It was suspected that the cause for the usability of electronic consumer products being under pressure may lie in practice. However, as current literature on usability in product development practice does not take an integrated approach, contains few case studies, and only a limited amount of studies investigate electronic consumer products specifically, the insight into how usability is dealt with in development of electronic consumer products is limited. Therefore the goal of this thesis was to obtain this insight as well as to identify variables in product development practice that contribute to or obstruct the usability of electronic consumer products, and to investigate how these variables are related.

Conceptual framework

Before conducting empirical studies, the primary concepts for studying usability in product development practice were identified by reviewing existing research as well as through exploratory interviews with experts on usability from academia and industry. Based on this a conceptual framework was created that provides an overview of concepts that may influence user-centred product development and usability, as well as a description of the (expected) relations between them. This included an analysis of the dependent variables of this research: interaction, user experience and usability.

Three case studies

In total three case studies were conducted. In each of the case studies an integrated approach was taken, focusing on the whole product development process (as opposed to just design) and including six roles that were considered to have the most influence on usability: the product manager, marketing specialist, industrial designer, interaction designer, usability specialist and development engineer.

Case Study I: Four adjacent sectors

In this study the goal was to explore how usability is dealt with in four sectors adjacent to the electronic consumer products market: high-end automotive, professional printers and copiers, office coffee makers and fast moving consumer goods. This study had a pilot-like character and served as an opportunity to test and refine the research method. The study had a multiple case design with holistic cases. A maximum variation case sampling strategy was used, with product complexity and business versus consumer products as the dimensions along which the cases varied. Using a topics guide, interviews with 19 product development practitioners were conducted, focusing on three topics: 1) the product development process, 2) multidisciplinary teamwork, and 3) attitude towards usability. Based on the interviews, context descriptions of the companies were created and barriers and enablers for usability were identified. A barrier is a property, situation or condition in the product development process, team or context that negatively influences the usability of a product. An enabler is the positive equivalent of this. To verify the findings and to discuss remaining issues a workshop was held in which the primary contact from each company participated. The results of this case study indicated that different product characteristics lead to differences in the attitude towards usability in the companies making these products, as well as to different methods for user-centred design being possible and relevant. What methods for user-centred design were used was also influenced by the attitude towards usability, which in addition influenced team composition. Finally, a number of implications for the method to be used in the next case study were identified.

Case Study II: Five development groups of electronic consumer products

In this study the goal was to learn how usability is dealt with in the development of electronic consumer products and to identify barriers and enablers for usability, and to study how these are related. The study had a multiple case design with holistic cases. The cases were selected based on a comparable case sampling strategy in which product type, efforts to improve usability, segmentation of roles, and whether product development was conducted in-house were the dimensions. The participating cases were five product development groups of major international companies, which created portable audio/video players, personal navigation devices, mobile phones, home controls and laundry care products. The primary data source was interviews with 31 product development practitioners fulfilling the roles that were identified earlier as relevant for usability. Based on the interviews, context descriptions of the cases were written and ‘jointly told tales≠ were constructed, consisting of interview quotes, the researcher≠s interpretations and the barriers/enablers. The 1700+ barriers and enablers thus identified were categorized using a categorization scheme that was based on the conceptual framework and that was developed further through open coding. The scheme formed the basis for the development of the Trace tool, which is an interactive software application that provides a categorized, interactive, browsable overview of the barriers and enablers. With the help of the Trace tool a cross-case analysis of mechanisms of barriers and enablers for usability was conducted. To verify the findings the primary contact of each company was asked to provide feedback on the context description and the description of mechanisms of barriers and enablers identified at his/her company. They were also given access to an online version of Trace so they could explore the barriers and enablers themselves. A final workshop was held in which the findings were presented, verified and in which emerging issues were discussed. The study provided insight into how usability is dealt with by product development groups in the electronic consumer products market. Secondly, mechanisms of barriers and enablers were identified, in the following categories: (1a) Process / creating usable products, (1b) Process / evaluating usability (2) Knowledge, (3) Team, (4) Project, (5) Company and (6) Market.

Case Study III: The origins of usability issues in three development projects

In the aforementioned study barriers and enablers were identified based upon interviews and at times it was only said that they influenced usability and it remained unclear how. Therefore the goal of the third and final case study was to trace the origins in product development of usability issues. This was done by investigating the development history of three electronic consumer products developed by one product development group, which makes this a single embedded case with three embedded units of analysis. For each of the products usability issues were identified based on usability tests and after sales feedback. Through a combination of the focused and semi-structured interview approach it was discussed with the product development team members (19 in total) that were involved in creating the products, how 35 of these usability issues had come into being and had been solved. Based on the interviews a context description of the development group and the projects was created, as well as a description of how the product development group dealt with usability. Secondly, based on the interviews causal networks per usability issue were created that indicated origins of usability issues and what caused them to be solved (or not). Based on the 35 causal networks two causal models of usability in product development practice were induced. The first model – the usability issue lifecycle – aims to explain what variables in product development influence usability issues. The second model shows what variables play a role in the generation of shared knowledge about usability issues. To verify the findings they were presented and discussed during workshops at the product development group where the study was conducted as well as at other organizational units of the parent company of the product development group.

Conclusions

The main conclusion of this thesis is that to be able to develop usable electronic consumer products, companies cannot suffice with adapting their process, but should make changes to their organization as well. Especially because the trends that cause the usability of electronic consumer products to be under pressure are expected to continue and even strengthen. Four primary drivers in product development for usability were identified: (1) User-centred design proficiency: the ability to execute a user-centred product development process; (2) Shared knowledge within the product development team (about users, design solutions and potential usability issues); (3) Design freedom: the extent to which a development team is able to make the most appropriate design, determined by design mutability and available resources; (4) Prioritization of usability: the extent to which usability is prioritized during product development and within the product development organization. For each of these drivers related mechanisms of barriers and enablers specific for the electronic consumer products sector are presented.

Recommendations for industry

Based on the insights gained through the case studies as well as existing research, 25 recommendations for industry were developed that describe how the author would organize a product development group if the goal is to make usable products. The recommendations were ‘user tested≠ and iterated by putting them on the weblog that the author kept while working on this research. The recommendations were published as a card set alongside this thesis.