Summary: Computer users experience frustration a shocking 11% of the time, a new study found. 2/3 of frustrations are caused by bad implementation (bugs and sloppy engineering), and 1/3 is due to bad design (usability problems).
It is still unacceptably unpleasant for people to use computers. Recent research asked users to report each instance in which the computer provoked feelings of frustration, and the logged frustration episodes amount to a staggering 11% of users’ computing time.
The study was conducted by Morten Hertzum and Kasper Hornbæk from the Universities of Roskilde and Copenhagen in Denmark. They collected data from 234 users who participated in a short, one-hour diary study. I would have preferred a more extended data collection period, but we must take what data we can get. At least this study provides fresh data about users’ authentic emotions while using today’s computer systems. During a mere hour of computer usage, the 234 participants reported 185 episodes of frustration. Even in such a brief period, scarce are the computer users who escape with their happiness intact.
A gobsmacking discovery was that, on average, users spent 6.63 minutes in a state of frustration during the one-hour computing session. This corresponds to 11% of users’ total time on the computer. Notably, this was a quasi-ethnographic diary study in which users worked in their natural environments performing their typical computing tasks. They were not exposed to newly designed or experimental software like we usually do in usability testing. Testing in-development software can be anticipated to induce difficulties, but people going about their everyday computer activities should encounter smooth sailing all the way. Not so!
The study participants reported spending an average of 7.9 hours per day using computers. Extrapolated across this amount of time, the frustration data in the study scale to nearly an hour of anguish and annoyance per day. An hour a day of being frustrated by tools meant to help us is technological purgatory.
Frustration visits computer users for about an hour each day. (Image of “frustrated user” generated by Midjourney.)
An inherent weakness in this methodology is its reliance on self-reported data. Researchers did not observe participants directly but instead allowed them to use their own computers in locations remote from the researchers. Although having participants work in their everyday environments lends ecological validity to the results, we lack an objective analysis of what precisely transpired during each irksome computing episode.
As for the principal finding that 11% of computer usage is spent being frustrated, the self-reported nature of the data is acceptable. Each person is the sole judge of his or her feelings: if a user claims to feel frustration, we should accept the veracity of his or her statement. If Joe says that he’s frustrated, he’s frustrated. If Mary says that she’s not frustrated, she’s fine. QED.
What Causes User Frustration?
Besides logging if and when they felt frustration, participants described the cause of each episode. Here is the breakdown of user-reported sources:
User frustration episodes categorized by (self-identified) cause.
Here the self-reported nature of the data does make me pause. Simply because a user identifies something as the source of their troubles does not make it the actual root cause. We must take this pie chart with a grain of salt. Too much salt may have ruined many a cherry pie, but does it spoil our data pie? Not necessarily, as long as we stay at a high level of granularity in our analysis. Detailed reasons are likely to be wrong, but the rough categories are apt to be quite stable.
Slightly over a third of the frustrations were caused by usability problems: cases where some design element was too difficult to use. We can lay the blame for this third of frustrations at the collective feet of the world’s user experience designers. We must do better because we hurt users when we underperform our role. On average, we subject users to distressing experiences once every 3 hours they use one of our designs. Do better, people!
On the other hand, almost two-thirds of the frustrations were caused by system problems. In these cases, the user interface design may have been excellent, but the implementation failed the users (and the designers, in failing to realize their vision). The largest proportion were instances where the system did not work at all: it froze, crashed, or didn’t work properly. In a smaller number of cases, the system functioned but at an unacceptably poor level of performance. Excruciatingly slow response times provoke frustration, partly because it’s unpleasant to be at the mercy of whenever the computer can be bothered to respond, and partly because unusually lengthy delays cause users to question whether the system is functioning at all or has crashed.
Comparing the sources of user woes, software developers are twice as culpable as user experience designers in aggravating users.
The Good News: Most Torments Terminate. The Bad News: They Keep Coming Back.
Users could resolve 74% of the frustrating experiences in the study, leaving only 1/4 of the frustrations unresolved (26%).
The study participants employed a variety of strategies to overcome the frustrating episodes. As shown in the following pie chart, the main approaches were to make an extra effort, to wait for the problem to resolve itself, and to repeat the previous steps:
How users resolved their frustration episodes in the study.
The fact that a popular strategy was “wait and see if it fixes itself” says everything about how pathetic software quality remains today. When the source of trouble remains a mystery, all one can do is wait and hope the problem evaporates. Similarly, repeating the same steps and expecting a different outcome is the supposed definition of insanity in some people’s books, but the approach does actually work with computers. For magical (i.e., inexplicable) reasons, it might work the second time you do the same thing. One might as well propitiate the machine gods with the ritual sacrifice of a goat at midnight for an equal likelihood of success.
Ultimately, only 26% of problems went unresolved. Thus, users could proceed with their goals 3/4 of the time, after enduring those roughly 6 minutes or so of vexing delay. While not measured in the current research, the ultimate impact on users’ productivity may be much worse than this relatively small amount of time. Interruptions of several minutes severely disrupt flow and concentration, and the recovery time to re-immerse in complex tasks after an interruption can dwarf the initial delay.
A dance of despair: round and around it goes, with the tech industry repeatedly subjecting users to the same aggravating experience. (“Dance of Despair” by Midjourney.)
Maybe the single-most damning number in the research is that users reported that a staggering 84% of the frustration episodes were repeats they’d endured before. The same issues arise again and again, inflicting recurring torment upon users.
It’s untenable that computers remain challenging to use and unreliable in their function. But it adds insult to injury that the prevailing expectation whenever something goes awry is that the problem will return to hurt you many times more in the future.
While a single 6-minute frustration might not make customers abandon a brand, being repeatedly hit by the same lousy computer design or implementation has the same impact as waves eroding the seashore: it will degrade brand perception over time and make customers angry:
The same problems re-manifest incessantly to frustrate users. That will make anybody angry. (“Angry user” by Midjourney, anime mode.)
Big Improvements Over 20 Years
On the one hand, it’s unacceptable that modern-day users spend 11% of their time with computers being frustrated. But believe it or not, this is a vast improvement compared to the past. Similar studies were published in 2004 and 2006 (meaning that the data was likely collected around 2003), and back then, between 44% and a whopping 50% of users’ time was spent being frustrated with their computers.
Unfortunately, the new and old studies used slightly different methodologies, so we cannot compare the precise figures. But the change in results is far greater than the change in study methodology, so I feel safe concluding that frustration has declined over the past two decades.
Is the glass of user frustration statistics half full or half empty? 11% of the time is still excessive to be frustrated, but it signifies a notable improvement over past studies. I choose the optimistic approach to believe the glass is half full and will fill further in the future: that is, the frustration count will continue to decrease. (Image of “half-full water glass” generated by Midjourney.)
This image above exemplifies user frustration: I had to battle my AI assistant for ages to generate that half-full glass — all it wanted to depict were full ones! Generative AI relies on stereotypes, and nearly all water glasses on the Internet seem to be full. Ultimately, I resolved my frustration by accepting a lower-quality outcome, joining the 5% of users who make such compromises, as my original vision for the illustration called for a still-life reminiscent of Paul Cézanne’s style.
Based on my experience observing users, I estimate that the major sources of user frustration have reversed over these 20 years. In the new research, slightly more than a third of the frustrations were due to usability problems (that is, caused by bad design). In contrast, almost two-thirds of frustrations were caused by system problems (that is, caused by bad implementation). Twenty years ago, this balance was roughly the inverse. I can’t provide precise numbers because we didn’t measure the exact same issues as were examined in the new paper. 20 years ago, usability problems (bad design) accounted for approximately two-thirds of user difficulties, whereas system problems (bad implementation) only constituted around one-third.
Why did this reversal occur? The answer lies in our success in convincing companies to prioritize user experience over the past two decades. The number of UX professionals in the world has increased tenfold, a staggering growth of 1,000%. This investment in UX has yielded tangible improvements in interface quality.
Conversely, while the number of programmers has also risen, the number of quality engineers has seen meager growth. More projects now eschew robust software engineering methods in their development practices. Instead of rigorous discipline, many companies chase agile methods, striving to release software of the lowest acceptable quality. One of the high-tech industry’s most illustrious figures has even advocated “move fast and break things.” Breaking things breeds bugs, accumulates tech debt, and is not the hallmark of rigorous software engineering.
Thus, the last 20 years have seen more rigorous usability engineering and more sloppy software engineering. The result of which is enhanced user interfaces but shoddier, more failure-prone systems behind them.
Usability problems persist, and computers are still inadequately designed to accommodate human characteristics. UX must do better over the next 20 years. But software developers need to do much better because the side-effects of their work become part of the total user experience, on a parallel footing with the official “design.”
I would like a two-year moratorium on new features, during which programmers worldwide focus on rectifying existing bugs and fortifying their code. It’s time to make tech work: the war on frustration begins today.
Quiz: Check Your Understanding of This Article
Check your comprehension. Here are 6 questions about ideas and details in this article. The correct answers are given after the illustration below.
Question 1: What percentage of computing time do users reportedly spend feeling frustrated?
A. 11%
B. 26%
C. 39%
D. 84%
Question 2: What is the average amount of time spent on computers per day by the study participants?
A. 6.63 minutes
B. 4.1 hours
C. 7.9 hours
D. 9.2 hours
Question 3: What percentage of user frustrations were caused by usability problems according to the study?
A. 11%
B. 26%
C. 39%
D. 84%
Question 4: What percentage of user frustrations went unresolved according to the study?
A. 11%
B. 26%
C. 39%
D. 84%
Question 5: Roughly what percentage of user frustrations were repeats of issues they had experienced before?
A. 11%
B. 26%
C. 39%
D. 84%
Question 6: Compared to studies from around 2003, what percentage of time do users now reportedly spend feeling frustrated according to the new study?
A. More
B. Less
C. Roughly the same
Reference
Morten Hertzum and Kasper Hornbæk. 2023. “Frustration: Still a Common User Experience.” ACM Transactions on Computer-Human Interaction, Vol. 30, No. 3, Article 42 (June 2023), 26 pages. Available at https://doi.org/10.1145/3582432 (warning: inordinately long PDF document, paid subscription required).
Quiz Answers
Question 1: What percentage of computing time do users reportedly spend feeling frustrated?
Correct answer: A. 11%
Question 2: What is the average amount of time spent on computers per day by the study participants?
Correct answer: C. 7.9 hours
Question 3: What percentage of user frustrations were caused by usability problems according to the study?
Correct answer: C. 39%
Question 4: What percentage of user frustrations went unresolved according to the study?
Correct answer: B. 26%
Question 5: Roughly what percentage of user frustrations were repeats of issues they had experienced before?
Correct answer: D. 84%
Question 6: Compared to studies from around 2003, what percentage of time do users now reportedly spend feeling frustrated according to the new study?
Correct answer: B. Less