Summary: Many-Shot prompts likely to improve some UX tasks | Prompt library for educational use of AI | LinkedIn’s useless UX advice | 10 K Midjourney images | Linking to a social media site for comments on your articles | The downside of tourist users, especially for AI applications | Dark design as blues and country songs
UX Roundup for May 6, 2024. (Midjourney)
Many-Shot AI Prompts May Improve Some UX Tasks
I recommend the use of zero-shot prompting for the explore phase and few-shot prompts for the refine stage of the prompting diamond in UX. However, new research suggests that many-shot prompts may deliver better results for a few key uses of AI in UX.
As a reminder, the term “shot” refers to the number of examples you include in a prompt to help the AI understand the desired properties of the output you’re requesting.
Zero-shot means that you don’t give any examples of what you want.
Few-shot prompts typically have around 5 good examples for the AI to emulate.
Many-shot prompts include a hundred, or even thousands, of good examples.
A bunch of researchers from Google DeepMind compared the results of few-shot vs. many-shot prompting on Google’s proprietary Gemini 1.5 Pro foundation model. While this study hasn’t been replicated, to my knowledge, on more commonly-used AI tools like ChatGPT or Claude Opus, it’s quite likely that the findings (if not the exact numbers) will transfer since all these LLMs seem to be basically the same right now.
The paper reports the improvements from few-short to many-shot prompting on a wide range of tasks, but the ones of particular importance to UX are:
Translation: a lift of 4%. Will improve almost any international user research, and also help in UI localization.
Summarization: a lift of 5%. Useful for summarizing large amounts of verbatim transcripts and for converting large UX reports into digestible presentations.
Sentiment analysis: a lift of 18%. Useful for understanding unstructured user comments.
Classification: a lift of 21%. Useful for structuring user research findings, deriving initial information architecture designs (to be further tested with users!), and almost anything where we would have performed a thematic analysis.
Many-shot prompts have only become feasible recently, with the release of AI tools with large context windows. For example, Claude 3 Opus can be used with a context window of up to 200,000 tokens, which should allow you to stuff around a thousand short examples (of less than around 100 words each) into the context window as part of the prompt. GPT-4 Turbo 128K has a context window of 128,000 tokens, whereas the standard GPT-4 is restricted to 4,000 tokens, which allows for few-shot prompts but not many-shot prompts.
You are charged extra for the large-window models, but the new research suggests that this price is worth paying for analyzing many forms of UX data. (Even larger context windows can be expected for future models, and the surcharge can also be expected to drop with larger and more efficient compute chips, such as NVIDIA Blackwell.)
You can give AI a small bucket of examples to work from (“few-shot prompt”) or a large bucket (“many-shot prompt”). For some tasks, the big bucket gives better results because the AI gets a broader understanding of what you’re looking for. (Ideogram)
Prompt Library for Educational Use of AI
One of my favorite business school professors, Ethan Mollick, has posted a prompt library with the prompts for ChatGPT (or similar large language models) he uses in his courses. The library includes both instructor aids and prompts for student exercises. For example, there’s a very elaborate prompt for creating a quiz from existing training materials and a prompt for students to simulate a negotiation (this is a business school, after all).
The prompt library may not be as grandiose as this illustration, but it’s more extensive than anything else that’s easily accessible. (Midjourney)
It's very useful, even if you’re not a professor. If you train or educate people, take a look.
The library contains interesting examples of “chain of thought” prompting, an approach to prompt engineering that improves the AI’s ability to handle complex tasks by guiding it through a logical sequence of steps, thereby producing more insightful and comprehensive outputs.
It’s useful to read some of the entries in Prof. Mollick’s prompt library, even if you’re not an educator. You can see how such elaborate prompts are structured with steps for the AI to follow. (Midjourney)
LinkedIn’s Useless UX Advice
In March, I referred to LinkedIn’s policy of degrading impressions for postings that contain links as “a canard” (complete with a very cute drawing of a French duck). It’s a misunderstanding that you grow your website by trying to force users to remain on the site and not go elsewhere.
You may think that I hold a grudge against LinkedIn, but I actually don’t. Still, I have to complain about another of their preposterous attempts to grow traffic: the AI-driven collaborative articles.
LinkedIn recently put the following in my feed:
“You’ve been selected to answer: You’re struggling to develop a digital content strategy. How can you use logical reasoning to overcome this?”
Leave aside the fact that the phrase “you’ve been selected” is a transparent ploy to make the user feel special, using the scarcity principle to create a sense of urgency. We all know that LinkedIn spams everybody, regardless of qualifications, to contribute fodder to its AI-written articles. Let’s also leave aside that false urgency is officially considered a dark design pattern.
Let’s just consider the actual question. It’s useless. Similar to providing sunglasses to moles who spend their lives underground.
Designing sunglasses for moles who live underground will create a useless product, no matter how good your design is. It's somewhat like answering some of LinkedIn’s collaborative questions. (Midjourney)
An interesting question, that might elicit interesting answers, would ask how to develop a content strategy under a variety of constraints or difficulties. “Using logical reasoning” is rarely the answer, but if it were, it would be something the content strategist had tried already.
Generating 10K Midjourney Images
I recently passed 10,000 images generated with Midjourney. (If you’re a Midjourney user, you can discover your lifetime image count through the /info command.)
Despite this large production, I don’t feel that my skills have improved much, after that initial boost of passing beyond the beginner stage. While 10K images are not the same as the 10K hours of practice that are often said to build expertise, it’s surely enough that my skills ought to have improved. (My images are better, but that’s mostly due to improved image quality and prompt adherence with each new release of the AI image-generation tools.)
My first reaction after getting this insight was to blame myself. Users always blame themselves for usability problems, and I’m no different. I have never been a good visual designer, so maybe I’m just hitting the low ceiling of my limited talent.
On the other hand, I’ve always deplored users’ tendency to blame themselves for usability problems that are actually the designers’ fault. In this case, it’s the AI image tools that are poor at growing users’ skills over time. Continuous improvements in user skills should be a core goal for long-term success of generative platforms.
Generating a flow of images with AI tools, but not improving skills at a sufficient pace. (Midjourney)
Offshoring Article Comments to Social Media
I recently noticed an interesting ending for a Baymard Institute article about the usability of apparel size filters. At the end of the article, they said, “If you have any comments on this article, you can leave them on LinkedIn,” with a link to their LinkedIn post about that article.
This is a smart way to encourage comments and social participation without the overhead of building a commenting feature for your website. Especially if you run a narrowly-targeted website, you may not get a large number of comments within your site, making a comment section feel deserted on many pages.
The one downside is that you don’t have the LinkedIn URL until after you post on that site, which makes this idea useless for an email newsletter, which is the dominant publication format for my own articles.
But if your Internet presence is mainly oriented around a website, consider this idea to add the opportunity for debate and user engagement without the overhead of designing, building, maintaining, and moderating a commenting feature. (Given the predominance of spam and flames, moderation quickly becomes expensive, especially since you have to monitor the comments on old pages like the proverbial hawk to avoid “commenters” hawking porn.)
Since Baymard is a B2B site targeting UX professionals as their users, LinkedIn is an appropriate social media platform to host their comments. For other websites, Facebook, Instagram, or X might be better. I recommend picking one social platform for comments, even if you feature many for sharing. You want to concentrate the discussion in one place.
I used “offshoring” in my headline as a metaphor for hosting comments on an external website. Here’s a metaphorical illustration of this metaphor. Meta-metaphor. (Midjourney)
Tourist Users Are Expensive in the AI Age
Tourists are users who come by your website or online service to sample it but then never become paying customers. Back in 1997, I warned against getting seduced by tourists when assessing your website traffic analytics. But in the old days of regular websites, tourists users were not that expensive. Even then, it only cost a minute fraction of a cent to serve up several web pages to a site tourist.
Now, in the AI age, Olivia Moore from the VC firm A16Z posted some interesting statistics on X aggregating data from hundreds of companies. At least for now, AI services are plagued by a high degree of tourists: users who sign up to try out the service but never become paid users.
The median conversion rate from trial to paid user is 1.8% for AI services and 4.9% for non-AI services. However, once somebody pays for AI, they stay paying users: 47% continue to pay after 6 months for AI services, compared to 45% for non-AI services.
Two explanations for the many AI tourists come to mind:
Because AI is new and exciting, people like to try out a wide variety of new services, but they can only afford to be paid customers of a few services.
Many new AI services are disappointing. The smaller number of good ones do keep their paying users.
As I mentioned, tourists were not a real problem in the old days, except for their tendency to inflate vanity metrics and thus possibly distort design decisions in favor of chasing more non-paying users.
However, for AI, interference compute is costly, so it’s expensive for an AI service to cater to people who never become paying customers.
Two solutions to the tourism problem for AI services are:
Describe your value proposition realistically up front, to avoid attracting trial users who won’t benefit enough from your service to ever pay for it.
Improve the usability of your service so that people are likely to realize true value within a very short time of starting a trial. I can’t tell you how many AI services I have tried where it’s too difficult to create anything better than what I already get from my existing paid-for subscriptions.
Tourist users who never become paying customers are expensive for AI services, so take steps to avoid them. (Midjourney)
Live Keynote by Jakob Nielsen Next Week: 10 Foundational UX Insights
I will present a live keynote on ADPList on May 15, 2024 in support of their mentorship mission on the topic of “10 Foundational UX Insights.” I thank my good friend (and stellar UX researcher) Alita Joyce from Google for being the session chair and managing the Q&A.
The event is free, but advance registration is required.
My speech is live on the Internet at these times:
San Francisco: 10 AM USA Pacific Time
New York: 1 PM USA Eastern Time
São Paulo: 2 PM Brazil Time
London: 6 PM British Summer Time
Paris/Berlin: 7 PM Central European Time
Dubai: 9 PM Gulf Standard Time
New Delhi: 10:30 PM India Standard Time
Singapore/Beijing: 1 AM the next day (May 16) Singapore Time/China Time
Here’s the summary of the talk:
UX has come a long way since its early beginnings at Bell Labs in the 1940s. As we enter Year 2 of the AI revolution, it’s clear that the core principles of UX design are not being replaced but are evolving with AI integration. This journey through the 10 foundational insights of UX reminds us that while “there's nothing new under the sun,” the way we apply these insights is constantly changing. By embracing AI as a tool for enhancing these time-tested principles, we can create more personal user experiences than ever before. Let’s honor the past as we design for the future, ensuring that the essence of UX — empowering humans while making technology subservient — remains at the heart of everything we do.
Jakob Nielsen presents a live keynote on ADPList on May 15. (Midjourney)
Dark Design as Blues and Country Songs
For some of my precious AI-generated songs about UX, some comments have noticed that the lyrics weren’t always a great match for the music.
I am now trying a new approach, which is to make the lyrics in specific poetic meters that are known to be good with certain styles of music. Here are examples of songs about the 12 main types of dark design patterns:
Iambic Tetrameter (country music style)
12-bar Blues Meter (jazz music style)
Balad Meter (rockabilly style)
All three songs were made with Udio. I added light AI animation with Leonardo to the first two music videos and used Runway to animate the third, more realistic image for the Rockabilly video. We’re clearly not talking Pixar-level animation yet, though Runway is much better than Leonardo. Let me know in the comments whether you prefer light animation or a simple still picture while the music plays.
Country song about dark design patterns: listen on YouTube. (Thumbnail by Midjourney, video animation by Leonardo)
Jazz song about dark design patterns: listen on YouTube. (Thumbnail by Midjourney, video animation by Leonardo)
Rockabilly song about dark design patterns: listen on YouTube. (Thumbnail by Ideogram, video animation by Runway)
The Benefits of Taking a Provocative Stand on UX
I was recently interviewed by Brave UX, and the full interview is now available on YouTube (86 min. video). The interviewer, Brendan Jarvis, was uncommonly well prepared and seemed to have read many of my articles and other interviews from the last 30 years. He dug into why I had been making provocative statements that some people don’t like.
I acknowledge my role as a provocateur, aiming to challenge prevailing norms and stimulate deeper thinking about UX practices. I am known for my candid and often controversial remarks, which have historically sparked discussions and even backlash within the industry. In the interview, I emphasized the importance of being truthful and direct in my communications, regardless of how my views are received by others.
In reflecting on my career, I noted that controversial statements are not made to simply provoke but to encourage critical evaluation and advancement in the field of UX. I highlighted that true progress often involves challenging established ideas and pushing for better solutions, which can upset those who are invested in the status quo.