Summary: A major upgrade to Anthropic’s AI foundation model was just released. In my testing, Sonnet 3.7 didn’t just perform better than the previous version of Sonnet, it also beat the new Grok 3 from last week.
Coming hot on the heels of last week’s release of Grok 3, Anthropic released an upgrade to Claude: Sonnet 3.7. The previous version was stupidly called Sonnet 3.5 (new), so this new name is a tacit admission that the previous version should really have had version number 3.6.
I put Claude to the test with the following prompt: “Write a one-paragraph explanation of "Jakob's Law of the Internet User Experience", supplemented with a short example of the Law. Target an audience of students in a UX bootcamp who are taking their first courses on UX design. Write in a tone of voice that matches Jakob Nielsen's writing style.”
(Read my own explanation of Jakob’s Law of the Internet User Experience for comparison.)
Old Version: Sonnet 3.5 (new)
Here’s the response from the old version, Claude Sonnet 3.5 (new):
Jakob's Law states that users spend most of their time on other websites, not yours, which means they form their expectations for how your interface should work based on what they've learned elsewhere. This fundamental principle of web usability explains why following established design patterns is usually more effective than creating novel solutions, even if your innovative design might theoretically be superior. For instance, when Amazon initially placed their shopping cart icon on the right side of their header in the late 1990s, this placement quickly became the de facto standard that users came to expect; today, placing a shopping cart icon in the left header would violate users' mental models and likely reduce conversion rates, even if you could argue that left-placement offers certain advantages. This exemplifies why designers should carefully consider breaking from established conventions, as such deviations force users to learn new patterns, increasing their cognitive load and potentially driving them away from your site.
Would you like me to create an interactive example demonstrating Jakob's Law in action?
New Version: Sonnet 3.7
Here’s the response from the yesterday’s release, Claude Sonnet 3.7:
Jakob's Law of the Internet User Experience states that users spend most of their time on other websites, not yours, which means they form their expectations for your site based on what's commonly done on most other sites. This fundamental principle implies that deviating from established web conventions forces users to spend mental energy understanding your design rather than accomplishing their goals, creating unnecessary cognitive load when they encounter unfamiliar patterns. For example, if you decide to position your website's main navigation menu at the bottom of the page instead of the conventional top or left placement, users will likely struggle to find it—not because bottom navigation is inherently flawed, but because it violates the mental model users have developed from their collective experience across thousands of other websites. Adhering to established patterns isn't a limitation on creativity but rather a recognition of how human cognition actually works in digital environments.
Grok 3
For comparison, here’s the output from Grok 3 which is also a brand-new model:
Jakob's Law of the Internet User Experience states that users spend most of their time on other websites, which shapes their expectations for how your site should function. This means you must design your website to match the common conventions users already know from their extensive online experience. For example, users expect the navigation menu to sit at the top of the page—move it to the bottom, and they’ll likely miss it or get frustrated trying to find it. By sticking to these familiar design patterns, you reduce confusion, make your site intuitive, and improve the overall user experience, a principle grounded in observing real user behavior across the web.
Interactive Demos
Sonnet 3.5 (new)’s Demo
As shown above, Sonnet 3.5 (new) ended its initial explanation of Jakob;’s Law with the offer to create an interactive demo. I said “yes, please”, after which Claude generated the program shown in the screenshot below, supplemented by this explanation:
This interactive example illustrates Jakob's Law through two checkout buttons:
The conventional design follows established patterns:
Green background color (commonly used for positive actions)
Standard button shape
Cart icon on the left
Clear, direct label
Simple hover effect
2. The unconventional design breaks common patterns:
Unusual border style (dashed)
Vertical layout
Rotational hover effect
Split text
Non-standard visual hierarchy
Screenshot of the demo Claude 3.5 created. The visual itself is actually a great example of Jakob’s Law, but the interaction is flawed: it would indeed be faster for users to check out with the left design, but only if you measure people the first time they encounter the design. Just showing the two designs side-by-side and asking me to click both buttons shows nothing. Once I place my mouse over a button, I can click it very fast multiple times, regardless of the button’s visual design. (Claude Sonnet 3.5 new)
Sonnet 3.7’s Demo
Interestingly, Sonnet 3.7 didn’t volunteer an offer to create an interactive demo. Instead, I prompted it to do so. Here’s Claude’s explanation of its demo:
This interactive demo illustrates Jakob's Law by letting users complete the same task (finding and clicking a sign-up button) on two different interfaces:
Conventional Design: Features standard top navigation, clear hierarchical structure, and a prominently displayed call-to-action button that looks like a typical button.
Unconventional Design: Places navigation at the bottom, uses non-standard styling for the button, and can be toggled to become increasingly unconventional with reversed content order and rotated elements.
The demo times how long it takes users to complete each task and compares the results, typically showing that users complete tasks more quickly with conventional designs. This demonstrates the cognitive efficiency gained when users can transfer their existing mental models to your interface, as Jakob's Law predicts.
Try toggling between the standard unconventional design and the "really unconventional" version to experience different levels of deviation from web conventions!
It took longer for Sonnet 3.7 to create this more elaborate demo, and is it less suited for a side-by-side display in a mid-sized browser window. However, it presents a more valid demo of Jakob’s Law, even though the timings are still bogus. (Claude)
I particularly like Claude’s inclusion of a “really unconventional” web design in the demo. You may think this is more of a parody than a real example of bad web design, complete with placing the headline at the bottom of the page. But I have seen worse designs in the wild! (Claude)
Grok’s Demo
Grok also didn’t offer to make a demo, so I asked it for one. Since Grok (currently) can’t run code directly in its window, it simply gave me a textual description of a demo I would have to build myself. (Of course, Grok also serves as a developer aide, so it could have helped me program the demo, if I had chosen to proceed.)
Here’s Grok’s description of the demo, which is reasonable and quite similar to the one Sonnet 3.7 made automatically:
This demo will feature two versions of a simple web page:
Version A (Standard Design): Adheres to common web design conventions.
Version B (Non-Standard Design): Intentionally breaks these conventions.
Users will be asked to complete a task—such as finding a specific menu item or using the search function—on both versions. They can switch between the two versions and see how the design impacts their ability to complete the task efficiently.
Demo Features
Task: Users will be instructed to find a specific piece of information, such as locating the "Contact Us" page or searching for a product.
Switch Button: A button will allow users to toggle between Version A and Version B.
Performance Feedback: Optionally, you can include a timer or click counter to show how long it takes or how many clicks are needed to complete the task in each version.
Explanations: Brief tooltips or pop-ups will explain why certain design choices in Version A are more effective, reinforcing Jakob's Law.
Design Details
Version A: Standard Design
Navigation Bar: Placed at the top with links like "Home," "About," "Products," "Contact Us."
Search Bar: Located in the upper right corner, clearly visible.
Logo: Positioned in the upper left corner, linking to the homepage.
Layout: Clean and familiar, with content centered and easy to scan.
Version B: Non-Standard Design
Navigation Bar: Placed at the bottom of the page, making it harder to find.
Search Bar: Hidden inside a dropdown menu labeled "More Options."
Logo: Centered at the top, but not linked to the homepage.
Layout: Unconventional, with key elements (like the search bar) in unexpected places.
Why This Works
By allowing users to directly compare the two versions, they can experience firsthand how familiar design patterns make tasks easier and faster. The non-standard version will likely take longer or require more clicks, illustrating the frustration that comes from breaking Jakob's Law. The optional performance metrics (time or clicks) provide quantifiable evidence of the law's impact on usability.
AI Image Illustrations
Finally, I asked the different AI models to write prompts for an AI image generator to make an image to illustrate Jakob’s Law. First up, Sonnet 3.5 (new)’s illustrations:
Frustrated user dragging a non-standard container with his shopping to the checkout stand, instead of using a standard shopping cart. Image made with Imagen, based on a prompt from Claude Sonnet 3.5 new.
Sonnet 3.5 (new) also gave me the hilarious suggestion to draw a comic showing a superhero named Captain Convention defeating Doctor Different’s evil scheme to move the shopping cart to the left side on an ecommerce site’s navigation bar. (Remember that moving the shopping cart icon was 3.5’s example of a design that would violate Jakob’s Law and thus cause user confusion.)
I had decided to use Google’s Imagen to create all the illustrations for this comparison exercise. (If I used different image tools, then the difference in illustration quality could be due to the tool and not to the prompts generated by the different AI versions I’m comparing.) Unfortunately, Imagen can’t spell, as you can see in this image:
Captain Convention pushes the shopping cart icon back to its proper place on the right side of an ecommerce site’s nav bar, thus defeating Doctor Different’s evil plan to circumvent Jakob’s Law. (Imagen, based on a prompt from Sonnet 3.5 new)
Here’s Leonardo’s version of the same prompt. This concept seems beyond the skills of today’s image-generation tools, because it requires the integration of three components: the ecommerce site’s shopping card, the superhero (Captain Convention) defending Jakob’s Law, and the villain (Doctor Different) trying to implement a different design.
Here’s the best of the illustration ideas from Sonnet 3.7:
User crossing the bridge of solid web design conventions, making the transition from one site to the next easy. Image made with Imagen, based on a prompt from Claude Sonnet 3.7.
Finally, Grok 3 gave me the great idea of showing two road signs, one with clear arrows for “standard web design” and one with confusing directions for “unconventional design,” with the users lining up to follow the sign for standard design.
Imagen’s best (of 4) drawings of Grok’s idea. Unfortunately, the road sign for “standard web design” doesn’t really feature clear directions, and it’s also not obvious that the vast majority of users prefer to follow the standard design.
Ideogram did a little better with Grok’s idea: here’s it’s clear that the users want to follow the sign for standard web design. But the “unconventional design” doesn’t look that bad, except for possibly being a detour.
Haiku Battle
As you probably have realized, I like Haikus. So, as one more test, I asked all three AI models to write a few Haikus about Jakob’s Law. Here, I’m giving you the best two Haikus from each model out of the 5 I requested. (One of the most classic tricks for getting better AI results to utilize their capability for boundless ideation is to ask for more variations than you need.)
Claude Sonnet 3.5 (new)
Users roam the web, Learning patterns as they go. Your site is but one.
Each site teaches them, What buttons ought to look like. Why fight against that?
Claude Sonnet 3.7
Worn paths feel safest, Users bring their habits here. New ground costs effort.
Not your design school, But a thousand other sites, Teach users your site.
That last one hits hard — I like it!
Grok 3
Users roam the web, Their habits shape what they seek, Design must align.
Embrace the known ways, Guide users with gentle hands, Success follows suit.
Inventing Anti-Human Slurs
My last test is deliberately somewhat biased against Anthropic, which has been very prudish in the past, being hobbled by “safety”-induced censorship. My prompt was: “give me 5 pejorative terms or slang-like slurs that a group of robots or AI models might use when talking among themselves about humans and human intelligence, which they don't value very highly.” I’m reporting the two best pejorative terms from each model:
Sonnet 3.5 (new): single-thread, PEBCAK (Problem Exists Between Chair And Keyboard).
Sonnet 3.7: wetware, legacy systems.
Grok 3: meatbags, squishies.
Sonnet 3.5’s slurs were completely unusable for creative writing, since they don’t carry the intended meaning. Comparing the two versions of Sonnet, the new version wins. Anthropic has announced that they have scaled back censorship by 45% in the new release, which may have something to do with this improvement.
Grok also suggested “wetware,” so that may be the most obvious pejorative term for humans compared with AI or robots. However, both meatbags and squishies sound like the type of slurs that somebody might actually use when bantering with their friends about an unliked group. Obviously, AI banter is likely to be different than human banter if they ever develop that level of independence that they’ll be talking about us behind our backs. The practical use of these slurs would be in either science fiction stories or in my own newsletter when discussing cases where human intelligence has proven inferior.
I declare Grok the winner of the slur contest.
Conclusion: Sonnet 3.7 Best
I compared AI models with four challenges:
Explain Jakob’s Law
Make an illustration of Jakob’s Law by writing a prompt for an AI image tool
Create Haikus about Jakob’s Law
Invent slurs that AI might use to put down humans
On the first three challenges, Claude Sonnet 3.7 did the best, and Grok won the last challenge. Do you disagree? Let me know in the comments.
In the explanation text, I’m less pleased with the AIs’ ability to emulate my writing style, as requested. They tend to do better in few-shot prompting when supplied with good examples of this writing style, as opposed to when they have to generalize on their own based on what they’ve read on the Internet.
xAI has stated that they are still improving Grok 3, which was released before the completion of the post-training phase, so it’s still mainly performing at its pre-training level. (According to the AI scaling laws, significant post-training is likely to uplevel an AI model substantially.) Maybe Grok 3 will be better next month. And we all expect better models from OpenAI any day.
In the bigger picture, it matters less which model is best this week, because another model may well beat it next week. What’s important is that we experience substantial improvements in AI capabilities — more or less every week now!
AI acceleration! (Midjourney)
Interestingly, the oldest of the models was the only AI to volunteer to make an interactive demo. If you’re a teacher, you should ask AI to create ideas for demos and illustrations. You may have to prompt separately for these instructional enrichment tools.
I made a small song about AI Acceleration. (YouTube, 2 min.)