Post

Turns out users were grinding scenes without rewards. Progression was too slow. Users were unsure how to unlock new scenes.

For secondary research, playtests were held with the team. This is valuable, but solutions are often thought of too quickly. There is a risk of confirmation bias, and to not solve the actual underlying problems.

It still created valuable data though. You spot things you may not have otherwise spotted, also because you know what to look for from the initial user data.

Example: Even a screen with two buttons can be confusing for a new player. What does a blue button mean? What is a green button? Is the left button the most important button? Even the basics are hard!

Goal: Increase retention by 1%

They made the start screen just one button. Moved connection button to top right, much smaller. This improved retention by 2.5%

They added a hint button to help players. This reduced retention by 2.2%. Players would just hit the button all the time, and the game became a chore instead of playing an actual game and having to think.

They added an overlay on how to play the game. This increased retention by 1.8%

So did it work? Yes! But not 1% total. They hit 0.81%

Key takeaways: Keep it simple. User stories can save your sanity. Always include your team. Build your research process. So that when you need to apply research, you can do so right away.

(I missed the last two talks as I had a long and good discussion with someone about hyped technology systems and confirmation bias. I'll link some things here that I think are important to relate: 1. mikelovesrobots.substack.com/p/wheres-the... 2. metr.org/blog/2025-07...

(3. @edzitron.com's work: www.wheresyoured.at/ai-is-a-mone... 4. The video I made recently in relation to blockchain, metaverse, and LLM AI in gamedev: youtu.be/CeZ_rcDQg6Q?... The earlier talk today that mentioned confirmation bias made me realize this is all related to those hypes.)

Next, another talk about World of Warships. How do you measure player satisfaction? How do you make sense of it?

They have 4 facets of satisfaction: Early, immediate, in-depth, and long-term. Each has different data that is important depending on the questions they need to answer.

By having this data, they can readily see if released content is too difficulty or too drastic. Gathering the data always means you have a baseline to compare to. So questions can be answered right away. (Reminding me of a previous talk, of having the user research systems always ready to go)

Same with data to identify a satisfaction drop after a feature launch, and a 'trinket' problem. In this case by adding a question to the system right after a match. So players would get a single question, and that gave them enough data to infer impact from when they launched content.

This question to the player can be a range, e.g. 1-5, but also open ended. Then a word cloud can be made from the open ended questions, which gives them a quick answer about most often used words without too much digging into the raw data.

Through the combination of all these elements, the entire 'ship' of user research can be built. If one element were missing, it would not float. Having all elements allows them to have the views they need to make accurate decisions.

Q: Do you ever have a hard time relating data to a particular change? A: (very long pause) ...yeah.

Next talk: Demystifying Sensitivity Reading to make games better (not 'just' inclusive)

'Sensitivity reading' is not about forcing DEI, censorship, judging, or woke/broke. It is about immersion, being memorable, avoiding backlash, and a virtuous cycle. It is about creating quality content. Like you would with historical projects.

That work can cross many different parts of a videogame, such as language, character design, communication, world building, stereotypes, and creative choices.

Showing an example of Sifu, which did not have Chinese employees and was not looked over by an expert in Chinese culture, which caused some mistakes to be made.