Wikipedia:WikiProject User warnings/Testing/Twinkle
Documentation for testing of user warnings in Twinkle. See more notes and working rewrites for Twinkle deletion notices here.
Background research
Frequency of Twinkle use
We also ran a query to see how many Twinkle user talk edits there had been in recent months. According to this query, there have been 161,934 Twinkle messages posted on user talk pages between the beginning of July and October 19th 2011. Next, we intend to run the same query for Huggle and Igloo edits and to count total user talk page edits during the same time period, in order to find out which of these tools is used most frequently on user talk pages, and what percentage of all user talk page messages are made using these tools.
Most frequently used TW warnings
In order to figure out which templates to focus on in our A/B experiment with Twinkle templates, we first need to know which Twinkle templates are most used most frequently on user talk pages. To gather this sample, we queried the Enwiki database for a random sample of 1000 user talk page revisions made using Twinkle (identified by ([[WP:TW|TW]]) which is populated in Twinkle edit comments by default) since October 2010. Then we grabbed the whole list of user warning templates used by Twinkle and queried the API to get the text of each of these revisions. Then we pulled out all the template names from the HTML comments in the template messages and counted up the ones that appeared on the official list of Twinkle user warning templates.
Below is the list of the most-used Twinkle user warning templates from the last year. Should give us a good start to narrowing down which templates to update when we run the Twinkle study.
| twinkle user warning template | number of uses in random sample of 1000 recent Twinkle user talk edits |
| {{Uw-vandalism1}} | 226 |
| {{Uw-vandalism2}} | 124 |
| {{Uw-vandalism3}} | 115 |
| {{Uw-vandalism4}} | 57 |
| {{Uw-block}} | 43 |
| {{Uw-unsourced1}} | 41 |
| {{Uw-test1}} | 40 |
| {{Uw-delete1}} | 37 |
| {{Uw-spam1}} | 28 |
| {{Uw-delete2}} | 25 |
| {{Uw-vandalism4im}} | 23 |
| {{Uw-error1}} | 17 |
| {{Uw-speedy1}} | 17 |
| {{Uw-vblock}} | 17 |
| {{Uw-speedy2}} | 15 |
| {{Uw-unsourced2}} | 14 |
| {{Uw-unsourced3}} | 14 |
| {{Uw-editsummary}} | 13 |
| {{Uw-spam2}} | 13 |
| {{Uw-test2}} | 12 |
The final counts delivered through this method is not completely accurate, because any other, previous messages on the user talk pages these revisions were pulled from which ALSO used one of the listed user warning templates (whether or not it was left using Twinkle), will also be counted. There may not be an easy way around this using the API Query. There is a way to pull up just the text of a single diff by passing non-Query API arguments, but we couldn't make that method work this time. However, this count is probably pretty close. And after all, any user warning template that is in active use is a good candidate for A/B testing, whether or not it's used solely by Twinklers.
Proposed deletion and Articles for deletion notifications
General principles we are shooting for...
- Simplicity/clarity: just focus on communicating the one or two actions you want the person to take. It doesn't have to be insanely short, just very clear about what you are encouraging people to actually do. Ultimately there's no way to teach people everything they need to know about deletion policy/process in one warning.
- Lack of bite. Which means, because you're giving someone bad news, you need to sugar coat it a little extra to not make it bite-y. And because it's a notification, not a warning, don't use exactly the same language or icons as uw templates.
- Letting people know where to ask for help. In the uw templates we've tested, that is usually the talk page of the reverting editor. This time we want to try the Help Desk, because it's already pretty normal for people to ask deletion-related questions there. Plus, I'm not sure if every nominating editor wants to teach the article creator how to navigate AFD etc.
- Encouraging people to improve their articles. A lot problems with articles might be fixed if authors kept at it and used deletion discussions as a way to elicit constructive feedback. Let's ask them to do so.
Templates tested
The templates tested were:
- PROD: new version, default version
- AFD: new version, default version
Analysis
Now that the templates are finished running, things we want to know are...
- Does the new version increase or decrease retention of the new editor compared to the default?
- Does the new AFD version increase the number of article creators who participate in the relevant AFD?
- Does the new PROD version increase the number of article creators who contest the proposed deletion by removing the template or using the talk page?
- Do either the old or new templates draw editors who have stopped contributing back into the project? E.g. Let's measure how many people receiving notifications didn't edit at least 30 days prior to getting the message, the age of their accounts (from date of first and last edit) and whether there was a difference between those people and newer accounts.
Feedback
- First it was hard for me to track down the item referred to, I had to use "what links here" for
{{Z82}}or whichever it was. Secondly that was back in December - you need to ask for feedback much faster than that. As far as the wording goes, I probably didn't read it, I just see the template, and generally wonder why people don't leave a proper message about the page, if it actually needs deleting I will do usually it on the spot, otherwise I can explain why it doesn't and we can avoid an unnecessary deletion debate. The first line is a bit irritating to regular editors "Hello, Rich Farmbrough, and thanks for contributing to Wikipedia!" The phrase "some editors are discussing" implies that something heavy is going down. "I have started a discussion" would be better. (Or "an editor has started...." ideally the nominating editor should be leaving the message.) The rest is pretty good, but targeted at newbies which may or may not irritate old hands. Rich Farmbrough, 00:47, 17 February 2012 (UTC).
- Hey Rich, sorry to make you hunt. When I messaged you I realized that this page didn't have a direct link to the new version. Fixed now. Thanks for the feedback. Steven Walling (WMF) • talk 00:58, 17 February 2012 (UTC)
- I was about to reply that I have not received any notification. The last AfD notification I received was 29 November 2011. My beef with the deletion processes is that they are quite happy to decide the fate of a wikipedia item and leave the Discussion/Talk page unbannered, red, unassigned to projects etc. (whatever you want to call it). Some wording that items must be tagged with projects on the talk page (to allow article alerts) to notify relevant projects would be nice. Argolin (talk) 01:35, 17 February 2012 (UTC)
- I was called here because I got an AfD notification, created with a use of new template. It was the third AfD notification in my life and I frankly didn't notice any difference as I cared most about the issue, not the notification's text. Still, as I'm already here, I would like to leave a couple of comments regarding these new templates:
- They are more personal and friendly, which is definitely good.
- New PROD notification lacks the information about consequences of declined PROD (I mean WP:CSD and WP:AFD). I think that this information is needed, as an editor would not be all that happy to learn that after he saved his article it was still deleted or nominated for deletion. The problem here is not with the possibility of deletion, but with the lack of awareness of the receiver.
- The case with AFD is even worse: just imagine the motivation to stay civil, which is imposed on Joe, who reads in AfD discussion the statement by Jack that the article Jack encouraged Joe to edit a couple of minutes ago is a blatant advertising which qualifies for speedy deletion. Such a start can damage the consensus building and fuel the uncivil debate by far easier then make Joe feel friendliness in a notification about his work to be deleted soon.
- So my summary is: the PROD template should be loaded with the WP:CSD- and WP:AFD-related information and the AfD template should be reduced to something like:
- P.S.: I want to note specifically that I don't think the experienced editors should be taken in account here. Anyway, people used to receiving template-based notifications read only keywards ("deletion" in these cases) and links.
- Dmitrij D. Czarkoff (talk) 01:47, 17 February 2012 (UTC)
- I found my PROD message to be friendly and not intimidating at all. It was calm, rational, and was easy to understand. As a person who likes Wikipedia, I hope that more PROD messages are like this could be used in the future to help new editors transition into become frequent editors of Wikipedia. GVnayR (talk) 01:49, 17 February 2012 (UTC)
Feedback from Anna Frodesiak
- Love 'em!
- Achieves the "General principles we are shooting for..."
- The absence of the amber triangle thingy is good. Those make the user from feel like they've done something wrong, or triggered some sort of software warning.
- The first person writing conveys a more human tone.
- In the AFD one, the softening of "Nomination of X for deletion" into "Deletion discussion about", makes the user feel that a few others are discussing, and not some large committee with some overlord nominating. So, that's less embarrassing and more private.
- In the PROD one, I like the clear bullets. The user will be freaking out, and will want clear step-by-step instructions on what to do.
- In the PROD one, only 2 blue links is better than the current, and overwhelming, 7.
- Friendlier, so better user retention. Anna Frodesiak (talk) 01:40, 17 February 2012 (UTC)