Jump to content

Wikipedia:Wikipedia Signpost/2011-08-01/Research interview

Add links
From Wikipedia, the free encyclopedia
< Wikipedia:Wikipedia Signpost | 2011-08-01
This is an old revision of this page, as edited by Skomorokh (talk | contribs) at 20:28, 20 July 2011 (initial questions). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.
The Signpost
← Back to Contents
View Latest Issue


Research interview

The Huggle Experiment: an interview with the team

Contribute  —  
Share this
By Skomorokh


How did the idea to experiment with the standardised warning system originate?

Name:




How do lofty strategic goals like "Support the recruitment and acculturation of newer contributors" get translated into practical initiatives such as this?




An experiment of this kind seeks to understand social phenomena using technical methodologies. Does this involve coordination between, for instance, the Community Department and the Technology Department, or is the experiment conducted by researchers proficient in social statistics or the digital humanities? Can you talk a little about the backgrounds of those involved?




How were the parameters of the experiment – number of warnings delivered, proportion of changed warnings – decided upon?




The Huggle experiment is not the first attempt to investigate the interactions of patrollers and new page creators. A notable community-lead effort was the Newbie treatment at Criteria for speedy deletion experiment in 2009, where experienced editors (this interviewer included) posed as inexperienced article creators in order to gain an insight into how new contributors were treated in the patrolling process. The experiment attracted significant controversy, due to ethical concerns surrounding informed consent of the subjects. To what extent did the research team consider or engage with the relevant subjects from the editing community (i.e. Huggle patrollers, new contributors) prior to this experiment?




What do the researchers hope to learn from the experiment and what are the preliminary expectations or hypotheses to be tested?




What sorts of approaches we might expect from the Foundation in testing and improving usability, reader engagement and editor retention in the months to come?



Next "Research interview" →
+ Add a comment

Discuss this story

These comments are automatically transcluded from this article's talk page. To follow comments, add the page to your watchlist. If your comment has not appeared here, you can try purging the cache.

"Fewer than half of the newbies investigated received a response from a real person during their first 30 days". I think we really dropped the ball here. Interaction is a major way to recruit newbies and hopefully turn them into "regulars". OhanaUnitedTalk page 05:18, 2 August 2011 (UTC)[reply]

I agree: personal mentoring is the key, but to allocate such resources requires the identification of the most likely newbies. How to do that? Perhaps some more focused research questions? I wonder whether the research project will involve the gathering and coding of qualitative data from newbies/anons. Tony (talk) 08:43, 2 August 2011 (UTC)[reply]
One does have to be careful, though; I fully hope that we drive the vandals and SEO upstarts away, which I'm guessing is probably over half of all new users at this point. I suspect the percentage will be dramatically better once we implement a requirement to become autoconfirmed to create articles; there's no possible way we can leave customized messages for all of the people we encounter. The Blade of the Northern Lights (話して下さい) 19:09, 2 August 2011 (UTC)[reply]
Hi, thanks for the comments on this topic! We are definitely going to be qualitatively analyzing the edits which all of these users make after receiving each of the different warnings. This is actually our primary way of evaluating the success of each of the templates -- a simple 'do they continue to edit' isn't good, because we don't want persistent vandals and spammers to keep editing. Then we will be able to run a bunch of interesting analyses on how different kinds of new users react to these different messages. It will be interesting to see if, for example, the more personalized warnings drive away vandals but not link spammers, or if the warnings with teaching messages are better at "converting" users who make test edits into good content contributors.
As to the time needed to personally interact with new users, this is a definite problem that we are very interested in, and we are working on trying to model which new users are more likely to become good contributors in the future. We are thinking of a new user welcoming suite like Huggle, but where you can look at a newbie's first few edits and then leave one of a dozen or so targeted welcome messages. So if you see a user fixing a lot of spelling errors to articles about Canada, you'd be able to thank them for copyediting and invite them to join WikiProject Canada in just a few clicks. And if you have any other comments, questions, or suggestions, I'd be happy to hear them. StuGeiger (talk) 20:20, 2 August 2011 (UTC)[reply]
Thanks, Stu. You said, "we are working on trying to model which new users are more likely to become good contributors in the future" – this is the most important thing I've heard in this discussion. I think we'll be hoping you can find sufficiently distinctive patterning as early as possible in the editing history of the newbie-pluses (the ones we want to keep) and the newbie-minuses (vandals, link-spammers, and paid political/corporate operators). I suppose it will be a combination of factors such as (i) the linguistic patterns, (ii) the locational distribution of the edits (which pages are edited), and (iii) the temporal distribution of the edits. How these three aspects interact could do with some heavy-duty stats analysis, and of them, the linguistic is likely to be the most challenging and deepest (a research delimitation is required, I think).

Perhaps two critical concerns will govern the efficiency with which the problem can be addressed: (i) how long into a newbie's edit-history the patterns become clear, and (ii) the extent to which they can be identified by a bot (including whether a bot could do the initial "easy" filtering and pass a minority on to human eyes for higher-level sorting to identify the promising newbie-pluses for human interaction – a three-tiered filtering, as it were). Of particular interest might be the grey area of newbies – not those who will clearly stay and those who clearly won't (or who we clearly do or don't want to stay), but those where final stage, human interaction, has a reasonable likelihood of making the difference, of bringing them over the line. Finding the best bot/human mechanism for rationing the supply of "newbie mentors" to this prioritised editorial demographic, IMO, is the challenge. After that, a future project could work on developing guidelines for the best ways in which to interact with newbie-pluses. Tony (talk) 02:41, 3 August 2011 (UTC)[reply]


Want the latest Signpost delivered to your talk page each month?
Retrieved from "https://en.wikipedia.org/w/index.php?title=Wikipedia:Wikipedia_Signpost/2011-08-01/Research_interview&oldid=440540077"