View source for Wikipedia:WikiProject Citation cleanup/Repairing algorithmically generated citations

{{shortcut|WP:AGCR}}

A number of tools can be used to generate citation templates algorithmically. The functionality is built into the [[Wikipedia:VisualEditor|VisualEditor]] and [[Wikipedia:RefToolbar|RefToolbar]]. [[User:Citation bot|Citation bot]] can be run on a page, perhaps summoned by the [[Wikipedia:Citation expander|Citation expander]] gadget. The user scripts [[Wikipedia:reFill|reFill]] (formerly reflinks), [[Wikipedia:ProveIt|ProveIt]], and [[Wikipedia:Miscellany for deletion/User:BrandonXLF/ReferenceExpander|ReferenceExpander]] are able to generate citations from a url.

This can be a great timesave, but the citations are not always correct or complete. '''Cautious editors will always double check the output of these tools''' to verify the correctness and completeness of the generated citations, but even very experienced editors may place too much trust in the tools, and create citations that must be repaired or cleaned up afterwards.

==How citations are generated==
Most of the tools listed above rely on the Mediawiki library [[:mw:Citoid|Citoid]]. The remainder, and Citoid itself, rely upon code snippets from the Zotero community called "[[:mw:Citoid/Creating Zotero translators|translators]]", that scrape metadata from individual domains, which Citoid then converts into cite templates. When the Zotero translators work accurately – like for large academic publishers or the ''New York Times'' – we get great citations. When an appropriate translator cannot be found (whether it doesn't exist or isn't in the suite of translators Citoid uses), Citoid will use Zotero's generic translator or, rarely, its own fallback functionality. 

These general purpose translators will sometimes introduce errors, omit parameters, or process dead links, creating citations that require repair or cleanup. Even the domain specific translators can fail if the linked page is populated on the client side with javascript instead of on the server side.

The tools relying on Citoid then insert the reference somewhere in the article being edited, without examining Citoid's output. Certain values in certain parameters (like numeric data in {{code|1={{!}}last2=}}) will cause an article to be placed in one or more applicable maintenance categories, which is done at the time of edit by [[Module:Citation/CS1]].

==Known issues==
===Failing to locate applicable parameters===
Authorship is commonly not determined by the functions that generate citations. Publication dates are second most frequently missed out if the page header doesn't have it in a meta tag but it is somewhere within the HTML body element. Sometimes titles are missed, which will generate an error from [[Template:Cite web|Cite web]] at the time of edit, and a visit from [[User:Qwerfjkl (bot)]] to the editor's user talk page. 

For book citations: volume, edition, and page number are never generated, even if the url processed by the citation generator links directly to a single page in a book. If a citation is generated to a book chapter that has been digitised or transcribed into html anywhere other than Google books or Internet Archive, the citation will be created using [[Template:Cite web|Cite web]] instead of [[Template:Cite book|Cite book]]. Usually the title of the book, the title of the chapter, and the name of the website will be mixed up or partially missing. Chapter or entry authorship will always be lost if the book is a compiled volume with many contributors.

===Incorrect parameter values===
Sometimes, a citation generator will find what it thinks to be one piece of information, which is actually something different. Common failure modes include putting the website or publisher into an author field, not removing the website or author from the end of a title, putting contact information into an author field, and replacing the title with the name of a website. Almost any kind of confusion is possible.

There are known special cases for book citations, depending on the domain. For '''Google books''', if a volume has editors, they will be misattributed as authors. Chapter or entry contributors are not usually possible to determine algorithmically, so these will always have to be added manually (if the information is even available), and the parameter names of the editors manually changed from {{code|1=last1=}}, {{code|1=first1=}}, etc., to {{code|1=editor1-last=}}, {{code|1=editor1-first=}} and so on. The misattribution of editorial contribution is a Zotero translator error.

For books hosted at '''Internet Archive''', the version information may not match the actual digitised work, place of publication will be forced into the {{code|1=publisher=}} parameter, and the {{code|1=others=}} field will be filled with the name of a library, or "{{tq|Internet Archive}}", or "{{tq|unknown library}}". The title of the book will be changed from title case to sentence case. Direct links to pages will be converted into links to the general volume information. Periodicals hosted at Internet Archive will be misidentified as books, and their authorship will be attributed to their publisher. Sometimes a direct page link will require "borrowing" the book from Internet Archive like a library. If restoring a direct page link to a work that requires "borrowing", the parameter {{code|1={{!}} url-access= registration}} should be added to the citation template. Most of the Internet Archive related issues are on their end, as a result of how their information is stored and presented.

===Processing dead links===
Sometimes, a link will no longer point to its original content, but will still serve a page. The site may have restructured, the content may be paywalled, or the domain may have been usurped. The citation generating tools will treat any webpage as a working link, even when obviously and explicitly labeled as a 404 error (page not found). In these cases the automatic reference should be removed entirely, and the bare URL dead link retained and tagged with [[Template:Dead link]], or an archived version located, per the guidance at [[Wikipedia:Link rot]].

===Incorrect template calls===
In almost all circumstances, citation generating tools will format their citations using [[Template:Cite journal|Cite journal]], [[Template:Cite book|Cite book]], or [[Template:Cite web|Cite web]]. Citations to journals are very likely to be correct and have very high overall accuracy. Rarely [[Template:Cite magazine]], [[Template:Cite patent]], or [[Template:Cite news]] will be used, almost invariably correctly. [[Template:Cite book|Cite book]] is very frequently correct, except for periodicals hosted at Internet Archive, journals compiled into volumes, conference proceedings, and occasionally others.

[[Template:Cite web]] is frequently problematic. It should often be one of the following:
*[[Template:Cite book]]
*[[Template:Cite journal]]
*[[Template:Cite AV media]]
*[[Template:Cite news]]
*[[Template:Cite conference]]
*[[Template:Cite press release]]
*[[Template:Cite encyclopedia]]
*[[Template:Cite magazine]]
*[[Template:Cite court]]

The source will always have to be inspected and the appropriate template substituted for [[Template:Cite web|Cite web]], which may require renaming parameters.

===Suboptimal website values===
Wikipedia citation templates are used to create [[COinS]] metadata, and the {{code|1=website=}} parameter is supposed to hold the human readable name of the website, except in cases where it is known primarily by its domain name. Citation generation tools will fill this parameter with the domain name very frequently. This information is not wrong, but it's also not helpful, since it can be trivially determined by inspecting the URL, and it goes against the intended function of the parameter for metadata reuse.

===Removal of internal anchors from URLs===
An internal anchor is what can appear after the {{code|#}} character at the end of a URL. For very long webpages, these can be essential for easy navigation, and act similarly to chapters in a book. They are removed from URLs in the generated citations, so they'll need to be readded manually afterwards, and the {{code|1=title=}} parameter updated appropriately, since [[Template:Cite web|Cite web]] does not support the {{code|1=chapter=}} parameter, and the {{code|1=work=}} parameter (the name of the larger work in which the title appears) is aliased to {{code|1=website=}}, so the two cannot coexist. In some cases the template would be more appropriate converted to [[Template:Cite book|Cite book]].

==Current repair projects==
The [[Wikipedia:Miscellany for deletion/User:BrandonXLF/ReferenceExpander|ReferenceExpander]] script originally (and currently, {{asof|lc=yes|June 2023}}) processed the first URL it located between a pair of ref tags, and replaced the entire content of whatever was between the ref tags with the citation generated by processing the URL. In addition to the issues listed above, this sometimes had the effect of deleting a second reference bundled with the first, deleting quotes or explanatory notes, removing correctly cited authorship and editorial attribution, removing page numbers, and other associated damage to manually formatted references. It also unnecessarily escaped the {{code|%}} character in URLs, which can cause them to break if they contain non-ASCII characters.

Around 2,500 edits using this script between January and April 2023 have been identified, and a cleanup and repair effort is underway at [[User:XOR'easter/sandbox/ReferenceExpander]]. A further 800 or so mainspace edits using the script in the second half of 2022 are identified at [https://quarry.wmcloud.org/query/72745 query 72745], and have been processed into worksheet format at [[Wikipedia:WikiProject Citation cleanup/ReferenceExpander batch 2]].

'''Current progress:'''
*[[User:XOR'easter/sandbox/ReferenceExpander|Task 1]]: {{bxt|checked: {{string count|page=User:XOR'easter/sandbox/ReferenceExpander|search={{((}}y{{))}}}}}}; {{!bxt|remaining: {{string count|page=User:XOR'easter/sandbox/ReferenceExpander|search={{((}}n{{))}}}}}}.
*[[Wikipedia:WikiProject Citation cleanup/ReferenceExpander batch 2|Task 2]]: {{bxt|checked: {{string count|page=Wikipedia:WikiProject Citation cleanup/ReferenceExpander batch 2|search={{((}}y{{))}}}}}}; {{!bxt|remaining: {{string count|page=Wikipedia:WikiProject Citation cleanup/ReferenceExpander batch 2|search={{((}}n{{))}}}}}}.

==Potential damage tracking and future repair efforts==
The final phase of cleaning up edits made using the ReferenceExpander script is foretold by queries [https://quarry.wmcloud.org/query/75209 75209] and [https://quarry.wmcloud.org/query/75208 75208], totaling approximately 2100 diffs. Edits of this age are more likely to have been corrected in the course of normal editing, and early users of the script were more conscious of its output.

ReFill / Reflinks seems to be a more popular tool with a lower associated degree of damage due to not overwriting existing references apart from bare URLs. It does identify itself in edit summaries, and could be approached similarly to ReferenceExpander edits, but that may not be worth the effort.

[[Module:Citation/CS1]] maintains tracking categories of possibly erroneous citation information, the most promising of which include:
*{{clc|CS1 errors: generic name}} (low false positive rate)
*{{clc|CS1 errors: missing name}}
*{{clc|CS1 maint: numeric names: authors list‎}} (very low false positive rate)
*{{clc|CS1 maint: multiple names: authors list‎}} (high false positive rate due to corporate authors)

Citoid bugs [https://phabricator.wikimedia.org/tag/citoid/ tracked at phabricator]

The overall scope of the problem is not yet clear.

==Suggestions for improvement==
At its root, to obtain more accurate algorithmically generated citations, the Zotero translators must be improved, which cannot be handled on Wikipedia, and requires a high level of technical skill (proficiency in JavaScript, regular expressions, CSS, and HTML).

Citoid can be improved by writing translators for its fork of the Zotero library, which can be done in-house but requires the exact same technical skills as coding for Zotero. Right now Citoid does not surface which translator it uses, which could aid in tracking problem cases. Requested at [[:mw:Talk:Citoid#Improving citation quality]]. Citation templates would have to be updated to hold a parameter for the translator used, and tracking categories created.

Citoid could generate warning messages for inaccurate translators, or general reminders to double check its work. Requested at [[:mw:Talk:Citoid#Improving citation quality]].

The citation templates could be updated to hold a hidden parameter that is set to a value whenever they are generated algorithmically instead of manually. This is like tracking the translator used, but less useful and very slightly less work. It could be removed or given an opposite value after citations are double checked by humans. Alternatively, citation generators could add a hidden comment to the wikicode like {{code|<!–– tool-generated citation ––>}}.

[[:Category:CS1 maint: numeric names: authors list‎]] could be elevated to error status, alerting editors that their author fields contain bad information, and a bot could place a notice on the editor's user talk page after publishing such an edit.

Whenever repairing algorithmically generated citations, practice leaving an edit summary that links this page, or a specific cleanup project, or a note along the lines of "{{xt|please always double check automated references.}}"