Module talk:Excerpt/Archive 2

Archive 1

Archive 2

Archive 3

Archive 4

Lua errors in portals

Here is a useful list of portal Lua errors, which I fixed occasionally by having the module handle more cases. The current crop is:

• Romney Classical Institute in Portal:American Civil War/Featured article doesn't remove {{Infobox school}}. This causes a Lua error in the infobox, "Lua error in Module:Location_map at line 414: No value was provided for longitude.". (Schools have longitude; portals don't.) This will affect Portal:American Civil War only occasionally but may indicate a wider problem. Module talk:Location map/Archive 10#Please limit error checking to main (article) space may be relevant.
• Battle of Vukovar (protected) in Portal:Croatia/Indices. This is also a longitude error but there is no unwelcome infobox.
• Portal:Liquor: I can't find an error even after expanding the multitude of collapsed sections. It may be transient and have gone away when a different random selection was made.
• Portal:San Francisco Bay Area has too many expensive function calls. This is not a module issue and requires that we simplify the portal. I've had a grumble on its talk page.
• Forth Road Bridge in Portal:Scotland/Selected articles/34 and Portal:Scotland/Selected articles doesn't remove {{Infobox bridge}}, causing another longitude error.
• Portal:Speculative fiction/Selected works: The time allocated for running scripts has expired. This is not a module issue and could be solved by simplifying the portal subpage. (I'm not going to do that, as the list has been the subject of heated discussion.) The portal itself is unaffected, as it processes only one randomly selected work.

– Certes (talk) 16:43, 22 May 2020 (UTC)

Thanks! Tomorrow I'll pause with the development of new features and focus on testing and debugging. Cheers! Sophivorus (talk) 18:32, 22 May 2020 (UTC)

@Certes: It seems that the only errors directly related to this module are the Longitude ones, correct? I created a test case at Template:Transclude lead excerpt/testcases2#Longitude error. As you said, it seems that for some reason the module isn't removing the infobox, but I haven't figured out why yet. Kind regards, Sophivorus (talk) 16:00, 23 May 2020 (UTC)

There may be two problems: one that causes infoboxes to be retained, and one in Module:Location map or in the way it's being used, seen even in Portal:Croatia/Indices which correctly hides the Vokovar infobox. Certes (talk) 17:25, 23 May 2020 (UTC)

Non-free files not being removed

The instructions on Template:Transclude lead excerpt indicated "Non-free images are removed." Except, they aren't. This template is now causing nearly all of the 112 non-free content violations on the daily nfcc9 report. --Hammersoft (talk) 23:42, 24 May 2020 (UTC)

Thanks for the report. This problem sounds serious and might merit temporarily reverting recent enhancements. Previous versions of the module devoted a substantial proportion of their code lines and CPU time to identifying and excluding non-free files. Certes (talk) 11:23, 25 May 2020 (UTC)

@Hammersoft and Certes: Indeed, thanks for the report. I looked into it and fixed it. What a single character can do! Sophivorus (talk) 13:14, 25 May 2020 (UTC)

Well done. That also seems to fix this report of spurious infoboxes appearing. Presumably a wanted templates lurk within the unwanted infobox. Certes (talk) 13:32, 25 May 2020 (UTC)

Thanks everyone! --Hammersoft (talk) 15:25, 25 May 2020 (UTC)

Revision history

Moved from Template talk:Excerpt

 – * Pppery * _{it has begun...} 23:17, 15 May 2020 (UTC)

Another issue I'm running into: excerpts don't seem to reflect accurately in revision histories, making it hard to see what an article was like at a given time. {{u|Sdkb}} ^talk 08:07, 8 April 2020 (UTC)

This is actually a MediaWiki core issue. If you view any page in the past, you'll notice that all the templates are transcluded as they are now, rather than as they were before. Same with files. Sophivorus (talk) 22:24, 7 May 2020 (UTC)

Yeah, I've noticed that before, but it's greatly exacerbated when excerpts come into the picture. Is it a solvable problem (i.e. should I file or find a phab ticket), or is it something that'd be technically impossible without significant changes to file storage/etc.? The issue is that changes to less-watched pages with poorer oversight have been showing up at more important pages through excerpts, and it's hard to detect them without them showing up in the revision history. {{u|Sdkb}} ^talk 23:06, 7 May 2020 (UTC)

A transcluded page may change several times between two adjacent entries in the edit history of the transcluding page. A page which was transcluded at the time may have been deleted. A portal which rotates content by including subpage /Selected Thing/month will transclude different pages from time to time without any edit. This may be a can of worms. Certes (talk) 23:39, 7 May 2020 (UTC)

@Certes: agreed — I recognize this is more in the "wish list" category. I guess the bigger theme I'm trying to get at is that excerpts introduce dynamics where changes to less sensitive areas can end up impacting more sensitive ones, and just as with templates, this introduces some risks for vandalism/etc. that we should be thinking about strategies to address. In my utopic vision for Wikipedia, we'd have a type of cascading protection, where you wouldn't be able to edit an excerpt unless you have the permission needed to edit the places it's excerpted. {{u|Sdkb}} ^talk 23:56, 7 May 2020 (UTC)

Recent research has produced old page versions processed with templates as they were at the time. (Search that link for WikiHist). Certes (talk) 20:49, 31 May 2020 (UTC)

Protection

@JJMC89: I see that you've protected this module at template-editor level. Unfortunately, that effectively ends maintenance and development on a module which was still being actively changed to handle new cases. (It parses wikitext, and a bulletproof implementation which handles every intricacy is not feasible in a Lua module.) I see no vandalism. Is pre-emptive protection at this level really necessary? Certes (talk) 11:20, 23 April 2020 (UTC)

@Certes and JJMC89: Agreed. Sophivorus, myself, and several others actively working on this are not (yet) template editors. ECP would be fine (any lower than that is a vandalism risk, given how widely transcluded this already is). - {{u|Sdkb}} ^talk 04:46, 24 April 2020 (UTC)

Protection does not stop development. Development should be happening in the sandbox, which is not protected. If no one working on it has the necessary rights to edit directly, then make an edit request (or request the rights if qualified). It is standard to protect modules at least the same level as a template that uses it, {{excerpt}} in this case. — JJMC89 (T·C) 05:18, 26 April 2020 (UTC)

Thank you for the reply. This change does seem ironic in view of the fact that 90% of the use is in portals and we have spent the last year being constantly lectured about how portals are unimportant because no one reads them. I hope that I have provided a good start on which others can build. Certes (talk) 09:46, 26 April 2020 (UTC)

Thank you for all your work, Certes. I really appreciate this module and countless editors and readers will benefit from it, hopefully well into the future. This protection is really annoying, because although I can still edit the module per being a global interface editor, I may not always respond as quickly as required or desired. I wish and recommend you request the template editor permission. I'm not sure about the process here but you can count with my support. Cheers! Sophivorus (talk) 18:31, 28 April 2020 (UTC)

@JJMC89: thinking about it, given the potential impact disruption of the template would have, even given its somewhat modest adoption so far, TEP is probably warranted. I've been considering applying for WP:PERM/TE for a while given how active I am in the template space, and this moves me toward that. {{u|Sdkb}} ^talk 20:25, 28 April 2020 (UTC)

@Certes and Sdkb: Hi! I just requested the template editor permission because it seems that using my global interface editor permission for editing this template and module is misuse. Maybe you'd like to check out my request, leave a comment, and perhaps request permission yourselves? Sophivorus (talk) 13:43, 13 May 2020 (UTC)

Congratulations on obtaining the grant and thank you for the ping. I've been following events here but, as I have collected no hats beyond the automatic "extended confirmed", I doubt that my backing would carry much weight. Certes (talk) 14:06, 13 May 2020 (UTC)

@Sophivorus and Certes: Congrats on the grant, Felipe — I'm so happy to hear about that! Regarding the template editor perm, as it happens, I applied for and received that perm a few days ago, but an editor who I have disagreed with some in the past launched an ANI thread about my template editing, which was closed two hours later before I saw it with the PERM revoked. It's now awaiting additional comment below the initial close before it can be reclosed. {{u|Sdkb}} ^talk 14:40, 13 May 2020 (UTC)

They've even protected the sandbox now. Certes (talk) 18:28, 14 May 2020 (UTC)

@Certes: the transclusion search (here) shows the sandbox is being used on thousands of pages, so a bot protected it. Whatever is causing this should be resolved. — xaosflux ^Talk 19:07, 14 May 2020 (UTC)

Yeah, someone borked something - there's zero reason why the sandbox should be transcluded onto more than 3k pages. Will look into this. Primefac (talk) 19:11, 14 May 2020 (UTC)

@Primefac: Looks like this is related to Module:Excerpt/templates - @Sophivorus: why are you requiring the sandbox version in to this module? — xaosflux ^Talk 19:12, 14 May 2020 (UTC)

Which is getting pulled in via Template:Transclude lead excerpt and who know where else - in to thousands of pages. I don't have time to work on this right this min. — xaosflux ^Talk 19:14, 14 May 2020 (UTC)

Sophivorus, you've borked things good, and I don't have the time or patience to sort through everything you've done. As near as I can figure, you created Module:Excerpt/templates and then forced {{transclude lead excerpt}} to use it; the module calls Module:Excerpt/sandbox and when I removed that requirement it all went to hell. Rather than break everything, kindly get things to a point where the sandbox is not being called from Module:Excerpt/templates. Primefac (talk) 19:18, 14 May 2020 (UTC) Looks like things have been rolled back to use the main template. For what it's worth the sandbox was never more than semi-protected, but I've removed that now as well. Primefac (talk) 20:49, 14 May 2020 (UTC)

Thanks for the explanations, and sorry for opening a can of worms. Certes (talk) 22:27, 14 May 2020 (UTC)

I apologize for the unnecessary confusion. My request was declined, but no problem, it just means I'll have to request the edits like everyone else. Talking about that, since the update I was trying to make was not the simplest, and by now JJMC89, Primefac and Pppery seem most acquainted with it, may I request your help on this one? For your convenience, here's a briefing: the latest changes aim to split the module in two: one part for the general, abstract, cross-wiki functionality (the main module), and another part for the enwiki-specific functionality (the /templates submodule). To avoid any breakings during the update, my strategy was:

1. Put the general, abstract, cross-wiki functionality in Module:Excerpt/sandbox
2. Put the enwiki-speceific functionality in Module:Excerpt/templates
3. Make /templates call /sandbox (temporarily)
4. Update the templates so that they call /templates
5. Once all the templates are calling /templates which calls /sandbox, update the main module to the latest /sandbox code (here I was when I was reverted)
6. Finally, edit /templates to call the main module rather than /sandbox

Maybe I could have created /temp and use that instead of /sandbox and that would've prevented all this trouble? Or maybe you can come up with a better way still? If you don't feel like helping on this one, I'll understand and I can request someone else or simply forget the split, though it would be a shame since it's already coded and it would be useful. In any case, thanks for your time, gentlemen! Sophivorus (talk) 23:21, 15 May 2020 (UTC)

I think the only wrong you did was failing to revert yourself on step (4) after I reverted you on step (5), and the general strategy for rolling the change out was fine. * Pppery * _{it has begun...} 04:05, 16 May 2020 (UTC)

@Pppery, JJMC89, Primefac, and Xaosflux: Hi! After some more debate, today I was granted the template editor permission. Thanks everyone! I'm now testing and debugging the latest features. If everything goes well, then in a couple days I'd like to deploy the changes so that editors may start to benefit. To avoid any trouble while deploying, I think I'll create staging submodules at Module:Excerpt/staging and Module:Excerpt/i18n/staging to avoid going through the sandbox. Kind regards, Sophivorus (talk) 14:33, 23 May 2020 (UTC)

You're always welcome to use a sandbox (that's what they're there for), the issue was that the "live" template ended up calling the sandbox when it shouldn't have been. Granted, you can call a "sandbox" anything you want, so I suppose carry on for now... Primefac (talk) 21:02, 23 May 2020 (UTC)

More non-production protections

FYI, I've reverted Sophivorus' protection on Module:Excerpt/i18n/staging and Module:Excerpt/staging, and asked that they no longer use GIE access. Whenever testing is done, any admin is welcome to roll tests to production and/or adjust protection levels as appropriate. — xaosflux ^Talk 16:11, 30 May 2020 (UTC)

@Pppery, JJMC89, Primefac, Xaosflux, and Swarm: Well, after days of testing and consensus-building, today I thought it'd be a good day for deployment. So I created the staging submodules I announced above, necessary due to breaking changes, and protected such staging submodules because during the update they were going to be live for some minutes on very high-traffic pages such as COVID-19 pandemic. I thought this would be reasonable, but it seems that protecting submodules is beyond my jurisdiction as a global interface editor. So I was reverted and asked to stop. Apologies for that, it seems my common sense was off. I don't want to screw up again, so I'd like to request someone else more familiar with the procedures of the English Wikipedia to please update the module to the latest version. I have explained above the procedure I recommend to avoid any breakings and I've done enough testing to be quite confident that the new version is stable, but please let me know if further explanations are required. Kind regards, Sophivorus (talk) 16:25, 30 May 2020 (UTC)

I consider myself too involved to implement this edit request (and I'm generally afraid of implementing large complicated edit requests), so won't do this myself, but I don't see why you can't do this yourself: you're trusted by the English Wikipedia community to edit template-protected templates, and the problems only arose when you tried to use rights you had not earned. * Pppery * _{it has begun...} 16:35, 30 May 2020 (UTC)

I am a template editor and willing to help, but I have no familiarity with this module or the background discussions. Can someone please spell out the specific edit request (example: "Copy the contents of Module:X/sandbox to Module:X") and explain how to determine if something is broken after the request is carried out? Thanks. – Jonesey95 (talk) 17:41, 30 May 2020 (UTC)

@Pppery: No worries, I understand, I'm also scared now. @Jonesey95: Thanks! The remaining steps would simply be:

1. Update the 6 templates using the module to their sandbox version. The sandbox version calls /templates, which in turn calls /staging which calls /i18n/staging
2. Once all the templates are calling /templates, update /i18n to /i18n/staging
3. Then update the main module to the latest /staging code and make it call /i18n instead of /i18n/staging
4. Edit /templates so that it calls the main module rather than /staging

I'm confident nothing will break, but I'd check the most visited articles and portals using the templates to make sure, in particular COVID-19 pandemic and Portal:Coronavirus disease 2019. Thanks for your help, Sophivorus (talk) 19:07, 30 May 2020 (UTC)

@Jonesey95: Please hold on, I found a (minor) issue I'd like to solve, thanks. Sophivorus (talk) 19:52, 30 May 2020 (UTC)

 Bureaucrat note: @Sophivorus: please note, the injunction is only on your use of GIE permissions (such as the protect function) - you are free to make any productive edits, including edits to protected pages that your local access allows. — xaosflux ^Talk 21:11, 30 May 2020 (UTC)

@Pppery and Xaosflux: Very well, I'll deploy myself then after I fix an issue or two. Feel free to protect the staging submodules if you want to, since they'll be live for a few minutes. Kind regards, Sophivorus (talk) 13:39, 31 May 2020 (UTC)

@Pppery, Xaosflux, and Jonesey95: Deployment complete! Thanks everyone for your help and support. Protecting Module:Excerpt/templates would probably be a good idea since it's now used by all templates. Future deployments should be simpler and smoother. Kind regards, Sophivorus (talk) 13:42, 1 June 2020 (UTC)

I added TP to match the module. — xaosflux ^Talk 13:47, 1 June 2020 (UTC)

Deployment

Hi everyone! The new version is almost ready! Besides the split between the main module and the /templates submodule, the new version fixes references automatically (yay!), introduces the new (and still rather immature) tablesOnly and listsOnly features, includes considerable code restructuring aimed at making the module more modular, and other minor features and fixes. Once this new version is deployed, I'm confident that with the new separation of concerns and the improved testcases, development will become faster, easier and better than ever! But before that, we should ensure this deployment goes smoothly. All the testcases are looking good to me. The output for the new version is always identical, trivially different, or better. In some cases it may differ due to randomized content, but I think that's normal and not a problem. Tomorrow I'll do some more tests, but any additional testing by someone else would be most welcome. If everything goes well and no one objects, I'd like to deploy on Wednesday 27. Kind regards, Sophivorus (talk) 15:17, 25 May 2020 (UTC)

Hi! I'm finding some minor issues while testing the code on the Spanish Wikipedia, so I'll delay the deployment a few days until the code is stable. Kind regards, Sophivorus (talk) 16:59, 27 May 2020 (UTC)

 Done Sophivorus (talk) 14:55, 1 June 2020 (UTC)

Multilingual Templates and Modules

The following page may be of interest to editors adapting this module for use on other Wikipedias: mw:Multilingual Templates and Modules. Certes (talk) 10:00, 22 May 2020 (UTC)

Thanks, I'll do my best to follow that approach! Sophivorus (talk) 17:49, 22 May 2020 (UTC)

@Certes: Yesterday I started with this process at mw:Module:Excerpt, thanks for the heads up! Sophivorus (talk) 13:27, 8 June 2020 (UTC)

Next steps

Hi! After finally deploying the latest changes, I'd like to outline what I think will be my next steps, for the record and in case someone wants to comment or suggest changes.

• Properly document the new features
• Fix bugs that may arise with the new version
• Improve the test cases of all the templates and modules using this module to ease future development
• Refine the getTables and getLists methods to allow to get specific tables and lists
• Encapsulate the code to get specific files and paragraphs into getFiles and getParagraphs methods
• Continue making the module and methods more modular

So in short, I think the immediate roadmap is to improve existing functionality rather than adding new one. Due to my grant goals for June, I'll also now focus more on spreading this module to other wikis. Kind regards, Sophivorus (talk) 14:55, 1 June 2020 (UTC)

The |only= feature is neat! Right now, I'm trying to get only an infobox (taking the infobox (minus the photos) from COVID-19 pandemic to use at COVID-19 pandemic by country and territory). Any idea on how to do that? {{u|Sdkb}} ^talk 03:31, 12 June 2020 (UTC)

@Sdkb: I'm working on a general solution to this and other requirements. Your comment suggests I'm thinking in the right direction. Thanks for the motivation, I'll do my best to get it ready soon. Sophivorus (talk) 14:19, 12 June 2020 (UTC)

Doctor Who (season 1)

Is transclusion of the episode list rather than the lead from page Doctor Who (season 1) intentional? This was originally found at Portal:Doctor Who with template {{Transclude list item excerpts as random slideshow}}. Here's demo of plain {{Excerpt}}:

Also, the rightmost column header is displayed incorrectly by {{Transclude list item excerpts as random slideshow}}, showing up as <abbr style="color:white" title=" '"`UNIQ--nowiki-00000008-QINU`"' ">AI instead of AI. —⁠andrybak (talk) 09:33, 13 June 2020 (UTC)

@Andrybak: Doctor Who (season 1) contains <onlyinclude> tags, so a transcluding page sees only the episode list, which is therefore treated as the desired section. We could change that behaviour – it's explicitly coded that way and has caused similar queries elsewhere – but that might negatively impact other excerpts which rely on it.

I don't know what is happening to the AI column. For me, it displays correctly above but not in the portal. On my screen, the table just squeezes into the available width of of the portal column and may be overflowing slightly. {{Transclude list item excerpts as random slideshow}}, which uses a different module, may not handle horizontal scrolling well. Certes (talk) 11:03, 13 June 2020 (UTC)

Certes, thank you. The onlyinclude tags are there for generating List of Doctor Who episodes (1963–1989).

{{Transclude list item excerpts as random slideshow}} links to Module:Excerpt slideshow, which uses Module:Excerpt. I haven't looked at the code too closely, but the excerpt generation seems to be done by Module:Excerpt. —⁠andrybak (talk) 11:13, 13 June 2020 (UTC)

As elsewhere, onlyinclude presents a dilemma. A purist might argue that the selection of the episode list is specific to the 1963–1989 page, which should use labeled section transclusion or {{Excerpt}} instead so that other pages can transclude different parts of the same source pages.

I previewed the portal, replacing {{Transclude list item excerpts as random slideshow}} by {{Transclude lead excerpt | paragraphs=1-2 | files=1 | more=| Doctor Who (season 1)}}, which calls Module:Excerpt directly. The result is ugly and overflows its box on my screen (so the AI header is hidden behind another snippet in the right column) but (when I make that overlapping item semi-transparent) the AI header does render correctly. I think the problem we see in the portal must have something to do with the gallery assembly in Module:Excerpt slideshow. Certes (talk) 11:44, 13 June 2020 (UTC)

Bug reports and feature requests

As the use of excerpts multiplies, I'm starting to get some bug reports and feature requests, as well as ideas I come up with myself to improve this module. How should we track all these? For now I'm writing them down here, cheers! Sophivorus (talk) 19:48, 5 April 2020 (UTC)

• Finish internationalization. Done at Module:Transcluder
• Move localization to a separate JSON subpage.
• Refine caption detection. Done at Module:Extract
• Detect and remove self-links, especially redirects. When article A transcludes article B, if B contains a link to A, MediaWiki will detect that, mark it as a self-link and show it bold. This is fine, and the bold text can be removed via CSS. However, if article B contains a link to some redirect to article A, then MediaWiki will not detect it and the link will be shown blue, which is undesirable. The current solution is to just edit B and change the redirect for a straight link to A, but such changes are not always desirable and it would be better to detect and fix this from the module. Done at Module:Transcluder
• Append references to the returned text, to avoid broken references. This is probably difficult to code, but fixing broken references is one of the main difficulties when using excerpts.
• Prefix reference names with the included article's title to avoid conflicts with same-name references in the excerpting article.
• Add a way to transclude fragments marked with <section begin=Name of fragment/> and <section end=Name of fragment/>, intended to be transcluded via the #lst parser function, in order to take advantage of all the extra goodies of this module.
• Add a way to transclude a section including all subsections.
• Add a way to transclude tables only, like there's a filesOnly option. Tables are excerpted quite a lot, so it would be awesome to be able to reuse them without having to worry about the text around them or having to add <section> or <includeonly> or <noinclude> tags.
• Add a way to transclude lists only, like there's a filesOnly option. Lists are excerpted quite a lot, so it would be awesome to be able to reuse them without having to worry about the text around them or having to add <section> or <includeonly> or <noinclude> tags.
• Improve table handling generally This is too general and difficult, it's better to improve the handling as needed.
• Use a consistent style for naming functions, variables, etc. (some are camelCase, some are not)
• Terminate prepended images by newline, so {|table, *list, :indent, etc. work at the start of the following excerpt.

@Certes and Sdkb: I created a Phabricator project for this module and the related templates in order to better track bug reports and feature requests. However, I'd like to ask if you also think this is the best way forward before migrating the open issues. Cheers! Sophivorus (talk) 13:54, 8 May 2020 (UTC)

@Sophivorus: I've subscribed to the project, but I barely use Phabricator myself. Certes (talk) 14:02, 8 May 2020 (UTC)

@Certes: I think in the end there was no real need or use for the Phabricator project for now. Sophivorus (talk) 14:57, 1 June 2020 (UTC)

@Certes, Sdkb, and Sophivorus: I agree that making people go to another site (where they have to create a login ID, and learn a new system) isn't how to handle template management on Wikipedia.  — SMcCandlish ☏ ¢ 😼  02:37, 17 July 2020 (UTC)

Attention! Please leave this section as-is and create new sections for bug reports and feature requests. Sophivorus (talk) 15:35, 18 July 2020 (UTC)

Transition to Module:Transcluder

Hi all! As I hinted above, I've been working on a general solution to many of the difficulties and requests I've been receiving and perceiving with excerpts: mw:Module:Transcluder (copied at Module:Transcluder) is a general-purpose transclusion engine, inspired on this module but largely rewritten to be fully abstract and cross-wiki. For a time I resisted creating a new module (I'm a big fan of reusing rather than remaking, as you may guess) but eventually I had to give up, there were just too many English and portal-specific assumptions for my skill level. The new module still lacks some functionalities and has some hacks and bugs I'm working on, but I wanted to share it already to maybe get some feedback and so that we may start discussing the best way towards a transition, if any.

My thoughts are as follows: I'm working on another submodule, which for now I call Module:Extract, which is basically an implementation of Template:Excerpt using Module:Transclude rather than Module:Excerpt. If the template is made to use that module, then that would only leave the "portal templates" (Template:Transclude lead excerpt, Template:Transclude linked excerpt, etc.) using Module:Excerpt, so I think it can be re-merged with Module:Excerpt/templates to perform the function it was originally intended to, namely to implement those templates and no more. Transitioning the portal templates to Module:Transclude is possible, but goes beyond my current capacity. I may eventually try it though.

What are the benefits of this transition to Module:Transclude? First, it would mean several new options, such as the possibility of including or excluding tables by id, templates by name, and others (see the module documentation for the full potential). But the main reason for this effort has been to make this whole idea truly cross-wiki. The new module was designed following mw:Multilingual Templates and Modules, so that users from all wikis will soon be able to collaborate on a shared module. Furthermore, the design of the new module is quite simple and "modular", which should further facilitate contributions by new developers. The transition, then, will also bring us many future benefits, fruit of the work and ingenuity of the wider community.

So anyway, thanks for reading and I look forward to your reply! Kind regards, Sophivorus (talk) 20:43, 24 June 2020 (UTC)

Hi again! The new Module:Transcluder is already live at the Spanish Wikipedia and no one noticed (that's good). Now I'm working towards the same goal here at the English Wikipedia, as explained above. I'm testing the new module by previewing articles with the new module and fixing bugs and issues as I notice them. In a few days, when I'm satisfied, I'll announce it here and give any interested user a chance to run some extra tests before deployment. Cheers! Sophivorus (talk) 16:03, 18 July 2020 (UTC)

Leading and trailing spaces

If you space apart the parameters, as many people do, e.g.: {{Excerpt| Wikipedia talk:Manual of Style/Capital letters | Concluded | subsections=no}}, then the thread name is misparsed as literally " Concluded", with a leading space. But we don't do section headings with leading spaces (if you encounter one, fix it), and this is not expected behavior of WP templates.  — SMcCandlish ☏ ¢ 😼  03:13, 17 July 2020 (UTC)
Update: It's actually worse than that, and renders as " Concluded ", with a trailing space too, which is just wrong: MW itself ignores such whitespace at the end of a section name.  — SMcCandlish ☏ ¢ 😼  03:18, 17 July 2020 (UTC)

@SMcCandlish: Thanks for the report. I just fixed it at the sandbox, as can be seen at Template:Excerpt/testcases#Sections. Soon it'll be live, hopefully. Cheers! Sophivorus (talk) 15:52, 18 July 2020 (UTC)

Huzzah!  — SMcCandlish ☏ ¢ 😼  01:10, 22 July 2020 (UTC)

Collapse templates not transcluded

I'm finding that in at least one case, this template is dropping a template from what it transcludes. At WT:NCCAPS, the top section is sectionally transcluded from the top section of WT:MOSCAPS, but it is losing the {{Collapse bottom}} at the end of it. So, the {{Excerpt}} ends up collapsing the entire WT:NCCAPS talk page after it. I had to manually put an extra, "orphan" {{Collapse bottom}} after the {{Excerpt}}, along with an HTML-comment explanation, for the interim at WT:NCCAPS. Same problem is also going to come up at WT:MOS, probably.  — SMcCandlish ☏ ¢ 😼  02:37, 17 July 2020 (UTC)
PS: This is with {{Excerpt|PageName|SectionName|subsections=yes}} format.  — SMcCandlish ☏ ¢ 😼  03:18, 17 July 2020 (UTC)

@SMcCandlish: Hi! Today I did some debugging on this issue (among others) and it looks like the new version doesn't suffer from it. Deployment should take place in a few days, hang on! Sophivorus (talk) 15:52, 19 July 2020 (UTC)

Great. Please let me know, so I can to remove the workaround {{Collapse bottom}} from that page.  — SMcCandlish ☏ ¢ 😼  01:11, 22 July 2020 (UTC)

Duplicate section names

When using {{Excerpt|PageName|SectionName|subsections=x}} format, what will this do when the page has two+ instances of the same SectionName? It would be desirable if it always used the first, because that is what MW itself does, e.g. when resolving a link to PageName#SectionName.  — SMcCandlish ☏ ¢ 😼  03:24, 17 July 2020 (UTC)

@SMcCandlish: Module:Transcluder, which aims to supersede this module, transcludes only the first section whenever there're two or more sections with the same title, as can be seen at mw:Module:Transcluder/testcases. I agree this is the desired behavior. As to the current Excerpt module, I'm not 100% sure but I think it also transcludes only the first section. If you know of any article with a repeated section name, let me know and I'd be happy to set up a test case. Sophivorus (talk) 13:34, 18 July 2020 (UTC)

It's easier to test with busy talk pages, which often have multiple sections named "Comments" and "Discussion".  — SMcCandlish ☏ ¢ 😼  01:12, 22 July 2020 (UTC)

XHTML style

Question: Does <section begin=Name of the fragment/> also work as <section begin="Name of the fragment" /> ?

The habit of doing that dies hard, and arguably should not die, for WP:REUSE reasons (parsers geared for that more detailed syntax may break without the quotes and space). This is why we tend to change <ref name=foo/> or <ref name="foo"/>, and <br> or <br/>, to (respectively) <ref name="foo" /> and <br />, even though MW's own parser does not require this.  — SMcCandlish ☏ ¢ 😼  02:48, 17 July 2020 (UTC)

@SMcCandlish: Yes, this module as well as Module:Transcluder can handle section tags with or without spaces, quotes or double quotes. Sophivorus (talk) 13:36, 18 July 2020 (UTC)

Schweet.  — SMcCandlish ☏ ¢ 😼  01:13, 22 July 2020 (UTC)

[ and ] in section titles

If the section to be sectionally transcluded by {{Excerpt}} has [ or ] characters in its heading title, there appears to be no way to use it. It barfs on the bare characters, and it also will not accept [ ] HTML character-entity escapes for them.  — SMcCandlish ☏ ¢ 😼  02:41, 17 July 2020 (UTC)
PS: This also was discovered while using {{Excerpt|PageName|SectionName|subsections=yes}} format.

@SMcCandlish: I just did a test case for this at mw:Module:Transcluder/testcases and it seems like the new version will not have this issue. If you know of any article here at the English Wikipedia that has square brackets at the title, please link me to it so I can add a test case to Template:Excerpt/testcases. Cheers! Sophivorus (talk) 14:26, 19 July 2020 (UTC)

I was seeing the problem when the section name had the characters. Not sure of page titles; might be testable in that parameter with a temporary userspace sandbox page.  — SMcCandlish ☏ ¢ 😼  01:10, 22 July 2020 (UTC)

@Sophivorus: PS – I'm not sure it's actually possible to put one of those characters in a page title, without URL-escaping it or something.  — SMcCandlish ☏ ¢ 😼  03:45, 25 July 2020 (UTC)

[ and ] can't appear in page titles, per WP:NCHASHTAG and mw:Manual:Page title. Of course, they can appear in section titles, as demonstrated in this section's header. Certes (talk) 09:22, 25 July 2020 (UTC)

New version

Hi everyone! I've been working hard on excerpts and I'd like to brief you on the situation. Thanks to Certes, I was made aware of mw:Multilingual Templates and Modules, a set of standards, conventions, techniques and tools to make modules and templates truly multilingual and cross-wiki. So after gathering some courage, I decided to rewrite excerpts to comply with said cross-wiki standards. This led me to create Module:Transcluder, a generic transclusion engine, as well as Module:Extract, a module that implements Template:Excerpt but using Module:Transcluder and complying with the cross-wiki standards.

So in a way, my goal has been to reimplement Template:Excerpt with something that on the surface does exactly the same, but on the background is fully multilingual and able to be deployed exactly the same on every wiki. The new version will also introduce several new features that you can read about in the documentation of the new modules (it would be too long to share everything here). The one key difference I should maybe mention is that the new implementation won't remove all "block level templates" by default like the current one does. Instead, undesired templates will have to be blacklisted at Module:Extract/config or through the upcoming 'templates' parameter. This will cause some output changes to some excerpts right after deployment, but I trust any inconvenience this may cause can be quickly fixed by blacklisting the offending template.

While I work on the final details, I'd like to share my preferred deployment strategy, to hopefully gather some feedback and consensus. Having Module:Excerpt, Module:Excerpt/templates and Module:Extract can be really confusing. Therefore, I'd like to merge Module:Excerpt and Module:Excerpt/templates into Module:Excerpt/portals, edit Template:Transclude lead excerpt &co to use Module:Excerpt/portals, and then move Module:Extract to Module:Excerpt. This would simplify the situation enormously.

What do you think? Thanks! Sophivorus (talk) 13:41, 24 August 2020 (UTC)

That sounds good as long as portals continue to behave as before, with the currently removed templates getting blacklisted and ideally with the special handling for awkward cases such as {{Nihongo foot}}. Thanks to Sophivorus and other contributors for all the good work. My version of Module:Excerpt was aimed solely at portals, but I agree that there are much wider applications that I didn't even attempt to address. I'm happy to treat my work as a proof of concept which I hope has provided some useful ideas for the more generic modules which can replace it. Certes (talk) 14:02, 24 August 2020 (UTC)

Indeed your work has been my guide and basis! Yes, my intention is that portals continue to behave exactly as before, cheers! Sophivorus (talk) 15:24, 24 August 2020 (UTC)

Hi again! I merged Module:Excerpt, Module:Excerpt/templates and Module:Excerpt/i18n into Module:Excerpt/portals and edited Template:Transclude lead excerpt &co to use it, all seems to have gone smoothly. Tomorrow I'll update the main Module:Excerpt to use the new version currently at Module:Extract. Kind regards, Sophivorus (talk) 18:49, 27 August 2020 (UTC)

Well, after much testing, I just started the final deployment, temporarily going through the Module:Excerpt/staging submodule. Kind regards, Sophivorus (talk) 12:41, 28 August 2020 (UTC)

Issues

Thanks for the update, Sophivorus. Unfortunately, I'm seeing some bugs still present. Looking at College_and_university_rankings#U.S. News & World Report college and university rankings, there's (a) a line-break issue that wasn't there before, (b) a non-free image being transcluded that wasn't before, and (c) possibly a displaytitle issue occurring (based on this edit that came up in my feed; Bdijkstra can hopefully clarify for us). If those things can't be resolved quickly, the new version should probably be rolled back until it's stable enough for deployment. {{u|Sdkb}} ^talk 19:06, 28 August 2020 (UTC)

@Sdkb: Thanks for the report, I'm on it. Sophivorus (talk) 19:28, 28 August 2020 (UTC)

The displaytitle issue was indeed fixed by that edit. I've now also added some line breaks and the image to the <noinclude> section, I hope that resolves the "bugs". --bdijkstra (talk) 19:30, 28 August 2020 (UTC)

@Bdijkstra and Sdkb: I improved Module:Transcluder so that it now filters out non-free images. I also added Template:DISPLAYTITLE and its aliases to the template blacklist at Module:Excerpt/config so that should also fix the display title issue. Let me know if you notice any other issues, I'm not able to do much testing myself right now. Sophivorus (talk) 19:37, 28 August 2020 (UTC)

Ah, the non-free image is still there. Give me a minute plz. Sophivorus (talk) 19:41, 28 August 2020 (UTC)

@Bdijkstra and Sdkb: Now yes, I think I got it. Thanks for your help and patience, and let me know if I still missed something! Sophivorus (talk) 19:53, 28 August 2020 (UTC)

@Sophivorus: There's another bug in this template: the "edit" link doesn't point to the section of the article that was transcluded. Jarble (talk) 20:04, 8 September 2020 (UTC)

Delete obsolete modules

Is there some reason that Module:Excerpt/templates and Module:Excerpt/i18n shouldn't be deleted, given that they've become unused as a consequence of this update. * Pppery * _{it has begun...} 23:47, 5 September 2020 (UTC)

@Pppery: Yes, given that the update went well, they should be deleted, thanks! Sophivorus (talk) 00:42, 6 September 2020 (UTC)

I've tagged Module:Excerpt/templates for deletion per G7. However, Module:Excerpt/i18n was also edited significantly by Ahecht and Certes, so isn't eligible for G7 deletion unless they agree. * Pppery * _{it has begun...} 02:13, 6 September 2020 (UTC)

My only edit is a one-line bug fix and I'm happy for its attribution to disappear. If the modules are no longer used and all of their useful code remains in other modules then let's delete them. Certes (talk) 10:33, 6 September 2020 (UTC)

If it's no longer being used, I have no issues with deletion. It took me a while to figure out that its function has basically been replaced by a combination of commons:Data:I18n/Module:Transcluder.tab and Module:Excerpt/config -- we should link to the Commons data somewhere in the documentation. --Ahecht (TALK
PAGE) 23:26, 6 September 2020 (UTC)

list parameter

List parameter in mobile view is expanding by default. Portal:Australia.--Moxy 🍁 07:03, 13 September 2020 (UTC)

@Moxy: Hidden sections don't hide on mobile. Development for other purposes has resulted in the Lua to support portals effectively being copy-paste moved from Module:Excerpt to Module:Excerpt/portals. Can you remember how Module:Excerpt handled hidden lists on mobile? (I stopped following its progress when it became protected in April 2020.) #New version above explains the process, but I can't simulate the old version because its submodules have been deleted. That also makes it hard to see what's changed beyond reintegration of the deleted modules. Sophivorus may be able to help. Certes (talk) 10:41, 13 September 2020 (UTC)

One option is to enclose the list in some wrapper which prevents it from appearing at all on mobiles. I think <div class="navbar"> would do the job. That might be controversial, as the content is debatably not a navbar. Certes (talk) 12:24, 14 September 2020 (UTC)

Bug: Templates calling Table of Contents (TOC)

Hey, I was excerpting Ocean fertilization to Enhanced_weathering and it pulled over the template Template:TOC limit, which created a new TOC. Probably worth excluding anything that calls __TOC__ in any way. Sadads (talk) 12:02, 3 October 2020 (UTC)

@Sadads: Fixed, thanks for the report! Sophivorus (talk) 21:12, 4 October 2020 (UTC)

Denglisch bug

When I excerpted the content of International Space Station#Fleet operations to International Space Station programme#Fleet operations using the template, I found that a hatnote generated from the template is written in mixed German and English (Denglisch). Can someone fix this bug? --Soumya-8974 ^talk _contribs ^subpages 06:32, 27 September 2020 (UTC)

I found the reason of this. If you add "only=paragraphs" in the template, the hatnote renders "Diese Absätze sind an excerpt from foo" instead of the intended "This paragraph is an excerpt from foo". --Soumya-8974 ^talk _contribs ^subpages 09:48, 27 September 2020 (UTC)

@Soumya-8974: Fixed, thanks for the report! Sophivorus (talk) 15:13, 27 September 2020 (UTC)

Also, "Diese paragraphen sind" should be replaced with "Diese Absätze sind" to increase Germanism. --Soumya-8974 ^talk _contribs ^subpages 10:27, 4 October 2020 (UTC)

Done, thanks again! Sophivorus (talk) 21:16, 4 October 2020 (UTC)

Propersal for re-styling the hatnote

I made a series of modification to improve the display the hatnote, to make it closer to {{Transcluded section}}. The new implementation was made in these steps:

• Replace the squared brackets by rounded brackets, and add a comma and a space at the end of note and before the edit link.
• Add additional class for styling.
• Resize the edit link.

They are very simple changes, so feel free to apply my modification. -- Great Brightstar (talk) 05:20, 29 September 2020 (UTC)

 Not done for now: please allow a couple of days for other editors to comment on this proposal — Martin (MSGJ · talk) 10:39, 29 September 2020 (UTC)

@Great Brightstar: Thanks for the contribution! It isn't that simple because this module is now used on many wikis and it's desirable to keep cross-wiki compatibility. Please give me a few more days to evaluate your proposal and see if I can come up with a cross-wiki solution. Cheers! Sophivorus (talk) 15:15, 7 October 2020 (UTC)

7 October 2020

When I used the template and used "only=paragraphs" to excerpt no image. However, when I used "paragraphs=1" alongside this, the image becomes excerpted. It is problematic if the image is non-free. Plz fix it. --Soumya-8974 ^talk _contribs ^subpages 09:37, 7 October 2020 (UTC)

@Soumya-8974: Thanks for the report! Can you link me to the problematic paragraph and file? Sophivorus (talk) 15:13, 7 October 2020 (UTC)

History of spaceflight#NASA --Soumya-8974 ^talk _contribs ^subpages 15:38, 7 October 2020 (UTC)

@Soumya-8974: Fixed! Sophivorus (talk) 14:36, 8 October 2020 (UTC)

Transcluding non-free media

I just removed three WP:NFCC violations added by using this module. (Special:Diff/977651999, Special:Diff/977652158) This module should not be transcluding non-free media. — JJMC89 (T·C) 04:23, 10 September 2020 (UTC)

@JJMC89: Indeed it should not. Thanks for the report and the fix, I'll look into it asap! Sophivorus (talk) 22:30, 10 September 2020 (UTC)

@JJMC89: I think I fixed this in the sandbox version. I'll deploy it over the next few days, cheers! Sophivorus (talk) 15:14, 7 October 2020 (UTC)

Deployed! Sophivorus (talk) 14:37, 8 October 2020 (UTC)

"Inline" parameter not functioning correctly

At Pomona College#Other traditions, the inline parameter is not functioning properly within a list, and is causing an unwanted line break. {{u|Sdkb}} ^talk 08:13, 26 October 2020 (UTC)

Jonesey95 fixed the issue by removing the bullet points, but the broader issue remains. See it at Special:Permalink/985495147#Other traditions. {{u|Sdkb}} ^talk 21:06, 26 October 2020 (UTC)

Provide more information to readers and editors?

Hi User:Sophivorus, I've discovered the excerpt tool now and am quite excited about it. I have two questions: (1) Would it be possible to somehow indicate where the excerpt starts and ends (i.e. not just in source editor but also visible for a normal reader)? I had that problem here, and had to insert a (non needed) section header to indicate where the excerpt ended: https://en.wikipedia.org/wiki/WASH#Diarrhea,_malnutrition_and_stunting . (2) Is it possible that for the page that is being excerpted that there is a note somehow somewhere (maybe on the talk page) to say: "this section (xxx) of this article has been excerpted to the following other articles: xxx". I think this could be useful so that if I am going to edit the lead of diarrhea then I would be made aware that the same text will also appear here. Thanks again for the work you are doing on this tool! EMsmile (talk) 01:49, 29 October 2020 (UTC)

Oh and a third question: what happens if an editor changes a section heading title and that breaks the excerpt on another page? i.e. when the excerpt doesn't take from the lead but from a section in the article. I think it would be useful if there was a warning in the article which is being excerpted so that an editor knows they have to fix also the link on the other side. E.g. if this is used: {{Excerpt|Sustainable development goals|Targets and indicators}} and then someone changes the section heading to "Targets", then the other command wouldn't work anymore. The editor wouldn't even know that they have just "broken" something. EMsmile (talk) 03:50, 29 October 2020 (UTC)

@EMsmile: Hi, I'm glad you're finding excerpts useful! (1) To mark the start and end of an excerpt, you can use either the "indicator" or "quote" parameters of the Template:Excerpt. (2) We could create a template, maybe a sidebox, to be used in the talk page of pages that are excerpted elsewhere. However I don't think it's possible to have the template automatically list all pages that transclude the page. Rather, the pages would need to be manually added as parameters to the template call (like the Template:Lua). Furthermore, the template would need to be manually added to the talk pages where you want to display it (have it magically appear on all pages being excerpted is not really possible). All this makes it rather difficult to implement and maintain, but if you want to do it anyway, go for it! Alternatively, I noticed that some users add an HTML comment to the sections that are excerpted elsewhere, to let other users know that their changes there will affect other pages. I think this may be a more effective technique, because it warns the user right on time. However, I generally dislike HTML comments in wikitext. (3) Broken excerpts get automatically added to the Category:Articles with broken excerpts. You can add the category to your watchlist to monitor and fix broken excerpts. I think this solution is proving to be quite effective! Sophivorus (talk) 16:21, 29 October 2020 (UTC)

Hi, @Sophivorus: thanks for your reply (also @Sadads:, FYI). I like the indicator option (had overlooked that earlier). I've never seen it before in action and wonder if other people would find it a distraction? I think it's useful, especially when the excerpt is within a longer section (and not its own section). I would probably argue that it should be the default option (indicator=yes as a default). But this has probably already been discussed elsewhere. The idea with the HTML comment seems to be a simple solution to let people know which parts of the article are used elsewhere. I have added it now to the article on diarrhea which is transcluded to the article on WASH. I have added it like this: <!-- FYI: The first and last paragraph of the lead are transcluded in the article on [[WASH]], see the section on health -->, is that the correct way of doing it?EMsmile (talk) 05:53, 2 November 2020 (UTC)

@Sophivorus: The other option could be a bot-added template somewhere on talk page or in sections (per EMsmiles recommendatioN), that updates using the various bits of reuse whenever something is on a page for a while. Sadads (talk) 13:08, 2 November 2020 (UTC)

After giving this some more thought, I seriously doubt most editors will read a template in the talk page before editing the transcluded section. I know I wouldn't, except maybe on very high-profile articles like COVID-19 pandemic. Thus I think the inline HTML comment is the most effective solution. That being said, I wouldn't recommend such comments by default or as a good practice, but rather as a one-off solution when the editor feels the need for them. Most edits to transcluded sections will improve the transcluding article too. Sophivorus (talk) 13:45, 4 November 2020 (UTC)

hatnotes

Sports in New York (state) has a hatnote about an incomming redirect. The lead of this is excerpted into Sports in the United States by state including the hatnote which doesn't apply or belong in this article. MB 03:08, 23 October 2020 (UTC)

@MB: I'm not 100% sure what you mean, but I just replaced all calls to #lsth in Sports in the United States by state for calls to Template:Excerpt. Did this fix what you meant? Sophivorus (talk) 14:00, 4 November 2020 (UTC)

Sophivorus, This was fixed by this edit. MB 15:43, 4 November 2020 (UTC)

Documentation length

I just added a section in the template documentation on inclusion control. I'm noting that the documentation is already so long, however, that it's pretty likely to get buried and not be read.

It might be a good idea to go through the documentation and streamline it to improve the organization and get it down to a more readable length. Some initial thoughts:

1. The TemplateData and parameters list should be merged.
2. The long examples make the documentation seem longer than it actually is and thus might discourage people from reading it. Do we really need to actually show e.g. the full lead for science to communicate that {{Excerpt|Science}} will transclude it?
3. "Replacing Template:Main" and "Compared to #section" don't give a good sense of what that section is actually about. We also might want to separate the "here's the code to use to get this to work" information from the "here are good-practice editing flows to make best use of this" information.
4. As much as the excerpt trees are cool, I'm not sure they help editors understand how to use the template, which is what the documentation is supposed to be for. We might want to move them to another page and just link that page briefly from the documentation.

Cheers, {{u|Sdkb}} ^talk 23:29, 22 October 2020 (UTC)

@Sdkb: Hi! I agree that the documentation is getting rather long and would be good to streamline it. However, I'm not sure about some of your proposals. (1) TemplateData is quite limited as to the amount of information it can convey. Getting rid of the parameters list would imply sacrificing a lot of useful information. (2) Here I totally agree. Maybe we could find shorter leads and examples? Or get rid of the examples section altogether? (3) Are you suggesting a simple change in the titles? (4) There used to be a Wikipedia:Excerpts page, but I merged it into this one to centralize information where it's most likely to be read by interested editors. Excerpt trees are currently a small thing here, but they are much more developed in the Spanish Wikipedia and may one day be so here too. Until then, I think giving them their own page would be giving them too much prominence, and at the same time hide them from view and thus reduce their chances of getting implemented. Overall, I think the most efficient way of reducing the documentation length would be getting rid of the examples section altogether. Maybe we could replace the links to the examples in the parameters list, with links to actual articles using the parameters? Kind regards, Sophivorus (talk) 13:35, 4 November 2020 (UTC)

1) Yeah, I wish TemplateParameters was a little better-developed. The only thing it seems to lack that the parameters list has is links to examples, though. And I saw somewhere a template that managed to find a way to include links in its TemplateParameters.

2) I think getting rid of it would be fine, or just describing what happens.

3) Title changes would help, although some deeper reorganization might be warranted.

4) Getting rid of duplicated help pages is definitely good, but I don't think it's possible to fully communicate everything we might like to about excerpts in documentation here, which is supposed to only be about the simple how-to of using the template. I just don't see how excerpt trees, even if they were more developed, aid someone considering adding their first excerpt to a page. Maybe Help:Transclusions might be a better place for it? {{u|Sdkb}} ^talk 19:27, 4 November 2020 (UTC)