Help talk:Template limits

Template limit problem

I'm currently involved in helping to re-write the article general relativity, more specifically: in making a work-in-progress/sandbox version at Talk:General relativity/WIP for later inclusion in the article. Since we eventually want to bring the article to FA status, we're putting a lot of work into finding proper references, and since it's a complex topic, there is quite a number of references to be included for proper documentation (even if the article itself is, naturally, summary style, with lots of spin-off articles). I now appear to have hit the template limit, with a hundred-something references included, and the list of references not nearly complete (the sandbox so far contains only one of the article's sections, after all). From previous postings here, I gather that one attitude towards this is "if it has so many templates, it needs to be fixed anyway". In this case, I'm not sure how this is supposed to work. I'd certainly be loathe to throw out references and thus compromise the article's content to satisfy some technical limit that, as far as I can see, is rather arbitrary. Did I make a technical goof somewhere? Am I using more templates than I could be? If not, is there any way the template limit can be avoided, or changed? Any help on this would be greatly appreciated. --Markus Poessel 08:08, 17 August 2007 (UTC)[reply]

For the citation template the pre-expand include size is about 18500 bytes, so without any other templates you can call it a little over 100 times. You could do substitution (not only of the outer template but also of the core template and the ifs).--Patrick 09:32, 17 August 2007 (UTC)[reply]

Many thanks for your quick reply. A little over a 100 times sounds about right - though I've no idea why whoever set the limit thought that would be enough. While I've been using these templates, I know very little about how they work. I take it that by "substitution" you mean that I use Special:ExpandTemplates to transform the Citation thingies into something more fundamental? I have no idea what you mean by expanding not only the outer template, but also the core template and the ifs (unless ExpandTemplates does that automatically?); any pointers on where to read up on this would be appreciated. --Markus Poessel 14:49, 17 August 2007 (UTC)[reply]

Yes, ExpandTemplates does that automatically, the other way is using "subst:" (see m:Help:Substitution), but that seems more cumbersome here.--Patrick 00:08, 18 August 2007 (UTC)[reply]

Thanks for the helpful information! --Markus Poessel 15:59, 18 August 2007 (UTC)[reply]

I recommended to a friend to make a list of his used literature to have it ready for good citations in a lot of his articles.

I asked him to use the German citation templates to have the state of the art.

We were stroke by the limit and at first we wondered why.

The german template's comments were outsourced to save some bytes but we are still in trouble.

de:Benutzer:JEW/Literatur

Is there a chance that some bytes more will be allowed? I find this neccessary for a appropiate form of working in Wikipedia. Thank you very much! -- Simplicius 01:47, 8 November 2007 (UTC)[reply]

New preprocessor

Thanks for the updates to this page post the new preprocessor implementation. Can someone please explain how the preprocessor node count works? Geometry guy 11:59, 26 January 2008 (UTC)[reply]

Since I'm still waiting on the toolserver db to come back up, I'll look through the source and fix something up. Patrick did a nice job of trimming out the stuff that is no longer relevant, and correcting my date error. I think that the next step is to think about whether there is a better structure for the page given that the new limits are easier to understand. — Carl (CBM · talk) 14:55, 26 January 2008 (UTC)[reply]

Nice rewrite! One query: is it really true that all template arguments are fully expanded and their length added to the template argument size? This would imply that ignored #if's and #switches could still add a lot to the template argument size, whereas they don't seem to in my (admittedly anecdotal) observations. Geometry guy 22:11, 26 January 2008 (UTC)[reply]

Actually, as I have been playing around with things, I see that I don't understand everything as well as I thought I did. I'll let you know when I figure out what's going on. — Carl (CBM · talk) 22:39, 26 January 2008 (UTC)[reply]

Why does is UTF produce an error but not a template that produces the same UTF?

Resolved

OK it is against expectation. I am writing a template to produce possible Arabic verb forms from the verb root. I have run into the parser limit (the HTML source says so). Because Arabic is bidirectional and messes-up things when written within left-justified text, I created a subtemplate for the letter ta instead of using the unicode value for readability reasons. This worked partially, but but didn't parse for the more complex verb forms. So I tried to reduce template transclusion in my code by using UTF values instead. The strange thing is that even the verb forms that worked before don't work now. The current version is the older version with the {{/ta}} template. How can that be?.(I tagged the problematic code with ERROR HAPPENS FORM HERE)-- hɑkeem_¡ʇu^{ɐ^ɹ}ɯǝǝʞɐɥ 10:08, 21 May 2008 (UTC)[reply]

I was #ifeq'ing utf values with characters input from the keyboard. I learned that wiki doesn't process utf's but passes them to the browser. Thanks. -- hɑkeem_¡ʇu^{ɐ^ɹ}ɯǝǝʞɐɥ 05:25, 22 May 2008 (UTC)[reply]

Proposed use of a date metatemplate into citation templates

I'm shortly hoping to introduce optional date formating into the cite_XXX family of templates, with cite_web acting as the test case - see Template talk:Cite web#Working version and final discussion . In essence the various date parameters (date, archivedate & accessdate) may be optionally formated into non-linked dates (as per changes at WP:Dates) by use of a metatemplate (see proposed new coding at {{cite web/sandbox}} and its use of {{date style}}). Whilst I've been advised not to worry about use of metatemplates, I'm not so sure that is true, as cite_XXX templates are not used just once or twice in articles but sometimes a very large number of times and across a large part of all wikipedia articles.

So will formating citation template dates, using metatemplate {{date style}}, cause a problem (if so would very ugly repeated direct date-format coding within {{cite web}} help)? Further, would using a meta-metatemplate to simplify the coding in {{date style}} help or hinder ? David Ruben ^Talk 14:28, 17 July 2008 (UTC)[reply]

Why don't you just edit the date regexes in DateFormatter.php so that it matches unlinked dates? Then you wouldn't need templates. -- Tim Starling (talk) 13:46, 19 July 2008 (UTC)[reply]

Sorry, I don't understand PHP, so that totally lost me :-( Both that I'm unfamiliar with PHP coding, but also that is a onetime edit for the whole of wikipedia if I understand correctly, but there needs be flexibility for American or British/European/International date formats which would need be selected on an article-by-article basis. (or am I missing something here and wikilinked-dates do allow a formating-style parameter to be included ?)

The three wish-list items are 1) show user-preference style if this has been set (minority of editors) 2) else select a date style appropriate for the article in question 3) in eithercase show as an unlinked item. The later is need to follow WP:Dates now stating dates should not appear wikilinked.

So whilst wikilinking dates switches appearance of input for those few editors who have selected a dmy/mdy/ISO preference, all the output (for those with & without a preference set) is still blue, underlined and linked.

Now if MediaWiki allowed a magic-word like feature of the same input flexibility (unlike #time parser function's limited date range) but without linking then I agree no problem. Not sure how one might so flag that, perhaps use {{foo}} or perhaps if need to help the system recognise a date as {{@foo}} ?

Alternative or addition options for MediaWiki assistance with this might be:

Allow reading of a user's preference option (if no option set then editors could decide which style best suited the article in question)
Allow article variable on this (we have some magic words that MediaWiki recognises for its own use) so we might have __DateDMY__ or __DateMDY__ or equivalent {{DATEdmy}} or {{DATEmdy}} and then mediawiki follows that preference if no user preference already been set.

For now, I'm not sure whether/how to proceed. David Ruben ^Talk 14:38, 19 July 2008 (UTC)[reply]

what are the limits that cause "Category:Pages where template include size is exceeded?"

i understand the need for this, but was wondering how is it triggered and/or configured? Bud0011 (talk) 05:47, 21 June 2009 (UTC)[reply]
Me, too. How and where I can change the settings in any MediaWiki-Installation (not wikipedia)? --EnterpriseWiki (talk) 11:39, 29 July 2010 (UTC)[reply]
- bumping this.Bud0011 (talk) 05:29, 27 November 2011 (UTC)[reply]
- dito — Preceding unsigned comment added by 114.198.35.204 (talk) 07:10, 21 March 2012 (UTC)[reply]

You might like to look at Wikipedia:Avoiding MediaWiki expansion depth limit. The question is very technical though, and you'll probably get a better response at either meta:Meta:Babel or mw:Project:Support desk. --Redrose64 (talk) 14:44, 21 March 2012 (UTC)[reply]

Yeah, well...

If there are going to be such template limits, then there should be limits on templates being unnecessarily expanded/rewritten with no regard to bytage; one consequence of a thoughtless substitution of a "new, improved" template ({{cite bcgnis}}) for a completely easy-to-use and less-bytage one ({{BCGNIS}}, although what's at that link now isn't waht it originally was thanks to its predecessor being "deprecated") is that existing pages which use lots of it are now damaged and require revision see Wikipedia_talk:Canadian_Wikipedians'_notice_board#Why_are_templates_not_working_on_this_page.3F and template talk:cite bcgnis. The advocate/creator of "cite bcgnis" claimed that coders are told that they shoudl write templates as if there were all teh server space and processor space in the world; but so long as there are limits on the size of pages, and these new template limits, then that's a conundrum. Why can't CONTENT also be written without regard to server/processor space/power? Or is priority being given only to making the INTERFACE as big and bloated as it wants to be, and page-content is to be sacrificed to make room for more code????Skookum1 (talk) 20:15, 14 September 2010 (UTC)[reply]

Content has no limits, though page splits are suggested for really long articles. I'm not sure where you got your info about this. Content is generally straight text, or wikilinks and other wiki markup that is usually easy to parse and light on processing load. Templates, on the other hand, may make use of functions which could have recursion or loops, which are processor intensive. Mind matrix 15:28, 15 September 2010 (UTC)[reply]

Added: "Highest expansion depth"

I've added the counter Highest expansion depth as a section, but cannot describe it. Anyone? -DePiep (talk) 10:35, 7 August 2012 (UTC)[reply]

Increase post-expand limits

If you encountered

<!-- WARNING: template omitted, post-expand include size too large -->

you can increase $wgMaxArticleSize variable value, this will increase post-expand limits too.

Is there a potential security risk with the NewPP limit report?

At the end of the report it says something like: "Saved in parser cache with key xxxx:pcache:idhas..." etc, where "xxxx" is the database name. Does this represent a potential security risk?

Limit too small for all template documentation cases

See Draft:Template:Hybrid_motor_type/doc. 217.162.112.133 (talk) 22:57, 23 September 2019 (UTC)[reply]

So cut it down. Does it really need to show demonstrations of every possible permutation of parameters? --Redrose64 🌹 (talk) 17:54, 24 September 2019 (UTC)[reply]

I will not cut it down. They are the test cases for the template. I can check that the code is working properly in the preview before commit. 217.162.112.133 (talk) 18:08, 24 September 2019 (UTC)[reply]

The template must be too complex. Given the simplicity of what it does, I don't see why it should exceed the limit. Also: testcases are usually placed under /testcases, not /doc; lint errors must be eliminated if you plan to ever roll it out; I strongly recommend you create an account. Nardog (talk) 18:30, 24 September 2019 (UTC)[reply]

Update to WP:Template editor and WP:Template limits

I added this to WP:Template editor#Other considerations and this across several sections of WP:Template limits to encourage editors to pay attention to template limits before changing templates like {{flagicon}} that are used many times on one page, as they can push a page over the limit.

Module:Citation/CS1 and the templates and modules that call it are another example. On pages with hundreds of citations, a seemingly minor change to the citation templates can push a page that has hundreds of citations like Donald Trump (700+ citations) over one or more of the limits. davidwr/(talk)/(contribs) 17:31, 5 March 2020 (UTC)[reply]

Discussion on how to deal with PEIS limits at 2019–20 coronavirus pandemic

You are invited to join the discussion at Talk:2019–20_coronavirus_pandemic#Dealing_with_technical_limitations_of_WP:PEIS. {{u|Sdkb}} ^talk 00:16, 13 April 2020 (UTC)[reply]

Analyzing a page's PEIS

If you need to reduce a page's post-expand include size, it would help to understand where that PEIS comes from. So is there a tool that, given a particular page, tells you how much different parts of the page contribute to the PEIS? One possibility would be to break it down by section (so, like the "Section sizes" template, but for PEIS instead of source size), but another would be to break it down by template name. (E.g., all of this page's calls to the "cite" template contribute X to the page's PEIS). Jmdyck (talk) 18:12, 15 June 2020 (UTC)[reply]

There’s already a report in the HTML source, which shows template expand times, not PEIS, like this:

<!--
Transclusion expansion time report (%,ms,calls,template)
100.00%   13.596      1 -total
 47.24%    6.423      1 Template:Resolved
 25.82%    3.511      1 Template:Hmbox
 24.92%    3.389      9 Template:Tl
 18.80%    2.556      1 Template:Z48
-->

I can imagine something like this for PEIS as well. (I don’t know how feasible it technically is, though.) —Tacsipacsi (talk) 00:50, 17 June 2020 (UTC)[reply]

Necessity of the "post-expand include size"-limit, identifying the core problem and possible alternative solutions

I was running into a problem when trying to transclude multiple sections in something related to the article 2020 in science due to the Post-expand include size limit.

The problem seems to be that each of the transcluded sections had many references which caused a long Transclusion expansion time (unbelievably over 7 seconds for content that is only text and images which are supposed to lazy-load in 2020)
- The Transclusion expansion time can be checked in the source code of the article. For a tested version of the article it was (the total % being over 100% seems to be an additional problem):

Transclusion expansion time report (%,ms,calls,template)

100.00% 7733.218 1 -total

61.75% 4774.901 1 Template:Reflist

23.74% 1836.005 293 Template:Cite_journal

22.62% 1749.421 502 Template:Cite_news

5.67% 438.425 4 Template:Fix

5.50% 425.630 1 Template:Overly_detailed_inline

5.47% 422.810 12 Template:Category_handler

4.52% 349.686 1 Template:CVE

3.32% 257.085 80 Template:Cite_web

1.89% 145.880 7 Template:Convert

Due to these problems I created an issue on phabricator here.
However my recent questions there have not been answered which may be due to readers of the task not knowing any answer to the questions I posed. This is why I'm asking you, dear watchers/readers of this talk page.
The problem I'm having seems to be shared with a substantial number of other editors / other articles − including many COVID-19-related ones.
The questions:
- Is or can there be any near-term solution for including more text via transclusions (i.e. up to 12 section-transclusions with many references each)?
  - One alternative or near-term solution I could imagine would be a software change that allows for collapsed sections that e.g. only load the data after clicking a [show] button and/or when expanding a section in the mobile view. What do you think of this and is there already an issue for this? It could also detect if the client device is a mobile phone (and/or has a slow Internet-connection) and if not preload the content of these collapsed sections.
- According to #Why are there limits? the rationale for this artificial limit seems to be that a) the content is considered "large quantity of data" b) the amount is slow to load and c) that it can be "used to mount a denial of service (DoS) attack on the servers".
  - However, 2 MB is not a large quantity of data in 2020 − it's about the size of images commonly shared on image-sharing websites and it should be only a fraction of that size when compressed. Is compression used as much/efficiently as it could be? There are many websites with a) lot more data that b) load a lot quicker and c) often deploy modern Web solutions like lazy-loading content or infinite-scrolling etc.
  - If the Wikimedia servers can't handle requests of 2 MB of uncompressed data (mostly text) I think there's something really wrong with the hard/software, especially when considering the amount of donated money theoretically available to upgrade either or both.
  - I don't think more content in pages would allow for more problematic DoS attacks than otherwise. And if so this should probably be mitigated, just like any other DoS attacks, with adequate technical measures (like limiting the number of requests to a page of large size from a single IP).
- Are templates and transclusions not prerendered/preparsed after a change to the template or transcluded target section? Does including templates and transclusions always require servers to process every single request? There seems to be something really wrong with the templating architecture if this is the case − no wonder there's a high load on the servers if the templates cause computing-intensive parsing at every page-load. Is there a task and/or suggested solution to this somewhere? From this it seems that transcluded templates are updated after their changed and not somehow dynamically loaded at page-load.
- Why do the reference-templates take so long to load? Shouldn't they be just text (partly hyperlinks)? There seems to be something really wrong with the reference templating design or template-mechanisms in general if the transclusion expansion time (see the example above) show the real load-times.

I'd appreciate if somebody could answer it here or at the phabricator issue.

tl;dr: just read the text highlighted in bold.

Thank you.

--Prototyperspective (talk) 21:18, 26 September 2020 (UTC)[reply]

The limits really do need to be raised, it is 2020. There should also be higher limits on certain "white listed" pages, with the understanding that purging of these pages' caches may be de-prioritized if the work queue gets long. I know you were writing about limits on time but there are whole classes of articles, mostly sports-related, that run up against other limits, including post-expansion include size and expensive function count limits. Wikipedia needs a better way to handle these. davidwr/(talk)/(contribs) 23:43, 26 September 2020 (UTC)[reply]

Workaround to the partial-transclusion issue: Instead of marking off sections in article A to be transcluded by Article B, SPLIT Article A into parts, and have both Article A and Article B include the sub-parts. It's not the normal way of doing things, but sometimes when you hit hard limits you have to WP:Ignore all rules to get the job done. Just be sure to document what you did and why you did it so it can be undone when the limits are eventually raised. davidwr/(talk)/(contribs) 23:54, 26 September 2020 (UTC)[reply]

Thanks for the helpful comments. I also think they should be raised for at least whitelisted pages. However, I propose to solve the problems that caused people to create these artificial limits (or to have these limits as low as they currently are).

Afaik the "Post-expand include size limit" is directly related to the "Transclusion expansion time" which is why the latter is so long when a page hits the former's limit. I think solving whatever causes long "Transclusion expansion time"s would also allow for higher "Post-expand include size limit"s. From the research, it looks like the reference templates or rather the way templating works in Wikipedia are what cause long "Transclusion expansion time"s.

I'm not sure if I understood your suggested workaround: what I tried was having Article A transclude sections from Article B as well as sections from Article C. So far I haven't seen any temporary workaround in articles which have similar problems, such as COVID-19 pandemic in Japan.

--Prototyperspective (talk) 09:37, 27 September 2020 (UTC)[reply]

I don't remember exactly what they are, but I think there are two very large Middle-East-related templates that are each transcluded into more than one article. If limits and performance were not an issue, one or both would be better done as a "transcluded section" in one of the articles which they are currently transcluded into. I do NOT know if they are done up as templates for technical reasons or if that is just an editorial decision made by whoever made them.

Here is something you can try as a test:

Take several science articles and copy them to your personal userspace. Be sure to remove categories and the like.

Carve off sections into stand-alone pages that are in your userspace.

Copy the "big" article that transcludes from several science articles into your userspace, removing categories and the like. Have it transclude these carved-off pages instead.

Compare the parserdata against the parserdata for the "live" version and see if there is any significant difference.

davidwr/(talk)/(contribs) 15:55, 27 September 2020 (UTC)[reply]

Move to help namespace?

Looks like a help page to me — Martin (MSGJ · talk) 20:20, 21 October 2020 (UTC)[reply]

Constant expressions

Will constrant expressions only be expanded once (until the template code is modified)? Trigenibinion (talk) 18:22, 22 January 2021 (UTC)[reply]

Whitespace

It is very bad that whitespace is being counted. Trigenibinion (talk) 14:05, 27 January 2021 (UTC)[reply]

Discussion at Wikipedia:Categories for discussion/Log/2021 May 7 § Category:Pages where template include size is exceeded

You are invited to join the discussion at Wikipedia:Categories for discussion/Log/2021 May 7 § Category:Pages where template include size is exceeded. * Pppery * _{it has begun...} 17:12, 7 May 2021 (UTC)[reply]