Module talk:Params

New function `self`

I have added a new function to the module, {{#invoke:params|self}}, however I am not sure whether the most idiomatic name for it is self or title (especially considering that self has a special meaning in Lua). --Grufo (talk) 00:36, 27 September 2023 (UTC)[reply]

@Grufo: I would prefer a different naming. Based upon magic words like {{PAGENAME}} and friends, things along the lines of PARENTNAME, TEMPLATENAME (sadly we already have a {{TEMPLATENAME}} with a slightly different meaning), INVOKERNAME or similar come to mind, along with the possibilites of things like CHILDNAME, MODULENAME, INVOKEENAME or similar; potentially with modifiers for other encodings a la {{PAGENAMEE}} and comparison of page title encodings, etc. (e.g., by returning mw.title:partialUrl() or similar), not to mention other options in line with things like {{FULLPAGENAME}}, {{FULLPAGENAMEE}}, etc. That said, perhaps those names are not really the best either and it would be better to stick to naming closer to Scribunto naming along the lines of parentPageTitle/modulePageTitle and stick to modifiers provided by mw.title object and/or mw.url API like partialUrl, queryEncode, pathEncode, wikiEncode, anchorEncode. Incidentally, I have often considered developing a Scribunto replacement for base (or even all the extensions in the default and/or WMF MediWiki packaging) magic words and then potentially extending it from there. This begs the need for these types of things more generally but I digress. —Uzume (talk) 19:51, 6 March 2024 (UTC)[reply]

@Uzume Sorry for my very late reply. Your suggestions would be {{#invoke:params|INVOKERNAME}}, or {{#invoke:params|CHILDNAME}}, or {{#invoke:params|MODULENAME}}, or {{#invoke:params|INVOKEENAME}}. These would break the convention used for function names in this module, and also would not be so immediate to understand (i.e. if template {{foobar}} invokes this module I would expect “INVOKENAME” and the other names you propose to return Module:Params instead of foobar). There is a convention used by many programming languages according to which the parameter zero is the name of the routine executing the code (e.g. in Bash $1, $2, $3, … are the parameters, whereas $0 is the name of the script or the name of the subroutine, much like our {{#invoke:params|self}}). However in these languages parameters can only be non-zero positive numbers, whereas in template syntax 0 can also be a parameter name (e.g. {{foobar|0=hello|1=world}}), so another solution needs to be found. There used to be an arguments.callee property in JavaScript with a similar meaning, but today is deprecated, and if we named our function {{#invoke:params|CALLEENAME}} I believe it would create confusion in redirected templates. The name “self” is what for me comes the closest to “parameter zero” in those languages that use the latter for the same purpose. And so “self” inside {{foobar}} is foobar. --Grufo (talk) 19:20, 22 May 2024 (UTC)[reply]

@Grufo: To me self tends to lend its meaning from Lua in much the same way as as this does in C++, namely an object reference (although in C++ these are always class-based and in Lua they do not have to be as they could be prototype-based, etc.). It is true that in MW "title" has several meanings but its most common meaning is a textual name of a page, that said I would try to avoid it based on things like Manual:Modeling pages. That said page "title" and PAGENAME tend to be the name of a page without a namespace, as opposed to frame:getTitle() and FULLPAGENAME, etc. You asked how people felt about the naming of the current self invocation target and though I am not a fan of "self", I do think it is better than anything with "title" in its name. —Uzume (talk) 10:16, 23 May 2024 (UTC)[reply]

It's true I asked, but my concern was not about how understandable the term is, but about the fact that “self” has a peculiar meaning in Lua. And although this still holds true, it is also true that this module is not meant to be called by other modules (i.e. it is not meant to be used in Lua), but it is meant to be called by templates, where “self” has no special meaning whatsoever (plus, template writers are required to know nothing at all about Lua, and maybe one day modules will be written in more languages other than Lua). And even if a template writer knows about Lua, self would still make sense (i.e. the “Lua-style self from the point of view of a template is the template itself”). If you mention words that are already in use, they would carry the reference they currently have (e.g. “title” would refer to the transcluding page, not the transcluded template), so we do need to invent new words that carry no references, since no magic word or template currently offers the same functionality. For me the only possible contender would be callee (but see what I wrote earlier about the case of redirected templates). --Grufo (talk) 10:42, 23 May 2024 (UTC)[reply]

Be verbose to avoid any confusion, it costs you nothing. How about calling_template_name? 68.199.122.141 (talk) 11:20, 23 May 2024 (UTC)[reply]

But that would be very confusing! This function is meant to be used in transcluded templates (otherwise one uses{{FULLPAGENAME}}); and if template {{foo}} calls {{bar}}, and the latter contains {{#invoke:params|self}}, we get bar, which is the called template, not the calling template (which instead would be foo – of course, technically the called template is also invoking a module, but users need to know nothing about Lua and modules for using {{#invoke:params}}). It is not a coincidence that in JavaScript the same functionality was called arguments.callee (and not arguments.caller – but again, “callee”/“called” create confusion in redirected templates). --Grufo (talk) 13:08, 23 May 2024 (UTC)[reply]

In your example, template {{bar}} is calling {{#invoke:params|self}}, so {{bar}} is the calling template (of your function), and its name is calling_template_name. It could be this_template_name as well. The word 'self' is usually used for the very object (with methods and stuff), not only its name. 68.199.122.141 (talk) 13:39, 23 May 2024 (UTC)[reply]

this_template_name would also be possible. Let's see if others prefer it to self. Contra: although the funcion is meant to be used in templates, nothing forbids to use it in non-transcluded pages, wehere there is no such thing as “this template”; or imagine the page Template:Foobar/doc using this_template_name for referring to itself instead of the actual template it is documenting (i.e. Template:Foobar – that would also create confusion with {{TEMPLATENAME}}). --Grufo (talk) 21:09, 23 May 2024 (UTC)[reply]

this_function_caller, I don't think it can be more precise; self, though, is a bit confusing when a page transcludes a template invoking a template invoking the function, or any such mumbo jumbo. 68.199.122.141 (talk) 22:50, 23 May 2024 (UTC)[reply]

I cannot say I like this_template_name. Without doing strange manipulations like frame:newChild, etc. the parent frame should always be the frame that calls #invoke. Even when frame:callParserFunction('#invoke', ...) is called this is normally true as it will either refer to the calling module or if specifically called off the parent frame, the invoker's invoker frame (i.e., a grandparent but the actual invoker frame is hidden then). This is why from the Lua standpoint something like parent_fullpagename probably makes the most sense, however, as you said this is designed to be understood from the template or parent's point of view (the users of which should not need to understand Lua, etc.) and as such there are way too many confusing page names to begin with. To that end, I still like invoker_fullpagename. full should probably be included as FULLPAGENAME includes a prefixed NAMESPACE as this function also returns. If you really like the caller nomenclature then how about module_caller_fullpagename and if you really like the this, how about this_module_caller_fullpagename. —Uzume (talk) 15:06, 11 July 2025 (UTC)[reply]

FYI: {{#invoke:TEMPLATENAME|main}} now does this same thing too but I am not a fan of its naming either. I personally believe names like template_name or module_name should be considered in situations where they are made to be used with msg: or #invoke:, i.e, the namespace is not prefixed for template or modules but it is otherwise, including for the empty main namespace (for example when you want to transclude a main space page and need syntax like {{:mainspacepage}} or {{msg::mainspacepage}} but you don't need such for Template and in a similar fashion you don't need Module for #invoke:. This is exactly what things like {{TEMPLATENAME}} do because they are designed for constructions like {{{{TEMPLATENAME}}|...}} (in this case typically from a /doc subpage but one wants it to also render properly from the main template and/or its /sandbox subpage, etc., i.e., everywhere /doc is transcluded from). —Uzume (talk) 15:06, 11 July 2025 (UTC)[reply]

Wrapper template with pass-thru params

I'm a template writer and have newly discovered this Module, and am still getting on board with it, but have not used it yet. I think it probably will handle some use cases I have, but I'm not quite sure. One case that is typical for me, is a wrapper template with "pass-thru params", that is, the case of a wrapper template A, that calls template B passing thru all or almost all of A's params to B without alteration, and one or two more params with hard-coded values. For a concrete example, consider template {{sfnlink}}. I want to create new wrapper template {{sfnlinknb}}, that will have the identical params that sfnlink has, less param |nb=; the job of the wrapper, is to pass through all of its params to {{sfnlink}}, and add |nb=yes as well. The "pass-thru" part of this seems like the kind of thing I ought to be able to do with Module:Params, but I'm not quite sure. Maybe using for_each? Thanks, Mathglot (talk) 00:09, 3 December 2023 (UTC)[reply]

It looks like you have a working implementation using double transclusion and for_each in {{sfnlinknb}} already, but if it's useful as an alternative approach, I just used concat_and_call to do the same thing in {{autnum plain}}: Special:Diff/1188117126. DefaultFree (talk) 13:26, 3 December 2023 (UTC)[reply]

DefaultFree, thank you so much for this. Your solution is infinitely better, and I've rewritten my wrapper using your idea. I had made multiple attempts—you can see the numerous failures at the history of User:Mathglot/sandbox/Templates/params_for_each and at Template:Sfnlinknb. At one point, I thought I had it because it worked perfectly at Special:ExpandTemplates, but then I found it didn't work anywhere else—it spit out the correct code (as if it were nowiki'ed), but didn't invoke it, except at ExpandTemplates, for some reason. Then I went off on a tangent, trying to add {{Eval}} in front of the whole thing to get it to execute, and as you saw, I got tangled up in double transclusion, and, oh, what a mess! Yours is so much better, and would make a great addition to the documentation of this Module, either as an example, or as a "Pro tip" note. (You might be interested to see what I did with the (on-page) /doc, the /sandbox, and the /testcases pages for {{Sfnlinknb}}.) Thanks again, and I owe you one! Mathglot (talk) 22:29, 3 December 2023 (UTC)[reply]

15th January 2024: New changes

The Module:Params/ChangeLog subpage now keeps the record of the most important changes in the module's code. Furthermore four new modifiers – mapping_values_by_calling, mapping_values_by_invoking, mapping_values_blindly_by_calling and mapping_values_blindly_by_invoking – were recently added to the module. These allow to pass each argument one by one to a custom template or a custom module in order to be preprocessed before further actions. --Grufo (talk) 04:07, 15 January 2024 (UTC)[reply]

growth and sandboxes

Hi @Grufo: - following on from your request over at WP:PERM. Got a few questions:

Do you know what has recently caused the large growth in calls to this module? Is such a large use necessary?
This module doesn't appear to have a /sandbox; now that it is so heavily used where are changes being validated prior to deployment?

Thanks, — xaosflux ^Talk 13:34, 22 April 2024 (UTC)[reply]

xaosflux, the drastic usage increase looks to be due to Special:Diff/1214727698. Primefac (talk) 13:42, 22 April 2024 (UTC)[reply]

Numbers should start falling, I've undone the above. Primefac (talk) 13:50, 22 April 2024 (UTC)[reply]

FYI to @Uzume: — xaosflux ^Talk 14:19, 22 April 2024 (UTC)[reply]

@Xaosflux: Thanks for the heads-up. I just fixed a minor documentation bug. Using something like this module insulated it from potential moves (or someone copying it for as a template, etc.) but there are obviously other ways to fix that issue and I certainly did not mean for it to have such ramifications (I would not want to prevent the author from development; the sandbox still allows such if somewhat more cumbersome). —Uzume (talk) 14:31, 22 April 2024 (UTC)[reply]

Hi Xaosflux. I have no idea. I tried to find out, but I believe that whoever is responsible for the growth did not follow the suggestion of adding {{Lua}} to the documentation of the templates that start using this module. So finding out where all the calls come from is hard (btw, someone has to improve the “What links here” special page and allow to list direct transclusions only…). I had planned some important improvements for this module, but I have been very busy in the last months. And as soon as I had a bit of time for the module I found out that I can no longer edit it. Concerning the sandbox: it's not hard to create one, is it? So far every time I posted an update I tested it locally on my machine, but I agree that it would be good to have a sandbox here too. EDIT. Primefac gave the answer while I was writing mine (@Primefac: Thank you). --Grufo (talk) 13:48, 22 April 2024 (UTC)[reply]

Always happy to help. For the record, I used an insource search to find where it was being invoked, specifically insource:/invoke:params/, as seen here. Primefac (talk) 14:19, 22 April 2024 (UTC)[reply]

I've lowered the protection level on this, you should be able to edit again. Suggest you build up a sandbox and incorporate it to the testcases. — xaosflux ^Talk 14:21, 22 April 2024 (UTC)[reply]

Thank you both @Primefac and @Xaosflux! The next changes in the code will pass through a sandbox. --Grufo (talk) 14:30, 22 April 2024 (UTC)[reply]

Ready for general use

Improvements are always possible, but I believe time has come to mark this module as ready for general use. --Grufo (talk) 18:52, 22 May 2024 (UTC)[reply]

Inverting parameters

At {{WP:RSPUSES}}, I needed not |1=key|2=value but |1=value|2=key. I achieved this with {{#invoke:params|sequential|mapping_by_calling|duses|values_and_names|setting|i|<br>|list_values}} when call_for_each didn't work. Are there plans to add a shortcut like call_for_each_value_and_key? 174.92.25.207 (talk) 21:56, 4 July 2024 (UTC)[reply]

You followed exactly the right way. There are no plans to create a call_for_each_value_and_key function, because that will mean that we will have to create also an invoke_for_each_value_and_key function, a magic_for_each_value_and_key function, a call_for_each_key function, and so on. This module is already quite rich in functions; if new functions will be added in the future it will probably be to add new functionalities, not just syntax sugar. --Grufo (talk) 00:56, 6 July 2024 (UTC)[reply]

Code link on mobile

@Grufo: Are you aware that the mobile view looks like this?

----------------------------------------
|               Function               |
| |----------------------------------| |
| |              count               | |
| |----------------------------------| |
|                  (                   |
|                code                  |
|                  )                   |
| Num. of arguments 0                  |

instead of the proper desktop view like this?

              |-----|
     Function |count| (code)
              |-----|
--------------------------------
| Num. of    0                 |
| arguments                    |

Ideally the entire header should be on the same line like on desktop. But the parentheses in the code link are currently a monstrosity on mobile. 174.92.25.207 (talk) 06:03, 8 July 2024 (UTC)[reply]

@174.92.25.207: Thank you. I think the quirk is too infobox-specific to be integrated in the {{./doc/link to the code}} template. However this should fix it. --Grufo (talk) 18:45, 8 July 2024 (UTC)[reply]

Minor limitation currently affecting some functions

There is currently a design limitation in the child frame object that affects invoke_for_each, invoke_for_each_value, mapping_by_invoking and renaming_by_invoking. The limitation affects these functions only when used on a large number of parameters (> 99). For more information see:

--Grufo (talk) 20:03, 14 April 2025 (UTC)[reply]

@Grufo: It seems like the best solution would be to avoid your current :newChild hacks and instead do actual #invoke: parser function calls via :callParserFunction. That also solves a number of problems with the current solution (besides the 99 frames limit), e.g., the current "invocation" method does not always work correctly for modules that attempt to detect such things (e.g., ones that test their inbound ... to see how the module is being loaded, either as a #invoke: "main" or as a require sub-library module, mw.loadData, etc.). It also allows you to jettison some error code as true #invoke: already throws errors when the function cannot be found, etc. Don't try and reinvent the wheel (I ran into a similar situation with 954154594). It also allows you the ability to potentially consolidate your code as your template call interfaces could be moved from :expandTemplate to :callParserFunction with msg: and both #invoke: and msg: could be implemented with your magic functions which already use :callParserFunction. The invoke function of Module:Lua call is a good example. Most uses of :newChild are typically poorly designed. One probably legitimate use would be to create an artificial parent frame for calling :callParserFunction off of (for modules that look at their parent frame, etc.). —Uzume (talk) 19:05, 9 July 2025 (UTC)[reply]

Thank you, Uzume. You make a strong point when you talk about modules that detect whether they have been called via require() or #invoke. I did not even know that was possible (how does one do that?). I have been thinking for a while about replacing :newChild() with :callParserFunction() + #invoke in the functions listed above. I have not done it so far because I was hoping to trigger a rethinking upstream on how :newChild() is currently designed. However no discussions have emerged so far as far as I know (and so maybe now it is the time to get rid of :newChild()). At the moment, when facing the bug described here, it is already possible to use {{#invoke:params|mapping_by_magic|#invoke|...}} instead of {{#invoke:params|mapping_by_invoking|...}} (which is why this did not seem to be an urgent fix). For instance, if we wanted to use {{#invoke:string|replace}} to replace spaces with hyphens in values (and forgetting for a moment that we already have {{#invoke:params|mapping_by_replacing|...}}), we could already do

...|mapping_by_invoking|string|replace|4|%s+|-||false|...

or, equivalently,

...|mapping_by_magic|#invoke|values_only_as|3|let|1|string|let|2|replace|4|%s+|-||false|...

Interestingly, while #invoke is recognized as a parser function, the same seem not to apply to msg, and so it seems not possible to use the latter in combination with :callParserFunction(). In any case, :expandTemplate() seems to work just fine. --Grufo (talk) 11:19, 10 July 2025 (UTC)[reply]

@Grufo: I am not sure why msg would not work as a parser function. I wonder if that is also true for msgnw (which is sort of similar to Scribunto mw.title:getContent() because it does not process page source through the main wikitext parser). I wonder if there is an issue with the way you are building your argument list or calling the parser function that is limiting things. —Uzume (talk) 21:29, 10 July 2025 (UTC)[reply]

Anyway, I am not sure about all the ways for Lua modules to detect how they are called but one of the primary ones would be to check ... during the execution of the module initialization block (which executes during #invoke before the function call to the its first argument and why it is possible to catch any argument as the function name using a metatable with __index like in Module:Ustring; oddly enough that also means one can process the frame args before the function name is provided to the module but of course nothing can be returned until later). require() passes in the module's name while Scribunto #invoke does not (imitating command line lua but the frame args are not put into ... or global arg so they are always empty/nil; probably because frame args have non-sequential parameter keys). Checking ... can be a good way to have the module return a different package table (often denoted as just p) based upon whether it is loaded via #invoke or not. In fact, that style of coding allows one to do away with the awkward leading underscore API functions and even allows the module to require itself during #invoke thereby using its own API while also providing frame wrapper function(s) for #invoke calls from wikitext. The name of the module used during #invoke is of course provided via frame:getTitle() which can be accessed from anywhere, including the module initialization block, via mw.getCurrentFrame(). —Uzume (talk) 21:29, 10 July 2025 (UTC)[reply]

I created an example module for demonstrating how a module can detect how it is loaded at Module:Sandbox/Grufo. Feel free to do with it as you see fit. Notice how your require() plus frame:newChild() implementation likely would not work with that construction but a frame:callParserFunction('#invoke', ...) construction would. You should consider which frame to call that off-of though. Frequently it makes more sense to call that off of the parent so that the new invocation has the same parent as where you call it from (i.e., they are effectively "sibling" invocations) as arguments to your module might not be particularly interesting to the module you are invoking (if they are, you should probably pass them to the module directly via its own arguments) but arguments to the parent template would otherwise be lost (due to frame depth; a module can only access its frame and one parent—no grandparents, etc.). I am not sure if it matters which parent frame is available for other parser functions/magic words but I encourage you to consider using frame:callParserFunction() from the parent frame. —Uzume (talk) 21:29, 10 July 2025 (UTC)[reply]

FYI: another little known fact is that Scribunto has translations of #invoke so such can actually be accessed via alias keywords on for example a Korean or Russian Mediawiki, etc.; that makes it pain to search for all such invocations of a module but luckily few people know and use aliases. —Uzume (talk) 21:29, 10 July 2025 (UTC)[reply]

Hi Uzume. Thank you for the sandbox module. I think I understand now. As for msg, you can check yourself:

{{if|eq|a|a|x|y}}
↳ x
{{#invoke:params|new|concat_and_call|if|eq|a|a|x|y}}
↳ x
{{#invoke:params|new|concat_and_magic|msg|if|eq|a|a|x|y}}
↳ Lua error: callParserFunction: function "msg" was not found.
{{#invoke:params|new|concat_and_magic|#ifeq|a|a|x|y}}
↳ x

You can also experiment with a new module and :callParserFunction() to see if there is a problem in Module:Params. But there are no reasons to replace :expandTemplate(). As for replacing :newChild() with :callParserFunction() + #invoke, we need to see if in these language in which they use a local translation of #invoke the English #invoke still works (I think it should). Otherwise this module will become a nightmare to maintain. If the English #invoke does not work we should leave the module as it is and simply warn that in case of > 99 parameters or in case of modules that check how they have been invoked, people should use the local version of {{#invoke:params|mapping_by_magic|#invoke|...}} instead of {{#invoke:params|mapping_by_invoking|...}}. I wonder what the performance difference is between using :callParserFunction() + #invoke and :newChild(). I will run some tests. --Grufo (talk) 15:34, 11 July 2025 (UTC)[reply]

@Grufo: I think msg not being recognized as a parser function is likely a bug. The question is where the bug is: your module, the Scribunto extension, MediaWiki proper or some other weird place (i.e., unforeseen interaction somewhere). As shown below, the first argument can cause this error in some cases:

{{#invoke:params|new|concat_and_magic|canonicalurl|:ja}}
↳ https://en.wikipedia.org/wiki/Ja
{{#invoke:params|new|concat_and_magic|canonicalurl|::ja}}
↳ Lua error: callParserFunction: function "canonicalurl" was not found.

This definitely needs more investigation. I might have to look into that. —Uzume (talk) 16:52, 11 July 2025 (UTC)[reply]

Yes, Scribunto ensures #invoke always works even when there are localized alias names. This is in a fashion similar to MediaWiki base namespace names, e.g., Chinese Wikipedia might have another name for Template but that name always works everywhere (even here we have a different base name for the Project namespace but Project still works; i.e. {{ns:Project}} yields Wikipedia and {{ns:Image}} yields File, etc.) they just act like a redirect alias to the localized name. —Uzume (talk) 16:52, 11 July 2025 (UTC)[reply]

FYI: Another little known fact is that all Scribunto module pages are in content language English (I am not sure that is really the right choice but I am not planning to fight it anytime soon) even on Mediawiki instances where English is really used nowhere else. As an example, try going to zh:Special:EditPage/Module:Var#mw-scribunto-console and type into the debug console: mw.log(mw.title.getCurrentTitle().pageLang.code). You will get: en. —Uzume (talk) 16:52, 11 July 2025 (UTC)[reply]

@Uzume: Now I don't have the time to create a test module and check :callParserFunction() with msg. However writing ::ja as argument seems to make the canonicalurl parser function invisible even when called directly (see third example):

{{#invoke:params|new|concat_and_magic|canonicalurl|ja}} corresponds to {{canonicalurl:ja}}; the latter yields
↳ https://en.wikipedia.org/wiki/Ja
{{#invoke:params|new|concat_and_magic|canonicalurl|:ja}} corresponds to {{canonicalurl::ja}}; the latter yields
↳ https://en.wikipedia.org/wiki/Ja
{{#invoke:params|new|concat_and_magic|canonicalurl|::ja}} corresponds to {{canonicalurl:::ja}}; the latter yields
↳ Template:Canonicalurl:::ja

As for moving from :newChild() to :callParserFunction() + #invoke, as soon as I have a moment I will update Module:Params/sandbox and run a few tests. --Grufo (talk) 18:13, 11 July 2025 (UTC)[reply]

@Grufo: Yes, that is interesting how the parser decides that syntax could not be a parser function call (because the argument is not a legal pagename?) and then falls back to parsing it as a template invocation of that pagename. The error from frame:callParserFunction() is definitely coming from the Scribunto PHP code but the point you bring up may in fact be related (see below). As memory serves, when the parser is parsing normal wikitext, it handles parser function calls by parsing out the name and the chunk of wikitext to the end of the call (i.e., }}) only separating them out by | and then calling the parser function code itself giving it a chance to "claim" the provided wikitext arguments (which are not yet fully parsed) and the parser function then decides how to parse such (some use the frame parser but many do not, e.g., #switch depends on the order/positioning of its arguments so it cannot use the frame parser and this is also why Scribunto #invoke can never perfectly emulate it) but I believe the parser function can also return to the parser stating it does not claim the provided wikitext arguments and in such a case the parser must march on assuming that it was not in fact a call to that parser function. I believe that is what is happening in your example above. Maybe something similar is happening in the frame:callParserFunction() path. It would kind of make sense there would be no way to actually call the parser function with entirely pre-parsed arguments so that path perhaps reconstructs the pseudo-unparsed wikitext arguments from the provided arguments and passes it to the parser function just as it would during normal wikitext processing. Then if the parser function rejects it, the parser assumes it was not a parser function call and eventually Scribunto yields the error: function "function" was not found. since there is probably no way to differentiate this from someone trying to call a nonexistent parser function, meaning the "not found" in the error is deceptive and should probably say something more like failed to execute function "function". Maybe this is an issue with the way the pseudo-unparsed wikitext arguments are reconstructed that causes msg, msgnw and raw to always reject the call. There may also be something special about msg and friends as in the history of wikitext parsing it was the first—even before normal template syntax existed and template syntax was implemented as a simplification. I wonder if parsoid has the same issue. —Uzume (talk) 20:17, 11 July 2025 (UTC)[reply]

@Uzume: I agree that the error is deceptive. Let me know if you find out more. In the meanwhile I have updated Module:Params/sandbox with a new version that replaces :newChild() with :callParserFunction() + #invoke. It will still require extensive testing. --Grufo (talk) 05:17, 12 July 2025 (UTC)[reply]

@Grufo: Okay, yes, I have determined parser functions themselves can and apparently sometimes do return the PHP value [ 'found' => false ] which will ultimately lead to that error, however, I think the issue with msg, msgnw and raw is different. I notice they do not show up on Special:Version#mw-version-parser-function-hooks. They are clearly magic words that take parameters but it looks like they might be handled special by the parser and technically are not parser functions or something which would explain why they cannot be reached by MediaWiki\Parser\Parser()->callParserFunction() (and it does rebuild the arguments for parser function calls in a sort of pseudo-unparsed fashion and the first argument has some special handling) which is used by MediaWiki\Extension\Scribunto\Engines\LuaCommon\LuaEngine()->callParserFunction() and ultimately via its Lua library interface. —Uzume (talk) 09:13, 12 July 2025 (UTC)[reply]

@Grufo: Okay, I read through the main parser again and in braceSubstitution() it clearly shows that msg, msgnw and raw are NOT parser functions. It looks for and handles these in this order:

subst and safesubst prefixes
variables (which are sort of like argument-less parser functions)
msg, msgnw and raw prefixes
parser functions by actually calling the same interface Scribunto calls: callParserFunction
page transclusions, i.e, template syntax
special transclusions from "special pages"

Then that function finishes up with some other processing like loop checking and HTML DOM updates, etc.

The msg, msgnw and raw prefixes "work" because they are mostly just deleted and do nothing but the code continues on and eventually finds the template transclusion so it appears to do that. msgnw and raw do set some flags but the same flags can sometimes also be set by some of the parser functions which can be seen in the handling at the return of the parser function call. This means some weird syntax is allowed, e.g., {{msg:canonicalurl:ja}} and {{canonicalurl:ja}} both yield https://en.wikipedia.org/wiki/Ja but one can also do {{msgnw:canonicalurl:ja}} which yields https://en.wikipedia.org/wiki/Ja (notice no link; this is because "nw" means nowiki but not as in the strip-marker generating tag but as in HTML entity escaping). It also means that this type of functionality is not directly available from Scribunto Lua (at least not without frame:preprocess()). —Uzume (talk) 11:03, 12 July 2025 (UTC)[reply]

Here are some more examples of core parser functions (I do not plan to go searching through every extension) that themselves return the deceptive "not found":

{{#invoke:params|new|concat_and_magic|int|help}}
↳ Help
{{#invoke:params|new|concat_and_magic|int|}}
↳ Lua error: callParserFunction: function "int" was not found.
{{#invoke:params|new|concat_and_magic|#interwikilink|d}}
↳ d:
{{#invoke:params|new|concat_and_magic|#interwikilink|x}}
↳ Lua error: callParserFunction: function "#interwikilink" was not found.
{{#invoke:params|new|concat_and_magic|ns|project}}
↳ Wikipedia
{{#invoke:params|new|concat_and_magic|ns|xyzzy}}
↳ Lua error: callParserFunction: function "ns" was not found.
{{#invoke:params|new|concat_and_magic|ns|829xyzzy}}
↳ Module talk

Oddly this one never returns the error so long as it is passed a value that starts with an integer value other than 0 (and it also takes exactly 0 too) even when such an integer value corresponds to no such namespace, e.g., {{ns:-500xyzzy}} parses as a valid parser function call. —Uzume (talk) 09:13, 12 July 2025 (UTC)[reply]

One thing about parser functions "failing" that bothers me is that there does not seem to be anything that specifies to the parser this type of dependency. One of the key tenants of parsoid is to be able to (re)parse parts of a page potentially out of order doing the right thing based upon a dependency chain (sort of like a software build tool like make). A key example is the new #interwikilink which fails when it is provided an invalid interwiki prefix but that is dependent on MediaWiki configuration. This means that I could place some wikitext somewhere that calls #interwikilink and fails but at some future point it might suddenly succeed. Suppose I lobby for some new interwiki prefix at m:Talk:Interwiki map and it succeeds and soon afterwards is deployed. Does this mean all MediaWiki wikis at WMF (and whatever else uses those maps) needs to reparse all pages everywhere (Commons literally has millions) because they might contain wikitext that will now suddenly render differently (properly)? If this dependency was somehow stored then the parser would know which pages (and/or page parts) to reparse (and whatever depends on them in turn, etc.). But nowhere is it documented, let alone specified, when and how a parser function will choose to parse something (or lack thereof; i.e., fail and not parse something). So the main parser cannot know when such parser functions change their mind. Maybe I am overthinking things because there are clearly timestamp type of magic words that obviously change their minds on a regular basis and things are just kept "fresh" at a regular cadence. —Uzume (talk) 09:13, 12 July 2025 (UTC)[reply]

@Uzume: I think you should bring the part of this discussion that concerns parser functions to mw:Extension talk:ParserFunctions. Going back to the original point, here is the bad news about my latest experiments. If I write the following wikitext (which uses :newChild()) 12 times in a page and display the preview, I get an average “Lua time usage” of < 0.1 seconds:

{{#invoke:params|new|
	imposing|99||
	filling_the_gaps|
	mapping_by_replacing||true|1|strict|
	mapping_by_invoking|string|replace|4|^.*$|Hello!|1|false|
	for_each|[$#::$@]}}

On the other hand, if I write the following wikitext (which uses :callParserFunction() + #invoke) 12 times in a page and display the preview, I get an average “Lua time usage” of 0.7 seconds:

{{#invoke:params/sandbox|new|
	imposing|99||
	filling_the_gaps|
	mapping_by_replacing||true|1|strict|
	mapping_by_invoking|string|replace|4|^.*$|Hello!|1|false|
	for_each|[$#::$@]}}

Therefore now I am no longer so sure we should drop :newChild()… --Grufo (talk) 10:34, 12 July 2025 (UTC)[reply]

@Grufo: I do not think mw:Extension talk:ParserFunctions is appropriate at all. That would be for discussion just about that extension including the parser functions from that extension, e.g., #expr, #if, #ifeq, #titleparts, etc. but not about parser functions in general nor about core parser functions. —Uzume (talk) 11:39, 12 July 2025 (UTC)[reply]

As for your performance data, that is very interesting but remember, though {{#invoke}} does have nontrivial overhead (and there are some Phabricator issue tickets about such things) it also means your client code can run loops more than 99 times so long as they do not run out of resources and that they will run properly even with modules that detect how they are loaded and provide a different package interface when loaded via require than with #invoke (it is also trivial to detect mw.loadData but that is sort of a moot point from your perspective as it can only return static data that cannot be executed). So yes, your newChild hack is somewhat faster but it is also buggy and your clients will half to take that into account. I am assuming (Module:Params is long with lots of indirection; I did not read through all of it) you are calling both newChild and callParserFunction off the parent frame to keep that in tact for the target module. —Uzume (talk) 11:39, 12 July 2025 (UTC)[reply]

Module:Params claims it is not designed to be used from other modules via require. To that end it could detect how it is loaded and provided a different interface when loaded via require keeping people from accessing it in such a manner (or fixing it so it does provide useful functionality under such conditions). —Uzume (talk) 11:39, 12 July 2025 (UTC)[reply]

@Uzume: Uh right. With this module being invoked via wikitext, the information that the second module (e.g. Module:String in my examples) has not been called via require() should be propagated and inherited, right (even if we actually call that via require())? As for :newChild(), I was wondering if a solution like frameTools.makePseudoFrame() found here could solve everything. The 99 parameter limit comes from a limitation in the number of child frames that can be created, however we don't really need so many child frames, since they are all identical except for one or two parameters. The ideal solution would be that they make the frame.args metatable editable upstream, so that we stick to one. Lacking that, I was wondering if we could use the “pseudo frame” object mentioned in that wiki. --Grufo (talk) 10:32, 13 July 2025 (UTC)[reply]

@Grufo: You mean will the Fandom makePseudoFrame from Module:FrameTools (with the documentation you mentioned at Global Lua Modules/FrameTools) solve your problems? Yes...no and maybe. I am not exactly sure without testing but I think the concept is mostly sound. That said, it definitely introduces more caveats.

There is no guaranteed your target module will even use the frame argument it is provided in the call as it can always fetch the real frame from mw.getCurrentFrame() and you are almost guaranteed that it will use such during package initialization if it wants access to the frame at all, because there is no other method for access (at that point the frame argument is not available as it is before the target function has even been indexed much less called because it is the initialization block that returns the package table the target function is indexed from). One common idiom I have seen here is a package function that uses a construction like if frame == mw.getCurrentFrame() then allowing the package function to potentially work under #invoke or require (I do not consider this practice as good as the example module I gave you but it does something somewhat similar in attempting to handle both situtations). Currently, a search of all EN Wikipedia Scribunto module code for getCurrentFrame yields 620 hits. Of those, I found this programing idiom at:

These all use getCurrentFrame and soon compare it to something with == (oh, and be glad I did not spam you with all the /sandbox copies too). All of these will already fail with your newChild hack (unless you call callParserFunction('#invoke', ...) off the newChild frame but that sort of defeats your purpose for using it to begin with) because in Lua comparing two tables always fails unless they are the same table reference (i.e., == does not do a "deep compare" to compare the table contents so it is only true if they are in fact the exact same table not a copy). And there maybe other modules that do such as well but hold the getCurrentFrame value and compare it in some other fashion or compare it at some later time, etc. I only looked briefly at the 620 search hit results. I did not look directly at the module code from all 620 hits. Incidentally, I also looked for comparison via ~= but I could not find a single such usage which I find a bit odd as I would probably prefer that myself but apparently that is not popular. I almost never depend on the provided frame argument as it is not really reliable and getCurrentFrame is (although I suppose someone could attempt to monkey patch the getCurrentFrame field of the mw package table/global).

Scribunto newChild actually calls into the PHP code to create a new PPFrame object (notice its own PHP newChild method) that the MediaWiki parser and extensions, etc. can all use. Fandom makePseudoFrame does not appear to do this but just creates a fake one in Lua by copying fields from another Lua frame table. To that end, it is worse than newChild but from your perspective, I would say it is perhaps considerably better. Consider this: makePseudoFrame is probably no worse than newChild with regard to using it and require to fake a #invoke. Both have same caveat of not being the frame of an actual #invoke. However makePseudoFrame has the additional caveat of introducing potential issues for the target module if it uses the provided pseudo-frame to call back into MediaWiki (e.g., where do you think the parser will get arguments for parameter references like {{{1}}} in a preprocess call?). If you use newChild you are trapping those in the new PPFrame (which PHP does not let you change and this is why Lua does not let you change them either; this is also the source of the frame count limitation) so those references work (even if the target module could detect is is not actually being be called via #invoke). This is why I say makePseudoFrame introduces more potential caveats. On the other hand, it will likely be considerably faster as it does not call into PHP and it will not have the PHP frame count limitation (and you probably can write into its arguments although since it is a purely Lua table and its overhead is small you can probably just ensure it gets deleted and recreate it cheaply). My take on it is that if you are considering keeping your require based #invoke emulation, you would likely be better to switch from newChild to makePseudoFrame, however, I would encourage you to pursue a proper #invoke solution over the require based emulation. And at the very least you should rename and very clearly document that those functions in {{#invoke:Params|function}} are #invoke emulations that have lower overhead but only work sometimes when the target module does not do much discrimination on how it is actually loaded (e.g., initialization block ...), its provided frame arguments (e.g., comparing it to getCurrentFrame) or calling back into PHP in certain ways (e.g., preprocess with makePseudoFrame). —Uzume (talk) 16:12, 13 July 2025 (UTC)[reply]

Thank you for your incredibly detailed answer, Uzume. What I don't understand is this: how is possible that if I want to call a module via require() + :newChild() I hit the < 100 parameters limitation, whereas #invoke is able to bypass it? How does it do it? Shouldn't it suffer from the same PHP limitation too? I was thinking that maybe the best solution at the moment is to leave everything as it is and write about it in the documentation. I think so because this is the solution that gives the maximum freedom of choice. Currently if one wants to favour performance can use, for instance ...|mapping_by_invoking|..., whereas if one wants to have the possibility of having 1000 parameters (at the cost of a slower solution) can use ...|mapping_by_magic|#invoke|... --Grufo (talk) 17:35, 13 July 2025 (UTC)[reply]

@Grufo: I took a brief look at your module's usage of newChild as well as makePseudoFrame (which can be directly compared with Scribunto's own newFrame). First, you should really consolidate your #invoke code whether it is an emulation or not (and while you're at it also encapsulate expandTemplate and callParserFunction; FYI: I do not like your usage of the word "magic" as it only supports parser functions not all types of magic words so you should really consider renaming those too). There is no reason to have five copies scattered around needing different maintenance.

How about a single invoker that takes an emulation parameter and either does a real invoke or emulates one? Then the caller can choose, and if you really get daring, the parameter can be more than a boolean specifying the emulation type, i.e., to what lengths should the emulation employ. There are some things I am not too thrilled with about makePseudoFrame (and I think I found a minor bug in it too). If you are serious about trying to maintain a #invoke emulator, I have some ideas on how to build a better mousetrap.

I am not sure if there is a good way to fake the environment for the module initialization block (e.g., ..., etc.) as hacking that would probably have security implications and as such is likely locked down well but remember that it runs at a very different time to the execution of the package functions which get executed later either by Scribunto directly (i.e, via #invoke) or by Lua calls from module code, typically needing a require to get access to the package. If you look at newFrame, Scribunto's own frame object generation code, you might notice that they are generic tables without a metatable. Sure, the args field is another subtable and it is definitely metatabled but the main frame objects do not seem to be (of course someone might go and change things at some later date but at the moment that is true). To that end, it gave me an idea to help you with your emulation and potentiality alleviate your 99 frame limit whoas. Why not use real newChild but do it later only upon demand? Your code seems to proactively create a newChild at every emulated #invoke. If you instead, created a dummy table with a metatable and pass that in, you could wait until the target code actually tries to reference fields from the dummy frame and only then create the newChild upon demand and return values from there. This way if the target module never uses the dummy frame object, newChild is never actually called. You could also try and monkey patch the mw.getCurrentFrame to return the same dummy frame object fixing those situations I mentioned above. Of course if the target module does try to use its frame object, then you will be forced to call newChild and provide one so the number of iterations would be limited then. Since most modules at least attempt to use the frame args, you will likely want to handle that yourself avoiding the newChild call. You will likely want to handle getTitle and getParent (for its own args) too but in other cases you could punt to newChild upon demand. The code would likely be pretty tricky but it should work fairly well (except for possible initialization block issues referenced earlier). Just some ideas. —Uzume (talk) 18:30, 13 July 2025 (UTC)[reply]

Thank you again, Uzume. As for the nomenclature, I might agree that “magic” is a bit creative, but it has the advantage of being short. Any other solution was lengthier. But I am definitely open to suggestions, as long as they are coherent with the rest of the names. As for “consolidate my #invoke code” and “encapsulate expandTemplate and callParserFunction”, I am not totally sure I understood what you mean. As for creating a metatable and passing that, there is no way that the target module does not access its frame.args metatable in functions like mapping_by_invoking, so at least that will have to be a “fake” parameter table. I was thinking that I could create a single :newChild(), create a metatable, redirect all its fields except frame.args to that single child frame, and use a normal (always changing) table for frame.args. How does this sound? You might have just given me an idea to experiment with. --Grufo (talk) 18:58, 13 July 2025 (UTC)[reply]

@Grufo: By encapsulate I mean make a function or something so there is only one copy of the code to maintain and have everything that needs to accomplish that feat use that function (or class object, etc.). Your idea sounds like an interesting starting point but I do think you are going to run into many weird corner cases and you need to be very meticulous and methodical with your design and implementation. And I question the need for newChild if you are not planning to capture the args in an actual parser frame (because you keep changing them without a newChild for every change). What you are suggesting is very much like the Fandom makePseudoFrame (which does not bother with newChild because it does not capture the args in an actual parser frame). Of course doing that presents issues if the target module tries to call back into parser as the parser will then not know about those uncaptured args. Instead of "magic" why not just "pf" or "parserfunc" or some such. —Uzume (talk) 19:12, 13 July 2025 (UTC)[reply]

@Uzume: :newChild() would be needed because the callback module might want to access other fields of the frame object (e.g. :callParserFunction(), or :getParent(), and in this way it will have the entire suite of fields ready. These remain constant for the entire duration of this module's invocation, so it makes sense that they always refer to a single child frame. The only thing that the callback modules can see changing is their frame.args field. --Grufo (talk) 19:20, 13 July 2025 (UTC)[reply]

@Grufo: Even if they remain constant, why bother with newChild if you can get them from your own frame? Well the only thing they will see is the changing frame.arg if and only if that is all they look at. You mentioned the target module might want other fields. What if it wanted say preprocess? The parser does not know about your changed uncaptured args so the preprocess has the possibility of rending incorrectly (e.g., parsing {{{1}}}, etc.) The point is, either capture the args with a real #invoke or newChild or don't and then there is no real need for newChild. Besides the functions themselves, a frame is really only a page name and a set of args. Sure you might be able to capture the title but if you are tweaking the args all the time, it would be very little work to handle the title yourself too. If you handle both, I do not see the point. What does even a single newChild buy you. I say very little under those circumstances. Of course it it is only one call so feel free to explore, but I really do not see the need. —Uzume (talk) 19:46, 13 July 2025 (UTC)[reply]

@Uzume: There are many little reasons why :newChild() is necessary. For instance in the callback module frame.args should return the parameters passed to the callback module itself, whereas frame:getParent().args should always return the calling template's parameters (I mean the parameters passed to the template that invoked Module:Params); or frame:getTitle() should return, for instance Module:String, whereas frame:getParent():getTitle() should always return the calling template's name, and so on. As for frame:preprocess(), you are possibly making a valid point (I should test it). --Grufo (talk) 23:54, 13 July 2025 (UTC)[reply]

@Grufo: But you can just pass back your module's parent frame for getParent and you already know the target module's full pagename as you need that to call require so you can easily implement getTitle. Anyway, I have not tested this at all however, I made a first try at creating an invoke function in Module:Sandbox/Grufo. I am not sure how robust it is but it should spoof args, getParent and getTitle while also monkey patching getCurrentFrame and never creating any frames until the target module attempts to access one of the other fields. Then it calls newChild and updates all the fields to stop spoofing things but just passing calls to the on demand created child frame. This will still have a 100 frame limitation if the target module uses those fields, but if it sticks to just those three that I have spoofed, it should never hit any such limit. It might be possible to spoof some of the other fields but they do not seem to be in much demand besides the ones that probably require calling back into the parser anyway. It also can do nothing about load detection in the target modules initialization block. It certainly has some key caveats but I think this should work for the vast majority of cases and be a little lighter on resources than doing many actual #invoke calls. Let me know what you think and if you try it out, etc. —Uzume (talk) 01:17, 14 July 2025 (UTC)[reply]

Thank you for your code, Uzume. I have not tested it yet, but it seems to me that on demand (i.e. delayed) it creates a child frame for each callback, am I wrong?. This means that with functions like mapping_by_invoking it will hit the < 100 limitation for sure, because these callbacks can never ignore the frame.args metatable. P.S. I started playing around with my solution, but so far I did not find a way to make the methods of the pseudo frame object work (none of them). Only the fields work. This happens because of this restriction. So I will have to find a way somehow to bypass it. --Grufo (talk) 04:04, 14 July 2025 (UTC)[reply]

@Grufo: No, that is not how it designed to work. It creates the frameSpoof and passes that instead. It is filled with fields that are punting stub functions that will call newChild and then rewrite the stub functions to other stub functions that will use the created newChild. This way if the target module does access one of the unspoofed fields, newChild will be called from inside the target module by one of the punting stub functions but only once. But before I am done creating the frameSpoof, I overwrite the fields I want to spoof including args, getParent and getTitle. So target modules that only use those three fields will use those spoofed values and never have newChild called. In fact, technically the target module is calling newChild when it calls one of the punting stub fields that are not spoofed. This is why the newChild is delayed. It is also why target modules that do not touch those other punting fields will never actually call newChild. We cannot control what the target module will do but we can respond to what it does do. So I wrote it inside-out which makes the code a little weird and maybe hard to follow. Once frameSpoof is completely constructed with its one-time punting stubs and three spoofed fields, I monkey patch getCurrentFrame to return my frameSpoof and then call the target module function and then immediately unpatch it before returning the value from the call to the target module function. Remember, you are not just spoofing the target module function. You are spoofing that and everything it calls including other modules it loads and calls via require, etc. Your module's main use cases make it lean towards being used with other modules that have functions with fairly simple functionality (otherwise why call them repeatedly?). If those target module functions are simple enough to not trigger one of the newChild stubs in frameSpoof (i.e, one of the punt stubs vs. the spoof fields), they never need do any punting to newChild at all. —Uzume (talk) 06:46, 14 July 2025 (UTC)[reply]

@Uzume: I am amazed. But I have to admit that you lost me there. I will probably have to just study your code. But just as a quick understanding, if I were to implement mapping_by_invoking using your solution, and then I were to launch the following code (which calls {{#invoke:string|replace}} 101 times),

{{#invoke:params|new|
	imposing|101||
	filling_the_gaps|
	mapping_by_replacing||true|1|strict|
	mapping_by_invoking|string|replace|4|^.*$|Hello!|1|false|
	for_each|[$#::$@]}}

would I hit the < 100 parameters limitation? --Grufo (talk) 13:48, 14 July 2025 (UTC)[reply]

@Grufo: Based upon my understanding and having studied Module:String (so I know it hardly touches its frame for anything other than args), I believe you should not hit such a limit (although you might hit others I am not sure). But of course such things need testing. I did some basic testing to make sure it appears to work in general. But I certainly did not test it to that extent. Also it really needs more input validation, etc. For example, right now, I could pass in some nasty things to modargs that are not strings, etc. But assuming one does not do anything crazy, I believe it should handle your situations fairly well. —Uzume (talk) 16:04, 14 July 2025 (UTC)[reply]

@Uzume: Thank you. Some personal commitments got in the way, and I won't be free until the beginning of August (so I will just do ordinary admin tasks on Latin Wikipedia for the next days). But I want to get to the bottom of this. In the meanwhile, if you find a solution that never hits the < 100 limitation, please make a whistle. The ideal would be that we found a way to create a :newChild only once. But unfortunately, due to this restriction using a custom metatable instead of the original frame won't work. And honestly at the moment I am in lack of frash new ideas. --Grufo (talk) 10:55, 16 July 2025 (UTC)[reply]

@Grufo: That is an easy answer. MediaWiki owns the parser and the parser knows about the entire parse tree (well eventually it does anyway otherwise it cannot yield a completely parsed page and that is really the parser's main job). So even though you can use newChild to create a parser frame object, you do not really know when it is safe to destroy it and so Scribunto does not provide access to a frame destructor for frames. The parser does know so it merrily goes along creating and destroying them. If you call #invoke 200 times, so long as there is no dependency requiring the frames to continue to exist, MediaWiki destroys them (if you created a tree of recursions where each frame created more frames there would be dependencies and the frames would have to live longer). In this way, so long as nothing hits imposed resource limitations all is good. Since the main parser is unlikely to need many frames at any single point in time, there is a limit imposed on them and you found it: 100. Since you cannot pass back unparsed text from Scribunto, I imagine the parser will quite quickly destroy all the frames you created with newChild upon exit of the #invoke to your module. This means you can batch such things, e.g., you could assume you can create 50 newChild frames without issue, then you could have some code that processes no more than 50 iterations and have your main code {{#invoke}} back into your own module where that entry point just does the 50 limit max and returns. Then your main code could get control again and keep doing this until the request is complete no more than 50 at a time. In fact the callParserFunction('#invoke', ...) route could be views as a "do only one iteration at a time" batch processor. —Uzume (talk) 19:00, 13 July 2025 (UTC)[reply]

Now it is all clear. Thank you! --Grufo (talk) 19:04, 13 July 2025 (UTC)[reply]

New function self