View source for Module talk:Plain text

{{oldtfdfull|date= 2018 May 5 |result=no consensus |disc=Module:Plain text}}

== strip_apostrophe_markup ==
{{ping|Galobtter}} The function string.gsub() is quite forgiving, so you don't need to test for each case. Also ' doesn't need to be escaped when used as a search pattern. You can't sensibly export the strip_apostrophe_markup function, so it should be local, or could just go inline. You can simplify strip_apostrophe_markup to 
<pre>
local function strip_apostrophe_markup(txt)
	txt = txt:gsub("'''''", ""):gsub("''''", ""):gsub("'''", ""):gsub("''", "")
	return txt
end
</pre>
In the main function, text should be a local variable:
 local text = frame.args[1]
I don't like altering code while others are developing it, so I'll leave you to update it as you see fit. --[[User:RexxS|RexxS]] ([[User talk:RexxS|talk]]) 19:56, 14 April 2018 (UTC)
*{{u|RexxS}} the second point - yeah I forgot to localize - regarding the strip_apostrophe_markup(txt), yeah I was also wondering why there were so many ifs etc, but I was too lazy to look over it (as you can see, I just copied it from [[Module:Citation/CS1/COinS]]). Wonder if the same change should be done on [[Module:Citation/CS1/COinS]] - ping {{u|Trappist the monk}} on that [[User:Galobtter|Galobtter]] ([[User talk:Galobtter|pingó mió]]) 20:05, 14 April 2018 (UTC)
** It's best to use the ustring library (as you have done), mainly because the module is likely to be reused in other languages, so your new code ends up not quite as simple, but is still fine. Nice work! --[[User:RexxS|RexxS]] ([[User talk:RexxS|talk]]) 20:36, 14 April 2018 (UTC)
*** Thanks! Yeah it is good to allow easy reuse., plus we ourselves use unicode characters occasionally for places I believe [[User:Galobtter|Galobtter]] ([[User talk:Galobtter|pingó mió]]) 20:43, 14 April 2018 (UTC)
**** We do sometimes use unicode characters for places, but interestingly string.gsub copes perfectly well with all of the Latin diacriticals and Greek or Cyrillic script that I've tried: [[Module talk:RexxS #Test stripApost]]. There will almost certainly be some characters that trip it up, but they won't be common. --[[User:RexxS|RexxS]] ([[User talk:RexxS|talk]]) 21:16, 14 April 2018 (UTC)
*****That is rather interesting, I think perhaps as long there is nothing being done to the unicode characters themselves it may be ok [[User:Galobtter|Galobtter]] ([[User talk:Galobtter|pingó mió]]) 06:06, 15 April 2018 (UTC)
****** I suspect that it's a case of not using a function that makes use of absolute positioning within the string, because the byte count that the string library uses much of the time is obviously going to be incorrect with unicode characters. We probably just struck lucky with gsub {{smiley}}. --[[User:RexxS|RexxS]] ([[User talk:RexxS|talk]]) 20:55, 15 April 2018 (UTC)

I replaced the mw.ustring.gsub with plain gsub because ustring is a lot slower than gsub and is not needed in this module. The optimization is not necessary but since people are looking at the code I thought it worth mentioning that wikitext will always use [[UTF-8]] and that means Lua gsub with the patterns in this module will work well. Lua gsub works in any language with a pattern like <code>'[12]'</code> ('1' or '2') but mw.ustring.gsub would be needed for a pattern like <code>['১২']</code> (that might be used at the [[:bn:|Bengali Wikipedia]] to search for their equivalent). In the first case (Lua gsub), the pattern finds the first location matching any of the ''bytes'' between <code>[</code> and <code>]</code>. In the Bengali case, each digit is three bytes in UTF-8, so there are six bytes between the square brackets. If Lua gsub were used, it would look for any of those ''bytes''. [[User:Johnuniq|Johnuniq]] ([[User talk:Johnuniq|talk]]) 09:47, 18 April 2018 (UTC)

== Could remove indentations ==

Can be comnbined with leading spaces: {{code|gsub("^[:;%s]+", "")}} [[User:Guarapiranga|— 𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚]] ([[User talk:Guarapiranga|talk]]) 20:31, 24 May 2021 (UTC)

== Performance improvements (and other) in the sandbox ==

I made a few performance (and other) improvements to this module in the [[Module talk:Plain text/sandbox|sandbox]] based on the work with [[Module:User scripts table]] (for which I started using [[Module:Plain text]], and ended up forking and customising it for the needs there). The two performance improvements are:
# Use greedy {{mono|[^x]+x}} instead of ungreedy {{mono|.-x}} whenever possible; and
# Use a single gsub for all {{mono|File:}}, {{mono|Category:}}, {{mono|Media:}}, etc, instead of a gsub for each.
— [[User:Guarapiranga|𝐆𝐮𝐚𝐫𝐚𝐩𝐢𝐫𝐚𝐧𝐠𝐚]]&nbsp;[[User talk:Guarapiranga|☎]] 13:48, 21 June 2021 (UTC)

== nowiki text removed? ==

The documentation example has in its example: <code>&lt;nowiki>?&lt;/nowiki></code> (question mark in nowiki tag).

The module removes this wikitext altogether, including the question mark. Why is this "other stuff" to be removed? -[[User:DePiep|DePiep]] ([[User talk:DePiep|talk]]) 13:41, 2 September 2021 (UTC)

== Tag stripping ==

Currently, this module strips both the tags and their contents for all HTML-style tags, except for {{tag|span|open}}, {{tag|i|open}}, {{tag|b|open}}, {{tag|em|open}}, and {{tag|strong|open}} (and the last three only because I just added cases for them). However, there are a variety of other tags which are valid in wikitext, and which contents arguably should be kept after discarding the tags themselves, e.g. {{tag|h2|open}}, {{tag|dfn|open}}, {{tag|sup|open}}, {{tag|u|open}}. These could continue to be added here individually, but I think it's probably simpler to reverse the module's behavior, and only discard contents of tags for a curated list, and otherwise keep the contents.

The main issue I can see with that would be for {{tag|sub|open}} and {{tag|sup|open}}, where just stripping them often results in confusing text, e.g. stripping "2<sup>32</sup>" would produce "232", or "[[electron neutrino|v<sub>e</sub>]]" producing "ve"; in these cases it might be better to replace the tags with "^"/"_" (resulting, for the aforementioned examples, in output of "2^32" and "v_e") or other appropriate characters (though the suggested characters, I believe, are the ones most often used for indicating super/subscript when formatting options are limited). <span class=nowrap>「[[User:Dinoguy1000|<span style=color:#00f>ディノ</span><span style=color:#080>奴</span>]][[Special:Contributions/Dinoguy1000|<span style=color:#F90>千？！</span>]]」<sup>[[User talk:Dinoguy1000#top|☎ Dinoguy1000]]</sup></span> 04:13, 6 October 2021 (UTC)
:Reasonable, especially when whitelist/blacklist are argued well & systematically -- and so more stable. Module history shows that it was never approached this systematically. 
:Now, the documentation has this peculiar sentence "other stuff that needs removing from short descriptions". Looks like it was purpose-build for [[WP:SHORTDESC]], [[WP:SDFORMAT]] then. But it is actualy unused in {{tl|short description}}; ask [[WP:WPSHORTDESC]]? And, what effect on does ''existing'' 1M+ usage (that's module; {{tl|Plain text}} has 35k)? For this, the proposed extended removals be put in a separate function? -[[User:DePiep|DePiep]] ([[User talk:DePiep|talk]]) 06:06, 6 October 2021 (UTC)
::I saw that the original intention was for SHORTDESCs before editing the module and starting this discussion, though I didn't actually check to see if the module is currently used for that; it's mildly amusing to me that it isn't.
::To be honest, I have no idea how this change might affect current uses. If performance isn't too much of a concern, we could simply add tracking for cases where nonspecific tags are being stripped and give things a while to filter in before looking through them and seeing if anything interesting appears. That being said, I'd expect the vast majority of uses to be via other templates or modules; some [https://en.wikipedia.org/w/index.php?search=insource%3A%2Finvoke%3APlain+text%2Fi&title=Special:Search&profile=advanced&fulltext=1&ns10=1 quick] [https://en.wikipedia.org/w/index.php?search=insource%3A%2Frequire%5C%28%5B%27%22%5DModule%5C%3APlain+text%5B%27%22%5D%2Fi&title=Special:Search&profile=advanced&fulltext=1&ns828=1 searches] show it's only being directly used in ~3 dozen templates and modules, which shouldn't be too hard to look through by hand (though TBH I don't know what I'd be looking for).
::So what tags ''should'' be fully discarded? There's the obvious {{tag|br|void}} (which is already stripped, though not very robustly), and the currently-not-stripped {{tag|hr|void}}, {{tag|wbr|void}}, and most (almost all?) of the [[Help:Parser tag|parser/extension tags]] (and maybe <code>&lt;!-- comment --></code>, though that doesn't get displayed anyways); {{tag|table|open}} and {{tag|div|open}} are potential candidates, though I could definitely see arguments for keeping their contents at least sometimes (so maybe make them optional somehow?).
::Conversely, looking through [[WP:HTML]] reminds me of {{tag|abbr|open}}, which would probably also need some specific consideration akin to {{tag|sup|open}}/{{tag|sub|open}}. The contents of {{tag|bdi|open}}/{{tag|bdo|open}} might be able to just be presented as they are in the original string, but I'm not an expert in this area, so at the least it would probably require a bit of discussion.
::I'm getting away from this discussion at this point, but after thinking about it earlier I concluded that probably the best method for stripping templates while optionally keeping some of their contents would be for templates/modules to have some sort of "plain output" mode, that would be "safe" for applications like this or [[WP:NAVPOP]], which currently just strips most templates entirely. Though obviously this would require quite a bit of work, and some planning/consideration on the implementation (which I don't have many thoughts on myself, but maybe one idea would be adding some feature to TemplateData to indicate "safe" parameters to output directly, assuming TemplateData doesn't already have such a feature). <span class=nowrap>「[[User:Dinoguy1000|<span style=color:#00f>ディノ</span><span style=color:#080>奴</span>]][[Special:Contributions/Dinoguy1000|<span style=color:#F90>千？！</span>]]」<sup>[[User talk:Dinoguy1000#top|☎ Dinoguy1000]]</sup></span> 08:39, 6 October 2021 (UTC)
:::I've started {{tl|Navbox wikitext-handling templates}}, to see what is related. -[[User:DePiep|DePiep]] ([[User talk:DePiep|talk]]) 20:06, 6 October 2021 (UTC)

== Link stripping ==

Since I've been [[#Tag stripping|thinking about this module]] today anyways, I realized that the link stripping here duplicates the stripping done by [[Module:Delink]], albeit probably less robust and (I think?) not catching as many cases. Are there any major reasons (other than [[Module talk:Delink#Optimization of nested links|performance]] maybe) not to just use Module:Delink for that functionality in this module? <span class=nowrap>「[[User:Dinoguy1000|<span style=color:#00f>ディノ</span><span style=color:#080>奴</span>]][[Special:Contributions/Dinoguy1000|<span style=color:#F90>千？！</span>]]」<sup>[[User talk:Dinoguy1000#top|☎ Dinoguy1000]]</sup></span> 08:42, 6 October 2021 (UTC)
:We need a more general approach to any "non-ascii"-stripping. Think HTML, html-tags, wm-extension tags, wikicode like <nowiki>[[{{!}}]]</nowiki>,  parser-strips, etc. -[[User:DePiep|DePiep]] ([[User talk:DePiep|talk]]) 20:27, 6 October 2021 (UTC)

== Keeping contents of &lt;sup>/&lt;sub> ==

(starting a new topic because [[#Tag stripping]] is old and only touches on these tags as a side-comment)

At [[WP:VPT#Wikilinks from italic titles]], I came across this template as a solution for one editor's issue (italics in ship-names), but the lack of preservation for superscript/subscript text breaks for another related use (chemical names). The general goal is to convert a properly formatted visual text into a bluelink. But as simple examples that currently don't work correctly for that use, we would want {{code|H<sub>2</sub>O}} (H<sub>2</sub>O) to become "[[H2O]]" not "[[HO]]" and {{code|<sup>3</sup>H}} (<sup>3</sup>H) to become [[3H]] not [[H]]. [[User:DMacks|DMacks]] ([[User talk:DMacks|talk]]) 16:04, 26 June 2022 (UTC)
:This happens because of [[Module:Plain_text#L-21|line 21]].  Could be fixed by adding these lines ahead of line 21:
::<syntaxhighlight lanf="lua">		:gsub('<sub.->(.-)</sub>', '%1') --remove subscript markup; retain contents
		:gsub('<sup.->(.-)</sup>', '%1') --remove superscript markup; retain contents</syntaxhighlight>
:{{small|{{lang|la|caveat lector}}: not tested}}
:—[[User:Trappist the monk|Trappist the monk]] ([[User talk:Trappist the monk|talk]]) 16:25, 26 June 2022 (UTC)
::I came here to ask for this feature, once again. Given existing behaviour & usage, I suggest this should be a opt-in by parameter ({{para|"keep-tag-content"|2=T [F-by-default]}}. (alternative: by fork)
::Other tag content to be kept? {{tag|sup|open}}, {{tag|sub|open}}, {{tag|h''n''|open}}, {{tag|dfn|open}},  {{tag|u|open}}? {{code|1=<nowiki>}} content can ''not'' be kept because of unknown code injection.
::From [[#Tag stripping]]: replace {{tag|sup|open}}, ..  with {{angbr|^}} or generic whitespace, etcetera? As option? -[[User:DePiep|DePiep]] ([[User talk:DePiep|talk]]) 07:53, 21 February 2023 (UTC)
:::I have adjusted the sandbox and added underline, subscript, and superscript test cases to the testcases page. Is there further testing that needs to be done, or should I roll this out to a million pages? – [[User:Jonesey95|Jonesey95]] ([[User talk:Jonesey95|talk]]) 15:41, 23 July 2023 (UTC)
::::Looks good! [[User:Johnuniq|Johnuniq]] ([[User talk:Johnuniq|talk]]) 23:27, 23 July 2023 (UTC)
:::::{{done}}. Thanks to {{U|Trappist the monk}} for the code. – [[User:Jonesey95|Jonesey95]] ([[User talk:Jonesey95|talk]]) 05:29, 24 July 2023 (UTC)

== Strips plus signs. ==

Sadly I discovered this template strips plus signs. And [[Template:Strip tags]] only strips span and div. [[User:Johnjbarton|Johnjbarton]] ([[User talk:Johnjbarton|talk]]) 22:52, 8 October 2024 (UTC)

: fixed the stripping of + signs [[User:Dpleibovitz|Dpleibovitz]] ([[User talk:Dpleibovitz|talk]]) 01:03, 24 December 2024 (UTC)