Module talk:ScribuntoUnit

Test a string contains expected text

I think there is a need for an assertion that checks that a string contains expected text. Motivation is as follows: you should test behaviour, not test methods and each test method should only test one thing. Suppose you have a complicated module (pl:Moduł:Koordynaty) that 1) adds a formatted link 2) prints out a formatted text 3) adds the page to a calculated category (and maybe will do other things in the future we cannot predict, and we don't want people adding one new feature to fix assertions for all other features just because module output has grown). To write a proper unit test, you should write 3 separate test methods, each checking only one functionality and no other, so that when only 1 of the functionalities is broken, exactly 1 test fails and no other. To achieve this, you need a method like

assertThat(coordinates.invoke("53 45", "23 45"), containsString("geohack/53.45W_23.45E/type:city"))

where you check a part of the module output and don't care about the rest. Also, there are modules that create multi-line outputs. Tests need to be readable and check only what is relevant. Therefore an "contains string" assertion would be handy.

In JUnit, you can deal with the lack of "assertContains" by using assertTrue() with a custom message displayed when the assertion fails. Here, there is no way to pass the failure message. Hence, assertTrue from ScribuntoUnit is not suitable for implementing "contains string" assertions, because there will be no meaningful error message.

I suggest either adding a new method "assertContains" or extending "assertTrue" with a failure message - or perhaps performing both. --Derbeth ^talk 21:40, 1 January 2014 (UTC)[reply]

That sounded like a good idea to me, and it will be useful for Module:Documentation which I'm working on now, so I went ahead and added an assertContains method. Take a look and let me know if you have any suggestions for how it could be improved. — Mr. Stradivarius ^{♪ talk ♪} 16:32, 2 January 2014 (UTC)[reply]

It looks great, thank you :) Do you have a page where you check the assertions work correctly and that their diagnostic messages look good? I did not find Module:ScribuntoUnit/testcases. --Derbeth ^talk 20:01, 2 January 2014 (UTC)[reply]

Great. But perhaps we should name it assertStringContains in case we later implement a method that checks if a table contains a given table key? And yes, we should have tests for the assertions as well. I'm not really sure how to write good tests for a testing module, but I started experimenting on Module:ScribuntoUnit/testcases. – Danmichaelo (talk) 08:42, 3 January 2014 (UTC)[reply]

So renamed. Sounds like a good reason to me. — Mr. Stradivarius ^{♪ talk ♪} 14:05, 3 January 2014 (UTC)[reply]

Automated tests are needed, but I think there is also a need for a visual presentation of how the module works in practise. I have created such a page: Module:ScribuntoUnit/showcase (run on the talk page). If I did not invoke something wrong, it seems that assertWithinDelta ignores the custom message. Plus, test methods are run in no particular order - can we do something about it? --Derbeth ^talk 21:08, 3 January 2014 (UTC)[reply]

Agree, the showcase page is a good idea. Seems like the output is not so consistent between the different methods; on failure, some methods include "actual" and "expected", while others do not. If "actual" and "expected" are included, "message" is not shown. This leads me to thinking that we might be better of just replacing the two columns by a single column. I updated Module:ScribuntoUnit/sandbox and Module talk:ScribuntoUnit/showcase. What do you think? – Danmichaelo (talk) 18:21, 5 January 2014 (UTC)[reply]

I made small tweaks to the sandbox code. I like it now. --Derbeth ^talk 10:31, 6 January 2014 (UTC)[reply]

AssertWithinDelta

The docs for ~~assertDeltaContains~~ assertWithinDelta says " While 1/3 == 9/3 evaluates as false, ...". This is an obvious result, as they are completely different numbers. I assume this was a typo for "1/3 == 3/9 evaluates as false"; however, when I tried this in the debug console just now it evaluated as true. Is there a better example we can use to demonstrate floating point error here? Personally, I haven't seen it action in Scribunto yet, so I can't think of a good one. — Mr. Stradivarius ^{♪ talk ♪} 14:02, 3 January 2014 (UTC)[reply]

True, that was a completely wrong example :) I've added a new one, and tested it in the debug console. Btw., the reason I added the method was that I ran into the problem on no:Module:Coordinate/testtilfeller. – Danmichaelo (talk) 00:45, 5 January 2014 (UTC)[reply]

Thanks for clearing that up. :) It's interesting to see how the floating point error works in a real-world example. I've given the docs a copy edit and expanded them a little to note that integers can be represented exactly up to 2^53. Feel free to tweak that if you want. — Mr. Stradivarius ^{♪ talk ♪} 11:48, 5 January 2014 (UTC)[reply]

Table display

At the moment, if two different tables are tested for equality, the wikitable generated by the module simply lists them both as "table". It would be nice to actually show the contents of the table in a reasonably nice format. Perhaps we can do something with Module:User:Anomie/deepToString and/or mw.logObject? deepToString fails when trying to display mw.title objects, and while mw.logObject handles them ok, it only outputs to the log buffer. So we would need to do some tweaking to get something that suits our purposes, but it probably wouldn't be too hard. — Mr. Stradivarius ^{♪ talk ♪} 12:00, 5 January 2014 (UTC)[reply]

Hm, how does it fail? I tried using mw.title at Module_talk:ScribuntoUnit/showcase, and it seems to work, but I'm not familiar with it. – Danmichaelo (talk) 18:47, 5 January 2014 (UTC)[reply]