Section 46.11 Doctesting Sage Code
Adding computer code to your textbook is a tricky proposition. You can propose that it is merely an illustration, and not meant to have all the necessary details, or you can make it exact, correct and executable, and then risk inevitable changes to render your code obsolete. At least you have the option of editing and reposting online versions quickly and easily.
One of our main motivations for this project was mixing in code from the powerful, open source, mathematical software project, Sage (Section 3.17). When you add example Sage code to illustrate mathematical ideas, you are then encouraged to also include expected output in the
<output>element. Here comes one of the powerful advantages of XML source and XSL processing.
mathbook/xsl/pretext-sage-doctest.xslstylesheet, used in the usual way, will create one, or several file(s), in exactly the format Sage expects for automated testing. So all your words are gone, and all your Sage input and output is packaged so Sage can run all the
<input>and compare the results to the expected
<output>. See Subsection 44.1.1 for details on obtaining more than one file.
We have many years’ experience testing hundreds of non-trivial Sage examples from textbooks, for linear algebra and abstract algebra. Roughly every six months, we discover ten to twenty examples that fail. Frequently the failures are trivial (usually output gets re-ordered), but some are significant changes in behavior that leads us to re-word surrounding guidance in the text, and in a few cases the failures have exposed bugs introduced into Sage. It has been relatively easy to do this maintenance on a regular basis, and if it had not been done, the accumulated errors would be enough to greatly degrade confidence in the accuracy of the examples.
Exact details for this process can be found in Section 4.35. Note that Sage is really just a huge Python library, so it might be possible to test pure Python code with this facility, but we have not tested this at all. Similar support for other languages can be considered if requested for use in a serious project.