If you’ve used NGLess before (or read about it), you’ll know that every ngless script starts with a version declaration:
This indicates which version of NGLess should be running the code. Since the bug changed the results, I needed to make a new version (we are now at version 0.8).
The question is what should NGLess do when it runs a script that uses an older version declaration? I see three options:
1. Silently update everyone to the new behavior
This is the typical software behavior: the new system is better, why wouldn’t you want to upgrade? Because we’d be breaking our promise to make ngless reproducible. The whole point of having the version line is to ensure that you will always get the same results. We also don’t want to make people afraid of upgrading.
2. Refuse to run older scripts and force everyone to upgrade
This is another option: we could just refuse to run old code. Now, at the very least, there would be no silent changes. It’s still possible to install older versions (and bioconda/biocontainers makes this easy), so if you really needed to, you could still run the older scripts.
3. Emulate the old (buggy) behavior when the user requests the old versions
In the end, I went with this option.
The old behavior is not that awful. Some reads are handled completely wrong, but the reason why the bug was able to persist for so long is that it only shows up in a few reads in a million. Thus, while this means that NGLess will sometimes knowingly output results that are suboptimal, I found it the best solution. A warning is printed, asking the user to upgrade.