openstatsguide
Minimum Viable Good Practices for High Quality Statistical Software Packages
Scope
We encourage developers of statistical software packages to follow this minimum set of good practices around:
“Documentation, Vignettes, Tests, Functions, Style, Life cycle”
These keywords can be easily remembered with the mnemonic bridge sentence:
“Developers Value Tests For Software Longevity”
While the recommendations are rather generic, we focus on functional programming languages and give links to implementations in R, Python and Julia.
This guide primarily addresses developers of statistical packages. Users interested in assessing the quality of existing statistical packages will find complementary “validation” focused resources valuable, as listed in References.
Recommendations
Documentation
Documentation is important for both users and developers to understand all objects in your package, without reading and interpreting the underlying source code.
Use in-line comments next to functions, classes and other objects to generate their corresponding documentation.
Do also document internal functions and classes for maintenance by future developers.
Add code comments for ambiguous or complex pieces of internal code, but only after optimizing the code design as much as possible.
Vignettes
Vignettes are documents that complement the object documentation by providing a comprehensive and long-form overview of your package’s functionality from a user point of view, with particular emphasis on the connection to the statistical approaches.
Provide an introduction vignette that introduces the package to new users, such that they have an easy entry point for getting started. Make sure to include code examples and automatically compile the vignette to ensure reproducibility.
Include deep dive vignettes that go into depth on specific use cases, functionalities or underlying theory, in particular describing the underlying statistical methodology and how it is implemented in the package.
Host your vignettes on a dedicated website, which allows users to read the vignettes without installing the package, and simplifies citing the vignettes in other publications.
Tests
Tests are a fundamental safety net and development tool to ensure that your package works as expected, both during development as well as on user systems.
Write unit tests for all functions and classes in your package, to ensure that all building blocks work correctly on their own (“white box” testing). Expect to rewrite tests for internal code when you refactor it.
Write functional tests for all user facing functionality (“black box” testing). These tests ensure that the user API is stable when refactoring internal code.
In addition, ensure a good coverage of your code with your tests as a final check, but only after having unit and functional tests on all levels of the code.
Functions
Function definitions should be short, simple and enforce argument types with assertions.
Write short functions with less than 50 lines of code for a single and well-defined purpose, with few arguments, and low cyclomatic complexity, in order to reduce the risk of bugs, simplify writing tests and ensure that you can maintain them.
Use type hints or types to explain to the user which argument of the function expects which type of input.
Enforce types and other expected properties of function arguments with assertions, which give an early and readable error message to the user instead of failing function code downstream in a less explicable way.
Style
A consistent and readable code style that is language idiomatic should be used and enforced by style checks.
Use language idiomatic code and follow the “clean code” rules (use descriptive and meaningful names, prefer simpler over more complex code, avoid duplication of code, regularly refactor code), while allowing for exceptions only where needed.
Use a formatting tool to automatically implement a consistent and readable code format.
Use a style checking tool to enforce a consistent and readable code style.
Life cycle
Life cycle management is simplified by reducing dependencies, and should comprise a central code repository.
Reduce dependencies to simplify maintenance of your own package. Only depend on other packages that you trust and deem stable enough for the purpose, in order to avoid reinventing the wheel.
Ensure that you track dependencies and pin their versions so if another developer contributes then they can use the same environment to produce consistent results and behaviours. This could be tracked using more system level approaches like configuring a snapshot date in the package repository to language-specific tools that generate a file that tracks dependencies and versions that serves as a source of truth for all packages developers.
Give clear information to users about changes in the package API via maintaining the change log and first deprecating functionality before removing it.
Use a central repository for version control, collecting and resolving issues, and managing releases. Include the publication of a contributing guide to help onboard new developers and enable community contributions.