Please review the following if you are preparing to contribute to casteval.
Design goals
We strive to design casteval to be robust, flexible, and modular.
Robustness
casteval shouldn’t break easily, so we build in a lot of input validation and try to give informative error messages/warnings.
Flexibility
casteval should accept a variety of inputs easily and should function for a variety of realistic use cases. The user should not have to do a lot of wrangling of data/parameters to get the software to work for them.
For example:
-
accuracy()
’squant_pairs
argument can accept eitherNULL
, a pair of quantiles, or a list of pairs of quantiles, and it handles all of these inputs in a reasonable fashion. -
plot_forecast()
only requires a forecast as input, but can additionally accept observations, quantile intervals, and a scoring function, depending on what the user seeks to accomplish.
Modularity
It should be easy to combine, swap out, or customize casteval’s outputs.
For example:
- User-defined scoring functions are compatible with the existing plotting functions, as long as they follow the syntax and return format.
- Some of the
plot_*()
functions (e.g.,plot_ensemble()
,plot_observations()
) can be chained together via the pipe operator|>
, which enables the user to customize the order of their their plot layers more easily (compared to simply usingplot_forecast()
).
Setup
renv
This project uses renv
to manage dependencies.
For R package projects, renv
tries to intelligently
discover dependencies by looking at the fields in the DESCRIPTION file
that are relevant for users, like Imports
and
Depends
(see
renv::settings$package.dependency.fields()
). For
developers, the Suggests
field can be equally important to
track as it can include packages used in the process of development but
not in the actual source code of the package, like
devtools
, covr
, to ensure development
environments don’t vary across users.
The renv
FAQ page recommends
tracking development dependencies in Suggests
. It
doesn’t, however, seem to tell you that in order to use their
recommendation, you need to use the dev = TRUE
flag in
renv::status()
to accurately check the status of such a
project, as well as in renv::snapshot()
when recording
files (the default is dev = FALSE
). If you simply execute
renv::snapshot(dev = FALSE)
, renv
will
miss/remove the development dependencies in Suggests
, which
will break our CI pipelines. Once these dependencies have been recorded,
renv::status(dev = FALSE)
will report that the project is
out-of-sync, as it will see installed dependencies that are not used
anywhere (since it skips Suggests
).
The bottom line is to be sure to use the dev = TRUE
option when working in this repository. If you need to add any
development dependencies to the project, be sure to add them to the
Suggests
field of the DESCRIPTION
, run
renv::install(dev = TRUE)
to install them, then
renv::snapshot(dev = TRUE)
to record them.
A side effect of this default behaviour is that, when you open a new
R session in this project, renv::status()
will get called
and report that the project is out-of-sync. Don’t trust this message!
Run renv::status(dev = TRUE)
to check the true status of
the project. (There is a feature request
to enable a user to configure the default behaviour of
renv::status()
on a project-by-project basis specifically
to accommodate the above situation, but this issue is still open as of
this writing on 2024-08-26…)
Workflow
git
There are two important branches in the repository: main
and dev
.
We intend for main
to be the branch that users install
from, and so it should be well-test and free of work-in-progress
code.
dev
is the main development branch that acts as a
staging area for new changes going into main
:
- A developer should start new features or bug fixes by branching from
dev
, with a succinct branch name that describes the branch’s use (e.g.,plot-grouped-forecasts
). - Once work is complete on the feature- or issue-specific branch, the
develop should create a pull request into
dev
. - The package maintainer should then decide on the next release
version of the package (following the semantic versioning paradigm) and accept
the pull request(s) that is/are to be a part of this release into
dev
. - The package maintainer should then pull
main
intodev
to sync and resolve any merge conflicts. - The package maintainer should create a branch from
dev
calledvx.y.z
, wherex.y.z
is the desired version number and then update the version number inDESCRIPTION
as well. This last commit should be tagged with the version number. - The package maintainer should finish by creating and accepting a
pull request from the version-specific branch to
main
and by creating a release (if there is a change in the major or minor version number).
Documentation
Developers can use pkgdown::build_site()
to preview the
entire package website based on the current state of the package
documentation.
It is also useful to use the commands
pkgdown::build_reference()
and
pkgdown::build_articles()
to preview just the Reference
(function documentation) and Articles pages, respectively. However, in
the latter case at least, pkgdown
will use the currently
installed version of the package, not the current source version, so you
may want to run devtools::install()
first if you’re working
on a part of an article that depends on the latest version of the
casteval code.
The results will appear in public/
, which is only for
local previews and should not be pushed to the remote repository (it
should be in .gitignore
already). We rebuild and deploy the
package website automatically with GitHub Actions whenever changes are
pushed to main
.
README
README.md
is used as the README on GitHub as well as on
the landing page for the package website. Its source is
README.Rmd
; any updates to the README content should be
made in the .Rmd
and then README.md
should be
regenerated with devtools::build_readme()
.
(Alternatively, we could create a GitHub Action to build the README before deploying the website…)
Testing
We use testthat
for most unit tests and
vdiffr
(in particular,
vdiffr::expect_doppelganger()
) for tests involving plots as
outputs.
vdiffr
sometimes deletes test snapshots when tests fail,
which is not that great (you might be able to avoid this by putting each
expect_doppelganger()
in a separate
test_that()
block, although that sounds tedious).
Before release
You should use devtools::check()
,
renv::status(dev=TRUE)
, and covr::report()
to
verify that the package is in a good state, especially before releasing
a new version.
-
devtools::check()
checks that the package can be installed, that unit tests pass, and a few other -
renv::status(dev=TRUE)
checks that the virtual environment used for development is in a consistent state -
covr::report()
reports the current test coverage of the package (note that there is a known issue withcovr
andvdiffr
)