{mmrm}

A Robust and Comprehensive R Package for Implementing Mixed Models for Repeated Measures

Ya Wang

on Behalf of ASA BIOP Software Engineering Working Group (SWE WG)

2023-08-08

Outline

  • Introduction to SWE WG
  • Motivation: why do we need a package for MMRM?
  • mmrm Package
    • Why this is not “yet another package”
    • Comparing mmrm package to SAS
    • Benchmarking with other R packages
  • Closing and next steps

Software Engineering working group (SWE WG)

  • Official working group of the ASA Biopharmaceutical Section
    • Formed in August 2022
    • Cross-industry collaboration (>35 members from >25 organizations)
  • Goal
    • Engineer selected R packages to fill in gaps in the open-source statistical software landscape
    • Develop and disseminate best practices for engineering high-quality open-source statistical software
  • Homepage: rconsortium.github.io/asa-biop-swe-wg

Motivation: why do we need a package for MMRM

  • Mixed Models for Repeated Measures (MMRM) is a popular choice for analyzing longitudinal continuous outcomes in randomized clinical trials
  • No great R Package
    • Initially thought that the MMRM problem was solved by using lme4 with lmerTest, learned that this approach failed on large data sets (slow, did not converge)
    • nlme does not give Satterthwaite adjusted degrees of freedom, has convergence issues, and with emmeans it is only approximate
    • Next we tried to extend glmmTMB to calculate Satterthwaite adjusted degrees of freedom, but it did not work

Idea

  • We only want to fit a fixed effects model with a structured covariance matrix for each subject
  • The idea is then to use the Template Model Builder (TMB) directly - as it is also underlying glmmTMB - but code the exact model we want
  • We do this by implementing the log-likelihood in C++ using the TMB provided libraries

Advantages of TMB

  • Fast C++ framework for defining objective functions (Rcpp would have been alternative interface)
  • Automatic differentiation of the log-likelihood as a function of the variance parameters
  • We get the gradient and Hessian exactly and without additional coding
  • This can be used from the R side with the TMB interface and plugged into optimizers

Why it’s not just another package

  • Ongoing maintenance and support from the pharmaceutical industry
    • 5 companies being involved in the development, on track to become standard package
  • Development using best practices as show case for high quality package
    • Thorough unit and integration tests (also comparing with SAS results) to ensure accurate results

Features of mmrm

  • Linear model for dependent observations within independent subjects
  • Covariance structures for the dependent observations:
    • Unstructured, Toeplitz, AR1, compound symmetry, ante-dependence, spatial exponential
    • Allows group specific covariance estimates and weights
  • REML or ML estimation, using multiple optimizers if needed
  • emmeans interface for least square means
  • tidymodels for easy model fitting
  • Satterthwaite and Kenward-Roger adjustments
  • Robust sandwich estimator for covariance

Comparing SAS and R

  • To run an MMRM model in SAS it is recommended to use either the PROC MIXED or PROC GLM procedures.
  • Less model assumptions are applied in PROC MIXED, primarily how one treats missingness.
  • We will compare the PROC MIXED procedure to the mmrm package in the following attributes:
  • Documentation
  • Unit Testing
  • Estimation Methods
  • Covariance Structures
  • Degrees of Freedom
  • Contrasts

Documentation

Both languages have online documentation of the technical details of the estimation and degrees of freedom methods and the different covariance structures available.

{mmrm}

PROC MIXED

Unit Testing

  • Unit tests in mmrm are transparent, compared to PROC MIXED.
  • Uses the testthat framework with covr to communicate the testing coverage.
  • Unit tests can be found in the GitHub repository under ./tests.

Note

The integration tests in mmrm are set to a tolerance of 10e^-3 when compared to SAS outputs.

Estimation Methods

Method {mmrm} PROC MIXED
ML X X
REML X X

Covariance structures

  • SAS has 23 non-spatial covariance structures, while mmrm has 10.
    • 9 structures intersect with SAS
    • Ante-dependence (homogeneous) is only in the mmrm package.
  • SAS has 14 spatial covariance structures compared to the spatial exponential one available in mmrm.

Tip

For users that need more structures, mmrm is easily extensible - let us know via feature requests in the GitHub repository.

Covariance structures details

Covariance structures {mmrm} PROC MIXED
Unstructured (Unweighted/Weighted) X/X X/X
Toeplitz (hetero/homo) X/X X/X
Compound symmetry (hetero/homo) X/X X/X
Auto-regressive (hetero/homo) X/X X/X
Ante-dependence (hetero/homo) X/X X
Spatial exponential X X

Degrees of Freedom Methods

Method {mmrm} PROC MIXED
Contain X* X
Between/Within X* X
Residual X* X
Satterthwaite X X
Kenward-Roger X X
Kenward-Roger (Linear)** X X

*Available through the emmeans package.

**This is not identical to the KR(Lin) in PROC MIXED

Contrasts/LSMEANS

Contrasts and least square means estimates are available in mmrm using:

  • mmrm::df_1d, mmrm::df_md
  • S3 method that is compatible with emmeans
  • LS means difference can be produced through emmeans (pairs method)
  • Degrees of freedom method is passed from mmrm to emmeans
  • By default PROC MIXED and mmrm do not adjust for multiplicity, whereas emmeans does.

Benchmarking with other R packages

  • Compared mmrm::mmrm with nlme::gls, lme4::lmer, glmmTMB::glmmTMB
  • Highlights
    • mmrm has faster convergence time
      • Using FEV dataset as an example, mmrm took ~50 ms, while lmer ~200 ms, gls and glmmTMB >600 ms
    • mmrm and gls estimates have smaller differences from SAS PROC MIXED estimates
    • mmrm and gls are more resilient to missingness
  • Detailed results at the online comparison vignette

Impact of mmrm

Outlook

  • We still have major features on our backlog
    • Type II and type III ANOVA tests
  • CRAN version 0.3 is coming soon
    • Add argument covariance, different covariance estimators (empirical, empirical Jackknife, empirical bias-reduced), predict method, etc
  • Please let us know what is missing in mmrm for you! And try it out :-)

Thank you! Questions?