# Use domain knowledge to review prior distributions

At the Insurance Data Science conference, both Eric Novik and Paul-Christian Bürkner emphasised in their talks the value of thinking about the data generating process when building Bayesian statistical models. It is also a key step in Michael Betancourt’s Principled Bayesian Workflow.

In this post, I will discuss in more detail how to set priors, and review the prior and posterior parameter distributions, but also the prior predictive distributions with brms ().

The prior predictive distribution shows me how the model behaves before I use my data. Thus, I can check if the model describes the data generating process reasonably well, before I go through the lengthy process of fitting the model.

As an example, I will get back to the non-linear hierarchical growth curve model, which is also part of the brms package vignette on non-linear models.

First I load the relevant packages and data set.

library(rstan)
library(brms)
library(bayesplot)
rstan_options(auto_write = TRUE)
options(mc.cores = parallel::detectCores())
theme_update(text = element_text(family = "sans"))
library(data.table)
url <- "https://raw.githubusercontent.com/mages/diesunddas/master/Data/ClarkTriangle.csv"
loss <- fread(url)

It is the aim to use growth curves to model the claims payments of insurance losses from different accident years (1991 - 2000) over time (development months) and predict future payments.

# Prior predictive distribution

I will start with the same model as in the brms vignette, but instead of fitting the model, I set the parameter sample_prior = "only" to generate samples from the prior predictive distribution only, i.e. the data will be ignored and only the prior distributions will be used.

nlform1 <- bf(cum ~ ult * (1 - exp(-(dev/theta)^omega)),
ult ~ 1 + (1|AY), omega ~ 1, theta ~ 1,
nl = TRUE)
m1 <- brm(nlform1, data = loss, family = gaussian(),
prior = c(
prior(normal(5000, 1000), nlpar = "ult"),
prior(normal(1, 2), nlpar = "omega"),
prior(normal(45, 10), nlpar = "theta")
),
sample_prior = "only", seed = 1234,
)

I can use the same plots to review the prior predictive distribution as I would have done for the posterior predictive distribution.

conditions <- data.frame(AY = unique(loss$AY)) rownames(conditions) <- unique(loss$AY)
me_loss_prior1 <- marginal_effects(
m1, conditions = conditions,
re_formula = NULL, method = "predict"
)
p0 <- plot(me_loss_prior1, ncol = 5, points = TRUE, plot = FALSE)
p0dev + ggtitle("Prior predictive distributions") Perhaps this doesn’t look too bad, but the model generates also many negative claims payments, as I have assumed a Gaussian distribution. Yet, customers rarely pay claims back to the insurance company. In addition, this model simplifies the variance structure to a constant $$\sigma^2$$. In () Jim Guszcza suggested a variance that is proportional to the claims amount. ## Review model code and priors Let’s look at a part of the brm-generated Stan code that describes the priors to better understand the model:  // priors including all constants target += normal_lpdf(b_ult | 5000, 1000); target += normal_lpdf(b_omega | 1, 2); target += normal_lpdf(b_theta | 45, 10); target += student_t_lpdf(sigma | 3, 0, 1964) - 1 * student_t_lccdf(0 | 3, 0, 1964); target += student_t_lpdf(sd_1 | 3, 0, 1964) - 1 * student_t_lccdf(0 | 3, 0, 1964); target += normal_lpdf(z_1[1] | 0, 1); // likelihood including all constants if (!prior_only) { target += normal_lpdf(Y | mu, sigma); }  The code shows the default priors set by brm for sigma and sd_1, which I didn’t set explicitly. In addition, the last line shows the switch that turns off sampling from the posterior distributions. Putting the model in ‘Greek’ makes it more readable: \begin{align} loss_{AY}(t) & \sim \mathcal{N}\left( \mu_{AY}(t), \sigma^2\right) \\ \mu_{AY}(t) & = \gamma_{AY} \cdot G(t; \omega, \theta)\\ G(t; \omega, \theta) & = 1 - \exp\left(-\left({t/\theta}\right)^\omega\right)\\ \gamma_{AY} & = \gamma + \gamma_{AY}^0 \\ \mbox{Priors}&:\\ \gamma & \sim \mathcal{N}(5000, 1000)\\ \omega & \sim \mathcal{N}\left(1, 2^2\right)\\ \theta & \sim \mathcal{N}\left(45, 10^2\right) \\ \sigma & \sim \mbox{Student-t}\left(3, 0, 1964\right)^+\\ \gamma_{AY}^0 & \sim \mathcal{N}(0, \sigma_{\gamma^0}^2) \\ \sigma_{\gamma^0} & \sim \mbox{Student-t}\left(3, 0, 1964\right)^+\\ \end{align} # Utilise domain knowledge There are a few aspects that I would like to change to the original model: • Use the loss ratio instead of the loss amounts to make data from the different years more comparable and to bring values closer to 1 • Change the distribution family from Gaussian to lognormal to avoid negative payments being generated • Assume a constant $$\sigma$$ parameter, which means a constant coefficient of variation in case of the lognormal distribution, i.e. the standard deviation is proportional to the mean • Use development year instead of development month to reduce the scale of $$t$$ • Use parameter $$\phi=1/\theta$$, since I believe $$0< \phi < 1$$. • Shrink uncertainty and shift $$\omega$$, as I believe $$\omega$$ will be around 1.25 • With the move to loss ratios the standard deviations $$\sigma_{\gamma^0}$$ and $$\sigma$$ will be much smaller. Therefore, I reduce the scale parameter and increase the degrees of freedoms from 3 to 5 for the Student-t distribution My new model looks like this now: \begin{align} \ell_{AY}(t) & \sim \log \mathcal{N}\left(\log(\mu_{AY}(t)), \sigma^2\right) \\ \mu_{AY}(t) & = \gamma_{AY} \cdot G(t; \omega, \phi)\\ G(t;\omega, \phi) & = 1 - \exp\left(-\left(t\cdot \phi\right)^\omega\right)\\ \gamma_{AY} & = \gamma + \gamma_{AY}^0 \\ \mbox{Priors}&:\\ \gamma & \sim \log \mathcal{N}\left(\log(0.5), \log(1.2)^2\right)\\ \omega & \sim \mathcal{N}\left(1.25, 0.25^2\right)^+\\ \phi & \sim \mathcal{N}\left(0.25, 0.25^2\right)^+ \\ \sigma & \sim \mbox{Student-t}\left(5, 0, 0.25\right)^+\\ \gamma_{AY}^0 & \sim \mathcal{N}(0, \sigma_{\gamma^0}^2) \\ \sigma_{\gamma^0} & \sim \mbox{Student-t}\left(5, 0, 0.25\right)^+\\ \end{align} Assuming a constant coefficient of variation across development time seems broadly okay: loss <- loss[, :=(loss_ratio = cum/losspremium,
dev_year = (dev+6)/12)]
print(
loss[, list(CV(loss_ratio) = sd(loss_ratio)/mean(loss_ratio)), by=dev_year],
digits=1)
##     dev_year CV(loss_ratio)
##  1:        1           0.14
##  2:        2           0.08
##  3:        3           0.08
##  4:        4           0.15
##  5:        5           0.13
##  6:        6           0.09
##  7:        7           0.11
##  8:        8           0.14
##  9:        9           0.21
## 10:       10             NA

To review if my new model makes sense I start by generating samples from the priors again:

nlform2 <- bf(loss_ratio ~ log(ulr * (1 - exp(-(dev_year*phi)^omega))),
ulr ~ 1 + (1|AY), omega ~ 1, phi ~ 1,
nl = TRUE)
m2 <- brm(nlform2, data = loss,
prior = c(
prior(lognormal(log(0.5), log(1.2)), nlpar = "ulr", lb=0),
prior(normal(1.25, 0.25), nlpar = "omega", lb=0),
prior(normal(0.25, 0.25), nlpar = "phi", lb=0),
prior(student_t(5, 0, 0.25), class = "sigma"),
prior(student_t(5, 0, 0.25), class = "sd", nlpar="ulr")
),
sample_prior = "only", seed = 1234
)

## Prior parameter distributions

library(bayesplot)
theme_update(text = element_text(family = "sans"))
mcmc_areas(
as.array(m2),
pars = c("b_ulr_Intercept", "b_omega_Intercept",
"b_phi_Intercept",
"sd_AY__ulr_Intercept", "sigma"),
prob = 0.8, # 80% intervals
prob_outer = 0.99, # 99%
point_est = "mean"
) + ggplot2::labs(
title = "Prior parameter distributions",
subtitle = "with medians and 80% intervals"
)

The prior parameter distributions look very similar to what I would expect, given the plot above.

## Prior predictive distributions

me_loss_prior2 <- marginal_effects(
m2, conditions = conditions,
re_formula = NULL, method = "predict"
)
p1 <- plot(me_loss_prior2, ncol = 5, points = TRUE, plot = FALSE)

# Conclusions

The plot looks good, but the model underestimates the claims development for the 1992 and 1993 accident years.

• Are those years and the data outliers?
• Should the model allow the parameters $$\omega$$ and $$\theta$$ to vary by year?
• Should I consider a different growth curve family?

Tapping into expert knowledge can be invaluable. However, many domain knowledge experts will not be statisticians and will find it difficult to form an opinion on the ‘Greek’ model or the parameters.

Yet, often experts will have a view on the data generated by the model. Showing them the output of the prior and posterior predictive distributions can be a lot more fruitful.

And as I said earlier, don’t forget they are experts, and hence like/likely to disagree with the non-expert. Use this bias to your advantage!

# Update

Jake and I published a new research paper on Hierarchical Compartmental Reserving Models:

Gesmann, M., and Morris, J. “Hierarchical Compartmental Reserving Models.” Casualty Actuarial Society, CAS Research Papers, 19 Aug. 2020, https://www.casact.org/sites/default/files/2021-02/compartmental-reserving-models-gesmannmorris0820.pdf

# Session Info

sessionInfo()
## R version 4.0.3 (2020-10-10)
## Platform: x86_64-apple-darwin17.0 (64-bit)
## Running under: macOS Big Sur 10.16
##
## Matrix products: default
## BLAS:   /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRblas.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib
##
## locale:
## [1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8
##
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base
##
## other attached packages:
## [1] latticeExtra_0.6-29  lattice_0.20-41      data.table_1.13.6
## [4] bayesplot_1.7.2      brms_2.14.4          Rcpp_1.0.5
##
## loaded via a namespace (and not attached):
##   [1] minqa_1.2.4          colorspace_2.0-0     ellipsis_0.3.1
##   [4] ggridges_0.5.2       rsconnect_0.8.16     markdown_1.1
##   [7] base64enc_0.1-3      farver_2.0.3         DT_0.16
##  [10] fansi_0.4.1          mvtnorm_1.1-1        bridgesampling_1.0-0
##  [13] codetools_0.2-18     splines_4.0.3        knitr_1.30
##  [16] shinythemes_1.1.2    projpred_2.0.2       jsonlite_1.7.2
##  [19] nloptr_1.2.2.2       png_0.1-7            shiny_1.5.0
##  [22] compiler_4.0.3       backports_1.2.1      assertthat_0.2.1
##  [25] Matrix_1.3-0         fastmap_1.0.1        cli_2.2.0
##  [28] later_1.1.0.1        htmltools_0.5.0      prettyunits_1.1.1
##  [31] tools_4.0.3          igraph_1.2.6         coda_0.19-4
##  [34] gtable_0.3.0         glue_1.4.2           reshape2_1.4.4
##  [37] dplyr_1.0.2          V8_3.4.0             vctrs_0.3.6
##  [40] nlme_3.1-151         blogdown_0.21        crosstalk_1.1.0.1
##  [43] xfun_0.19            stringr_1.4.0        ps_1.5.0
##  [46] lme4_1.1-26          mime_0.9             miniUI_0.1.1.1
##  [49] lifecycle_0.2.0      gtools_3.8.2         statmod_1.4.35
##  [52] MASS_7.3-53          zoo_1.8-8            scales_1.1.1
##  [55] colourpicker_1.1.0   promises_1.1.1       Brobdingnag_1.2-6
##  [58] parallel_4.0.3       inline_0.3.17        shinystan_2.5.0
##  [61] RColorBrewer_1.1-2   gamm4_0.2-6          yaml_2.2.1
##  [64] curl_4.3             gridExtra_2.3        loo_2.4.1
##  [67] stringi_1.5.3        dygraphs_1.1.1.6     boot_1.3-25
##  [70] pkgbuild_1.2.0       rlang_0.4.10         pkgconfig_2.0.3
##  [73] matrixStats_0.57.0   evaluate_0.14        purrr_0.3.4
##  [76] labeling_0.4.2       rstantools_2.1.1     htmlwidgets_1.5.3
##  [79] processx_3.4.5       tidyselect_1.1.0     plyr_1.8.6
##  [82] magrittr_2.0.1       bookdown_0.21        R6_2.5.0
##  [85] generics_0.1.0       pillar_1.4.7         withr_2.3.0
##  [88] mgcv_1.8-33          xts_0.12.1           abind_1.4-5
##  [91] tibble_3.0.4         crayon_1.3.4         rmarkdown_2.6
##  [94] jpeg_0.1-8.1         grid_4.0.3           callr_3.5.1
##  [97] threejs_0.3.3        digest_0.6.27        xtable_1.8-4
## [100] httpuv_1.5.4         RcppParallel_5.0.2   stats4_4.0.3
## [103] munsell_0.5.0        shinyjs_2.0.0

# References

Bürkner, Paul-Christian. 2017. brms: An R Package for Bayesian Multilevel Models Using Stan.” Journal of Statistical Software 80 (1): 1–28. https://doi.org/10.18637/jss.v080.i01.
Guszcza, James. 2008. “Hierarchical Growth Curve Models for Loss Reserving.” In, 146–73. CAS Fall Forum; https://www.casact.org/pubs/forum/08fforum/7Guszcza.pdf.

### Citation

Markus Gesmann (Aug 02, 2018) Use domain knowledge to review prior distributions. Retrieved from https://magesblog.com/post/2018-08-02-use-domain-knowledge-to-review-prior-predictive-distributions/

BibTeX citation:

@misc{ 2018-use-domain-knowledge-to-review-prior-distributions,
author = { Markus Gesmann },
title = { Use domain knowledge to review prior distributions },
url = { https://magesblog.com/post/2018-08-02-use-domain-knowledge-to-review-prior-predictive-distributions/ },
year = { 2018 }
updated = { Aug 02, 2018 }
}