Package: OptHoldoutSize 0.1.0.0

OptHoldoutSize: Estimation of Optimal Size for a Holdout Set for Updating a Predictive Score

Predictive scores must be updated with care, because actions taken on the basis of existing risk scores causes bias in risk estimates from the updated score. A holdout set is a straightforward way to manage this problem: a proportion of the population is 'held-out' from computation of the previous risk score. This package provides tools to estimate a size for this holdout set and associated errors. Comprehensive vignettes are included. Please see: Haidar-Wehbe S, Emerson SR, Aslett LJM, Liley J (2022) <arxiv:2202.06374> for details of methods.

Authors:Sami Haidar-Wehbe [aut], Sam Emerson [aut], Louis Aslett [aut], James Liley [cre, aut]

OptHoldoutSize_0.1.0.0.tar.gz
OptHoldoutSize_0.1.0.0.zip(r-4.5)OptHoldoutSize_0.1.0.0.zip(r-4.4)OptHoldoutSize_0.1.0.0.zip(r-4.3)
OptHoldoutSize_0.1.0.0.tgz(r-4.4-any)OptHoldoutSize_0.1.0.0.tgz(r-4.3-any)
OptHoldoutSize_0.1.0.0.tar.gz(r-4.5-noble)OptHoldoutSize_0.1.0.0.tar.gz(r-4.4-noble)
OptHoldoutSize_0.1.0.0.tgz(r-4.4-emscripten)OptHoldoutSize_0.1.0.0.tgz(r-4.3-emscripten)
OptHoldoutSize.pdf |OptHoldoutSize.html
OptHoldoutSize/json (API)
NEWS

# Install 'OptHoldoutSize' in R:
install.packages('OptHoldoutSize', repos = c('https://jamesliley.r-universe.dev', 'https://cloud.r-project.org'))

Peer review:

Datasets:

On CRAN:

This package does not link to any Github/Gitlab/R-forge repository. No issue tracker or development information is available.

3.18 score 10 scripts 161 downloads 30 exports 9 dependencies

Last updated 3 years agofrom:923c183550. Checks:OK: 3 NOTE: 4. Indexed: yes.

TargetResultDate
Doc / VignettesOKNov 13 2024
R-4.5-winNOTENov 13 2024
R-4.5-linuxNOTENov 13 2024
R-4.4-winNOTENov 13 2024
R-4.4-macNOTENov 13 2024
R-4.3-winOKNov 13 2024
R-4.3-macOKNov 13 2024

Exports:add_aspre_interactionsaspreaspre_k2ci_mincostci_ohscov_fnerror_ohs_emulationexp_imp_fngen_base_coefsgen_predsgen_respgrad_mincost_powerlawgrad_nstar_powerlawlogisticlogitmodel_predictmodel_trainmu_fnnext_noptimal_holdout_sizeoptimal_holdout_size_emulationoracle_predpowerlawpowersolvepowersolve_generalpowersolve_sepsi_fnsens10sim_random_aspresplit_data

Dependencies:latticeMatrixmatrixStatsmle.toolsmnormtmvtnormrangerRcppRcppEigen

ASPRE example

Rendered fromASPRE_example.Rmdusingknitr::knitron Nov 13 2024.

Last update: 2022-02-09
Started: 2022-02-09

Comparison of algorithms

Rendered fromcomparison_of_algorithms.Rmdusingknitr::knitron Nov 13 2024.

Last update: 2022-02-18
Started: 2022-02-09

Simulated example

Rendered fromsimulated_example.Rmdusingknitr::knitron Nov 13 2024.

Last update: 2022-02-09
Started: 2022-02-09

Readme and manuals

Help Manual

Help pageTopics
Add interaction terms corresponding to ASPRE modeladd_aspre_interactions
Computes ASPRE scoreaspre
Emulation-based OHS estimation for ASPREaspre_emulation
Cost estimating function in ASPRE simulationaspre_k2
Parametric-based OHS estimation for ASPREaspre_parametric
Data for example on asymptotic confidence interval for OHS.ci_cover_a_yn
Data for example on asymptotic confidence interval for min cost.ci_cover_cost_a_yn
Data for example on empirical confidence interval for min cost.ci_cover_cost_e_yn
Data for example on empirical confidence interval for OHS.ci_cover_e_yn
Confidence interval for minimum total cost, when estimated using parametric methodci_mincost
Confidence interval for optimal holdout size, when estimated using parametric methodci_ohs
Covariance function for Gaussian processcov_fn
Data for vignette showing general exampledata_example_simulation
Data for 'next point' demonstration vignette on algorithm comparison using emulation algorithmdata_nextpoint_em
Data for 'next point' demonstration vignette on algorithm comparison using parametric algorithmdata_nextpoint_par
Measure of error for emulation-based OHS emulationerror_ohs_emulation
Expected improvementexp_imp_fn
Coefficients for imperfect risk scoregen_base_coefs
Generate matrix of random observationsgen_preds
Generate responsegen_resp
Gradient of minimum cost (power law)grad_mincost_powerlaw
Gradient of optimal holdout size (power law)grad_nstar_powerlaw
Logisticlogistic
Logitlogit
Make predictionsmodel_predict
Train model (wrapper)model_train
Updating function for mean.mu_fn
Finds best value of n to sample nextnext_n
Data for vignette on algorithm comparisonohs_array
Data for vignette on algorithm comparisonohs_resample
Estimate optimal holdout size under parametric assumptionsoptimal_holdout_size
Estimate optimal holdout size under semi-parametric assumptionsoptimal_holdout_size_emulation
Generate responsesoracle_pred
Parameters of reported ASPRE datasetparams_aspre
Plot estimated cost functionplot.optholdoutsize
Plot estimated cost function using emulation (semiparametric)plot.optholdoutsize_emul
Power law functionpowerlaw
Fit power law curvepowersolve
General solver for power law curvepowersolve_general
Standard error matrix for learning curve parameters (power law)powersolve_se
Updating function for variance.psi_fn
Sensitivity at theshold quantile 10%sens10
Simulate random dataset similar to ASPRE training datasim_random_aspre
Split datasplit_data