Objectives
The completed research programme separates three simulation questions:
- Simulation A compares covariance estimators while holding DR fixed.
- Simulation B1 compares SDR subspace estimation while holding OAS fixed.
- Simulation B2 compares structural dimension recovery while holding OAS fixed.
This separation avoids declaring one method universally best when the evaluation objectives differ.
Data-generating mechanism
simulate_risdr_data() generates
and constructs a response from the sufficient predictors
\mathbf{B}^{\mathsf{T}}\mathbf{X}.
library(risdr)
sim <- simulate_risdr_data(
n = 120,
p = 30,
d = 2,
rho = 0.8,
sigma = 1,
model = "linear_quadratic",
beta_type = "coordinate",
seed = 9201
)
dim(sim$X)
#> [1] 120 30
crossprod(sim$beta)
#> beta1 beta2
#> beta1 1 0
#> beta2 0 1A single replication
one <- run_one_simulation(
n = 100,
p = 20,
d = 2,
rho = 0.6,
sigma = 1,
model = "linear_quadratic",
sdr_method = "dr",
cov_method = "oas",
nslices = 6,
d_max = 5,
seed = 9202
)
one
#> n p true_d selected_d rho sigma model sdr_method cov_method
#> 1 100 20 2 4 0.6 1 linear_quadratic dr oas
#> selector subspace_distance RMSE MAE MAPE R2 Adjusted_R2
#> 1 cicomp 1.636901 2.12264 1.496609 339.7211 0.04941063 0.009385819
#> Correlation
#> 1 0.3081921Subspace distance is the Frobenius distance between the projection matrices:
The measure is invariant to rotations and sign changes within an estimated subspace.
A small reproducible study
small_study <- run_risdr_simulation(
R = 2,
rho_values = c(0.3, 0.8),
methods = c("sir", "dr"),
cov_methods = c("ridge", "oas"),
n = 80,
p = 15,
d = 2,
nslices = 5,
d_max = 4,
seed = 9203
)
summarise_simulation(small_study)
#> rho sdr_method cov_method subspace_distance RMSE MAE R2
#> 1 0.3 dr oas 1.497544 1.999751 1.568484 0.10707876
#> 2 0.8 dr oas 1.604443 1.962117 1.371217 -0.04389550
#> 3 0.3 sir oas 1.584844 1.989832 1.492512 0.09150760
#> 4 0.8 sir oas 1.723671 2.154795 1.591799 0.04424048
#> 5 0.3 dr ridge 1.805062 2.156424 1.672917 -0.10393320
#> 6 0.8 dr ridge 1.629352 2.116198 1.591837 0.16768560
#> 7 0.3 sir ridge 1.578103 1.723998 1.329183 0.05391467
#> 8 0.8 sir ridge 1.605170 2.134675 1.448731 -0.02267343
#> Adjusted_R2 Correlation
#> 1 0.09563105 0.33874075
#> 2 -0.05727878 0.39308889
#> 3 0.07389137 0.35729563
#> 4 0.03198715 0.35573130
#> 5 -0.13370648 0.09270968
#> 6 0.15024652 0.42669004
#> 7 0.04178537 0.43541348
#> 8 -0.03578462 0.32394481The thesis simulations use substantially more replications. Small values are used here only to keep package checks proportionate.
Legacy Simulation A fixture
simulation_a <- utils::read.csv(system.file(
"extdata",
"simulation",
"simulation_A_final_covariance_DR_summary_tidy.csv",
package = "risdr"
))
aggregate(
cbind(
mean_subspace_distance,
mean_RMSE,
mean_condition_number,
mean_runtime_seconds
) ~ cov_method,
data = simulation_a,
FUN = mean
)
#> cov_method mean_subspace_distance mean_RMSE mean_condition_number
#> 1 LW 1.894428 2.309378 2.134967e+02
#> 2 MEC 1.965678 2.704028 1.713922e+07
#> 3 OAS 1.895389 2.293333 2.123284e+02
#> 4 RIDGE 1.897106 2.388906 1.390600e+02
#> 5 SAMPLE 1.950800 3.732978 9.276394e+06
#> mean_runtime_seconds
#> 1 0.3466222
#> 2 0.3374833
#> 3 0.2410333
#> 4 0.2114944
#> 5 0.2414333OAS and ridge produce the lowest average condition numbers in the supplied Simulation A record. Its external-prediction columns require a corrected rerun under a shared true basis.
Simulation B1 fixture
ranking_b1 <- utils::read.csv(system.file(
"extdata",
"simulation",
"simulation_B1_overall_ranking.csv",
package = "risdr"
))
ranking_b1[order(ranking_b1$avg_subspace_distance), ]
#> sdr_method avg_subspace_distance avg_RMSE avg_R2 avg_condition_number
#> 1 DR 1.895500 2.309850 -0.35607222 211.3530
#> 2 SIR 1.895578 2.455239 -0.53662222 211.5089
#> 3 PHD 1.912528 2.168078 -0.19206111 212.3132
#> 4 SAVE 1.937872 2.078517 -0.07530556 210.7767
#> avg_runtime_seconds
#> 1 0.27097778
#> 2 0.08057778
#> 3 0.09651667
#> 4 0.17836667DR has the best average subspace ranking in this supplied summary, although the margin over SIR is small and scenario-level results should also be examined. This legacy table is retained for traceability pending the targeted rerun.
Simulation B2 fixture
ranking_b2 <- utils::read.csv(system.file(
"extdata",
"simulation",
"simulation_B2_overall_ranking.csv",
package = "risdr"
))
ranking_b2[
order(ranking_b2$avg_dimension_recovery_rate, decreasing = TRUE),
]
#> sdr_method avg_dimension_recovery_rate avg_selected_d avg_sd_selected_d
#> 1 DR 0.290833333 2.146667 1.16188889
#> 2 PHD 0.207500000 2.266667 1.34311111
#> 3 SIR 0.167777778 2.720000 1.23361111
#> 4 SAVE 0.004166667 1.002778 0.04033333
#> avg_condition_number avg_runtime_seconds
#> 1 210.5465 0.46459444
#> 2 210.9296 0.09940000
#> 3 211.0959 0.08013333
#> 4 211.7987 0.16837778DR attains an average structural dimension recovery rate of approximately 29.1 per cent in the supplied B2 ranking. Recovery remains difficult across the complete design, so the result should be interpreted comparatively rather than as evidence of uniformly accurate dimension identification. B2 uses one training sample per replication and is not affected by the train-test basis mismatch, but it should be regenerated alongside the corrected simulation package for a single auditable record.
Reproducibility requirements
The repository includes analysis/reproduce_simulations.R
and config.yml. The corrected workflow reuses each training
basis in its corresponding test sample while preserving the original
scenario order, test predictors, error draws, and seed rule. It writes
outputs with a _corrected_v0_3_0 suffix. From the
repository root, run:
The default configuration retains the original scenario grid and 200 replications. A copied configuration with one replication can be used for a preflight smoke run.
Each reported simulation should retain:
- the full scenario grid;
- the master seed and replication-specific seed rule;
- successful and failed replications;
- full replication-level outputs;
- the aggregation script;
- software and package versions;
- runtime and numerical-conditioning diagnostics.