All Possible Regression
All subset regression tests all possible subsets of the set of potential independent variables. If there are K potential independent variables (besides the constant), then there are \(2^{k}\) distinct subsets of them to be tested. For example, if you have 10 candidate independent variables, the number of subsets to be tested is \(2^{10}\), which is 1024, and if you have 20 candidate variables, the number is \(2^{20}\), which is more than one million.
model <- lm(mpg ~ disp + hp + wt + qsec, data = mtcars)
ols_step_all_possible(model)## Index N Predictors R-Square Adj. R-Square Mallow's Cp
## 3 1 1 wt 0.7528328 0.7445939 0.70869536
## 1 2 1 disp 0.7183433 0.7089548 0.67512054
## 2 3 1 hp 0.6024373 0.5891853 0.50969578
## 4 4 1 qsec 0.1752963 0.1478062 0.07541973
## 8 5 2 hp wt 0.8267855 0.8148396 0.78108710
## 10 6 2 wt qsec 0.8264161 0.8144448 0.77856272
## 6 7 2 disp wt 0.7809306 0.7658223 0.72532105
## 5 8 2 disp hp 0.7482402 0.7308774 0.69454380
## 7 9 2 disp qsec 0.7215598 0.7023571 0.66395284
## 9 10 2 hp qsec 0.6368769 0.6118339 0.52014395
## 14 11 3 hp wt qsec 0.8347678 0.8170643 0.78199548
## 11 12 3 disp hp wt 0.8268361 0.8082829 0.76789526
## 13 13 3 disp wt qsec 0.8264170 0.8078189 0.76988533
## 12 14 3 disp hp qsec 0.7541953 0.7278591 0.68301440
## 15 15 4 disp hp wt qsec 0.8351443 0.8107212 0.77102968The plot method shows the panel of fit criteria for all
possible regression methods.
model <- lm(mpg ~ disp + hp + wt + qsec, data = mtcars)
k <- ols_step_all_possible(model)
plot(k)
Best Subset Regression
Select the subset of predictors that do the best at meeting some well-defined objective criterion, such as having the largest R2 value or the smallest MSE, Mallow’s Cp or AIC.
model <- lm(mpg ~ disp + hp + wt + qsec, data = mtcars)
ols_step_best_subset(model)## Best Subsets Regression
## ------------------------------
## Model Index Predictors
## ------------------------------
## 1 wt
## 2 hp wt
## 3 hp wt qsec
## 4 disp hp wt qsec
## ------------------------------
##
## Subsets Regression Summary
## ---------------------------------------------------------------------------------------------------------------------------------
## Adj. Pred
## Model R-Square R-Square R-Square C(p) AIC SBIC SBC MSEP FPE HSP APC
## ---------------------------------------------------------------------------------------------------------------------------------
## 1 0.7528 0.7446 0.7087 12.4809 166.0294 74.2916 170.4266 296.9167 9.8572 0.3199 0.2801
## 2 0.8268 0.8148 0.7811 2.3690 156.6523 66.5755 162.5153 215.5104 7.3563 0.2402 0.2091
## 3 0.8348 0.8171 0.782 3.0617 157.1426 67.7238 164.4713 213.1929 7.4756 0.2461 0.2124
## 4 0.8351 0.8107 0.771 5.0000 159.0696 70.0408 167.8640 220.8882 7.9497 0.2644 0.2259
## ---------------------------------------------------------------------------------------------------------------------------------
## AIC: Akaike Information Criteria
## SBIC: Sawa's Bayesian Information Criteria
## SBC: Schwarz Bayesian Criteria
## MSEP: Estimated error of prediction, assuming multivariate normality
## FPE: Final Prediction Error
## HSP: Hocking's Sp
## APC: Amemiya Prediction CriteriaThe plot method shows the panel of fit criteria for best
subset regression methods.
model <- lm(mpg ~ disp + hp + wt + qsec, data = mtcars)
k <- ols_step_best_subset(model)
plot(k)
Stepwise Forward Regression
Build regression model from a set of candidate predictor variables by
entering predictors based on p values, in a stepwise manner until there
is no variable left to enter any more. The model should include all the
candidate predictor variables. If details is set to TRUE,
each step is displayed.
Variable Selection
# stepwise forward regression
model <- lm(y ~ ., data = surgical)
ols_step_forward_p(model)##
##
## Stepwise Summary
## --------------------------------------------------------------------------
## Step Variable AIC SBC SBIC R2 Adj. R2
## --------------------------------------------------------------------------
## 0 Base Model 802.606 806.584 646.794 0.00000 0.00000
## 1 liver_test 771.875 777.842 616.009 0.45454 0.44405
## 2 alc_heavy 761.439 769.395 605.506 0.56674 0.54975
## 3 enzyme_test 750.509 760.454 595.297 0.65900 0.63854
## 4 pindex 735.715 747.649 582.943 0.75015 0.72975
## 5 bcs 730.620 744.543 579.638 0.78091 0.75808
## --------------------------------------------------------------------------
##
## Final Model Output
## ------------------
##
## Model Summary
## -------------------------------------------------------------------
## R 0.884 RMSE 184.276
## R-Squared 0.781 MSE 38202.426
## Adj. R-Squared 0.758 Coef. Var 27.839
## Pred R-Squared 0.700 AIC 730.620
## MAE 137.656 SBC 744.543
## -------------------------------------------------------------------
## RMSE: Root Mean Square Error
## MSE: Mean Square Error
## MAE: Mean Absolute Error
## AIC: Akaike Information Criteria
## SBC: Schwarz Bayesian Criteria
##
## ANOVA
## -----------------------------------------------------------------------
## Sum of
## Squares DF Mean Square F Sig.
## -----------------------------------------------------------------------
## Regression 6535804.090 5 1307160.818 34.217 0.0000
## Residual 1833716.447 48 38202.426
## Total 8369520.537 53
## -----------------------------------------------------------------------
##
## Parameter Estimates
## ------------------------------------------------------------------------------------------------
## model Beta Std. Error Std. Beta t Sig lower upper
## ------------------------------------------------------------------------------------------------
## (Intercept) -1178.330 208.682 -5.647 0.000 -1597.914 -758.746
## liver_test 58.064 40.144 0.156 1.446 0.155 -22.652 138.779
## alc_heavy 317.848 71.634 0.314 4.437 0.000 173.818 461.878
## enzyme_test 9.748 1.656 0.521 5.887 0.000 6.419 13.077
## pindex 8.924 1.808 0.380 4.935 0.000 5.288 12.559
## bcs 59.864 23.060 0.241 2.596 0.012 13.498 106.230
## ------------------------------------------------------------------------------------------------
Detailed Output
# stepwise forward regression
model <- lm(y ~ ., data = surgical)
ols_step_forward_p(model, details = TRUE)## Forward Selection Method
## ------------------------
##
## Candidate Terms:
##
## 1. bcs
## 2. pindex
## 3. enzyme_test
## 4. liver_test
## 5. age
## 6. gender
## 7. alc_mod
## 8. alc_heavy
##
##
## Step => 0
## Model => y ~ 1
## R2 => 0
##
## Initiating stepwise selection...
##
## Selection Metrics Table
## -----------------------------------------------------------------
## Predictor Pr(>|t|) R-Squared Adj. R-Squared AIC
## -----------------------------------------------------------------
## liver_test 0.00000 0.455 0.444 771.875
## enzyme_test 0.00000 0.334 0.322 782.629
## pindex 0.00155 0.177 0.161 794.100
## alc_heavy 0.00172 0.174 0.158 794.301
## bcs 0.01025 0.120 0.103 797.697
## alc_mod 0.19286 0.032 0.014 802.828
## gender 0.20972 0.030 0.011 802.956
## age 0.39073 0.014 -0.005 803.834
## -----------------------------------------------------------------
##
## Step => 1
## Selected => liver_test
## Model => y ~ liver_test
## R2 => 0.455
##
## Selection Metrics Table
## -----------------------------------------------------------------
## Predictor Pr(>|t|) R-Squared Adj. R-Squared AIC
## -----------------------------------------------------------------
## alc_heavy 0.00065 0.567 0.550 761.439
## enzyme_test 0.00089 0.562 0.544 762.077
## pindex 0.07087 0.489 0.469 770.387
## alc_mod 0.10979 0.481 0.461 771.141
## gender 0.79395 0.455 0.434 773.802
## age 0.83908 0.455 0.434 773.831
## bcs 0.93062 0.455 0.433 773.867
## -----------------------------------------------------------------
##
## Step => 2
## Selected => alc_heavy
## Model => y ~ liver_test + alc_heavy
## R2 => 0.567
##
## Selection Metrics Table
## -----------------------------------------------------------------
## Predictor Pr(>|t|) R-Squared Adj. R-Squared AIC
## -----------------------------------------------------------------
## enzyme_test 0.00057 0.659 0.639 750.509
## pindex 0.00961 0.622 0.599 756.125
## bcs 0.55687 0.570 0.544 763.063
## age 0.58269 0.569 0.544 763.110
## alc_mod 0.91757 0.567 0.541 763.428
## gender 0.93799 0.567 0.541 763.433
## -----------------------------------------------------------------
##
## Step => 3
## Selected => enzyme_test
## Model => y ~ liver_test + alc_heavy + enzyme_test
## R2 => 0.659
##
## Selection Metrics Table
## ---------------------------------------------------------------
## Predictor Pr(>|t|) R-Squared Adj. R-Squared AIC
## ---------------------------------------------------------------
## pindex 1e-04 0.750 0.730 735.715
## bcs 0.21294 0.670 0.643 750.782
## alc_mod 0.75743 0.660 0.632 752.403
## age 0.77290 0.660 0.632 752.416
## gender 0.99197 0.659 0.631 752.509
## ---------------------------------------------------------------
##
## Step => 4
## Selected => pindex
## Model => y ~ liver_test + alc_heavy + enzyme_test + pindex
## R2 => 0.75
##
## Selection Metrics Table
## ---------------------------------------------------------------
## Predictor Pr(>|t|) R-Squared Adj. R-Squared AIC
## ---------------------------------------------------------------
## bcs 0.01248 0.781 0.758 730.620
## age 0.86220 0.750 0.724 737.680
## gender 0.96390 0.750 0.724 737.712
## alc_mod 0.97040 0.750 0.724 737.713
## ---------------------------------------------------------------
##
## Step => 5
## Selected => bcs
## Model => y ~ liver_test + alc_heavy + enzyme_test + pindex + bcs
## R2 => 0.781
##
## Selection Metrics Table
## ---------------------------------------------------------------
## Predictor Pr(>|t|) R-Squared Adj. R-Squared AIC
## ---------------------------------------------------------------
## age 0.74164 0.781 0.754 732.494
## gender 0.80666 0.781 0.753 732.551
## alc_mod 0.94086 0.781 0.753 732.614
## ---------------------------------------------------------------
##
##
## No more variables to be added.
##
## Variables Selected:
##
## => liver_test
## => alc_heavy
## => enzyme_test
## => pindex
## => bcs##
##
## Stepwise Summary
## --------------------------------------------------------------------------
## Step Variable AIC SBC SBIC R2 Adj. R2
## --------------------------------------------------------------------------
## 0 Base Model 802.606 806.584 646.794 0.00000 0.00000
## 1 liver_test 771.875 777.842 616.009 0.45454 0.44405
## 2 alc_heavy 761.439 769.395 605.506 0.56674 0.54975
## 3 enzyme_test 750.509 760.454 595.297 0.65900 0.63854
## 4 pindex 735.715 747.649 582.943 0.75015 0.72975
## 5 bcs 730.620 744.543 579.638 0.78091 0.75808
## --------------------------------------------------------------------------
##
## Final Model Output
## ------------------
##
## Model Summary
## -------------------------------------------------------------------
## R 0.884 RMSE 184.276
## R-Squared 0.781 MSE 38202.426
## Adj. R-Squared 0.758 Coef. Var 27.839
## Pred R-Squared 0.700 AIC 730.620
## MAE 137.656 SBC 744.543
## -------------------------------------------------------------------
## RMSE: Root Mean Square Error
## MSE: Mean Square Error
## MAE: Mean Absolute Error
## AIC: Akaike Information Criteria
## SBC: Schwarz Bayesian Criteria
##
## ANOVA
## -----------------------------------------------------------------------
## Sum of
## Squares DF Mean Square F Sig.
## -----------------------------------------------------------------------
## Regression 6535804.090 5 1307160.818 34.217 0.0000
## Residual 1833716.447 48 38202.426
## Total 8369520.537 53
## -----------------------------------------------------------------------
##
## Parameter Estimates
## ------------------------------------------------------------------------------------------------
## model Beta Std. Error Std. Beta t Sig lower upper
## ------------------------------------------------------------------------------------------------
## (Intercept) -1178.330 208.682 -5.647 0.000 -1597.914 -758.746
## liver_test 58.064 40.144 0.156 1.446 0.155 -22.652 138.779
## alc_heavy 317.848 71.634 0.314 4.437 0.000 173.818 461.878
## enzyme_test 9.748 1.656 0.521 5.887 0.000 6.419 13.077
## pindex 8.924 1.808 0.380 4.935 0.000 5.288 12.559
## bcs 59.864 23.060 0.241 2.596 0.012 13.498 106.230
## ------------------------------------------------------------------------------------------------Stepwise Backward Regression
Build regression model from a set of candidate predictor variables by
removing predictors based on p values, in a stepwise manner until there
is no variable left to remove any more. The model should include all the
candidate predictor variables. If details is set to TRUE,
each step is displayed.
Variable Selection
# stepwise backward regression
model <- lm(y ~ ., data = surgical)
ols_step_backward_p(model)##
##
## Stepwise Summary
## -------------------------------------------------------------------------
## Step Variable AIC SBC SBIC R2 Adj. R2
## -------------------------------------------------------------------------
## 0 Full Model 736.390 756.280 586.665 0.78184 0.74305
## 1 alc_mod 734.407 752.308 584.276 0.78177 0.74856
## 2 gender 732.494 748.406 581.938 0.78142 0.75351
## 3 age 730.620 744.543 579.638 0.78091 0.75808
## -------------------------------------------------------------------------
##
## Final Model Output
## ------------------
##
## Model Summary
## -------------------------------------------------------------------
## R 0.884 RMSE 184.276
## R-Squared 0.781 MSE 38202.426
## Adj. R-Squared 0.758 Coef. Var 27.839
## Pred R-Squared 0.700 AIC 730.620
## MAE 137.656 SBC 744.543
## -------------------------------------------------------------------
## RMSE: Root Mean Square Error
## MSE: Mean Square Error
## MAE: Mean Absolute Error
## AIC: Akaike Information Criteria
## SBC: Schwarz Bayesian Criteria
##
## ANOVA
## -----------------------------------------------------------------------
## Sum of
## Squares DF Mean Square F Sig.
## -----------------------------------------------------------------------
## Regression 6535804.090 5 1307160.818 34.217 0.0000
## Residual 1833716.447 48 38202.426
## Total 8369520.537 53
## -----------------------------------------------------------------------
##
## Parameter Estimates
## ------------------------------------------------------------------------------------------------
## model Beta Std. Error Std. Beta t Sig lower upper
## ------------------------------------------------------------------------------------------------
## (Intercept) -1178.330 208.682 -5.647 0.000 -1597.914 -758.746
## bcs 59.864 23.060 0.241 2.596 0.012 13.498 106.230
## pindex 8.924 1.808 0.380 4.935 0.000 5.288 12.559
## enzyme_test 9.748 1.656 0.521 5.887 0.000 6.419 13.077
## liver_test 58.064 40.144 0.156 1.446 0.155 -22.652 138.779
## alc_heavy 317.848 71.634 0.314 4.437 0.000 173.818 461.878
## ------------------------------------------------------------------------------------------------
Detailed Output
# stepwise backward regression
model <- lm(y ~ ., data = surgical)
ols_step_backward_p(model, details = TRUE)## Backward Elimination Method
## ---------------------------
##
## Candidate Terms:
##
## 1. bcs
## 2. pindex
## 3. enzyme_test
## 4. liver_test
## 5. age
## 6. gender
## 7. alc_mod
## 8. alc_heavy
##
##
## Step => 0
## Model => y ~ bcs + pindex + enzyme_test + liver_test + age + gender + alc_mod + alc_heavy
## R2 => 0.782
##
## Initiating stepwise selection...
##
## Step => 1
## Removed => alc_mod
## Model => y ~ bcs + pindex + enzyme_test + liver_test + age + gender + alc_heavy
## R2 => 0.78177
##
## Step => 2
## Removed => gender
## Model => y ~ bcs + pindex + enzyme_test + liver_test + age + alc_heavy
## R2 => 0.78142
##
## Step => 3
## Removed => age
## Model => y ~ bcs + pindex + enzyme_test + liver_test + alc_heavy
## R2 => 0.78091
##
##
## No more variables to be removed.
##
## Variables Removed:
##
## => alc_mod
## => gender
## => age##
##
## Stepwise Summary
## -------------------------------------------------------------------------
## Step Variable AIC SBC SBIC R2 Adj. R2
## -------------------------------------------------------------------------
## 0 Full Model 736.390 756.280 586.665 0.78184 0.74305
## 1 alc_mod 734.407 752.308 584.276 0.78177 0.74856
## 2 gender 732.494 748.406 581.938 0.78142 0.75351
## 3 age 730.620 744.543 579.638 0.78091 0.75808
## -------------------------------------------------------------------------
##
## Final Model Output
## ------------------
##
## Model Summary
## -------------------------------------------------------------------
## R 0.884 RMSE 184.276
## R-Squared 0.781 MSE 38202.426
## Adj. R-Squared 0.758 Coef. Var 27.839
## Pred R-Squared 0.700 AIC 730.620
## MAE 137.656 SBC 744.543
## -------------------------------------------------------------------
## RMSE: Root Mean Square Error
## MSE: Mean Square Error
## MAE: Mean Absolute Error
## AIC: Akaike Information Criteria
## SBC: Schwarz Bayesian Criteria
##
## ANOVA
## -----------------------------------------------------------------------
## Sum of
## Squares DF Mean Square F Sig.
## -----------------------------------------------------------------------
## Regression 6535804.090 5 1307160.818 34.217 0.0000
## Residual 1833716.447 48 38202.426
## Total 8369520.537 53
## -----------------------------------------------------------------------
##
## Parameter Estimates
## ------------------------------------------------------------------------------------------------
## model Beta Std. Error Std. Beta t Sig lower upper
## ------------------------------------------------------------------------------------------------
## (Intercept) -1178.330 208.682 -5.647 0.000 -1597.914 -758.746
## bcs 59.864 23.060 0.241 2.596 0.012 13.498 106.230
## pindex 8.924 1.808 0.380 4.935 0.000 5.288 12.559
## enzyme_test 9.748 1.656 0.521 5.887 0.000 6.419 13.077
## liver_test 58.064 40.144 0.156 1.446 0.155 -22.652 138.779
## alc_heavy 317.848 71.634 0.314 4.437 0.000 173.818 461.878
## ------------------------------------------------------------------------------------------------Stepwise Regression
Build regression model from a set of candidate predictor variables by
entering and removing predictors based on p values, in a stepwise manner
until there is no variable left to enter or remove any more. The model
should include all the candidate predictor variables. If details is set
to TRUE, each step is displayed.
Variable Selection
# stepwise regression
model <- lm(y ~ ., data = surgical)
ols_step_both_p(model)##
##
## Stepwise Summary
## ------------------------------------------------------------------------------
## Step Variable AIC SBC SBIC R2 Adj. R2
## ------------------------------------------------------------------------------
## 0 Base Model 802.606 806.584 646.794 0.00000 0.00000
## 1 liver_test (+) 771.875 777.842 616.009 0.45454 0.44405
## 2 alc_heavy (+) 761.439 769.395 605.506 0.56674 0.54975
## 3 enzyme_test (+) 750.509 760.454 595.297 0.65900 0.63854
## 4 pindex (+) 735.715 747.649 582.943 0.75015 0.72975
## 5 bcs (+) 730.620 744.543 579.638 0.78091 0.75808
## ------------------------------------------------------------------------------
##
## Final Model Output
## ------------------
##
## Model Summary
## -------------------------------------------------------------------
## R 0.884 RMSE 184.276
## R-Squared 0.781 MSE 38202.426
## Adj. R-Squared 0.758 Coef. Var 27.839
## Pred R-Squared 0.700 AIC 730.620
## MAE 137.656 SBC 744.543
## -------------------------------------------------------------------
## RMSE: Root Mean Square Error
## MSE: Mean Square Error
## MAE: Mean Absolute Error
## AIC: Akaike Information Criteria
## SBC: Schwarz Bayesian Criteria
##
## ANOVA
## -----------------------------------------------------------------------
## Sum of
## Squares DF Mean Square F Sig.
## -----------------------------------------------------------------------
## Regression 6535804.090 5 1307160.818 34.217 0.0000
## Residual 1833716.447 48 38202.426
## Total 8369520.537 53
## -----------------------------------------------------------------------
##
## Parameter Estimates
## ------------------------------------------------------------------------------------------------
## model Beta Std. Error Std. Beta t Sig lower upper
## ------------------------------------------------------------------------------------------------
## (Intercept) -1178.330 208.682 -5.647 0.000 -1597.914 -758.746
## liver_test 58.064 40.144 0.156 1.446 0.155 -22.652 138.779
## alc_heavy 317.848 71.634 0.314 4.437 0.000 173.818 461.878
## enzyme_test 9.748 1.656 0.521 5.887 0.000 6.419 13.077
## pindex 8.924 1.808 0.380 4.935 0.000 5.288 12.559
## bcs 59.864 23.060 0.241 2.596 0.012 13.498 106.230
## ------------------------------------------------------------------------------------------------
Detailed Output
# stepwise regression
model <- lm(y ~ ., data = surgical)
ols_step_both_p(model, details = TRUE)## Stepwise Selection Method
## -------------------------
##
## Candidate Terms:
##
## 1. bcs
## 2. pindex
## 3. enzyme_test
## 4. liver_test
## 5. age
## 6. gender
## 7. alc_mod
## 8. alc_heavy
##
##
## Step => 0
## Model => y ~ 1
## R2 => 0
##
## Initiating stepwise selection...
##
## Step => 1
## Selected => liver_test
## Model => y ~ liver_test
## R2 => 0.455
##
## Step => 2
## Selected => alc_heavy
## Model => y ~ liver_test + alc_heavy
## R2 => 0.567
##
## Step => 3
## Selected => enzyme_test
## Model => y ~ liver_test + alc_heavy + enzyme_test
## R2 => 0.659
##
## Step => 4
## Selected => pindex
## Model => y ~ liver_test + alc_heavy + enzyme_test + pindex
## R2 => 0.75
##
## Step => 5
## Selected => bcs
## Model => y ~ liver_test + alc_heavy + enzyme_test + pindex + bcs
## R2 => 0.781
##
##
## No more variables to be added or removed.##
##
## Stepwise Summary
## ------------------------------------------------------------------------------
## Step Variable AIC SBC SBIC R2 Adj. R2
## ------------------------------------------------------------------------------
## 0 Base Model 802.606 806.584 646.794 0.00000 0.00000
## 1 liver_test (+) 771.875 777.842 616.009 0.45454 0.44405
## 2 alc_heavy (+) 761.439 769.395 605.506 0.56674 0.54975
## 3 enzyme_test (+) 750.509 760.454 595.297 0.65900 0.63854
## 4 pindex (+) 735.715 747.649 582.943 0.75015 0.72975
## 5 bcs (+) 730.620 744.543 579.638 0.78091 0.75808
## ------------------------------------------------------------------------------
##
## Final Model Output
## ------------------
##
## Model Summary
## -------------------------------------------------------------------
## R 0.884 RMSE 184.276
## R-Squared 0.781 MSE 38202.426
## Adj. R-Squared 0.758 Coef. Var 27.839
## Pred R-Squared 0.700 AIC 730.620
## MAE 137.656 SBC 744.543
## -------------------------------------------------------------------
## RMSE: Root Mean Square Error
## MSE: Mean Square Error
## MAE: Mean Absolute Error
## AIC: Akaike Information Criteria
## SBC: Schwarz Bayesian Criteria
##
## ANOVA
## -----------------------------------------------------------------------
## Sum of
## Squares DF Mean Square F Sig.
## -----------------------------------------------------------------------
## Regression 6535804.090 5 1307160.818 34.217 0.0000
## Residual 1833716.447 48 38202.426
## Total 8369520.537 53
## -----------------------------------------------------------------------
##
## Parameter Estimates
## ------------------------------------------------------------------------------------------------
## model Beta Std. Error Std. Beta t Sig lower upper
## ------------------------------------------------------------------------------------------------
## (Intercept) -1178.330 208.682 -5.647 0.000 -1597.914 -758.746
## liver_test 58.064 40.144 0.156 1.446 0.155 -22.652 138.779
## alc_heavy 317.848 71.634 0.314 4.437 0.000 173.818 461.878
## enzyme_test 9.748 1.656 0.521 5.887 0.000 6.419 13.077
## pindex 8.924 1.808 0.380 4.935 0.000 5.288 12.559
## bcs 59.864 23.060 0.241 2.596 0.012 13.498 106.230
## ------------------------------------------------------------------------------------------------Stepwise AIC Forward Regression
Build regression model from a set of candidate predictor variables by
entering predictors based on Akaike Information Criteria, in a stepwise
manner until there is no variable left to enter any more. The model
should include all the candidate predictor variables. If details is set
to TRUE, each step is displayed.
Variable Selection
# stepwise aic forward regression
model <- lm(y ~ ., data = surgical)
ols_step_forward_aic(model)##
##
## Stepwise Summary
## --------------------------------------------------------------------------
## Step Variable AIC SBC SBIC R2 Adj. R2
## --------------------------------------------------------------------------
## 0 Base Model 802.606 806.584 646.794 0.00000 0.00000
## 1 liver_test 771.875 777.842 616.009 0.45454 0.44405
## 2 alc_heavy 761.439 769.395 605.506 0.56674 0.54975
## 3 enzyme_test 750.509 760.454 595.297 0.65900 0.63854
## 4 pindex 735.715 747.649 582.943 0.75015 0.72975
## 5 bcs 730.620 744.543 579.638 0.78091 0.75808
## --------------------------------------------------------------------------
##
## Final Model Output
## ------------------
##
## Model Summary
## -------------------------------------------------------------------
## R 0.884 RMSE 184.276
## R-Squared 0.781 MSE 38202.426
## Adj. R-Squared 0.758 Coef. Var 27.839
## Pred R-Squared 0.700 AIC 730.620
## MAE 137.656 SBC 744.543
## -------------------------------------------------------------------
## RMSE: Root Mean Square Error
## MSE: Mean Square Error
## MAE: Mean Absolute Error
## AIC: Akaike Information Criteria
## SBC: Schwarz Bayesian Criteria
##
## ANOVA
## -----------------------------------------------------------------------
## Sum of
## Squares DF Mean Square F Sig.
## -----------------------------------------------------------------------
## Regression 6535804.090 5 1307160.818 34.217 0.0000
## Residual 1833716.447 48 38202.426
## Total 8369520.537 53
## -----------------------------------------------------------------------
##
## Parameter Estimates
## ------------------------------------------------------------------------------------------------
## model Beta Std. Error Std. Beta t Sig lower upper
## ------------------------------------------------------------------------------------------------
## (Intercept) -1178.330 208.682 -5.647 0.000 -1597.914 -758.746
## liver_test 58.064 40.144 0.156 1.446 0.155 -22.652 138.779
## alc_heavy 317.848 71.634 0.314 4.437 0.000 173.818 461.878
## enzyme_test 9.748 1.656 0.521 5.887 0.000 6.419 13.077
## pindex 8.924 1.808 0.380 4.935 0.000 5.288 12.559
## bcs 59.864 23.060 0.241 2.596 0.012 13.498 106.230
## ------------------------------------------------------------------------------------------------
Detailed Output
# stepwise aic forward regression
model <- lm(y ~ ., data = surgical)
ols_step_forward_aic(model, details = TRUE)## Forward Selection Method
## ------------------------
##
## Candidate Terms:
##
## 1. bcs
## 2. pindex
## 3. enzyme_test
## 4. liver_test
## 5. age
## 6. gender
## 7. alc_mod
## 8. alc_heavy
##
##
## Step => 0
## Model => y ~ 1
## AIC => 802.606
##
## Initiating stepwise selection...
##
## Table: Adding New Variables
## -------------------------------------------------------------------------
## Predictor DF AIC SBC SBIC R2 Adj. R2
## -------------------------------------------------------------------------
## liver_test 1 771.875 777.842 616.009 0.45454 0.44405
## enzyme_test 1 782.629 788.596 626.220 0.33435 0.32154
## pindex 1 794.100 800.067 637.196 0.17680 0.16097
## alc_heavy 1 794.301 800.268 637.389 0.17373 0.15784
## bcs 1 797.697 803.664 640.655 0.12010 0.10318
## alc_mod 1 802.828 808.795 645.601 0.03239 0.01378
## gender 1 802.956 808.923 645.725 0.03009 0.01143
## age 1 803.834 809.801 646.572 0.01420 -0.00476
## -------------------------------------------------------------------------
##
## Step => 1
## Added => liver_test
## Model => y ~ liver_test
## AIC => 771.8753
##
## Table: Adding New Variables
## ------------------------------------------------------------------------
## Predictor DF AIC SBC SBIC R2 Adj. R2
## ------------------------------------------------------------------------
## alc_heavy 1 761.439 769.395 605.506 0.56674 0.54975
## enzyme_test 1 762.077 770.033 606.090 0.56159 0.54440
## pindex 1 770.387 778.343 613.737 0.48866 0.46861
## alc_mod 1 771.141 779.097 614.435 0.48147 0.46113
## gender 1 773.802 781.758 616.901 0.45528 0.43391
## age 1 773.831 781.787 616.928 0.45498 0.43361
## bcs 1 773.867 781.823 616.961 0.45462 0.43323
## ------------------------------------------------------------------------
##
## Step => 2
## Added => alc_heavy
## Model => y ~ liver_test + alc_heavy
## AIC => 761.4394
##
## Table: Adding New Variables
## ------------------------------------------------------------------------
## Predictor DF AIC SBC SBIC R2 Adj. R2
## ------------------------------------------------------------------------
## enzyme_test 1 750.509 760.454 595.297 0.65900 0.63854
## pindex 1 756.125 766.070 600.225 0.62163 0.59892
## bcs 1 763.063 773.008 606.379 0.56975 0.54394
## age 1 763.110 773.055 606.421 0.56938 0.54354
## alc_mod 1 763.428 773.373 606.704 0.56683 0.54084
## gender 1 763.433 773.378 606.709 0.56679 0.54080
## ------------------------------------------------------------------------
##
## Step => 3
## Added => enzyme_test
## Model => y ~ liver_test + alc_heavy + enzyme_test
## AIC => 750.5089
##
## Table: Adding New Variables
## ----------------------------------------------------------------------
## Predictor DF AIC SBC SBIC R2 Adj. R2
## ----------------------------------------------------------------------
## pindex 1 735.715 747.649 582.943 0.75015 0.72975
## bcs 1 750.782 762.716 595.377 0.66973 0.64277
## alc_mod 1 752.403 764.337 596.743 0.65967 0.63189
## age 1 752.416 764.350 596.755 0.65959 0.63180
## gender 1 752.509 764.443 596.833 0.65900 0.63116
## ----------------------------------------------------------------------
##
## Step => 4
## Added => pindex
## Model => y ~ liver_test + alc_heavy + enzyme_test + pindex
## AIC => 735.7146
##
## Table: Adding New Variables
## ----------------------------------------------------------------------
## Predictor DF AIC SBC SBIC R2 Adj. R2
## ----------------------------------------------------------------------
## bcs 1 730.620 744.543 579.638 0.78091 0.75808
## age 1 737.680 751.603 585.012 0.75030 0.72429
## gender 1 737.712 751.635 585.036 0.75016 0.72413
## alc_mod 1 737.713 751.636 585.037 0.75015 0.72413
## ----------------------------------------------------------------------
##
## Step => 5
## Added => bcs
## Model => y ~ liver_test + alc_heavy + enzyme_test + pindex + bcs
## AIC => 730.6204
##
## Table: Adding New Variables
## ----------------------------------------------------------------------
## Predictor DF AIC SBC SBIC R2 Adj. R2
## ----------------------------------------------------------------------
## age 1 732.494 748.406 581.938 0.78142 0.75351
## gender 1 732.551 748.463 581.978 0.78119 0.75325
## alc_mod 1 732.614 748.526 582.023 0.78093 0.75297
## ----------------------------------------------------------------------
##
##
## No more variables to be added.
##
## Variables Selected:
##
## => liver_test
## => alc_heavy
## => enzyme_test
## => pindex
## => bcs##
##
## Stepwise Summary
## --------------------------------------------------------------------------
## Step Variable AIC SBC SBIC R2 Adj. R2
## --------------------------------------------------------------------------
## 0 Base Model 802.606 806.584 646.794 0.00000 0.00000
## 1 liver_test 771.875 777.842 616.009 0.45454 0.44405
## 2 alc_heavy 761.439 769.395 605.506 0.56674 0.54975
## 3 enzyme_test 750.509 760.454 595.297 0.65900 0.63854
## 4 pindex 735.715 747.649 582.943 0.75015 0.72975
## 5 bcs 730.620 744.543 579.638 0.78091 0.75808
## --------------------------------------------------------------------------
##
## Final Model Output
## ------------------
##
## Model Summary
## -------------------------------------------------------------------
## R 0.884 RMSE 184.276
## R-Squared 0.781 MSE 38202.426
## Adj. R-Squared 0.758 Coef. Var 27.839
## Pred R-Squared 0.700 AIC 730.620
## MAE 137.656 SBC 744.543
## -------------------------------------------------------------------
## RMSE: Root Mean Square Error
## MSE: Mean Square Error
## MAE: Mean Absolute Error
## AIC: Akaike Information Criteria
## SBC: Schwarz Bayesian Criteria
##
## ANOVA
## -----------------------------------------------------------------------
## Sum of
## Squares DF Mean Square F Sig.
## -----------------------------------------------------------------------
## Regression 6535804.090 5 1307160.818 34.217 0.0000
## Residual 1833716.447 48 38202.426
## Total 8369520.537 53
## -----------------------------------------------------------------------
##
## Parameter Estimates
## ------------------------------------------------------------------------------------------------
## model Beta Std. Error Std. Beta t Sig lower upper
## ------------------------------------------------------------------------------------------------
## (Intercept) -1178.330 208.682 -5.647 0.000 -1597.914 -758.746
## liver_test 58.064 40.144 0.156 1.446 0.155 -22.652 138.779
## alc_heavy 317.848 71.634 0.314 4.437 0.000 173.818 461.878
## enzyme_test 9.748 1.656 0.521 5.887 0.000 6.419 13.077
## pindex 8.924 1.808 0.380 4.935 0.000 5.288 12.559
## bcs 59.864 23.060 0.241 2.596 0.012 13.498 106.230
## ------------------------------------------------------------------------------------------------Stepwise AIC Backward Regression
Build regression model from a set of candidate predictor variables by
removing predictors based on Akaike Information Criteria, in a stepwise
manner until there is no variable left to remove any more. The model
should include all the candidate predictor variables. If details is set
to TRUE, each step is displayed.
Variable Selection
# stepwise aic backward regression
model <- lm(y ~ ., data = surgical)
k <- ols_step_backward_aic(model)
k##
##
## Stepwise Summary
## -------------------------------------------------------------------------
## Step Variable AIC SBC SBIC R2 Adj. R2
## -------------------------------------------------------------------------
## 0 Full Model 736.390 756.280 586.665 0.78184 0.74305
## 1 alc_mod 734.407 752.308 583.884 0.78177 0.74856
## 2 gender 732.494 748.406 581.290 0.78142 0.75351
## 3 age 730.620 744.543 578.844 0.78091 0.75808
## -------------------------------------------------------------------------
##
## Final Model Output
## ------------------
##
## Model Summary
## -------------------------------------------------------------------
## R 0.884 RMSE 184.276
## R-Squared 0.781 MSE 38202.426
## Adj. R-Squared 0.758 Coef. Var 27.839
## Pred R-Squared 0.700 AIC 730.620
## MAE 137.656 SBC 744.543
## -------------------------------------------------------------------
## RMSE: Root Mean Square Error
## MSE: Mean Square Error
## MAE: Mean Absolute Error
## AIC: Akaike Information Criteria
## SBC: Schwarz Bayesian Criteria
##
## ANOVA
## -----------------------------------------------------------------------
## Sum of
## Squares DF Mean Square F Sig.
## -----------------------------------------------------------------------
## Regression 6535804.090 5 1307160.818 34.217 0.0000
## Residual 1833716.447 48 38202.426
## Total 8369520.537 53
## -----------------------------------------------------------------------
##
## Parameter Estimates
## ------------------------------------------------------------------------------------------------
## model Beta Std. Error Std. Beta t Sig lower upper
## ------------------------------------------------------------------------------------------------
## (Intercept) -1178.330 208.682 -5.647 0.000 -1597.914 -758.746
## bcs 59.864 23.060 0.241 2.596 0.012 13.498 106.230
## pindex 8.924 1.808 0.380 4.935 0.000 5.288 12.559
## enzyme_test 9.748 1.656 0.521 5.887 0.000 6.419 13.077
## liver_test 58.064 40.144 0.156 1.446 0.155 -22.652 138.779
## alc_heavy 317.848 71.634 0.314 4.437 0.000 173.818 461.878
## ------------------------------------------------------------------------------------------------
Detailed Output
# stepwise aic backward regression
model <- lm(y ~ ., data = surgical)
ols_step_backward_aic(model, details = TRUE)## Backward Elimination Method
## ---------------------------
##
## Candidate Terms:
##
## 1. bcs
## 2. pindex
## 3. enzyme_test
## 4. liver_test
## 5. age
## 6. gender
## 7. alc_mod
## 8. alc_heavy
##
##
## Step => 0
## Model => y ~ bcs + pindex + enzyme_test + liver_test + age + gender + alc_mod + alc_heavy
## AIC => 736.3899
##
## Initiating stepwise selection...
##
## Table: Removing Existing Variables
## ------------------------------------------------------------------------
## Predictor DF AIC SBC SBIC R2 Adj. R2
## ------------------------------------------------------------------------
## alc_mod 1 734.407 752.308 584.276 0.78177 0.74856
## gender 1 734.478 752.379 584.323 0.78148 0.74823
## age 1 734.544 752.445 584.367 0.78121 0.74792
## liver_test 1 735.878 753.779 585.255 0.77574 0.74162
## bcs 1 741.677 759.577 589.203 0.75032 0.71233
## alc_heavy 1 749.210 767.111 594.541 0.71294 0.66926
## pindex 1 756.624 774.525 600.014 0.67070 0.62059
## enzyme_test 1 763.557 781.458 605.318 0.62559 0.56861
## ------------------------------------------------------------------------
##
## Step => 1
## Removed => alc_mod
## Model => y ~ bcs + pindex + enzyme_test + liver_test + age + gender + alc_heavy
## AIC => 734.4068
##
## Table: Removing Existing Variables
## ------------------------------------------------------------------------
## Predictor DF AIC SBC SBIC R2 Adj. R2
## ------------------------------------------------------------------------
## gender 1 732.494 748.406 581.938 0.78142 0.75351
## age 1 732.551 748.463 581.978 0.78119 0.75325
## liver_test 1 733.921 749.833 582.951 0.77556 0.74691
## bcs 1 739.677 755.589 587.106 0.75032 0.71845
## alc_heavy 1 750.486 766.398 595.217 0.69499 0.65605
## pindex 1 754.759 770.671 598.530 0.66987 0.62773
## enzyme_test 1 761.595 777.507 603.950 0.62532 0.57749
## ------------------------------------------------------------------------
##
## Step => 2
## Removed => gender
## Model => y ~ bcs + pindex + enzyme_test + liver_test + age + alc_heavy
## AIC => 732.4942
##
## Table: Removing Existing Variables
## ------------------------------------------------------------------------
## Predictor DF AIC SBC SBIC R2 Adj. R2
## ------------------------------------------------------------------------
## age 1 730.620 744.543 579.638 0.78091 0.75808
## liver_test 1 732.339 746.262 580.934 0.77382 0.75026
## bcs 1 737.680 751.603 585.012 0.75030 0.72429
## alc_heavy 1 748.486 762.409 593.500 0.69499 0.66322
## pindex 1 752.777 766.700 596.959 0.66976 0.63536
## enzyme_test 1 759.596 773.518 602.553 0.62532 0.58629
## ------------------------------------------------------------------------
##
## Step => 3
## Removed => age
## Model => y ~ bcs + pindex + enzyme_test + liver_test + alc_heavy
## AIC => 730.6204
##
## Table: Removing Existing Variables
## ------------------------------------------------------------------------
## Predictor DF AIC SBC SBIC R2 Adj. R2
## ------------------------------------------------------------------------
## liver_test 1 730.924 742.858 579.087 0.77136 0.75269
## bcs 1 735.715 747.649 582.943 0.75015 0.72975
## alc_heavy 1 747.181 759.114 592.362 0.69104 0.66582
## pindex 1 750.782 762.716 595.377 0.66973 0.64277
## enzyme_test 1 757.971 769.905 601.477 0.62270 0.59190
## ------------------------------------------------------------------------
##
##
## No more variables to be removed.
##
## Variables Removed:
##
## => alc_mod
## => gender
## => age##
##
## Stepwise Summary
## -------------------------------------------------------------------------
## Step Variable AIC SBC SBIC R2 Adj. R2
## -------------------------------------------------------------------------
## 0 Full Model 736.390 756.280 586.665 0.78184 0.74305
## 1 alc_mod 734.407 752.308 583.884 0.78177 0.74856
## 2 gender 732.494 748.406 581.290 0.78142 0.75351
## 3 age 730.620 744.543 578.844 0.78091 0.75808
## -------------------------------------------------------------------------
##
## Final Model Output
## ------------------
##
## Model Summary
## -------------------------------------------------------------------
## R 0.884 RMSE 184.276
## R-Squared 0.781 MSE 38202.426
## Adj. R-Squared 0.758 Coef. Var 27.839
## Pred R-Squared 0.700 AIC 730.620
## MAE 137.656 SBC 744.543
## -------------------------------------------------------------------
## RMSE: Root Mean Square Error
## MSE: Mean Square Error
## MAE: Mean Absolute Error
## AIC: Akaike Information Criteria
## SBC: Schwarz Bayesian Criteria
##
## ANOVA
## -----------------------------------------------------------------------
## Sum of
## Squares DF Mean Square F Sig.
## -----------------------------------------------------------------------
## Regression 6535804.090 5 1307160.818 34.217 0.0000
## Residual 1833716.447 48 38202.426
## Total 8369520.537 53
## -----------------------------------------------------------------------
##
## Parameter Estimates
## ------------------------------------------------------------------------------------------------
## model Beta Std. Error Std. Beta t Sig lower upper
## ------------------------------------------------------------------------------------------------
## (Intercept) -1178.330 208.682 -5.647 0.000 -1597.914 -758.746
## bcs 59.864 23.060 0.241 2.596 0.012 13.498 106.230
## pindex 8.924 1.808 0.380 4.935 0.000 5.288 12.559
## enzyme_test 9.748 1.656 0.521 5.887 0.000 6.419 13.077
## liver_test 58.064 40.144 0.156 1.446 0.155 -22.652 138.779
## alc_heavy 317.848 71.634 0.314 4.437 0.000 173.818 461.878
## ------------------------------------------------------------------------------------------------Stepwise AIC Regression
Build regression model from a set of candidate predictor variables by
entering and removing predictors based on Akaike Information Criteria,
in a stepwise manner until there is no variable left to enter or remove
any more. The model should include all the candidate predictor
variables. If details is set to TRUE, each step is
displayed.
Variable Selection
# stepwise aic regression
model <- lm(y ~ ., data = surgical)
ols_step_both_aic(model)##
##
## Stepwise Summary
## ------------------------------------------------------------------------------
## Step Variable AIC SBC SBIC R2 Adj. R2
## ------------------------------------------------------------------------------
## 0 Base Model 802.606 806.584 646.794 0.00000 0.00000
## 1 liver_test (+) 771.875 777.842 616.009 0.45454 0.44405
## 2 alc_heavy (+) 761.439 769.395 605.506 0.56674 0.54975
## 3 enzyme_test (+) 750.509 760.454 595.297 0.65900 0.63854
## 4 pindex (+) 735.715 747.649 582.943 0.75015 0.72975
## 5 bcs (+) 730.620 744.543 579.638 0.78091 0.75808
## ------------------------------------------------------------------------------
##
## Final Model Output
## ------------------
##
## Model Summary
## -------------------------------------------------------------------
## R 0.884 RMSE 184.276
## R-Squared 0.781 MSE 38202.426
## Adj. R-Squared 0.758 Coef. Var 27.839
## Pred R-Squared 0.700 AIC 730.620
## MAE 137.656 SBC 744.543
## -------------------------------------------------------------------
## RMSE: Root Mean Square Error
## MSE: Mean Square Error
## MAE: Mean Absolute Error
## AIC: Akaike Information Criteria
## SBC: Schwarz Bayesian Criteria
##
## ANOVA
## -----------------------------------------------------------------------
## Sum of
## Squares DF Mean Square F Sig.
## -----------------------------------------------------------------------
## Regression 6535804.090 5 1307160.818 34.217 0.0000
## Residual 1833716.447 48 38202.426
## Total 8369520.537 53
## -----------------------------------------------------------------------
##
## Parameter Estimates
## ------------------------------------------------------------------------------------------------
## model Beta Std. Error Std. Beta t Sig lower upper
## ------------------------------------------------------------------------------------------------
## (Intercept) -1178.330 208.682 -5.647 0.000 -1597.914 -758.746
## liver_test 58.064 40.144 0.156 1.446 0.155 -22.652 138.779
## alc_heavy 317.848 71.634 0.314 4.437 0.000 173.818 461.878
## enzyme_test 9.748 1.656 0.521 5.887 0.000 6.419 13.077
## pindex 8.924 1.808 0.380 4.935 0.000 5.288 12.559
## bcs 59.864 23.060 0.241 2.596 0.012 13.498 106.230
## ------------------------------------------------------------------------------------------------
Detailed Output
# stepwise aic regression
model <- lm(y ~ ., data = surgical)
ols_step_both_aic(model, details = TRUE)## Stepwise Selection Method
## -------------------------
##
## Candidate Terms:
##
## 1. bcs
## 2. pindex
## 3. enzyme_test
## 4. liver_test
## 5. age
## 6. gender
## 7. alc_mod
## 8. alc_heavy
##
##
## Step => 0
## Model => y ~ 1
## AIC => 802.606
##
## Initiating stepwise selection...
##
## Table: Adding New Variables
## -------------------------------------------------------------------------
## Predictor DF AIC SBC SBIC R2 Adj. R2
## -------------------------------------------------------------------------
## bcs 1 797.697 803.664 640.655 0.12010 0.10318
## pindex 1 794.100 800.067 637.196 0.17680 0.16097
## enzyme_test 1 782.629 788.596 626.220 0.33435 0.32154
## liver_test 1 771.875 777.842 616.009 0.45454 0.44405
## age 1 803.834 809.801 646.572 0.01420 -0.00476
## gender 1 802.956 808.923 645.725 0.03009 0.01143
## alc_mod 1 802.828 808.795 645.601 0.03239 0.01378
## alc_heavy 1 794.301 800.268 637.389 0.17373 0.15784
## -------------------------------------------------------------------------
##
## Step => 1
## Added => liver_test
## Model => y ~ liver_test
## AIC => 771.8753
##
## Table: Adding New Variables
## ------------------------------------------------------------------------
## Predictor DF AIC SBC SBIC R2 Adj. R2
## ------------------------------------------------------------------------
## bcs 1 773.867 781.823 616.961 0.45462 0.43323
## pindex 1 770.387 778.343 613.737 0.48866 0.46861
## enzyme_test 1 762.077 770.033 606.090 0.56159 0.54440
## age 1 773.831 781.787 616.928 0.45498 0.43361
## gender 1 773.802 781.758 616.901 0.45528 0.43391
## alc_mod 1 771.141 779.097 614.435 0.48147 0.46113
## alc_heavy 1 761.439 769.395 605.506 0.56674 0.54975
## ------------------------------------------------------------------------
##
## Step => 2
## Added => alc_heavy
## Model => y ~ liver_test + alc_heavy
## AIC => 761.4394
##
## Table: Removing Existing Variables
## -----------------------------------------------------------------------
## Predictor DF AIC SBC SBIC R2 Adj. R2
## -----------------------------------------------------------------------
## liver_test 1 794.301 800.268 637.389 0.17373 0.15784
## alc_heavy 1 771.875 777.842 616.009 0.45454 0.44405
## -----------------------------------------------------------------------
##
## Table: Adding New Variables
## ------------------------------------------------------------------------
## Predictor DF AIC SBC SBIC R2 Adj. R2
## ------------------------------------------------------------------------
## bcs 1 763.063 773.008 606.379 0.56975 0.54394
## pindex 1 756.125 766.070 600.225 0.62163 0.59892
## enzyme_test 1 750.509 760.454 595.297 0.65900 0.63854
## age 1 763.110 773.055 606.421 0.56938 0.54354
## gender 1 763.433 773.378 606.709 0.56679 0.54080
## alc_mod 1 763.428 773.373 606.704 0.56683 0.54084
## ------------------------------------------------------------------------
##
## Step => 3
## Added => enzyme_test
## Model => y ~ liver_test + alc_heavy + enzyme_test
## AIC => 750.5089
##
## Table: Removing Existing Variables
## ------------------------------------------------------------------------
## Predictor DF AIC SBC SBIC R2 Adj. R2
## ------------------------------------------------------------------------
## liver_test 1 773.555 781.511 616.671 0.45777 0.43650
## alc_heavy 1 762.077 770.033 606.090 0.56159 0.54440
## enzyme_test 1 761.439 769.395 605.506 0.56674 0.54975
## ------------------------------------------------------------------------
##
## Table: Adding New Variables
## ----------------------------------------------------------------------
## Predictor DF AIC SBC SBIC R2 Adj. R2
## ----------------------------------------------------------------------
## bcs 1 750.782 762.716 595.377 0.66973 0.64277
## pindex 1 735.715 747.649 582.943 0.75015 0.72975
## age 1 752.416 764.350 596.755 0.65959 0.63180
## gender 1 752.509 764.443 596.833 0.65900 0.63116
## alc_mod 1 752.403 764.337 596.743 0.65967 0.63189
## ----------------------------------------------------------------------
##
## Step => 4
## Added => pindex
## Model => y ~ liver_test + alc_heavy + enzyme_test + pindex
## AIC => 735.7146
##
## Table: Removing Existing Variables
## ------------------------------------------------------------------------
## Predictor DF AIC SBC SBIC R2 Adj. R2
## ------------------------------------------------------------------------
## liver_test 1 748.167 758.112 593.257 0.67347 0.65388
## alc_heavy 1 755.099 765.044 599.321 0.62875 0.60647
## enzyme_test 1 756.125 766.070 600.225 0.62163 0.59892
## pindex 1 750.509 760.454 595.297 0.65900 0.63854
## ------------------------------------------------------------------------
##
## Table: Adding New Variables
## ----------------------------------------------------------------------
## Predictor DF AIC SBC SBIC R2 Adj. R2
## ----------------------------------------------------------------------
## bcs 1 730.620 744.543 579.638 0.78091 0.75808
## age 1 737.680 751.603 585.012 0.75030 0.72429
## gender 1 737.712 751.635 585.036 0.75016 0.72413
## alc_mod 1 737.713 751.636 585.037 0.75015 0.72413
## ----------------------------------------------------------------------
##
## Step => 5
## Added => bcs
## Model => y ~ liver_test + alc_heavy + enzyme_test + pindex + bcs
## AIC => 730.6204
##
## Table: Removing Existing Variables
## ------------------------------------------------------------------------
## Predictor DF AIC SBC SBIC R2 Adj. R2
## ------------------------------------------------------------------------
## liver_test 1 730.924 742.858 579.087 0.77136 0.75269
## alc_heavy 1 747.181 759.114 592.362 0.69104 0.66582
## enzyme_test 1 757.971 769.905 601.477 0.62270 0.59190
## pindex 1 750.782 762.716 595.377 0.66973 0.64277
## bcs 1 735.715 747.649 582.943 0.75015 0.72975
## ------------------------------------------------------------------------
##
## Table: Adding New Variables
## ----------------------------------------------------------------------
## Predictor DF AIC SBC SBIC R2 Adj. R2
## ----------------------------------------------------------------------
## age 1 732.494 748.406 581.938 0.78142 0.75351
## gender 1 732.551 748.463 581.978 0.78119 0.75325
## alc_mod 1 732.614 748.526 582.023 0.78093 0.75297
## ----------------------------------------------------------------------
##
##
## No more variables to be added or removed.
##
## Variables Selected:
##
## => liver_test
## => alc_heavy
## => enzyme_test
## => pindex
## => bcs##
##
## Stepwise Summary
## ------------------------------------------------------------------------------
## Step Variable AIC SBC SBIC R2 Adj. R2
## ------------------------------------------------------------------------------
## 0 Base Model 802.606 806.584 646.794 0.00000 0.00000
## 1 liver_test (+) 771.875 777.842 616.009 0.45454 0.44405
## 2 alc_heavy (+) 761.439 769.395 605.506 0.56674 0.54975
## 3 enzyme_test (+) 750.509 760.454 595.297 0.65900 0.63854
## 4 pindex (+) 735.715 747.649 582.943 0.75015 0.72975
## 5 bcs (+) 730.620 744.543 579.638 0.78091 0.75808
## ------------------------------------------------------------------------------
##
## Final Model Output
## ------------------
##
## Model Summary
## -------------------------------------------------------------------
## R 0.884 RMSE 184.276
## R-Squared 0.781 MSE 38202.426
## Adj. R-Squared 0.758 Coef. Var 27.839
## Pred R-Squared 0.700 AIC 730.620
## MAE 137.656 SBC 744.543
## -------------------------------------------------------------------
## RMSE: Root Mean Square Error
## MSE: Mean Square Error
## MAE: Mean Absolute Error
## AIC: Akaike Information Criteria
## SBC: Schwarz Bayesian Criteria
##
## ANOVA
## -----------------------------------------------------------------------
## Sum of
## Squares DF Mean Square F Sig.
## -----------------------------------------------------------------------
## Regression 6535804.090 5 1307160.818 34.217 0.0000
## Residual 1833716.447 48 38202.426
## Total 8369520.537 53
## -----------------------------------------------------------------------
##
## Parameter Estimates
## ------------------------------------------------------------------------------------------------
## model Beta Std. Error Std. Beta t Sig lower upper
## ------------------------------------------------------------------------------------------------
## (Intercept) -1178.330 208.682 -5.647 0.000 -1597.914 -758.746
## liver_test 58.064 40.144 0.156 1.446 0.155 -22.652 138.779
## alc_heavy 317.848 71.634 0.314 4.437 0.000 173.818 461.878
## enzyme_test 9.748 1.656 0.521 5.887 0.000 6.419 13.077
## pindex 8.924 1.808 0.380 4.935 0.000 5.288 12.559
## bcs 59.864 23.060 0.241 2.596 0.012 13.498 106.230
## ------------------------------------------------------------------------------------------------