= Bagging =

'''Bagging''' ('''b'''ootstrap '''agg'''regat'''ing''') is a non-parametric ensemble model using [[Statistics/DecisionTrees|decision trees]].

<<TableOfContents>>

----



== Description ==

Bagging is an iterative ensemble model. For each iteration, a [[Statistics/SimpleRandomSample|SRS with replacement]] of observations are selected. A [[Statistics/DecisionTrees|decision tree]] model is trained on this data. (This is the ''bootstrapping'' step.)

The final prediction of the bagging model is the average prediction across all trees. (This is the ''aggregating'' step.)

This model is designed to mitigate overfitting.



=== Out-of-bag Error ===

For any single tree, the cases ''not'' selected for a single tree's estimation are called '''out-of-bag'''. These can be used as a test set to evaluate the trained model.

For each single tree, the corresponding out-of-bag set is used to calculate an error rate. The weighted average (weighted by size of the out-of-bag set) of these error rates is the '''out-of-bag error'''.



----
CategoryRicottone