Random Forest

Random Forest is an ensemble that trains many decision trees and combines their predictions.

Each tree sees a random subset of the data. At each split it considers a random subset of features. That double randomization cuts the variance of a single tree. For classification the forest takes a majority vote. For regression it averages numeric guesses. Individual trees overfit or err; those errors dilute in the vote.

The method needs little parameter tuning, handles noisy and missing values, and trains quickly on current hardware. Uses include credit-risk scores, fraud detection, medical imaging, and recommenders. Built-in feature-importance scores show which variables drive predictions, which matters in regulated industries. Trees train in parallel, so clusters can cut training from hours to minutes.

Bootstrap the rows, randomize the columns at each split, grow a deep tree, repeat many times. Classification votes. Regression averages. The ensemble cancels idiosyncratic splits that a single tree would memorize. Missing values and noise are less fatal than in a lone tree. Feature-importance scores, how often a variable is used to split, are why banks and hospitals can explain a score.

Parallel training on a cluster is straightforward because trees do not depend on each other during growth. Credit, fraud, imaging, and recommenders still use that recipe when they need speed and an audit trail. Leo Breiman's 2001 paper: many decision trees, random features, vote. Still strong on tabular data.