Regression
==================

1. Draw the flow-chart diagram for making predictions using regression
   as ML algorithm.
   Explain briefly each box on the flow-chart: ML model, ML algorithm,
   Quality metric, Feature extraction.

2. Write down formula for simple linear regression model.
   How one can interpret coefficients.
   Write formula for defining cost function using RSS estimator.

3. Write down sequence of iterative gradient descent algorithm finding
   minimum of the cost function for simple linear regression.
   When do we stop iteration, how do we choose step-size.

4. How do we access performance? Explain what is the "training error",
   "validation error", "generalization error", "test error".
   What does it mean "cross-validation"?
   Draw illustrative plot how they typically behave with regression
   model complexity.

5. Explain what are the sources of errors on the prediction:
   noise, bias, variance.
   Draw simple illustration explaining it.
   What does it mean bias-variance trade-off.

6. What does it mean "over-fitting"?
   Explain how we can mitigate it adding extra term to the
   cost function:  "regge regression"  or "lasso regression".
   Write formula of the respective cost functions.
   Show illustrative plot how the coefficients w will behave in each case.

7. Explain procedure for selecting features for regression with
   the greedy algorithm.

8. What does it mean "non-parametric regression".
   Explain concept of: (1-NN) regression, (k-NN) regression,
   weighted regression, kernel regression.