Closed-form vs Gradient Descent

• In many cases, there is no closed form for the gradient, so Gradient Descent is required.
• In some cases, we have many, many coefficients, so calculating the the closed form is difficult even if there is a closed form.
• The advantage of closed form is that it is not iterative. We do not need to choose an initial value for the iteration and we do not need to choose a step size. We simply calculated it directly.