Newton's Method vs Gradient Descent Method in tacking saddle points in Non-Convex Optimization

For Convex functions, Newton’s method leads to faster convergence since it’s a second order optimization technique that takes the curvature of the cost surface in account. The curvature information at a given point is provided by the Hessian matrix of the cost function. Although Newton method is good for convex optimization having one global minima, […]

Monty Hall - The True picture with Simulation

Many of you might have come across the Monty Hall problem . Here is the wikipedia definition of the problem Suppose you're on a game show, and you're given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows […]

The Mathematics and Geometry of Gradient Descent

Gradient Descent is one of the most widely used Optimization techniques that has profound use in Machine learning. Although one of the more simpler methods because of its ease of use and less memory requirements it has been widely adopted in the Data Science and Machine Learning Community. If we have a cost function where […]