Machine learning has changed drastically over three decades. Initially, it was referred to the artificial intelligence (AI) tools of expert systems and deep learning (DL). Today, many algorithms used in business intelligence can analyze data and have been combined with older technologies under an umbrella term of ML. The complexity involved in training and deploying DL systems are useful under certain circumstances.
Optimization is a mathematical term for determining the maximum or minimum value of a function. For example, to maximize profit for a business, a mathematical formula can be created that incorporates all costs, materials, production, distribution, overhead, and more. The outcome can be used to calculate the maximum profit. The fixed formulas raise a question of what is minimum or maximum. People limit the potential conditions of a problem by using an experience-based bias while picking a data set or a range. Mathematicians define the maximum as local maximum and the minimum as a local minimum.
Problems arise when new information models are considered. DL has issues when applied to optimization problems because large data sets are being theoretically local. Stochastic gradient descent (SDG) is used to train DL systems. The results gathered by the training run of the model are used in the algorithms next run. However, SGD is susceptible to local minimums and maximums. Therefore, DL systems are not suited for complex and generalized optimization issues.
DL machines are inappropriate when applied to more general optimization, but they are still used for optimization. The reason behind this is that often the local minimum or maximum is better than the global minimum or maximum. For example, e-commerce websites list other products while a user is browsing for their products. These recommendations are based on the optimization problem. The site owners want those recommendations to be add-on purchases for the visitor. The power of deep learning systems does not reside in figuring out global optimization but in fine-tuning local minimums and maximums. It can take customization of offerings from universal generalities to specific individual interests.