So What is Machine Learning? A comprehensive body of work by Izenman (2008) outlines the basics of machine learning and through my research on the topic provides some of the most precise explanations of the topic.

The Idea of Machine Learning: Machine learning evolved out of the subfield of computer science known as artificial intelligence (AI). Whereas the focus of AI is to make machines intelligent, able to think rationally like humans and solve problems, machine learning is concerned with creating computer systems and algorithms so that machines can “learn” from previous experience.

Because intelligence cannot be attained without the ability to learn, machine learning now plays a dominant role in AI. The machine-learning community divides learning problems into various categories: the two most relevant to statistics are those of supervised learning and unsupervised learning.

Supervised Learning: Problems in which the learning algorithm receives a set of continuous or categorical input variables and a correct output variable (which is observed or provided by an explicit “teacher”) and tries to find a function of the input variables to approximate the known output variable: a continuous output variable yields a regression problem, whereas a categorical output variable yields a classification problem.

Unsupervised Learning: Problems in which there is no information available (i.e., no explicit “teacher”) to define an appropriate output variable; often referred to as “scientific discovery.” The goal in unsupervised learning differs from that of supervised learning. In supervised learning, we study relationships between the input and output variables; in unsupervised learning, we explore particular characteristics of the input variables only, such as estimating the joint probability density, searching out clusters, drawing proximity maps, locating outliers, or imputing missing data.


Izenman, A. (2008). Modern Multivariate Statistical Techniques: Regression, Classification, and Manifold Learning. New York, NY: Springer.