The first part of this book presents the essential topics for an introduction to deterministic optimal control theory. The second part introduces stochastic optimal control for Markov diffusion processes. It also inlcudes two other topics important for applications, namely, the solution to the stochastic linear regulator and the separation principle.