This monograph provides an overview of distributed online optimization in multi-agent systems. Online optimization approaches planning and decision problems from a robust learning perspective, where one learns through feedback from sequentially arriving costs, resembling a game between a learner (agent) and the environment. Recently, multi-agent systems have become important in diverse areas including smart power grids, communication networks, machine learning, and robotics, where agents work with decentralized data, costs, and decisions to collectively minimize a system-wide cost. In such settings, agents make distributed decisions and collaborate with neighboring agents through a communication network, leading to scalable solutions that often perform as well as centralized methods.The monograph offers a unified introduction, starting with fundamental algorithms for basic problems, and gradually covering state-of-the-art techniques for more complex settings. The interplay between individual agent learning rates, network structure, and communication complexity is highlighted in the overall system performance.