A characterization is given for the factorizations of almost simple groups with a solvable factor. It turns out that there are only several infinite families of these non-trivial factorizations, and an almost simple group with such a factorization cannot have socle exceptional Lie type or orthogonal of minus type. The characterization is then applied to study s-arc-transitive Cayley graphs of solvable groups, leading to a striking corollary that, except for cycles, a non-bipartite connected 3-arc-transitive Cayley graph of a finite solvable group is necessarily a normal cover of the Petersen graph or the Ho?man-Singleton graph.