This book constitutes the thoroughly refereed post-proceedings of the Second International Workshop on Statistical Methods in Video Processing, SMVP 2004, held in Prague, Czech Republic in May 2004 as an ECCV 2004 workshop.The 17 revised full papers presented were carefully selected and improved during two rounds of reviewing and revision. The papers are organized in topical sections on 3D geometry, tracing, background modeling, and image and video analysis.