This book constitutes the thoroughly refereed proceedings of the 4th International Workshop on Software Engineering and Middleware, SEM 2004, held in Linz, Austria, in September 2004.The 16 revised full papers presented went through two rounds of reviewing and improvement and were selected from 44 submissions. The papers are organized in topical sections on middleware services, ubiquitous computing, performance and QoS, and building distributed applications.