For time-harmonic scattering of electromagnetic waves from obstacles with uncertain geometry, we perform a domain perturbation analysis. Assuming as known both the scatterers' nominal geometry and its small-amplitude random perturbations statistics, we derive a tensorized boundary integral equation (BIE) which describes, to leading order, the second-order statistics, i.e., the two-point correlation of the randomly scattered electromagnetic fields. Perfectly conducting as well as homogeneous dielectric scatterers with random boundary and interface, respectively, are considered. Tensor equations for secondorder statistics of both, Cauchy data on the nominal domain of the scatterer as well as of the far-field pattern are derived, generalizing the work by Harbrecht et al. (2008, Sparse second-moment analysis for elliptic problems in stochastic domains. Numer. Math., 109, 385-414) to electromagnetic problems, and being an instance of the general program outlined by Chernov & Schwab (2013, First order k-th moment finite element analysis of nonlinear operator equations with stochastic data. Math. Comp., 82, 1859-1888). The tensorized BIEs are formulated on the surface of the known nominal scatterer. Sparse tensor Galerkin discretizations of these BIEs are proposed and analyzed based on the stability results by Hiptmair et al. (2013, Sparse tensor edge elements. BIT, 53, 925-939); we show that they allow, to leading order, consistent Galerkin approximations of the complete second-order statistics of the random scattered electric field, with computational work equivalent to that for the Galerkin solution of the nominal problem up to logarithmic terms.