We use imaging from the Next Generation Virgo cluster Survey (NGVS) to present a comparative study of ultra-compact dwarf (UCD) galaxies associated with three prominent Virgo sub-clusters: those centered on the massive red-sequence galaxies M87, M49, and M60. We show how UCDs can be selected with high completeness using a combination of half-light radius and location in color-color diagrams (u∗iKs or u∗gz). Although the central galaxies in each of these sub-clusters have nearly identical luminosities and stellar masses, we find large differences in the sizes of their UCD populations, with M87 containing ∼3.5 and 7.8 times more UCDs than M49 and M60, respectively. The relative abundance of UCDs in the three regions scales in proportion to sub-cluster mass, as traced by X-ray gas mass, total gravitating mass, number of globular clusters (GCs), and number of nearby galaxies. We find that the UCDs are predominantly blue in color, with ∼85% of the UCDs having colors similar to blue GCs and stellar nuclei of dwarf galaxies. We present evidence that UCDs surrounding M87 and M49 may follow a morphological sequence ordered by the prominence of their outer, low surface brightness envelope, ultimately merging with the sequence of nucleated low-mass galaxies, and that envelope prominence correlates with distance from either galaxy. Our analysis provides evidence that tidal stripping of nucleated galaxies is an important process in the formation of UCDs.