Spectral unmixing of urban land cover relies on representative endmember libraries. For repeated mapping of multiple cities, the use of a generic spectral library, capturing the vast spectral variability of urban areas, would constitute a more operational alternative to the tedious development of image-specific libraries prior to mapping. The size and heterogeneity of such a generic library requires an efficient pruning technique to extract site-specific spectral libraries. We propose the “Automated MUsic and spectral Separability based Endmember Selection technique” (AMUSES), which selects endmember subsets with respect to the image to be processed, while accounting for internal redundancy. Experiments on simulated hyperspectral data from Brussels (Belgium) showed that AMUSES selects more relevant endmembers compared to the conventional Iterative Endmember Selection (IES) approach. This ultimately improved mapping results (kappa increased from 0.71 to 0.83). Experiments on real HyMap data from Berlin (Germany) using a combination of libraries from different cities underlined the potential of AMUSES for handling libraries with increasing levels of generality (RMSE decreased from 0.18 to 0.15, while only using 55% of the number of spectra compared to IES). Our findings contribute to the value of generic spectral databases in the development of efficient urban mapping workflows.