Diervilla is beautiful as is Hypericum, probably not too fast spreading though. Liriope would be good as well.
Fragrant sumac, I've used honeysuckle on one bank, but that might get invasive as well. Chelone, Fallopia, Kalimeris. Not sure if any of these would be considered invasive where you are. Or Oenethera fruticosa or missouriensis are beautiful as well.
Aren't cotoneasters one step above weeds? (I know, I'm a plant snob)
