It would seem that there would be no problem right next to the driveway since they were able to build the driveway. If you build a short wall there it would allow you to place fill behind the wall and build a base over the shale without exposing the toe of the wall that you want to build 10' back. It may not work for a wall 10' back, but maybe you could bring that one forward a little bit as well.
Please excuse the cheesy diagram:
