I was there a few years ago by myself; I stayed in Avranches, which is a big enough town that there is affordable accomodation. There is a local bus every day in the high season that leaves in the morning, allows you to spend the day there, and then picks you up at like 3 pm. I was afraid it wouldn't be enough time there, but it is not a big place and it is pretty crowded as well; by three I was done. If you do want to stay longer, you can get a taxi back to Avranches, but it is kind of expensive. The other cool thing about Avranches it that that is where all the "treasures" of the Mont are; there is a museum with all the books that used to be housed at the monastery. It was a fairly important library and scriptorium (place of book production) in the Middle Ages, and the books themselves are no longer on the island, they are in Avranches. Good luck, it is beautiful.