Well in my experience the roads are insane but if you're adventurous you could do it. Windy mountain roads with blind turns, gigantic potholes that you just have to know are coming and slow down from 70 to 10, minimal compliance with traffic rules, and I don't know if the maps/signs are all that great. Plus you'd probably want decent spanish to deal with the police checkpoints.
Still, you'd get used to it after a week or two so it's possible. But busses are incredibly cheap down there. I couldn't really understand how they could afford to run them so cheap given that gas is just as expensive as it is here. 5 hours on a bus in Peru was about $3-4 (whereas a litre of gas was about $1.5). Busses are very frequent too.
Are you trying to save money, or just have more flexibility, or something else?