We lived in Japan for three years, and did a fair bit of traveling around, though mostly in the north of the country. It can be fiercely expensive, but there are ways to cut back a bit and still enjoy the experience.
For eating, a bowl of udon or soba is a cheap and filling option, with meat or fish and veggies as well as lots of noodles for energy and a full stomach. Alcohol is horrendously expensive, generally.
100 Yen stores are great places to pick up decent quality souvenirs and anything you suddenly require (toothbrush? straw hat? sunglasses? photo album? underwear? chopsticks? teapot? - just going in to the store is a trip.
We only stayed in one hostel, in Hakkodate (Hokkaido), which was impeccable and pretty much deserted when we went there in the off season - it was either spring or early fall - warm enough for shorts but all the kids were in school. I don't have my guide book handy, but check one out from the library for current prices and suggestions. Another interesting and budget concious option is to stay at a ryokan - we stayed in one in Tokyo (Ueno area - a good base, incidentally - ours was in an old cobblestoned residential district, but walking distance to the metro, Zoo and park with all the museums), and one on a tree lined canal street in Kyoto. In Osaka we stayed one night at the Holiday Inn Express, which was the cheapest option I could find at the time, and close to a park and tennis stadium, though a hefty walk from the train station in the dark, pelting rain, porting three small children and a backpack. Perhaps not my best decision ever...
There's a ton to see and do in south central Japan, but if you have time to get up north, you'll have quite a different experience of the country and culture. We lived in Aomori prefecture, which is has hiking trails around beautiful gorges and lakes, lots of rural farming areas, sandy coasts, colourful parades (Aomori's Nebuta festival and the lantern festival in neighbouring Akita prefecture are big summer draws) and little of the crowded neon frenzy you find in the south. You could search for glass fishing floats on Mutsu Bay after a stormy night, or check out the wild horses at the Cape. At the "tip of the hatchet" (look at a map, you'll understand what I mean), you can catch a ferry to Hokkaido, which is even more distinct from the typical Japanese experience. Sapporo is the capital, though we visited only Hakkodate, a 19th C. trading port with quite a bizarre mix of architectural styles, great restaurants, and a pleasant atmosphere.
We also visited Okinawa, which is quite crowded and is the most "American" part of Japan that I've seen - K-Mart and Walmart and every kind of fast food, because of all the military bases there, but really interesting once you get past that - different cultural traditions from the "mainland", and wonderful beaches. We swam in the ocean in December, and saw giant clams and barracuda and thousands of colourful fish from a glass bottom boat.
If you have a chance to get away from the main cities and check out the countryside I think you'll really enjoy it - hiking along a river path and suddenly coming to an isolated temple or torii you can almost feel like you're in an old woodblock print.