I lived in Astoria for about a year and a half while I was in the Coast Guard.
If you can cross the bridge into Washington to see the Cape Disappointment Lighthouse. There are some very stunning views from there and it is only like a 30 minute drive from Astoria.
I used to frequent the Wet Dog Cafe when I was there. They make their own microbrews on site and they were pretty good. This was a few years back though...There food was just not bad...Nothing amazing but better then an Applebees or something.
Also there is the Columbia River Maritime Museum which is the official Maritime museum of Oregon. It is very interesting especially with the history of Cape Disappointment. This is also where the Coast Guard Cutters dock and back then we would give tours to anyone that called ahead. The Lightship Columbia is also moored there.
http://www.crmm.org/# I am very much into Maritime history so this was a much bigger draw for me then the Flavel house. The Flavel house is really just a old house full of old furniture.
It sounds like you are driving down 101. Seaside makes a nice stop, but I wouldn't plan too much time there. It is basically a very touristy beach area. Seaside is before Cannon Beach. Another tip about 101...While it is a beautiful road to drive down, if you are going in the summer you will have hundreds of Motor Homes. The speed on 101 can be as slow as 30 mph in some locations.
The air Museum and Cheese factory are both worth doing and can be done in one day easily. You also have some great hikes around Tillamook and I suggest you do them. Well worth it, you will be blown away with the sights.
Newport is my favorite town along 101. Parking sucks here so find a spot and then walk the streets. It is a very charming little city. There is also a Ripleys believe it or not museum and a wax museum and an underwater garden. There is also a nice hike up to Yaquima Lighthouse, which is the tallest in Oregon and you can go to the top for some amazing views of Oregon's Coast.
Coos Bay & North Bend are not very impressive.
That is about as far south as me or my ship went.
Since I was in the Coast Guard at the time, I pretty much had a place to sleep in every port. I dont have any info on hostels...Camping areas are popular along 101.
Also if you are coming down I-5 you may want to take a quick detour to check out the Mt. St. Helens site.
Have you ever been to Portland? If not this city is simply amazing and may be worth staying an extra night. There is so much to see and do. Hope this helps...Let me know if you have any questions.