Boleslawiec is definitely off the beaten path for backpackers (ceramics shoppers from Germany flock there by the ton), but it's a neat little town with some cool stuff to see, and not very crowded because all the shoppers just do a circuit of the factory stores. It is, however, quite a long way from the rest of your path. It took pretty much the whole day to get from there to Krakow, and about 5 hours from Wroclaw. My favourite two cities in Poland are Gdansk and Szczecin - the one a major city and tourist draw, the other less touristy, smaller but really fun. Both are well worth visits. From Szczecin, you can spend the day in Swinoujscie (I may have spelled that wrong), a slightly decaying resort town with a lot of charm.
I don't know about many smaller towns/villages in Poland, we tended to stay more in the larger centres during our visits. I bet you could find something smashing in the Tatras, though - maybe one of the ski resorts.
Good luck, and have a great time.