Na, i don't think so, some people swim, some do not, not that big a deal...People on the tour might razz ya bit about it, but find something that you do well to take away from the fact you can not swim.
What tour are you going on?? Will you trip be all tour, or is some time on your own??? Alot of tour groups will not let you bring a backpack with a metal fram ( internal or external)
, check with them before you go, you might not have the option of bringing a backpack.