I take my SLR with me when I travel, but that's because high-quality photos are (one of) the point(s) of my trips. I'm not worried about sticking out like a sore thumb; chances are you will anyway. With the new camera bags designed to look like, well, NOT camera bags, it's easy to carry one around and not look like an asshole. Just remember to be aware of your surroundings, and most of all, be polite and discreet when snapping away.
However: unless you're REALLY set on investing the $$ into a big camera in order to potentially make money from your future photographic endeavors, just take your trusty point & shoot. They can't be beat for portability, compactness and stealth.
In order to get high-quality photos with an SLR, you need to invest in good lenses (cha-$$$$$-ching) and it's actually a lot harder to get a good shot than with a point & shoot that does all the work for you. It takes a lot of practice and know-how and almost always a lot of work in post-processing. That said, if you're willing to put in the time, effort and hard-earned cash into it, it's worth it.