I think I'm the resident snowboard nut here, so...
What you'll usually get with the rental: Snowboard, boots.
I dunno how it works in NZ, but here, even if you're learning, it's best to pay a few extra bucks for the "demo" or "premium" package. The typical rental boards, bindings and boots are generally crap, which make learning to ride that much more difficult.
Ok, you've already got the Gore-tex jacket. Good start. As mentioned, snow pants are a must. Looser cut pants are the norm for snowboarding, and considering's now the off-season in the Northern Hemisphere, there are good deals to be found. Make sure whatever you get has some zippered pockets - they come in handy.
Long underwear - some thermals or stretchy long undies for under the snow pants, they're pretty vital, IMO. And for the top, I'd recommend a Nike Dri-fit or UnderArmor type athletic top, to wick away the sweat that will inevitably start dripping under all that waterproofing. Layer with t-shirts over that, though I rarely need more than one layer between my undershirt and my jacket.
Gloves - mittens are cheaper, but get fingered gloves that fit you well. Why? When you're learning, you'll find it easier to strap/unstrap your bindings with decent gloves, rather than fumbling with them and getting frustrated.
Snowboarding socks - I loooooove my Wigwam Ultimax socks, and your feet will love them too. You can get away with reusing one pair, but get a couple if you can. Don't cheap out on socks. Your feet are the primary point of contact/stress on the board, so you want them to be dry, warm, and comfortable, with good circulation.
I've never found goggles to be necessary except when it's snowing moderately-hard. If you haven't worn them before, they can be a bit disorienting, but if you're planning on getting them anyway, go ahead. Personally, I'm more of a sunglasses guy - a holdover from my skiing days, I guess.
Protective gear - optional, but I recommend having wrist guards when you're learning (your natural tendency will be to incorrectly break your fall with your arms, which can bust your wrists...). A helmet, as dorky as it feels, is also pretty important. I don't wear mine on 'cruiser' days, but otherwise I do - I've had several friends of different levels suffer from concussions, one with a bout of amnesia. What's more uncool?
Anyway, have a blast - remember, the first few days might be REALLY frustrating, but don't get discouraged. Once you get the hang of making a couple of turns in a row, you're well on your way to getting good!