Mr Luxon left on the advice of his security team when a bunch of vocal attendees trailed the prime minister during a walkaround.
The group heckled Mr Luxon on his government's pledges to review hot-button issues that affect LGBT people, but primarily, called for him to support an unconditional ceasefire in the Israel-Hamas conflict in Gaza.
Mr Luxon said he didn't feel unsafe, calling it "a bit of a shame".
"People are free to protest as they wish," he said.
The right-leaning government has come under attack from rainbow communities for its plans to axe and replace New Zealand's school-based relationship and sexuality education, and to review public sport funding based on their gender policies.
The policies were only campaigned on by the smallest of the government parties - NZ First - which received 6.1 per cent of the vote.
However, in a concession to the populist minor party, Mr Luxon's National party included the pledges in a coalition agreement needed to form government.
Labour has launched a petition to keep the relationship and sexuality education, which were overseen, ironically, by former NZ First MP Tracey Martin.
"Changes will discourage meaningful and respectful conversations around sexuality, consent and gender," Labour education spokeswoman Jan Tinetti said.
"These resources were created to answer the call of teachers and communities for more action against bullying, violence and child abuse. Taking the axe to them will be incredibly damaging for years to come.
"I know I am not alone when I share my hope for our schools to be inclusive and safe places for all our students whatever their gender, race, nationality, beliefs, or sexual orientation."
Mr Luxon said Labour's campaign was "embarrassing".
"We want sex education and good sex education in New Zealand schools, it ain't going anywhere," he said.
Mr Luxon said "all we've asked for is that the curriculum is age-appropriate and parents get consulted on it", although those are two features of the current guidelines.
Parents able to feed into what is taught - with mandatory consultation windows - and are able to withdraw their children from the classes.
A Labour spokeswoman said the petition had received more than 1000 signatures in the first 24 hours.
5°C