Prince Albert on Twins Jacques and Gabriella: They're Starting to Say, 'Are We There Yet?'

Years from now, Monaco’s Prince Jacques and Princess Gabriella may recall their 2019 visit to Japan as the family trip that included a local school visit, toy shopping and that time “Dad took us to Disneyland — and left us there.”

The royal twins, who turn 5 in December, were “unofficial” entourage members during Prince Albert’s voyage to the enthronement of Japan’s Emperor Naruhito. Their presence during the 12-day long visit was intentionally kept away from the press with only the briefest sightings, largely on social media.

At Disneyland, the royal dad tells PEOPLE exclusively that he had to leave the twins “because I had another appointment that day and they wanted to keep going [with their nannies and security]. It was okay, but I can tell you that they were really tired at the end of that day!”

Albert says he and wife Princess Charlene are committed to easing their children into royal duties as comfortably as possible. “When they were in Japan they didn’t do anything official there,” he says, “except go visit a school on Naoshima Island. That was more to play with other kids than to really cut a ribbon or anything like that.”

The school visit, a coloring-and-painting play date revealed in a post on Princess Charlene’s Instagram, was an informal stop occurring while their father “was visiting the Tadao Ando museum, seeing art pieces I didn’t think they would be too interested in,” he says. “So they went to visit the school, and there’s some great pictures of them with the other kids there.”

“I think it’s always better to bring them in slowly to official situations,” he adds. Sometimes, says the royal dad — drawing on his own youth — “it’s a little difficult.”

“Sometimes it’s been a little flustering for them when they enter a room full of people staring at them. We’ve had some moments where understandably they were a little taken aback by that, so we have to try to accompany them in these kinds of situations.”

All in all, he says, “they’ve become very used to dealing with public appearances if it’s not too long, too taxing and not too overbearing for them.”

• Can’t get enough of PEOPLE‘s Royals coverage? Sign up for our free Royals newsletter to get the latest updates on Kate Middleton, Meghan Markle and more!<

With the enthronement to attend and an intensely packed diplomatic and cultural agenda, he explains his days were split: “It was usually mornings with the kids. Mostly at the beginning of the trip and the end, and those mornings when I didn’t have to do official functions like the hour meeting with Japanese Prime Minister Shinzo Abe, going to different activities with them.”

Those activities included “the Aquarium, Legoland and Tokyo Disneyland,” he says. “Things like Kidzania and teamLab, which is a great interactive delight with different games in different rooms. Tokyo is unbelievable. They have loads for kids, and the range of activities they can propose for young kids is incredible.” In addition, the twins and their mother caught the consecutive weekend World Cup Rugby victories of South Africa’s Springboks.

“And of course the kids wanted to go shopping a fair amount — and shopping for them is not just any shopping, it’s toys!”

The distance and length of the trip, which coincided with the twins’ Monaco school term break, tested their travel skills, though with trips since New Year’s to Chile and their mother’s native South Africa, their father says the children “are pretty good travelers. Especially on long plane flights.

“Okay, they run around a little bit; jump up and down — but they handle long trips pretty well, I must say. They’re now at the age when they’re starting to say, ‘How much longer?’ and ‘Are we there yet?’ You know, the usual. That was virtually non-existent before. Now it’s starting to be a standard feature.”

The in-flight entertainment program helps, he admits. “There’s more superhero videos and SuperMario and Sonic the Hedgehog and all those other characters than full-length features onboard now. They’re not too much into playing video games yet, and that’s a pretty good thing, but they love watching cartoons and love watching the characters.”