The couple of five years treated close friends and family to a night of festivity, food and plenty of dancing when they wed Saturday at the Nihi Sumba Resort on the island of Sumba, off the coast of Indonesia.

“It was the most dreamy, incredible night ever,” Carter, 29, tells PEOPLE exclusively in this week’s issue, on stands Friday. “And it was about us, and a celebration of the relationship we’ve always had.”

The long weekend began with a jungle-themed welcome party for guests that included Jenner’s brothers Burt and best man Brandon Jenner as well as mom Linda Thompson. (Dad Caitlyn Jenner was traveling for work, and his sisters Kendall and Kylie Jenner did not attend).

The next day was the rehearsal, followed by a white party and an activity that drew inspiration from the local culture.

“Sumbanese people are especially talented at horseback riding,” says Carter. “They’re amazing equestrians. So we decided to have horse racing on the beach. Everyone placed bets and the money went to benefit the Sumba Foundation [an organization that provides aid to the local community].”

During the wedding ceremony, which took place on the resort’s spa property, Nihi Oka, Carter and Jenner, 34, exchanged personalized vows against a dramatic backdrop of the ocean and surrounding rice fields.

“She was the most beautiful bride I’d ever seen,” says Jenner. “And the best part was having our friends and family to witness our special moment.”

Adds Carter: “Saying our vows I just felt so happy and excited. A lot of our guests cried. I didn’t cry, but my mom said Brody got teary-eyed when he saw me!”

At the beachfront “jungle feast” reception, candlelit tables were decorated with coconuts and abalone shells, and guests dined on local fare including fresh fish with a sambal sauce, coconut rice and papaya salad. For the big finish, the couple served “a really funky” pineapple upside down cake, says Carter, who was inspired by a photo she found on Pinterest. Lounge areas were set up on the sand, and guests also received maple candies as a treat, a nod to Carter’s home state of New Hampshire.

• Want to keep up on the latest from PEOPLE? Sign up for our daily newsletter to get our best stories of the day delivered straight to your inbox.

Jenner chose Ed Sheeran’s “Perfect” for the first dance, and just after the song was complete, fireworks went off as a surprise from a friend to the newlyweds.

Friends and family made speeches toasting the couple — Jenner’s mom Linda read a poem — before the party really started. At 10 p.m., guests headed up to a friend’s villa, where the party went strong into the wee hours. “It was nonstop dancing until 4 a.m.,” says Carter of the afterparty, complete with a DJ and sushi. “It was a really fun night.”

Now, Jenner and Carter, who will honeymoon in Africa, are focused on starting their lives together — and yes, kids are on the horizon.

“I’m very excited about kids,” says Jenner. “I think I’ll be a good dad. A fun dad!”

• For more on Brody and Kaitlynn’s wedding, pick up this week’s issue of PEOPLE, on stands Friday

Adds Carter: “Our kids will never be bored! It will happen exactly when it’s supposed to.”

And for now, the newlyweds are happily settling into married life.

“I look forward to just spending the rest of my life with Kaitlynn,” says Jenner. “She makes me a better person and she’s just as beautiful on the inside as she is on the outside. I feel like the luckiest man on earth to have her by my side.”