Stephen Colbert and his wife, Evelyn “Evie” McGee Colbert, have been happily married for over 30 years, but their love story didn’t begin with sparks flying at first sight.

During an appearance on Andy Cohen Live, the couple looked back on their initial meetings and shared lighthearted anecdotes about their early impressions of one another.

Both Stephen, 60, and Evie, 61, grew up in Charleston, South Carolina, and first crossed paths in their youth.

However, romance didn’t blossom right away. “We met several times,” Evie revealed.

Stephen jokingly chimed in, “The first couple of times, it did not take.”

Their earliest encounter was at a college party where they were introduced by a mutual friend.

Reflecting on that moment, Stephen quipped, “I was chemically impaired, and Evie was a distant frost queen on a mountaintop.

” Laughing at the memory, he admitted, “That first time, I thought, ‘This is going nowhere.’ It was literally a 30-second conversation.”When asked for her impression of Stephen during that time, Evie didn’t hold back.

“An immature, skinny boy,” she said, laughing. Their true connection happened later, in the lobby of a theater, where the pair discovered they had grown up in the same town.

In a 2016 interview, Stephen recounted that pivotal moment, saying he had an immediate realization: “Her. That’s your wife. You’re going to marry her.”

The couple tied the knot in October 1993 and built a life together, raising three children: daughter Madeline and sons Peter and John.

By all accounts, their family life is a cornerstone of their happiness.

“Stephen and his family are very happily entrenched,” a New Jersey neighbor shared with PEOPLE in 2015.

“You can always see him with his family. Family and community have always been important to him.”