Jimmy Kimmel was amazed by Paul McCartney’s party with Taylor Swift and Travis Kelce

• Jimmy Kimmel reveals to Howard Stern some celebrity observations he made at Paul McCartney’s star party
• He shared that Taylor Swift, Travis Kelce, Bruce Springsteen, Steven Spielberg, Mick Jagger and many more were present
• The late-night host admits he was impressed by the celebrity guests

Jimmy Kimmel talked about what it was like to celebrate at Paul McCartney’s house with other A-list celebrities.

During a conversation with Howard Stern on Wednesday, June 26, Kimmel and his wife Molly McNearney said they celebrated with Taylor Swift, Travis Kelce, Bruce Springsteen, Steven Spielberg, Mick Jagger, Jennifer Aniston and Courteney Cox.

Stern, 70, had previously heard about the star-studded party and asked Kimmel, 56, how the event came about.

“We were invited to dinner that night and then the host of the dinner was invited to this party (at Paul McCartney’s house) and said, ‘Why don’t we all go to the party?’ And so we went to the party,” Kimmel explained.

Although there were rumors that Swift DJed at the event, Kimmel said she simply “had her iPhone with her and somehow hooked herself into the house’s system,” clarifying, “But it wasn’t like she was hired to work there.”

Stern asked if she was at the party with “her husband” Kelce.

“Yes, indeed! Yes, indeed,” Kimmel replied, and McNearney, 46, added, “He looks very good.”

“Boy oh boy! Did you see the big guy carrying her around onstage like he was Frankenstein carrying the little girl down to the river?” Stern asked, referring to Kelce’s stage appearance at Swift’s Eras Tour concert on Sunday, June 23 in London.

The radio host then asked if there was a “jam session.” There wasn’t, but Kimmel recalled, “People were dancing in the kitchen, including Molly.”

Stern, who was fascinated by the soirée with about 100 guests, asked the couple what they had said to the 80-year-old Jagger.

“Nothing, I haven’t spoken to him,” Kimmel laughed. “I think what you do at a party like that is you gravitate towards the people you already know and say, ‘Can you believe this is happening and that we’re here?’ And you leave it at that.”

McNearney added that she was “so relieved to see Aniston and Cox,” and that she spent time “in the corner” with the two. “I didn’t know how to hold my body. I didn’t know where to look. I didn’t know what to do. It was like another famous person came after another.”

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from celebrity news to compelling human interest stories.

Kimmel chatted with Springsteen, 74, he told Stern. “He’s funny, you know? We had a good conversation,” Kimmel said, noting they talked about Elvis and LA. “We even had this moment where we were like, ‘Can you believe this party?’ Even Bruce Springsteen was like, ‘This is a party!'”

“Did Courteney Cox and Bruce Springsteen dance?” Stern asked, referring to Cox’s appearance in Springsteen’s 1984 music video “Dancing in the Dark.”

The PEOPLE Puzzler crossword puzzle is here! How fast can you solve it? Play now!

The couple said that was exactly what they wanted and tried to convince Cox, 60, to attend his show in Los Angeles the next day and dance on stage with the singer.

Stern’s co-host Robin Quivers asked if they had spoken to Swift. Kimmel confirmed she had, while McNearney spoke to Spielberg’s wife, Kate Capshaw.

“My God, how did Mount Rushmore survive that night with so many heads falling off it,” joked Quivers, 71.