Sean Penn is setting the record straight. 

The Mystic River star recently addressed the long withstanding rumors that he physically assaulted his ex-wife Madonna—to whom he was married from 1985 to 1989—with a baseball bat, noting it has followed him around for years.

In fact, the Oscar winner described a date who asked him point blank about the supposed incident, telling The New York Times, "She's looking at me like I killed her dog."

In addition to denying the rumors, Sean also emphasized how they initially caught him off guard.

"I didn't know what the hell she was talking about," Sean explained in the interview published June 24. "I think it's fair to say that I'm not the biggest guy in the world. But if I hit Mike Tyson in the head with a baseball bat, he's going to the hospital."

But while the Milk actor—who has also been married to Robin Wright and Leila George—denied any incident involving a bat, he did share insight into his and Madonna's turbulent marriage.

"I had a freaking SWAT team come into my house," he recalled of one incident when Madonna feared there were guns in their home. "I said: ‘I'm not coming out. I'm going to finish my breakfast.' The next thing I knew, windows were being broken all around the house and they came in." 

As for Madonna—who later married Guy Richie for eight years—she's previously shared insight into her marriage to Sean, even shutting down the rumors, which began circulating in the ‘80s. After Sean filed a defamation lawsuit against Lee Daniels in 2015 for remarks he made in an interview alluding to Sean engaging in domestic violence, Madonna issued an affidavit denying the rumor.

"While we certainly had more than one heated argument during our marriage," the "Popular" singer's affidavit read, according to court files obtained by E! News at the time. "Sean has never struck me, 'tied me up,' or physically assaulted me, and any report to the contrary is completely outrageous, malicious, reckless and false."

The I Am Sam star later settled the $10 million lawsuit after Daniels issued an apology and extended an unspecified charitable donation to an organization of Sean's choice, per The Hollywood Reporter.  

And while Sean and Madonna may have cut all romantic ties back in 1989, he noted he has a lot of love for the Material Girl. 

"It turns out it's a lot quicker to repair a friendship after divorce if there are not kids involved," he noted to The New York Times. "She's someone I love."

E! News has reached out to reps for Madonna regarding Sean's recent comments but has not yet heard back. 

For the latest breaking news updates, click here to download the E! News App