In his latest cover interview with TIME for on Sept. 25, the legendary musician, 76, opened up about how success felt “exhilarating and terrifying” for him, especially after his first appearance of the outlet’s cover back in 1975. But at the same time, his worries about fame changing him as a person only grew from there.

“It’s making you very, very different than all the people you grew up with,” he said. “[Fame is] a very distorted lens to live your life through.”

He also recalled his sister Pam telling him about the paparazzi outside looking into the kitchen at his family home, which only added to the singer’s discomfort with his newfound status. “You have to be very protective of yourself, of what matters dearly to you,” he noted.

How Bruce Springsteen’s Roots Influenced His Career

Before Springsteen became the rock legend, he grew up in a working-class family with the expectation that he would work in his hometown at the textile mill. But once his fame grew, Springsteen took his understanding of the working class Americans and channeled it into some of his biggest hits in his album “Born in the U.S.A,” “My Hometown,” and other tracks.

Bruce Springsteen on How His Kids Feel About His Fame

The rocker previously opened up more about fame in his interview with The Times back in June, where he revealed that his children — Evan, 34, Jessica, 33, and Samuel, 31 — don’t display any items from his career in their New Jersey family home.

“Except for a few guitars and a piano, you wouldn’t know musicians lived here,” he told the outlet. “The kids grew up in another house in town, a midsized home not unlike the ones their friends lived in, and we tried to keep a very natural domestic existence for them. Anything else is a burden they don’t need.”

He noted that his kids did not care much about his fame for the most part and noted that they mainly “ignored” it. “They might come to a show, bring their friends, but it’s never been a central part of their lives,” he shared.