It works very similarly to a religion actually. When something stays with you from childhood it's impossible to separate it from your identity. I'm a big Star Wars fan, for instance. I might claim that I've outgrown it, but listening to John Williams' music moves me deeply, so it's impossible for me to make that claim with a straight face. The real problem is not when some work of art, a sports team or a celebrity is part of your identity, but when you start feeling that the object of your fandom somehow belongs to you. Entitlement is what breeds toxicity.