For me, this is in two parts:

The Screamin' Cheetah Wheelies
Magnolia
It took me almost three months of listening to it at least once a day to "get it". Since then, it is definately one of my favorites.

Frank Zappa
You Can't Do That on Stage Anymore VOL 6
Got it as a birthday present because I liked Joe's Garage Album (although the first time I saw Zappa on Saturday Night Live I thought he sucked and that wasn't good music, you know how it is when you're raised on Top 40)
Anyway, I listened to it once. Hated it. Let it sit for a year, pulled it out and wow, there was actually some good stuff musically. Listened to it non-stop for quite awhile and then it was always in the rotation. This started my "Zappa phase" where I bought at least two Zappa albums a week.

Name some albums that fit in this category for you.