Good taste can be idiosyncratic, in fact, it's expected to be. You're supposed to like what you like for your own well-thought-out reasons, and not just like what everyone else likes. (There are also shared cultural and class standards of “good taste,” but those aren't what I'm talking about.)
Someone with taste has a well fleshed-out theory about what makes a work of art good or bad. The cultivated observer is supposed to be able to see something new and rigorously scrutinize it according to their code…
Having coherent reasons for your preferences is integral to the concept of good taste. You're supposed to be able to recognize a band that swings hard, or a rocking baseline, or witty lyrics, or whatever you think is important in music.
You gain status for your good taste if you can reliably pick stuff that other people will like. You can't be capricious. If you recommend songs strictly because they have sentimental value for you, they're unlikely to appeal to other people. You have to appeal to shared musical values.
“Guilty pleasures” are things people like but can't justify liking. The concept of a guilty pleasure only makes sense if you try to live by an aesthetic code in the first place.