What I do on my iPod, I created a playlist based on my "eclectic mix" that only includes music with a last played date longer than 6 months ago. That way I have a playlist that's a manageable size for my 8GB iPod, and once I hear a song, it scrolls off the playlist and makes room for something I haven't heard recently. You could vary the date depending on the size of your library, the capacity of your phone and how often you listen to music.

There's also a setting now that automatically puts a lower-resolution file on your portable device. Used to be if I didn't want hi-res copies taking up space on my iPod, I had to make a lo-res duplicate on my computer and check that one to sync. No need for that any more.