I've gone the other way with CD/data Cd based MP3 players. In my experince they have better features, are more adaptable, and have a better display.
Since they dont have to put all that money into a mechanical hard drive, they can put it into the display.
One CD will hold 700+ MB of Music, that's a ton. skips are non exsistant and they're fast to acces.. they also play regualr CD's.
If that's too big for you and you are set on a small one look into the mini disc type, or a flash type.
Make sure that you can also upgrade the flash memory. If the $100 is 512 MB that's a ton of room but with no display how you gonna find your songs?
Ipods have a heck of a name, but in My Very humble opinion the names what makes 'em so spendy, not the features...
I also like USB connectivity or the ability to burn my own disks.