Snag an Axim X50 off of Ebay, a cheap 2 gig SD card, a cheap 2 gig CF card, and you have nearly 4GB of flash memory at your disposal. Grab GSPlayer, and you have MP3, WAV, OGG, WMA, and with a plugin provided from the GSPlayer site, MIDI playback. You'll also have WiFi/Bluetooth connectivity, be able to access emails and such, etc etc. I've seen used X50's go for bout the same price as Creative's 4-6 gig flash players or the ickpuke nano 4gb models. Add a really cheap card reader, and you can store anything on the cards you want. If you like to listen to MIDI's, the PPC platform is the only one that I've seen be able to play them back. No flash player can do this to my knowledge.
Of course, this will be much more complicated than just a flash player, but it provides flexibility beyond compare.