First off, there's FL Studio, but it's not free. The other alternative I know about is Open ModPlugTracker, which is free, but can be a royal pain to learn.
I don't know about other 'costly' programs, but FLStudio has a heap of preset instruments and samples, and also has many synthesisers, and you can find free packs like Timmy Techno Ultimate that has 300 synths. Erik McCLure uses it, and so do I, but I haven't produced anything useful with it yet...
FL may take a bit more time to learn, but depending on what standard you wan't to get to, you can get the $99 version or the $299 "producer's edition". Free trail is the producer's edition, you can't save .flp, but you can export your song as wav or mp3. (Leave it running for days at a time and you can create a good song eventually)
ModPlug Tracker can use MIDI and wav samples, and now has VSTi support, but I only know how to use the basic features. I think it's fairly easy to learn the basics, + it's free. I use this too...
And this: MuseScore is also free (I think) and, if you know how to read music, then you're lucky, because you just drag and drop notes onto the staff. (I think it uses wav samples, but has basic instruments included like piano, violin, etc) + it has a tutorial.