Trying to do the same myself and have had no luck with "computer" microphones.

I'm going to try borrowing a proper mic and see if that makes a difference

Forgot to say I'm using a griffin imic to plug the mic into

I tried using the mic builtin to the firewire isight
but that seems to be only good for voice.

I used a basic computer mic, which is ancient (like, 10 yearsish) and I believe came with a weird computer camera, and says "quickshot" on it, and the "recordpad" portion of the trial version of wavepad. I had to record an audition cd last fall. my saxophone friend did it with me on the same set-up, and both sets of recordings came out fine.

i think the program has just as much to do with the quality as the mic- with this same mic, if i use the basic sound tool on windows, it has poor scratchy sound quality.

I have some for of Sony Voice Recorder. I just record my music, plug it into the computer with the software it came with and burn it to a CD. When you play the songs you recorder on the recorder, they sound like crud. When you put them on a computer, then they sound good.

Quite a few programs can be used to change formats

http://audacity.sourceforge.net/ is a free one