You really don't need very expensive equiptment to do this.
Back when I didn't have anything fancy, I'd just use a digital recorder that I used to record lectures and that worked fine. Sound quality wasn't quite studio quality, but hey, it worked, and it was what I had. (I acually mainly used that method to make like a ghetto loop pedal...so I could pratice improving over it, lol)
Eventually I came around to recording on my computer for fun. And I just used freeware. In my case, I use Audacity. Its great. I still use it even though I have a copy of Adobe Audition (which I haven't installed yet.) And I just used my computer's built in mic. Worked fine.
Now that I have a bit more gear, I can plug into my combo amp and put some reverb on it or whatever, and use the aux out from the amp to go into my computer using a quarter inch to stereo headphone cable. That works great too.
But you really don't need anything fancy. Just get Audacity and you'll be good to go.