Oh what a pain this can be.
I've gone through several different methods to find one that works and I recently migrated all the CD's to DVD.
Originally I just burned stuff to CD as I got enough to fill a disk. Then I would use Iview Media Pro to catalog that disk with a thumbnail and add keywords to those photos so that I could search them later. This was impractical but I didn't know a better way.
Then I found a nice little program called PhotoMechanic that would batch keyword and caption photos so that the information was in Photoshop's file info and imbedded in the photo.
When I migrated the CD's to DVD I had to re-key word the photos because the key wording that I had done before was in the catalog not the photo.
You can also use a program like Extensis Portfolio to catalog you photos, but I prefer Iview. I can search by filename, any of the standard keywords that I have decided to use, and by disk.
Anyway, make sure you do the keywords from the beginning. My catalog now has something like 8,000 images in it. I burn 2 disks of everything; keep one set at home, one set in my office. Hard drives are pretty delicate instruments; I don't trust them unless I have them mirrored in a raid system. That means that they are set up in a way that what you save to one of them gets saved to the other. You don't get access to the slave unless the master fails.
Hope that makes sense.