These items are NOT recyclable with your bottles and cans. Tapes cause terrible tangling problems at the sorting faciities. CDs, DVDs and cases are not the kind of plastic they can sort and sell, and will be landfilled or mis-sorted.
For noncommercial (taped, burned, or home recorded) CDs, DVDs and cases, mail to CD Recycling Center of America, 68H Stiles Road, Salem, NH 03079.
GreenDisk accepts electronics and electronic media "technotrash", including difficult to recycle VHS tapes and microfilm, for a fee.
Vinyl records can also be made into party bowls.
Otherwise, these items go in the trash.