Unfortunately, compostable and biodegradable are often used interchangeably. But they're not the same thing. Biodegradable means that a product can be broken down without oxygen and turn into carbon dioxide, water, and biomass within a reasonable amount of time. Biodegradable products usually break down into something natural, but can leave behind trace metals. Everything will biodegrade eventually, so labeling something as biodegradable means the consumer really has to do some research. For example, aluminum cans will biodegrade in the ocean in about 176 years while hard plastic bottle caps will take approximately 400 years -- not the positive image you may have been picturing.
Compostable items break down to carbon dioxide (which can be stored, instead of released into the atmosphere), water, and other forms in approximately 90 days and leave no toxic residue. Composting occurs in certain conditions and results in a rich hummus full of nutrients. All compostable items are biodegradable, but not all biodegradable items are compostable.
Composting can remove 20-50 percent from your household waste stream, save you money, reduce the burden on landfills and replenish your lawn, trees or garden for free. Compost helps retain moisture, makes soil easier to work (clay soil anyone?), suppresses plant diseases and pests, all while reducing the need for chemical fertilizers. In addition to trapping carbon dioxide, it also prevents the creation of methane that occurs in landfills. Methane contributes to climate change and displaces breathable oxygen.
Note that some items are listed as "commercially compostable." These items need more heat to break down, which means a bigger pile than what you have in your yard. The process and results are the same. Items are generally more heavy-duty in composition, and meat/bones also fall into this category.
Composting at home doesn't have to be difficult or time consuming, but there are some things you need to know before getting started. You need the right mix of "browns" (dry leaves/plants and twigs) and "greens" (food scraps, coffee grounds) and water is needed. If you compost correctly, you won't attract vermin and it won't stink. Try the resources here and here to get started.