You could use a staple gun to shoot another staple

Or drive a couple small finish nails

Or buy a package of Glaziers Points from the hardware store that you push-in with a screwdriver

Pro Help: The frame shop has a tool that shoots a plain glazing point into the wood frame

duct tape, or any similar heavy tape like strapping tape should work on small pieces. Driving nails or using a staple gun on a frame with glass can result in broken glass. And take a guess how I know that . . .

I worked as a picture framer for years, and still do my own framing. If you plan to re-frame more than a few pictures, it might be worth investing in one of these tools: http://www.framing4yourself.com/shop/products/4-8/
I've got the point inserting tool and love it. You can use any size brad, depending on picture size.

Thanks for the advice. I only have a few frames that need an update and don't want to invest in a tool. I actually considered duck tape before I posted :). I will look for the glazier points. Are they simple to insert or is there a danger of glass breakage with them too?

Glazier points are easy to use as long as its soft wood you're pushing them into. You just push them in with a flat head screw driver. Just make sure the screw driver is as level as possible with the back of the picture so you're not putting downward pressure on the glass and you should be fine. Any hardware store should have them.