If you are really concerned about damage I would call a granite shop and talk to them. That way you could tell them the type of granite and that may also influence their decision as to which method is best for removal.

They make a label adhesive remover you can get at office supply stores and other places. It's really just a kerosene based product and I've used kerosene (NOT gasoline) in a pinch. Do not use Goo Gone or any of the xylene/tolulene stuff).