You should use a silconized caulk between the counter and backsplash tile. It is flexible (even if it doesn't feel flexible) so won't crack. You should be able to easily dig out your grout and do caulking. The caulk we got was right next to our grout and came in colors that matched but I've heard other people have used the all-purpose kitchen and bath caulk. Good luck!

Yup. I agree.

And hopefully it's all coming out--if not, you'll have to get one of those little grout saws and the job gets unpleasant.

siliconized acrylic caulk, not pure silicone.

Depending on the grout brand, some actually make sanded siliconized sanded caulk that will match the grout color. A few brands I am familiar with are Tec, Super Tech and Custom (sold at Home Depot). The first two may be difficult to find unless you go to a dedicate tile store.
Just to reiterate do not use 100% silicones of any colorâ¦. they are not water soluble and unless you are skilled with a caulk gun you wonâÂÂt be happy with the results.
Cheers