Not that difficult. Now is a good time to invest in a multi-tool (eg. Fein multimaster or similar.) Remove the grout with it (grout blade). Then cut an X in the stone with a grinder, catching the dust with a shopvac....or use a masonry bit and make several holes in the stone and remove the stone in pieces. Once you got a few out, the rest will come out with the aid of a 5-in-1 painter's tool and a hammer. After all the pieces are out, use your multi-tool with the rasp blade to grind out the remaining thinset. Now it's ready to try again.

thanks! However, buying a multi-tool isn't going to happen. I discovered that my angle grinder will take a wet/dry blade and seems to cut the stone ok, so I may try it in the way you suggest. Hopefully, then, I can grind down and chip out the thinset. What do you think?

You can get an inexpensive multi-tool at Harbor Freight for about $30.

I second the Harbor Freight recommendation. I thought I would just be trying the tool to see if I liked what it did and then I'd buy the Fein if I did. Several years later.....the knockoff is still going strong. And it's the most used tool that I own, because it's the most versatile. It is SO worth the money!