If you were to regrout or patch, don't use your leftover grout. It only has a certain shelf life so your results (durability and color matching) would most likely disappoint.

A recommendation that many have had excellent results with would be to use a colorant like Aquamix Grout Colorant.

You clean the grout with Aquamix's grout cleaner. Then apply the colorant. It's epoxy-based, so it will be a film-type of colorant over your existing grout. Not a penetrating-type of sealer.

It does work very well for recoloring problematic grout, and being epoxy, it's an excellent sealer as well.

If you want to remove the existing grout, I don't chisel it out. I use a diamond bit either in a dremel-type of tool, or in a multimaster-type of tool. Depends on the situation. You want to remove the exiting grout at least as deep as the grout joint is wide.

Vacuum/clean it out well. Then regrout.

Good luck!