I would use a putty knife and scrape it off. I suspect that you will have more of a problem getting off the glue holding the foam down. Is the window made of wood or metal? Is it painted?

Mineral spirits might take off the remaining glue - pour some on a rag and rub it. But you have to do this gently because it may also take off the paint.

It's not good to breathe in foreign particles, but I do this kind of stuff all the time and am still alive and breathing.

The foam that is disintegrated is pressed into a horizontal gap between the upper and lower window sashes. There is no glue/adhesive. I had cut the insulating foam to proper width and pushed it between the gap in the window sashes, since the lower sash had to be raised for the installation of the AC (and a gap was created). So all of the foam disintegrates into dust-like particles when touched lightly. I have to find a way to get all of that foam out of there without making a huge mess.