How concerned should I be about the zonolite asbestos my roofer spread all over the outside of my house, flower beds, and lawn?

Venting needed to be opened up in my soffits when the new roof was installed. I was told it was unlikely for the zonolite to be in the soffits since it is poured between the joists and any small amounts that would be there would be caught by the tarps they would set up. Well, there was allot and for some reason after the roof was stripped the tarps were packed up before the soffits were opened up and it got everywhere. By the time I noticed I could see them already leaf blowing it off the deck and patio. There are thousands of vermiculite pieces under the deck and in the flower beds.

What I have done so far is hose it off the house and cracks in the deck and patio as best I could. On a rainy day I plan to crawl under the deck and scoop the top layer of soil off with the pellets to be disposed of. When the growing season is over and the plants have died off I plan to scoop out all the flower beds. I will hand pick anything I can see in the lawn area.

Does that sound like a reasonable plan? I know it is outside the house by my concerns are the lawn mower will suck them up and powderize them, they will be stepped on, and animals would disturb it. Anything else I can do?