getting grout off slate tips

hey... put down some 8x16 slate and applied two coats of 511 sealer and enhancer prior to grouting. sealed in sections so I could apply and wipe excess within 3-5 minutes as specified.. did second coat 1-2 hours after first.. I tested the sealer in a few spots and water beaded up and it some dirt on the floor came right up... Sealer was applied 24 hours before the grout..

Grout was applied last night... followed manufacturer instructions to the letter.. but I can't seem to get the tile as clean as I would have expected... So what gives? I know slate has nooks and crannies which doesn't help.. but there are other spots that seem to be holding on to the grout.. do I just need to keep at it? Any tips to get it completely cleaned up finally?