Your idea of dowels is a good start.
You can get a plug cutter from woodworking supply stores such as woodcraft. Box stores won't have them, or at least good ones. It is a bit that you use in a drill (ideally a drill press) to drill tapered plugs from a scrap board. You pop the plugs out of the board with a screwdriver.
You then drill a corresponding hole in the deck board with a Forstner bit.
The plugs are glued into the hole and tapped down with a hammer (lightly). Once the glue is dry, you use a sharp chisel, flush cut saw, block plane, or sander to make the top of the plug flush with the deck board.
Total investment in the two bits would be under $40.
I would strongly suggest using someone's drill press to mount the plug cutter. Cut the smallest diameter plugs you can get away with. The Forstner bit will work in a hand drill just fine.