Definitely carpenter bees IMO. Yes, they normally drill perfectly round holes, but I think they make chambers to place their larva in and those areas that look like skid marks are either where something tried to get to them for a quick meal or the larva chamber somehow breaks down from the moisture/excrement, etc. causing the divots. Carpenter bees seem to especially like cedar, and that wood looks like cedar in the pic.