They are sold as masonry, concrete and tile bits. I have used mine mainly for drilling into masonry and concrete to fit anchors for hanging art. Has always worked flawlessly. I see there is a 3/16" bit for about $15.
Those are tile bits, and I doubt they have enough length for what the OP is doing.
You may well be right but the OP has not told us what he is trying to accomplish with such small holes in a foundation? As mentioned, the bit I have works just fine in masonry and concrete for drilling holes into which screw anchors go to hang art.