Cold draft at baseboards and wood plank floors

I've noticed a cold draft coming along my baseboards of one wall but also through the 1/4 inch gap in probably the last 3-4 inches of my wide plank wood floors. The house sits on a fieldstone foundation. I've felt some cold drafty areas in the basement field stone. Is it worth sealing behind the baseboard? where the drywall meets the floor there's a small gap with air flow.