You need something called a cold chisel. It's available at any hardware store or Harbor Freight. By hitting the joint with a small sledge hammer, you can fracture the joint and remove the scrapped piece of concrete, likely without any collateral damage.