Here's how a submersible pump control box works; when power is applied, the yellow wire that goes to the motor is energized by L2. This wire is common to both the start and run windings in the motor.
The black wire that goes to the motor is the run winding, common to the yellow. It is energized by L1, through the overload. If too much current flows, the overload will disconnect L1 from everything.
During starting, the relay allows current to flow to the start winding (red wire to the motor) through the start capacitor. Once the motor is at about 1/2 to 2/3 speed, there'll be enough voltage on the start winding to energize the coil of the relay. This causes the start winding to disconnect from the capacitor. If the relay fails to disconnect the start capacitor, the motor won't come up to speed, excessive current will flow, thus tripping the overload.
If there's a run capacitor, it's always connected to the start winding. The start capacitor is paralleled across the run capacitor during starting to 'give it an extra kick', so to speak. (It's more complex than that, involving mechanical and electrical phase shifting, but that's a bit deep for the discussion at hand).
There are several things that will cause the overload to trip.
1) Mechanical problems with the pump. This can best be determined by current readings.
2) Failure of either capacitor, or the relay. Capacitors can best be tested by a meter designed for the purpose. There are other ways involving current vs. resistance, or resistance vs. time, but I don't recommend them. The relay can be tested by measuring current on the red wire going to it. Lots of current during starting, none while running. The start relay stays on for less than a second. Sometimes you can hear it click, but that doesn't mean it actually dropped out.
3) A fault in the wires going from the control box to the motor. This is best done with a meggar. In fact, it's the only sure-fire way to know that there's not a problem here.
4) A fault in the motor. Pretty rare. This can be tested with an ohmmeter. If you don't know the condition of the wiring to the motor, bad ohm readings can be either the wiring or the motor.
Based on the drawing you gave, I'll guess that this is a 1-1/2 HP motor. If so, you should have 1.7-2.2 ohms from yellow to black. From yellow to red, you should have 8.0-9.7 ohms. Add 1.6 ohms for every 500' of #12 wire from the control box to the motor. Add 1 ohm for 500' of #10. Current should be less than 11.5 on yellow, 11.0 on black, and 1.3 on red. All these readings are plus or minus about 20%.
The current measurements are the most important. They will tell if a capacitor is bad, or the relay is bad, and most of the time if the wiring from the box to the motor is OK. A clamp-on ammeter is best.
Rob