As long as you either drive the car long enough to evaporate the water that the fluid absorbs, or change it regularly there will not be a problem, The problem with Dot 3 or Dot 4 fluid is that it absorbs moisture out of the air. Back when people actually drove the cars regularly (and actually performed routine maintenance) the brake fluid would reach a high enough temp to evaporate the water. As clutches never get hot enough, it was recommended by most manufacturers to flush and refill the hydraulic systems on an annual, or bi-annual schedule. (depending on climate)... With headers, the fluid's probably getting hot enough, Warren!!
The reason that Dot 4 fluid was originally specified, was that English systems used natural rubber, which Dot 3 fluid would dissolve. In the 90's the rage was to go to Dot 5 as it doesn't absorb water. The problem with this is two-fold... you cannot mix Dot 5 with either Dot 3 or Dot 4 fluids, so you have to start with a completely new (virgin) system. They have different swelling properties and mixing them will cause rapid seal deterioration and ultimate failure. In brake systems it can cause the seals to swell to the extent that the brakes will lock on and the only way to move the car is to open a bleed screw and use the e-brake to stop.
The other problem is, silicone brake fluid is compressible... Dot 3 and Dot 4 are not. This results in a spongey pedal not unlike the sensation of an improperly bled system. Definitely not the ideal condition if the car is to be used in "anger".
The simple solution is proper maintenance.... and the use of the correct specification fluid.