I should have thought 3ft x 2 ft was big enough if the 3ft dimension is vertical.
I assume it is not furling at this point (tail still on its stop). If the tail is at right angles to the prop then it will normally run at a bit of an angle to the wind. You need to set the tail away from furling direction 15 to 20 deg to balance the prop thrust.
The fact that you can make it speed up by pulling it into the wind makes me think that it is at a very considerable angle ( 20 deg has no obvious effect on speed or output).
Have you got about 4ft between the yaw pivot and the start of the tail fin?
Any idea of prop speed and power out and wind speed at the point you are bothered about?. A bigger tail will help but with your 1/2" ply you may push the furl point too high if you make it much bigger.
I take it that it is a consistent offset, not twisting about from turbulent wind. If you can manage it a picture may help spot something silly.