You could also look on the web for a very simple adjustable power supply, based on the LM317 voltage regulator IC. This thing is adjustable, so no need to bother with zeners.
It's a voltage source though, which in theory is not ok for your application (you need a current limit too), but, since you have only a few & small PV panels, the output current is limited by your panels. Here are some example schematics:
http://www.tkk.fi/Misc/Electronics/circuits/lm317.html
http://www.national.com/pf/LM/LM317.html
http://www.uoguelph.ca/~antoon/circ/vps.htm
Now, to be really honest; I think you can take a shortcut here, and skip the regulator all together, since you've got such a limited PV supply: max 100mA. When you charge 4 nicads at 100mA, if they're modern nicads of, say, 1000mAh, you need to charge for 15 hours. But, at such a low charge current (1/10 of the capacity), you can almost permanently let them be charged; It's called trickle charge.
Many of the simple & cheap battery chargers do just that: supply a current of 1/10C, and expect the user to stop the charge after the required time. And even if you forget to stop it, no great harm is done to the batteries.
In that case, all that would be needed would be a diode, to prevent the batteries unloading themselves via the PV panel, during the night...
That's the shortcut, and the one I'd take. But, if you want to do it the proper way, build a small charge controller or voltage regulator. If only because it's fun and you'll learn a lot about electronics...
I know curiosity killed the cat, but what's the special project?