Jim,
I think that would probably be fine. I had a friend of mine build my ridders, as he has a lot of experience in composites as well as building foils. He worked for Sperry Boats as well as Larry Tuttle who is/was pretty well known for his foils in these parts. My friend recommended a wood core, mainly because he likes working with it better than foam. I am concerned about saving weight, so after I swayed him to foam, he settled on 10lb/cubic foot density. The rudders are semi-balanced, with a 1 1/2" SS pipe for the rudderpost. I welded on 3 tabs of 1/4" x 2" SS flat bar spread apart to distribute the load in the foam. He hogged out the foam halves to fit the shaft and tabs, leaving room for thickened epoxy and cloth, spanning I think, from tab to tab, which further spreads the load out. He did a fairly heavy layup on the outside skin. I don't think he used fabric as heavy as 24oz though on the inside, so I would think you would be ok seeing as you will have a thick skin not only inside, but outside as well. Spanning from tab to tab ought to balance out the lower density foam.
For what it's worth, bear in mind that I haven't put the boat in yet, so I have no feedback for you on how the construction has held up. It seems to me though, that after much research and advice that they are of sound design.
Good Luck!
Lincoln