Dan, 4hp on your boat will be too small if theres any tide running. It'll keep you out of trouble at best. I've a 21ft Tremlett and a 4hp 4 stroke mariner. It will just push the thing along at slack water. Ok I never really got it as an aux. It was purchased for the dinghy that gets used as a tender when the family want to play.
Realisticaly a 6 will just about do it, but I would err on the safe side and go for 8horses as a min. Get a bracket and offer it up to determine wether you need a long or short shaft. If the prop isn't in deep enough you'll loose propulsion when the boat lifts and rolls in a swell. Steering with an aux on your type of boat is a right royal pain. Been there done that. I'got hold of an extention handle which makes things a bit easier.