Ken,
I assume you were only on holiday for a couple of weeks or so. In which case, your batteries should not have gone flat, even without the solar panel running.
As Mike says, you may have a small leak to earth, but most probably you will find that something was accidently turned on.
Having said that, the solar panel will need to get direct sunlight to work efficiently. You never know, it might have been the flashing red led that ran down your batteries.
BF