Ha! Your dad's not totally wrong, but neither are you. Banana peels give a solid potassium boost, just mix some store feed in for the other stuff they need.
Worry about the fruit flies and ants first. Those peels rot fast and draw every bug in a five mile radius. Dry them out completely or bake them into a powder unless you want a side of pests with your compost. Even good ideas go bad with poor setup.