Is it bad that I often find myself spending more time on program structure than actually writing code inside methods? Is this common?

I feel I spend more time laying the foundation than actually building the house (metaphorically). While I understand that without a good foundation the house will cave in, but does it legitimately need to take half of the project to finalize code structure?

I understand design patterns, and I know where to go if I need help on choosing one, but often I find myself doubting my own choices.