I'm leading a largish development team (~35 developers). We are doing primarily Web Development work on a number of sites.

Historically the knowledge on the teams has been pretty siloed. If you worked on Site A you will know how to troubleshoot it, but you would not be a lot of help on Site B.

We also have a few cross-cutting concerns, i.e. common components used between sites which require specialized knowledge to troubleshoot.

With all this in mind, I'm trying to understand the best way to setup an on-call team.

This would be a team of programmers who would be available to deal with out-of-hours emergency issues occasionally (say one call every 2 weeks). They may be required to deploy emergency fixes.

Part of me is saying we can't have a big on-call team with shallow knowledge, instead we need a smaller team with deep knowledge who can expect to be on-call more often and remunerated as such.

Does anyone have any suggestions based on experience on how to setup this team?

Thanks in advance.