What is the preferred way to configure Amazon Security Groups to achieve a multi-tier architecture?

Each of my instances has its own Security Group, which I only want to use for rules specific to an instance. I'd like to keep any rules which apply to multiple instances in a separate Security Group, which can then be assigned to instance Security Groups as necessary.

As an example, I've setup a group called "admin", which allows administrative access from my IP. I added the "admin" group as the source to each of my instance security groups. However, I still can't access the instances from my IP without adding the rules directly to the instance's group.

Am I missing something? Although it seems a multi-tier security architecture should be possible, it doesn't seem to be working.