In Django, how do I construct a COUNT query for a ManyToManyField?

My models are as follows, and I want to get all the people whose name starts with A and who are the lord or overlord of at least one Place, and order the results by name.

class Manor(models.Model):
    lord = models.ManyToManyField(Person, null=True, related_name="lord")
    overlord = models.ManyToManyField(Person, null=True, related_name="overlord")
class Person(models.Model):
    name = models.CharField(max_length=100)

So my query should look something like this... but how do I construct the third line?

people = Person.objects.filter(
       Q(name__istartswith='a'), 
       Q(lord.count > 0) | Q(overlord.count > 0) # pseudocode
).order_by('name'))