In the first molecular study of a member of the threatened avian family, Mesitornithidae, we used nine polymorphic microsatellite loci to elucidate parentage, patterns of within-group kinship and occurrence of extra-group paternity in the subdesert mesite Monias benschi, of southwest Madagascar. We found this cooperatively breeding species to have a very fluid mating system. There was evidence of genetic monogamy and polygynandry: of the nine groups with multiple offspring, six contained one breeding pair with unrelated helpers and three contained multiple male and female breeders with related helpers. Although patterns of within-group kinship varied, there was a strong positive relationship between group size and relatedness, suggesting that groups form by natal philopatry. There was also a strong positive correlation between within-sex and between-sex relatedness, indicating that unlike most cooperatively breeding birds, philopatry involved both sexes. In contrast to predictions of kin selection and reproductive skew models, all monogamous groups contained unrelated individuals, while two of the three polygynandrous groups were families. Moreover, although between-group variation in seasonal reproductive success was related to within-group female relatedness, relatedness among males and between the sexes had no bearing on a group's reproductive output. While kin selection may underlie helping behaviour in females, factors such as direct long-term fitness benefits of group living probably determine helping in males. Of the 14 offspring produced by fully sampled groups, at least two were sired by males from neighbouring groups: one by a breeding male and one by a nonbreeding male, suggesting that males may augment their reproductive success through extra-group paternity.