Old, climate-buffered infertile landscapes (Ocbils) have attracted increasing levels of interest in recent years because of their exceptionally diverse plant communities. Brazil’s campos rupestres (rupestrian grasslands) are home to almost 15% of Brazil’s native flora in less than 0.8% of Brazil’s territory: an ideal study system for exploring variation in floristic diversity and phylogenetic structure in sites differing in geology and phytophysiognomy. We found significant differences in floristic diversity and phylogenetic structure across a range of study sites encompassing open vegetation and forest on quartzite and on ironstone substrates, commonly termed canga. Substrate was key in structuring floristic diversity with physiognomy having a secondary effect. In contrast physiognomy was more important than substrate in structuring phylogenetic diversity, with neither substrate nor its interaction with physiognomy accounting for significant variation in phylogenetic structure. Phylogenetic clustering was significant in open vegetation on both canga and quartzite, reflecting the potential role of environmental filtering in these exposed montane communities adapted to multiple environmental stressors. In forest communities, phylogenetic clustering was significant only at relatively deep nodes of the phylogeny in forest on quartzite while no significant phylogenetic clustering was detected across forest on canga, which may be attributable to proximity to the megadiverse Atlantic forest biome and/or comparatively benign environmental conditions in forest on canga with relatively deep, nutrient-rich soils and access to edaphic water reliable in comparison to those for open vegetation on canga and open or forest communities on quartzite. Clades representing relatively old lineages are significantly over-represented in campos rupestres on quartzite, consistent with the Gondwanan Heritage Hypothesis of Ocbil theory. In contrast, forested sites on canga are recognized as Yodfels. To be effective, conservation measures must take account of the distinct communities which are encompassed within the broad term campos rupestres, and the differing vulnerabilities of Ocbils and Yodfels.