The world is banking on a major increase in food production, if the dietary needs and food preferences of an increasing, and increasingly rich, population are to be met. This requires the further expansion of modern agriculture, but modern agriculture rests on a small number of highly productive crops and its expansion has led to a significant loss of global biodiversity. Ecologists have shown that biodiversity loss results in lower plant productivity, while agricultural economists have linked biodiversity loss on farms with increasing variability of crop yields, and sometimes lower mean yields. In this paper we consider the macro-economic consequences of the continued expansion of particular forms of intensive, modern agriculture, with a focus on how the loss of biodiversity affects food production. We employ a quantitative, structurally estimated model of the global economy, which jointly determines economic growth, population and food demand, agricultural innovations and land conversion. We show that even small effects of agricultural expansion on productivity via biodiversity loss might be sufficient to warrant a moratorium on further land conversion.