The conserved oligomeric Golgi (COG) complex is essential for establishing and/or maintaining the structure and function of the Golgi apparatus. The Golgi apparatus, in turn, has a central role in protein sorting and glycosylation within the eukaryotic secretory pathway. As a consequence, COG mutations can give rise to human genetic diseases known as congenital disorders of glycosylation. We review recent results from studies of yeast, worm, fly and mammalian COG that provide evidence that COG might function in retrograde vesicular trafficking within the Golgi apparatus. This hypothesis explains the impact of COG mutations by postulating that they impair the retrograde flow of resident Golgi proteins needed to maintain normal Golgi structure and function.