The phrase “groves of academe” has been in English since at least the eighteenth century. The OED cites it as deriving from Horace’s Odes:

groves of Academe [translating classical Latin silvās Acadēmī in Horace: see the etymology] (literary): a place of studious seclusion; the academic world, viewed as sheltered from the demands of everyday life.

In post-war America the phrase came into common use due largely to Mary McCarthy’s eponymous 1952 novel, whence the word academe as a frequent designation for the academic world or academic life. Lately, however, it is being supplanted by the hypertrophic variant academia, often mispronounced to rhyme with macadamia (as in “macadamia nuts”). This variant is on the cusp of consigning the correct form to oblivion and is even to be heard emanating from the lips of academics, who should know better.