The book takes its title from a verse from Andrew Marvell's "To His Coy Mistress": "The grave's a fine and private place, / But none, I think, do there embrace." The setting, accordingly, is the fictional Yorkchester Cemetery, where one Jonathan Rebeck, a homeless and bankrupt pharmacist who has dropped out of society, has been living, illegally and unobtrusively, for nearly two decades. He is maintained by a raven who, like the legendary ravens who fed Elijah in the wilderness, supplies him with food in the form of sandwiches thieved from nearby businesses.
The protagonist exhibits the peculiar ability to converse with both the raven and the shades of the dead who haunt the cemetery. Beagle portrays ghosts as being bound to the vicinity of their burial, with their minds and memories slowly fading away as their mortal forms return to the dust. As the plot proceeds, Rebeck befriends two recently arrived spirits, those of teacher Michael Morgan, who died either from poisoning by his wife or suicide (he can't remember which), and of bookshop clerk Laura Durand, who was killed by a truck. The two ghosts fall in love, and each pledge themselves to each other "for as long as I can remember love."
Rebeck soon finds himself subject to another's attentions as well, in the form of a widow, Mrs. Klapper, who discovers him while visiting her husband's mausoleum. The quiet existence of this unlikely quintet is diverted by philosophical conversation and the poisoning trial of Morgan's wife, word of which is regularly provided by the raven from the local newspapers.
After she is ultimately found innocent and her husband's death ruled suicide, Morgan faces separation from Laura when his body is removed to unhallowed ground. Rebeck, under the encouragement of Mrs. Klapper, is driven to find a way to reunite them, and finally takes leave of his unusual abode.