In November 1970, Simon Morley, an advertising sketch artist, is approached by U.S. Army Major Ruben Prien to participate in a secret government project. He is taken to a huge warehouse on the West Side of Manhattan, where he views what seem to be movie sets, with people acting on them. It seems this is a project to learn whether it is feasible to send people back into the past by what amounts to self-hypnosis—whether, by convincing oneself that one is in the past, not the present, one can make it so.
As it turns out, Simon (usually called Si) has a good reason to want to go back to the past—his girlfriend, Kate, has a mystery linked to New York City in 1882. She has a letter dated from that year, mailed to an Andrew Carmody (a fictional minor figure who was associated with Grover Cleveland). The letter seems innocuous enough—a request for a meeting to discuss marble—but there isa note which, though half burned, seems to say that the sending of the letter led to "the destruction by fire of the entire World", followed by a missing word. Carmody, the writer of the note, mentioned his blame for that incident. He then killed himself.
Si agrees to participate in the project, and requests permission to go back to New York City in 1882 in order to watch the letter being mailed (the postmark makes clear when it was mailed). The elderly Dr. E.E. Danziger, head of the project, agrees, and expresses his regret that he can't go with Si, because he would love to see his parents' first meeting, which also occurred in New York City in 1882. The project rents an apartment at the famous Dakota apartment building, which did not actually exist in 1882. (It was completed two years later, but Finney explains that he took a few liberties with the timeline due to his fascination with the building.) Si uses the apartment as both a staging area and a means to help him with self-hypnosis, since the building's style is so much of the period in which it was built and faces a section of Central Park which, when viewed from the apartment's window, is unchanged from 1882.
Si is successful in going back to 1882, at first very briefly, and then a second time he is able to take Kate with him. They travel by horse-drawn bus down to the old post office, and watch the letter being mailed by a man. They follow him, and learn that he lives at 19 Gramercy Park. Then they return to their base at the Dakota apartments and return to the present.
Si is debriefed and carefully examined after each trip to the past, and as far as the project organizers can tell, his activities in the past are making no difference to the present. He is encouraged to go back again. He presents himself at 19 Gramercy Park as a potential boarder. He is accepted, begins living there and learns that the man who mailed the letter is named Jake Pickering. He explores the Manhattan of the past for several days, sketching all the while—he is an illustrator, and Finney inserts illustrations from the period into the book as Si's own. He goes on to learn that Pickering is blackmailing Carmody. Si finds himself falling for the landlady's niece, Julia Charbonneau. But he has a rival—Pickering. Eventually, Pickering makes a scene,having tattooed the name "JULIA" on himself, and Si soon leaves, to return to the present.
Things aren't going as well in the present. One of the other participants in the project, having gone back to Denver some seventy years in the past, has made some unknown change in the past and thus a friend, whom he remembers, was never born. Danziger insists that the project be stopped. When he is overruled, he resigns. After Prien talks to him, Si sees no alternative other than to return to the past again, though he is troubled by Danziger's resignation.
He is accepted back at Gramercy Park cheerfully, with even the dour Pickering happy. It seems Pickering and Julia are now engaged. Si (casting himself as a private detective) tells Julia that Pickering is a blackmailer. They go to Pickering's office and conceal themselves to watch the blackmail money being turned over by Carmody. Carmody brings only $10,000, rather than the demanded million dollars for the incriminating files. After knocking him out, Carmody ties up Pickering and sets out to look for the papers. He realizes they are concealed amid many other files. He patiently thumbs through the files, as Si and Julia agonize as the hours pass. Finally, Carmody decides on a scheme—burn the files. He does so. Pickering tries to save the files, but burns himself badly in the process. To the pair's astonishment, Si and Julia burst forth, urging them to flee, and flee themselves.
It is a huge fire, and Si and Julia find themselves trapped. They barely escape. Si learns that the building used to house the newspaper the New York World and one piece of the puzzle fits in—the missing word in Carmody's note was "Building". After watching the efforts to fight the fire, in which many die, the shaken couple returns to Gramercy Park. There is no sign of Pickering. [The burning of the New York World building is a factual historical event].
Two days later, the two are picked up by Police Inspector Thomas Byrnes, and then taken to Carmody's house. The terribly burned man there accuses them of murdering Pickering and starting the fire. After they leave, Byrnes expresses indecision and lets them walk away—only to yell "The prisoners are escaping" to the sergeant who accompanies him. It is a set-up, the two are to prove their guilt by "attempting to escape". As it turns out, police all over the island have already been provided with their description and photographs. They are able to flee, but haveno money and nowhere to go. They shelter in the as-yet-unassembled Statue of Liberty's arm, then standing in Madison Square. (Again, the arm standing in Madison Square Park prior to the statue as a whole being erected is a factual event). Si tells Julia the whole story, but she takes it as entertaining fantasy. She is soon convinced otherwise, as Si brings them both into the present, and she observes the dawn from high inside the long-assembled statue, seeing a totally strange New York.
They spend a day in the present, with a shocked Julia observing the things that have changed in ninety years, from clothing to television. At last, they settle into Si's apartment. He is ashamed to tell her the history of what has happened in the past ninety years, the horrible wars and the fact that there are areas of the city where no law-abiding citizen can safely go. Julia must return home. The two realize that the man whom they met at Carmody's house was in fact Pickering—Carmody died in the fire. Armed with that fact, Julia can keep Pickering from having her arrested, lest he be exposed. As 1882 is far more real to her than 1970, she returns to the past without needing any help from Si.
Si goes to report in, and tells most of the story, concealing Julia's visit to 1970. They then give him an assignment—to intentionally alter the past. Research has confirmed that Carmody (actually Pickering) was an acquaintance of Grover Cleveland's--and talked Cleveland out of buying Cuba from Spain. The military men now in effective control of the project conclude that if Pickering is exposed, he might never have influence with Cleveland, and the U.S. might never have to worry about Fidel Castro. But after talking with Danziger, Si worries about the other effects the change might have, and Danziger makes him promise not to carry out the scheme. Si returns to 1882. Having been told how Danziger's parentsmet, he aborts the meeting. Danziger will never be born, and the project will never happen. Si walks away towards Gramercy Park and Julia, and away from 1970.