The events of the novel take place between 1946 and 1948, primarily on the Near Northwest Side of Chicago. The title character is Francis Majcinek, known as "Frankie Machine", a young man who is a gifted card dealer and an amateur drummer. While serving in World War II, Frankie is treated for shrapnel in his liver and medicated with morphine. He develops an addiction to the drug, although initially in the story he believes he can control his habit.
Frankie lives in a small apartment on Division Street with his wife, Sophie (nicknamed "Zosh"). Sophie has been using a wheelchair since a drunk-driving accident caused by Frankie (although the novel implies that her paralysis is psychological in nature). She spends most of her time looking out the window and watching the nearby elevated rail line. She takes out her frustrations by fighting with her husband, and she uses his guilt to keep him from leaving her. The turmoil in their relationship only spurs on his addiction.
He works nights dealing in backroom card games operated by "Zero" Schwiefka. He aspires to join the Musicians' Union and work with jazz drummer Gene Krupa, but this dream never materializes. His constant companion and protégé is "Sparrow" Saltskin, a feeble-minded thief who specializes in stealing and selling dogs; Frankie gets Sparrow a job as a "steerer", watching the door to the card games and drawing in gamblers.
Often referring to his drug habit as the "thirty-five-pound monkey on his back", Frankie initially tries to keep Sparrow and the others in the dark about it. He sends Sparrow away whenever he visits "Nifty Louie" Fomorowski, his supplier. One night, while fighting in a back stairwell, Frankie inadvertently kills Nifty Louie. He and Sparrow attempt to cover up his role in the murder.
Meanwhile, Frankie begins an affair with a childhood friend, "Molly-O" Novotny, after her abusive husband is arrested. Molly helps Frankie fight his addiction, but they soon become separated when Frankie is imprisoned for shoplifting and she moves out of the neighborhood. Without Molly, he begins using drugs again when he is released.
Nifty Louie owed money to politically connected men, and finding his killer becomes a priority for the police department. Sparrow is held for questioning by the police, and he is moved from station to station to circumvent Habeas corpus requirements. Eventually he breaks down and reveals what he knows, and Frankie is forced to flee.
While on the run, Frankie manages to find Molly at a strip club near Lake Street. He hides in her apartment and beats his addiction, but in the end the authorities learn where he is hiding. He barely manages to escape and gets shot in the foot, leaving Molly behind. He flees to a flophouse, but without any hope of reuniting with Molly or staying free, he hangs himself in his room on April Fools' Day, 1948.
The novel ends with a transcript of the coroner's inquest, as well as a poem for Frankie entitled "Epitaph."