The Sound of Waves is the story of Shinji Kubo, son of a Japanese soldier killed in World War II. Shinji becomes enamored of the beautiful pearl diver Hatsue, but her father's exacting standards render relations between the two difficult. The title of the book is a reference to Hatsue's father's test of her suitors' mettle. He arranges for them to work aboard one of his ships during a towering storm, a test that Shinji passes.
Shinji Kubo lives with his mother, a pearl diver, and his younger brother, Hiroshi. He and his mother support the family because Shinji's father died in World War II after the fishing boat he was on was strafed by an American bomber. However, the family lives a somewhat peaceful life and Shinji is content to be a fisherman along with his master, Jukichi Oyama, and another apprentice, Ryuji.
Things change when Terukichi Miyata, after the death of his son, decides to bring back the daughter he adopted away to pearl divers from another island. Raised as a pearl diver, the beautiful Hatsue wins many admirers, including Shinji. The prospect of marrying Hatsue becomes even more attractive when the wealthy Miyata intends to adopt the man who marries Hatsue as his own son. Shinji and Hatsue soon fall in love.
When Chiyoko, the daughter of the Lighthouse-Keeper and his wife, returns from studying at a university in Tokyo, she is disappointed to discover Shinji, whom she has affections for, has fallen in love with someone else. She takes advantage of the jealous Yasuo Kawamoto, an arrogant and selfish admirer of Hatsue, and uses him so Yasuo will spread vicious rumours of Shinji stealing away Hatsue's virginity. Yasuo is jealous of Shinji because he thinks Shinji will take Hatsue's virginity. He tries to rape Hatsue at night when it's Hatsue's turn to fill out the bucket of water. The attempt is unsuccessful, however; he is stung by hornets as he tries to strip her clothes off. Humiliated, he makes a deal with Hatsue: He will refill the bucket and carry it down the stone steps for her, and Hatsue must not tell anyone he was trying to rape her.
When Terukichi finds out the rumor, he forbids Hatsue to see Shinji, but through Jukichi and Ryuji, the two manage to continue communicating with one another by means of secret letters. Terukichi steadfastly refuses to see Shinji for an explanation and when Shinji's mother, who knows her son will never deliberately lie, goes to see Terukichi, Terukichi's refusal to see her only increases the tension between Shinji and Hatsue. Chiyoko, before returning to Tokyo, becomes filled with remorse after Shinji off-handedly replies that she is pretty when she asks him if he thinks she is unattractive. She returns to Tokyo with guilt that she ruined Shinji's chance at happiness
Terukichi mysteriously employs Yasuo and Shinji on one of his shipping vessels. When the vessel is caught in a storm, Shinji’s courage and willpower allow him to brave the storm and save the ship. Terukichi's intentions are revealed when Chiyoko's mother receives a letter from Chiyoko, who refuses to return home, explaining that she feels she cannot return and see Shinji unhappy because she was the one who started therumors. The Lighthouse-Keeper's wife confronts Terukichi, who reveals that he intends to adopt Shinji as Hatsue's husband. Employing the boys on the ship had been a test to which one was most suitable for his daughter and Shinji's act to save the vessel had earned Terukichi’s respect and permission to wed his daughter.