Allen, who grew up in Pittsburgh, entered the Naval Academy but was injured and returned home. He finished his education at the University of Pennsylvania, and later joined the Pennsylvania National Guard in the signal corps. His unit saw duty near El Paso, Texas, where Mexican bandit Pancho Villa was raiding American settlements. Finally his unit was sent to France during World War I.
During his time in the military, Allen wrote poetry, at first creating verse inspired by Rudyard Kipling's Barrack Room Ballads. When he returned from the war, Allen became something of a poet vagabond, traveling up and down the East Coast. He settled in Charleston, South Carolina, and collaborated with poet DuBose Heyward, whose novel and play Porgy inspired the opera Porgy and Bess. Allen and Heyward wrote Carolina Chansons (1922).
Beginning in 1924, Allen taught English at Columbia University and Vassar College for two years. There he met Ann Hyde Andrews, one of his students. They created a minor scandal when they married after her graduation. He wrote a biography, Israfel: The Life and Times of Edgar Allan Poe (1926). With proceeds from the book, he bought a plantation in Bermuda, and lived there five years. The couple's three children were born there.
Allen's 1,224-page novel, Anthony Adverse (1933), became an instant success, rivaled only by Margaret Mitchell's Gone With the Wind that was published the same year. It was the sweeping saga of an illegitimate child of a French noblewoman and an Irish soldier who grows up to face a difficult life. With exotic settings in Cuba, Africa, Paris, the Italian Alps, New Orleans, and the American West, as well as colorful characters and, for its time, risque sex scenes, the book became a bestseller. A 1936 epic film starring Frederic March and Olivia de Havilland was based on the book.
Allen went to Miami in the early 1940s to visit his aunt. By then he was editor of the Rivers of America series published by Rinehart and Company. He asked Marjory Stoneman Douglas to write a book about the Miami River. "You can't write about the Miami River," she replied. "It's only about an Inch long." She suggested, instead, that she write about the Everglades, and include in it the story of the Miami River. Allen encouraged her to do so, and the idea consumed her for the rest of her life.
Allen later moved to an estate he called The Glades in Coconut Grove, then a small town near Miami, and became good friends with Douglas and with poet Robert Frost, who lived there, too.