The Friedbergers' grandmother was a musician and choir director at a Greek Orthodox church near the family's home in Oak Park, IL; their mother, who had a penchant for Gilbert & Sullivan, played piano and guitar and sang; and throughout school, Matthew played standup bass. While the Friedbergers weren't the closest of siblings growing up, after college and separate trips abroad they returned to Oak Park and began working on music together. The pair mixed simple, poppy melodies with a dizzying array of wordplay, sounds, and influences, including the Who, Captain Beefheart, Os Mutantes; dashes of folk, blues, and garage rock; and Eleanor's adventures in Europe. In 2000, they moved to Brooklyn, took day jobs, and began playing as the Fiery Furnaces late in the year.

The Furnaces played their initial gigs at a small club called Enid's and branched out from there, going through several lineups of supporting musicians as they played gigs with the French Kicks, Sleater-Kinney, and Spoon. In 2002, they began working on their first album. By the time they signed to Rough Trade on the basis of their demo, their debut, Gallowsbird's Bark, was completed and the Fiery Furnaces were already at work on the follow-up. Gallowsbird's Bark arrived in fall 2003 and won critical acclaim for its charming kitchen-sink feel, but the band gained more momentum the following year, when praise for the debut album dovetailed with the release of the group's even more diverse and challenging sophomore album, Blueberry Boat, that summer.

The Fiery Furnaces spent much of 2004 touring with Ted Leo & the Pharmacists, Franz Ferdinand, and the Shins, but were already working on their third and fourth albums. Early in 2005, the Furnaces released the simply titled EP, a mini-album gathering most of their B-sides along with a few new songs. That fall, the band returned with Rehearsing My Choir, a challenging stream-of-consciousness album featuring the Friedbergers' grandmother, Olga Sarantos. They moved to Fat Possum for Bitter Tea, a collection of poppier songs that arrived in spring 2006. The following year, the Fiery Furnaces switched to Thrill Jockey and spent the summer and fall touring in anticipation of the '70s rock-influenced Widow City, which was released in October 2007 and was mixed by Tortoise's John McEntire. Remember, a sprawling live album that spliced different versions of the same songs together into collage-like tracks, followed in 2008. That fall and into winter 2009, the Fiery Furnaces recorded I'm Going Away, a simpler, more stripped-down collection that drew on the warm sounds of '70s and '80s TV show theme songs like Taxi for inspiration. The album came out in summer 2009, by which point the band was already working on other ideas, including their "silent record," an album in book form, and their "Democ-Rock" project, which allowed fans of the band to vote on the group's creative process. ~ Heather Phares