Armed with a four-track recorder and a wealth of lo-fi pop songs, Stoltz began recording his own material, performing all the parts himself and drawing comparisons to artists like Brian Wilson and Captain Beefheart. Those songs caught the ear of Monte Vallier, who helped Stoltz clean up and sweeten the recordings for release as The Past Was Faster. After that, Stoltz upgraded to an eight-track and self-released Antique Glow in a limited quantity of 200 vinyl copies, each one housed in a different, originally designed sleeve by Stoltz himself. Antique Glow was then picked up by Jackpine Social Club for wider release in 2003, which both raised his profile and allowed him to quit his teaching job.

Stoltz found a more permanent home for his music in 2005, when he signed with Sub Pop and released the Sun Comes Through EP. A full-length album, Below the Branches, followed in March 2006. Stolz's band toured that summer as the opening act for the Raconteurs and returned to the studio in 2008 to record his lushest production yet, Circular Sounds. Two years later, he toured with Echo & the Bunnymen and released another layered pop album, To Dreamers, which featured his live band on two tracks. His affiliation with Sub Pop having run its course, Stoltz next spent time behind the scenes as a member of Sonny & the Sunsets and producing albums for the Mantles and Tim Cohen. He returned in 2013 with an album for Third Man called Double Exposure. In 2015, Stoltz found a new sponsor in Castle Face Records, the label founded by John Dwyer of Thee Oh Sees. Castle Face released In Triangle Time, a set of pop gems performed and produced largely by Stoltz, in early November 2015. ~ Sean Westergaard