What makes a popular podcast? Panoply, despite producing dozens of successful shows, doesn’t have the perfect formula, so it roped in some outside input — from listeners themselves.

In late March, the company announced the Panoply Pilot Project, a three-week experiment that put listeners in the producer’s chair by letting them vote on which of four podcast pilots they wanted to see become full shows. Panoply ultimately went with two of them: By the Book, hosted by Kristen Meinzer and Jolenta Greenberg, attempts to figure out which self-help books actually work, while Sam Dingman’s Family Ghosts investigates the mysteries of families’ pasts. Listeners were able to vote on Panoply’s website and by subscribing to their preferred shows.

We are so excited to announce that #ByTheBook w @kristenmeinzer
& @JolentaG & #FamilyGhosts w @samdingman has been greenlit! #PanoplyPilots pic.twitter.com/us4cxmlZKP

— Panoply (@Panoply) April 21, 2017

Panoply is always looking for ways to get listeners involved in the creation of its podcasts, Andy Bowers, the company’s chief content officer, told me. Much of that feedback, however, comes after the shows have already been recorded and produced, which limits its impact. “We always try to think of the listeners when we’re creating a new show, but this was a way to bring the actual listeners into the process early on,” Bowers said. “We’re basically asking them: Are we way off base? Are we a little off base? You tell us before we make a whole season of something drive you away.”

The Panoply Pilot Project is inspired by a similar effort from Amazon Studios, which since 2014 has let viewers vote on which of its TV pilots would be developed into full shows. Many of these shows have been both popular among viewers and critics: The Man in the High Castle and Transparent, for example, have won multiple Emmys. Amazon’s viewers, as it turns out, are decent arbiters of taste.

Beyond helping companies determine which programming is worth producing, these efforts also serve as effective marketing for the shows themselves. The people that vote to greenlight their favorite podcast or TV pilot tend to become those program’s biggest fans and often advocate that others tune in as well. In the case with Panoply, each of the greenlit shows will launch with the kind of subscriber fanbase that might take multiple episodes to build normally.

Listener input wasn’t the only factor in Panoply’s decision to green light By The Book and Family Ghosts. Bowers said that, as with all the shows that Panoply creates, producers also considered other factors — such as the replicability of the format, the hosts’ chemistry, the overall complication of the production process. By The Book’s first official episode could come as soon as the next few weeks, according to its hosts, while Family Ghosts will go live later this year. (Host Sam Dingman plans to stay in touch with listeners via newsletter until then.)

Roping in listeners early on has helped in other ways as well. With Family Ghosts, for example, listeners have already offered their own family mysteries that they think are worth investigating. Listeners to By The Book suggested that the hosts should interview experts in the subjects that they cover — and swear a little less. Cohost Jolenta Greenberg has taken this feedback to heart. “Having time to go back to the drawing board after the pilot was released is a luxury that not many people get,” she said.

Getting input from listeners early on is vital, particularly when there are so many podcasts vying for their attention. Producers often don’t know if a show is resonating with listeners until after multiple episodes have been released. Getting early feedback offers producers more confidence early on. “Generally, there is a danger that you create programs that producers think are a good idea or something that appeals to them as professionals,” Bowers said. “But it’s even more important in podcasting than radio that you make something that is of interest to the listeners from the get-go. They have a lot of choices out there, so what you create has to connect with them.”

Bowers said it’s too soon to say whether Panoply will repeat its pilot project with another set of pilots next year. The first test is to see whether this year’s greenlit shows will continue to be popular once their full seasons air.

“We enjoyed the process enough, and the audience enjoyed it enough, that it’s certainly a possibility that we would do it again,” Bowers said.