Barry Williams Explains How Producer Lloyd Schwartz Stood Up for ‘The Brady Bunch’ Child Actors

Making The Brady Bunch involved keeping the kids lively.

The cast and crew of the beloved sitcom look back in the CW’s docuseries, TV We Love. Barry Williams credited producer Lloyd Sherwood, whose dad, show creator Sherwood Schwartz, put him in charge of the child actors on the show, for understanding what they needed.

“The focus really was on the kids and their lives. This is, too, a part of the genius of Lloyd Schwartz and Sherwood Schwartz both. The way to keep us connected to what we were doing here, was to let us be kids.”

Lloyd recalled being tasked with minding the kids and how that leaked into the show’s production. “I basically would look at the show from the kids’ point of view and my dad would have the point of view of the adult.”

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories.

When it came to dealing with the kids, Lloyd says, “I just treated it a little bit like summer camp. We just wanted them to be kids, which caused problems later on for me.”

Williams noted that filming involved “a lot of waiting around,” during which he and costars Christopher Knight, Mike Lookinland, Maureen McCormick, Eve Plumb and Susan Olsen would roam around the Paramount Studios lot.

That wasn’t something that everyone on the lot loved, however, leading to some complaints.

“I got a call from the president of the studio, saying, ‘The kids were all running around. You can’t have them do this. They’re very expensive property,’ ” Lloyd recalled.

“And that just put my hackles up and I said, ‘They’re not property, they’re kids. That’s what kids do.’ He said, ‘You’re not gonna change that?’ I said, ‘No, I’m not gonna change that. If you want that done, you’re gonna have to fire me.’ Well, he didn’t. It was pretty ballsy.”

It was something that the cast was grateful for, Williams says. “Lloyd, to his credit, and Sherwood would both make sure the studio wouldn’t intervene and they let us go, and we had a ball.”

TV We Love is airing on Mondays on the CW at 8:00 p.m. ET/PT.