Nicole Richie is mixing and matching items from every generation to make her stylish outfits, thanks to her family’s open closet policy.

The House of Harlow founder, 44, attended the Fashionphile flagship store opening at The Row in Los Angeles in her ambassador role on Wednesday, Oct. 8. Her passion for secondhand shopping led to her partnership with the reseller of pre-owned luxury items.

However, Richie revealed while speaking exclusively to PEOPLE that she has no hesitations when it comes to putting outfits together using items from family members’ closets before buying something new.

“Black ’80s Chanel bags,” says Richie, when asked about the items she stolen most from mom Brenda Harvey-Richie’s closet. “She has been telling me since I was five years old that they are mine.”

Lionel Richie and Harvey-Richie, his first wife, became Nicole’s legal guardians when she was 3 years old and adopted her at age 9. She is the biological daughter of Lionel’s drummer, Peter Escovedo and Karen Moss.

The fashion designer — who was holding a navy Birkin bought from Fashionphile at the event — told PEOPLE that there are no limits on what she can keep from her mom’s opulent closet.

“If I borrow, she’s like, ‘Keep it, keep it.’ She’s very loose with them,” Nicole says of Harvey-Richie. “She’s down to part. She just loves (it) when I wear her things, she really enjoys it. It’s open for me to take what I please.”

The Simple Life alum shares daughter Harlow17, and son Sparrow16, with her husband, Joel Madden46.

Having been shown generosity by her mom, Nicole said she now lets Harlow choose as she pleases from her closet. The mother of two added that her “Balenciagas and my Goyards” are among the most borrowed items.

The actress said that she’s never had an issue with borrowing an item from her daughter if she changes her mind and wants it back.

“We have a real easy back-and-forth with each other’s closets, so if I need to go and get something, I will,” Nicole says. “And she keeps everything in pristine condition.”

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories.

“She has really good jeans and she wears baggy jeans and I don’t own a lot of baggy jeans, so when I do want to wear baggy jeans, I go to her and she’s got good sweatshirts,” she continues of Harlow. “She has really good streetwear sneakers. She knows she can come and borrow anything from me and it’s fine if I do the same.”

Richie added that Sparrow and her dad have “pretty similar styles,” with every person in their household influenced by each other’s sense of dress.

“I would say that my kids also go in each other’s closets, because they wear a lot,” she tells PEOPLE. “They love Chrome Hearts and they love … like Gallery Dept and a lot of streetwear brands. Actually, now that I think about it, it’s all very open throughout the house.”