As a discreet pagan, I've always believed that best hiding places are in plain sight. So many of the accessories we surround ourselves with as pagans have mundane and/or decorative uses. Brooms are found in every home. Wreathes, decorated with ribbons and flair, routinely hang on our walls and doors. My favorite cauldron is our trusted cast-iron Dutch oven (I have a smaller, more discreet one for burning offerings, but that mostly only comes out around Samhain). Incense is a personal fragrance preference. Candles look good anywhere. Crystal balls, prisms, statuary can all be part of the decor. Jewelry is jewelry. People see what they want to see.
My own altars, when I use them, tend to be very discreet. Even so, they all contain something representing Earth, Fire, Water and Air. Sometimes it's as simple as a stick of incense (air and fire) in a bowl of wet sand (earth and water). Sometimes, like during the holidays, I can go all out, since a certain amount of creative license is expected in holiday decor. I save my formal altar for special occasions. It consists of a statue of the Mother, a small tealight in a color appropriate for the work I'm doing, a large piece of quartz crystal (my all-purpose stone in magick), a stick of incense (my go-to is sandalwood), a chalice of water or wine and an athame. My chalice is the lone surviving goblet from my wedding crystal. My athame is a really cool letter opener I found at Pottery Barn many years ago. All of this is placed on a lovely silk scarf I found in a resale shop and wear often.
The formal altar gets put away when not in use, not because I am ashamed of it, but because my personal beliefs are my own. I would sooner go skyclad at the mall than wave my religion around like a banner. And frankly, I really don't want to risk that last crystal goblet.