You just need to make sure there is no open space where the wind can be funneled through and allowed to cause serious destruction. Your home won’t be inundated by a massive internal pressure buildup because air is still flowing in and out from the home at a safe rate. Moreover, it’s key to note that a house isn’t an airtight system - there is no pressure differential being created because houses are ventilated systems.
The resulting set of forces would lift the roofs off and make it look like buildings exploded due to forces from inside - which was technically true it just wasn’t because of a massive buildup in pressure. Eventually, scientists began realizing the houses were being blown up because of the open windows - gusts of wind would push in homes and streak upward towards roofs, while wind from the outside would thump endlessly on the roof. Buildings often look like they were blown up from the inside after a tornado.