Do you have a pile of coins or some other placeable next to the wall (or building piece) that you’re trying to put back in? It might be blocking you from replacing your building piece.