There are currently two ways to accomplish that. One is to use flotsam ceilings:
Blends in really nicely with certain wooden building pieces.
However, if you don’t have Siptah or you’re not building wooden floors, the other good alternative would be to use a hatch frame in whatever style you’re using for your floors and then put a reinforced hatch door (variant A) in it:
This one can blend into any other building style and it doesn’t require any DLCs.
Be cautious using these Reinforced Hatches as walkways as followers will
derp out when teleporting,
will not freely walk across or down an aisle of them,
when aggroed will derp partially into them and are tough to reset.
They look great, and can support planters. Just be advised if you’re building “sparely” for PvP, you will cry a little when your fighters are all stuck in hatches because someone tried to creep up on your castle.