Seriously, it feels odd to enter doors which vanish behind you as soon as you walk through them.
Your suggestion allowed anybody to enter the world I was coming from, not just "me". Having one (or more??) defined places where one could enter a world and, when wanting to go back, return through them to the original world might help. Of course that would be "view"-side "take-me-back"-doors, with different target places for each user.
So kind of an interplay between trapdoors (anybody can enter) and "back doors"/"take me back doors" which allow me to return to my last microverse.
Otherwise the "back button" would always have to stay outside of the microverses, which would kind of break the symmetry of the door metaphor, no? If you want to stick to "one-way" portals I'd suggest to use something which is visually different to doors...