My advice is to keep the camera still and then just composite images for the window and whatever's outside. If people walk in front of the window, use a green screen. If not, there's no need for the green and you won't have to worry about the color spill.
If the camera has to move, make a 3D...