You have to understand my perspective - I have individual mics worth over $1,000 going into quality pre-amps; I listen on $3k worth of speakers in an isolated, treated room. In addition, I have 30+ years of critical listening experience. So the H4 sounds like dung to me already. It has a fairly high self noise floor that adds hiss to the sound. To my ears it also sounds brash and that's if it is being used skillfully.
From where I'm sitting you're pounding in a nail with a wrench instead of a hammer; it may get the job done, but it's not going to be pretty, and the odds are you're going to bend a hell of a lot of nails before you pound a few in straight. In other words, you are going to have to work extra hard to get any decent results at all because you are using the wrong tool for the job.
If you pay LOTS AND LOTS of attention to making sure that one of the mics is always aimed at the person speaking you have a chance at almost passable audio. Remember, there are TWO mics on the H4, so capturing the audio equally on both mics means that you will be picking up a lot of background noise. No matter what you do, however, it will probably sound better than what you have, but it will be FAR from great.