Fixed lens cameras tend to have smaller sensor sizes, which is the major difference. There's nothing inherently wrong with wanting (or having) a fixed lens, assuming that lens is of good enough quality - in the 'good ol days' of DVCAM, HDV et al we shot a lot with fixed lens cameras - albeit the cameras were in the $10k range at the time (which you can now pick up an FS7 for!).
Even television ENG cameras are designed so that the lenses themselves can be interchanged, but for a vast majority of work - they don't (or didn't).
Being able to change lenses is handy, helpful, and in general allows you to use specialised lenses that will overall give you better IQ (and much more interesting/versatile shots) - for example, you might use a vintage lens set on one project, and a sharp, modern, contrasty set on another. Or a couple of zooms on one fast-paced project, then a set of 12 primes on a slower-paced narrative.
That doesn't mean, however, that fixed/single lens cameras can't be workhorses, and indeed they traditionally have.
The other drawback is the smaller sensor - however, these days 'smaller sensor' tends to mean 1" rather than the 1/4, 1/3, 1/2, 2/3" sensors that used to be the norm. A 1" sensor is just smaller than a 4/3 sensor (and is pretty close to 16mm film). That's entirely useable and you can get nice images out of it, but some prefer the aesthetic that you get out of S35mm (or full frame), as evidenced by the abundance of '35mm converters' for fixed lens cameras back in the 'old' days.