Why is it seemingly not used very much in most films?
It is, but sometimes it may be for a short amount of time. If the scene is well edited, you might not even notice that it was used.
Some posters on this thread are overly associating "Point Of View" shots with "Handheld" shots. P.O.V. can be static, steadicam or handheld. As exampled the steadicam shot that opens HALLOWEEN. The steadicam killer P.O.V. when the guy with the knife goes after Sean Connery, in THE UNTOUCHABLES.
I like the desert island device where one hungry guy looks at his buddy, who appears as a big chicken leg.
Without P.O.V. movies like JAWS would not have had such a psychological impact. How about that Hyperspace jump in STAR WARS? Here is a similar sequence - notice the P.O.V. itself is camera static:
https://www.youtube.com/watch?v=1xkYtWXqSfI
How about RETURN OF THE JEDI and the steadicam forest shots for the speeder chase. The mine car tracks in INDIANA JONES AND THE TEMPLE OF DOOM. Those are the kind of shots that the audience loves. They get to "share the experience" and put wear the characters' shoes.
How about those Doc Ock tentacles in SPIDERMAN 2? I'm talking about the hospital scene, where they attack the surgeons. Of course, Sam Raimi is especially known for his use of P.O.V. In the EVIL DEAD trilogy, death is invisible and glides through the forest. The P.O.V.
is the monster. Of course, the view can be from an inanimate object like an arrow flying at its target.
I'm a big horror fan, so the P.O.V. shot is very important to the genre. In the original THE FLY, we got the classic bug P.O.V.
In movies like WOLFEN or FRIDAY THE 13th, there would be no twist as to who the killer was, if there wasn't the P.O.V. shots of the attacker/stalker. As someone said, it is a tool in the DP's kit.
2001 used a famously long psychedelic sequence, seen from the astronaut's view as he descended near Jupiter and evolved into the Star Child.
Movies like BRAINSTORM and STRANGE DAYS used other people's memories and let us see what they saw. These movies were about the P.O.V. The latter movie's director, Katherine Bigelow, used it very effectively when Keanu Reeves chased Patrick Swayze through houses and yards in POINT BREAK. She used it well in THE HURT LOCKER, when she put us in the bomb suit, along with the heavy breathing.
Whenever you want to convey something other than human, you may get the enhanced P.O.V.s of THE TERMINATOR or the thermoscan view of ROBOCOP seeing criminals through walls. These shots are usually the most talked about shots in the movies. This technique has never gone away, though it might be used economically, like the views from the eye of Sauron, in LORD OF THE RINGS.
I love P.O.V.s. I don't like shakycam. They don't have to be the same. After all, our real eyes are wide angle and stabilized. Our head turns left, right, up and down; we walk or run, yet the vision is smooth. Shaky cam isn't necessarily accurate for a character, as much as it is for a camera's view.
If you want to see a movie with a little too much point of view, see FROM DUSK TILL DAWN 2. Even the oscillating fan has a view, which is pretty cool. One of the best P.O.V.s I've seen was the bullet view, when the trapped girl in the kitchen shoots a guy through a tiled wall, in LA FEMME NIKITA (French version).