Because Shatner conceived the story and directed the film. I mean, it really is that simple, unfortunately.
As to the "taking away the pain" thing, the novelization of TFF makes an attempt to explain exactly what was going on. Essentially, according to the book, it was a type of telepathy/mind meld that was more akin to mind control than simply erasing someone's pain. It's been many, many years since I read it, so I am fuzzy on the details, but there was at least an attempt to explain why everyone went along with Sybok more than just "oh, he made me feel better, so I'm going to betray all my friends." (The novel also explains why Spock calls them "mashmelons.")