I have to admit, I've always wondered about something: is there any possible correlation between the "canon" policy in regards to tie-ins and the fact (or at least the assumption) that the target audience is only a tiny minority of the fans, or that it should be?
That's part of the reason. But there's a more fundamental reason than that. The tie-ins aren't canon because they aren't created by the same people who make the show. The concept is theirs; anyone licensed to do adaptations of it is simply borrowing it. Now, maybe that line is vaguer in a case like ST, which has had so many producers and showrunners, than in a case like
Babylon 5 or
Buffy/Angel where there's a single clear "owner" of the creation. But the principle is the same.
I mean, when I read expansion stuff from a series like Star Wars, I like the fact that there's a lot of effort being made to treat these stories as being just as valid as the movies, even though the movies are recognized as the standard for SW canon.
Valid in what sense? Yes, the various tie-ins are all obligated to acknowledge and conform to one another (albeit glossing over the contradictions that exist), and the onscreen material does borrow a lot of ideas and characters from them, but it also ignores and contradicts them without hesitation. Just last month,
The Clone Wars did a 3-episode Mandalore arc that, I gather, invalidated a whole multi-book series of popular
Star Wars novels by an author named Karen Traviss.
So "just as valid?" Not at all. The tie-ins are a source of ideas for the canon, but are not binding upon it.
With Trek, I've sometimes felt like the PTB are indifferent at best, and at worst wouldn't mind excluding stuff or ignoring a potentially good link just because they can.
It's their show. It's their creation and their right to make whatever choices they want. We're just borrowing their toys. So they're free to use, ignore, or contradict anything we come up with. If you go to your friend's house and they let you play with the big toy-railroad diorama in their basement, that doesn't mean they're sharing ownership of the trains with you. It doesn't mean you have a right to tell them how to rearrange the tracks or the miniature houses, or that you can make such changes without their permission, no matter how cool your ideas are. If they like a suggestion you make and choose to incorporate it, that would be a gratifying bonus, but that doesn't make it something you should expect as an entitlement, and it doesn't make it wrong if they choose to do something different instead. Because you have no entitlement to something that belongs to somebody else.
But I also know that for a lot of fans who would defend the idea that, for better or worse, stuff like novels is "non-canon" or "fluff" or what have you, that idea didn't spring from nowhere. It sprung from that fact that the PTB (specifically Gene) said it was the case in terms of what should be regarded and what shouldn't be.
That was his right as long as he was running the show, though he made a much bigger deal out of it than most producers would. After he passed on, his successors weren't as concerned about defining canon in such proscriptive terms, so that attitude pretty much became obsolete. Except in the minds of fans who don't realize that it's no longer binding.