Everyone has different tastes, obviously, so it's a little difficult to create a list of hard-and-fast rules beyond the basic rules of storytelling that apply to all genres. That being said:
OBSERVE THE BASIC RULES OF STORYTELLING THAT APPLY TO ALL GENRES.
I'm sure you know the drill. Characters should speak and act like believable people. (I find it helpful to say the dialog back to myself to be sure it sounds like something that would come from the mouth of an actual person) Be consistent: don't introduce a super-powerful alien creature like Q then have him defeated in a fistfight. Make sure your story gives your characters some kind of obstacle to overcome or problem to grapple with--don't just have them standing around and talking. This is especially true if your story is populated entirely by characters you created yourself. It may be enjoyable to see Kirk and Spock shooting the breeze, provided it's written well, because those are characters we have a preexisting affection for. Nobody will care about Captain Quigley and Mr. Kl'plorth unless you give them something interesting and exciting to do.
Apart from that, here are a few other things that I find helpful:
Use the vastness of space to your advantage
Space is big, and the things in it are really really really really really REALLY far apart. Use this to your advantage. If you're writing a story about a starship on an exploratory mission, they'd realistically be too far away for the Captain to call Starfleet Command and request instructions when he gets into a sticky situation. This way, the tough dramatic choices are forced upon your Captain, not delegated to some Admiral on a viewscreen.
Place realistic limits on sensing devices
The idea that a character can look at his magic sensor display or tricorder and learn everything about a situation, down to the DNA of the aliens aboard an approaching ship that's thousands of miles away, has robbed many a Star Trek story of the mystery and danger of space exploration from which the franchise theoretically derives much of its appeal. Don't let this happen to you. So what if it happened on Enterprise or Voyager? Don't let yourself be bound by other people's lazy writing choices. And don't feel obliged to "explain" via technobabble infodump why your ship's sensors can't instantly tell what color underwear the people on an approaching vessel are wearing. Figure out the parameters in your head and then simply have the story abide by them without overexplaining.
Avoid alien stereotypes and monocultures
The only country on Earth where everyone dresses, acts, and speaks alike is North Korea. So unless you're writing about an alien planet that's a similar kind of dystopic totalitarian hellhole don't make everyone on it act, speak, think, and dress alike. And keep the same thing in mind when writing about established Star Trek races. All Klingons are not war-obsessed Space Vikings; they can't be, or there'd be nobody to design starships, build buildings, or fix the plumbing. All Romulans are not sneaky humorless guys with soup-bowl haircuts. Again there is no reason for your to abide by the lazy writing choices that other people have made.
Canon? Schmanon!
For understandable reasons, anyone who writes an officially-licensed Star Trek novel or comic has to abide by the Offical Canon Policy. But you're writing a fanfic, which means you can do whatever the heck you want. The important thing is to stay true to the spirit of Star Trek and the essence of the preexisting characters, organizations, and alien races that make up the Star Trek universe. If Douchey McNitpick wants to point out your story's "canon violations" don't let it bother you. If you're writing good stuff, people will enjoy it.
lol cool.
