Do you mean the 10,000 lightyear range transporter, which only works with the planet's unique crystaline composition? The transporter which Voyager stole, but couldn't use for that and other reasons?
No. There was another episode where a group of aliens were using their long range transporters by beaming 1 person at a time over a distance of 10 light years so they can slowly take over a starship and use it for themselves - the former crews were put into 'habitats' that suited their biology most - the aliens in question were very intolerant to cold.
Anyway, I agree with your point on lack of progress, although the million km emergency beaming seems to go against that. However, this is why I like the idea behind Scotty's transwarp beaming. He knows it will work, but gets sidelined because he works on it in an overeager manner which gets him punished. He puts the project on the back burner for decades. No one thinks it will work so no one else pursues the idea.
In the mean time, dimensional beaming and subspace beaming both seem more promising. They both are workable but dead ends in their own ways. In that regard they act as distractions from the actually more promising transwarp beaming. Eventually Scotty gets around to working on it again, and actually makes it work sometimes between 2379 and 2386.
The bad side is, none of the writers are going to be smart enough, or none of the producers will be willing, to explore the full implications of transwarp beaming. For one thing, a lot of the functions of ships get obsoleted by its existence.
It would appear that TNG was originally supposed to have been set 150 years after Kirk... and was stated that instead of starships, the Federation would simply beam from one planet to another - though I would imagine that deep space exploration would likely be done via a combination of starships and automated probes.
Hence why they set TNG 75 years from Kirk.
I for one find a great deal of realism in the "lack of progress": there could be plenty of it, but none of it would make any difference. Increasing range from 40,000 km to 400,000 km or 4 million km would simply not be useful and would not see any action in the episodes; inventors pushing for 40 million kilometers would not get funding, not unless they would be able to promise 40 billion kilometers later on.
I don't find any realism in the lack of progress.
I find it highly unrealistic.
There is no money in the Federation. Scientific research is likely conducted by allocating necessary resources (raw materials, technology, etc.) to labs and people doing the research in whatever quantity is necessary.
You cannot apply the artificial constrictions of the monetary system as it exists today on the Federation, because it was stated such limitations do not exist.
The writers simply didn't advance the Federation enough 75 years down the line after TMP - which realistically, for a civilization as advanced and progressive as theirs, technical and scientific breakthroughs would occur much faster than they do even in real life - especially with over 150 different species working together.
Technological stagnation occurs in real life due to socio-economic reasons which are based on artificially induced scarcity.
The Federation has no such issues - and the writers were idiots for not making every planet and colony self sufficient (because quite frankly, we had the ability to do so since 1974 in reality... let alone the Federation that uses replicators to convert energy into matter, not to mention what was seemingly mentioned to be subspace based computers.
The writers (except perhaps Roddenberry) had relatively little idea on how to progress technology adequately because they had little to no understanding of it to begin with, and when presented with something 'advanced', they had little to no idea how to wrap their brains around it.
There were ample examples from real life to give them proper indications... but even if they did this, it is likely the network executives or someone else could have stopped this as it would easily defeat the purpose of the show.
But we do know that Scotty developed a formula for Transwarp beaming in the late 24th century... likely sometime after Voyager came back to the Federation.
Though, I 'guess' that one of the reasons SF rarely portrays its technical prowess is because they don't necessarily want their potential aggressive adversaries who might misuse it... so they keep themselves intentionally limited?
However, if I remember correctly from one of the episodes of TNG, the Federation was pretty open in sharing a lot of the technical breakthroughs it made with their neighbours - which begs the question, why did the writers persist in writing those races close to them as adversaries?
It would seem to me they simply wanted to keep some adversaries for the sake of it and create 'typical drama'.
Mind you, Tuvok wasn't exactly surprised about the 'extremely long range of 10 lightyear'... then again he's a Vulcan. But Janewa's reaction wasn't that much of a big deal either.
Still, it might have been possible that the Federation was into the millions or billions of KM range with transporters in the late 24th century (possibly even across the entire solar system), we just never got to see this in use.
And their transporters do appear to be quite versatile. So what they might be lacking in range, they could make up in other areas when it comes to practical use.
Mass beaming of this level (solar system wide) might present certain limitations, so transporter cycles would likely be allocated based on pressing time constraints, while others would simply use starships.