DS9: Redefined 2.0 - The Ultimate DS9 Upscale Project

[Tosk]

Not really related to this specific topic, but I wonder if theres a consensus out there in the tech world if advancements in technology is expected to continue to improve when it comes to enhancing SD content. Could it be even better in 20, 30 years? Or is there a ceiling on how much something from a SD source can be improved artificially? End of the day I do realize you are limited to the data/pixels available from the SD source.

Eventually AI's ability to make up video imagery based on existing footage will progress to the point where it's insanely hard to tell what is artificial. You'll be able to turn SD into full HD with ease...except it won't actually be the original footage any more, it'll be a synthetic approximation.

 

[Cryogenator]

The potential of generative upscaling and filling is unlimited. Artificial intelligence can already go far beyond source pixel interpolation by using extensive training on vast datasets to generate totally new information and can also very accurately convert 2D to 3D (I now watch all movies and shows in 3D using Moon Player's realtime stereoscopic conversion in my Vision Pro, and prerendered autoconversions can look even better), somewhat accurately colorize, regrade from standard to high dynamic range, interpolate frames, generate audio for silent video, generate video from audio, and even generate audio and video from text. In two or three decades, making a DVD look like native UHD could be possible and generative filling could be stable enough to reliably expand the aspect ratios of whole shows and movies. In the distant future, novels and scripts will be instantly convertible to flawless holodeck experiences.

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

 

[trekfan_1]

Wow thars kind of mind blowing stuff potentially on he horizon . I wonder if this technique can soon help shows that were shot and edited on video tape look like it was done UHD. Like old episodes of Threes Company . Now that would be impressive . Perhaps feeding AI with high-quality reference material could also significantly improve its ability to upscale SD content. For example, using TNG HD datasets to help A.I predict the overall look of Trek centric astetehics like starship design panels, props , alien skin tones and appearance etc . Also referencing HD renderings of actors likeness from other sources where HD footage of those actors exists to help A.I to beter predict and improve flesh tones on SD content.

 

[David cgc]

Perhaps feeding AI with high-quality reference material could also significantly improve its ability to upscale SD content. For example, using TNG HD datasets to help A.I predict the overall look of Trek centric astetehics like starship design panels, props , alien skin tones and appearance etc .

I haven't looked into it, but I've seen AI upscaling, AI frame expansion, and AI style transfer, and I've been thinking it should be possible to use those technologies in concert to merge the 16x9 SD version of Babylon 5 and the 4x3 HD version of the show to to make a faithful 16x9 HD version that'll look much better than upscaling the SD version, and won't have the artifacts and hallucinations of just doing a frame expansion on the HD version. Unfortunately, there are differences in color timing and frame distortion that mean doing it "conventionally" by overlaying both sources directly is impractical.

The remastered footage from What We Left Behind and "Birthright" feel intuitively like they should be able to improve upscales of DS9, specifically, but I don't know enough about the technology to feel as confident that it would be useful as I am about the idea of being able to give frame-to-frame example data.

 

[Cryogenator]

I haven't looked into it, but I've seen AI upscaling, AI frame expansion, and AI style transfer, and I've been thinking it should be possible to use those technologies in concert to merge the 16x9 SD version of Babylon 5 and the 4x3 HD version of the show to to make a faithful 16x9 HD version that'll look much better than upscaling the SD version, and won't have the artifacts and hallucinations of just doing a frame expansion on the HD version. Unfortunately, there are differences in color timing and frame distortion that mean doing it "conventionally" by overlaying both sources directly is impractical.

Various fan projects such as the Wizarding World Ultimate Definitive Editions by icebox616 (AKA Klexos) have already done this with AutoOverlay and manual color correction. He merged three aspect ratios (4:3 DVD, 16:9 digital, and 2.39:1 4K Blu-ray) with contextual blur filling the corners to come as close as possible to the full 35mm frame while also upscaling, HDR regrading, and reintegrating all available deleted scenes and improving them with 7.1 surround sound upmixes from Amadian and visual effects from JJPotter. If a skilled faneditor wanted to merge the 16:9 DVDs and 4:3 Blu-rays of Babylon 5, he certainly could!

 

[trekfan_1]

A good long term "test" on how well a future A.I can enhance a SD source would be to have it try to upscale/enhance TNG sourced from a DVD and have it side by side compared to TNG that was reedited into HD via the orginal film elements.

 

[Cryogenator]

Eventually, AI's ability to make up video imagery based on existing footage will progress to the point where it's insanely hard to tell what is artificial. You'll be able to turn SD into full HD with ease... except it won't actually be the original footage anymore; it'll be a synthetic approximation.

Wow! That's kind of mindblowing stuff potentially on the horizon. I wonder if this technique can soon help shows that were shot and edited on videotape look like they were recorded in UHD.

Sisko

SiskoGPT

 

[Richard S. Ta]

Within five years this kind of technology will be able to do things easily which we can’t even imagine right now.

Many decry images such as the ChatGPT generated one above and yes, it is slightly off. Sisko’s face ends up looking like a very good painting of Sisko’s face.

However the technology is in relative infancy and is improving exponentially.

The next-next generation of kids will be able to remould and redefine media as they say fit. AI is really incredible in terms of its potential as an editing tool.

Don’t like a scene? Rewrite it and let AI do the rest. How about giving SNW a TOS skin or vice-versa?

I don’t know if I’m in favour of not but in a world where a viewer can conceptually alter the media they watch, remasters become redundant. We could all have our own, tailor made.

Ramblings over. :-)

 

[Cryogenator]

I love your prescient realizations which are a refreshing rarity amidst a morass of delusional doomerism and impotent indignation. Seeing such vitriol from the masses is unsurprising but is strange coming from science fiction fans who love Starfleet computers and Mr. Data.

The new Black Mirror episode "Hotel Reverie" depicts a fictional Casablancaesque film remade into an holodeckesque interactive simulation, and the earlier "San Junipero"

Spoiler

is set in a simulated reality in which its immortal inhabitants can time travel at will because the laws of physics can be almost totally ignored in digital worlds.

This is the ultimate future of storytelling.

Training artificial intelligences on copyrighted works is no different from training natural intelligences on copyrighted works; just as we are free to create original works inspired by copyrighted works we've studied, so are we free to have artificial intelligences create original works inspired by copyrighted works we've directed them to study. An AI creating an original image in the style of Studio Ghibli is no different from a human creating an original image in the style of Studio Ghibli. No legal nor moral right to exclusive use of a general style exists. Generating a Star Trek fanfilm is no different from shooting or animating one without AI.

If an artificial intelligence or robot can do a job for me as well as or better than a human for the same or lower cost (or worse but for much less), I will use it instead of hiring a person who has absolutely no right to my money. I have no obligation to provide anyone with employment. No one has any obligation to provide employment and no one has any right to employment, either. No one is still fighting for the nonexistent right of lamplighters and switchboard operators to continue their work because automated streetlights and telephones are vastly superior. Automation and simulated reality will eventually enable universal postscarcity and freedom from working for survival (like in the Culture). While there are difficulties in this ongoing transitional phase, insisting on makework jobs for humans when AI and robotics can do them better is a fool's errand, and the quixotic quest to rebottle the genie is doomed to fail.

 

[evilchumlee]

Within five years this kind of technology will be able to do things easily which we can’t even imagine right now.

Many decry images such as the ChatGPT generated one above and yes, it is slightly off. Sisko’s face ends up looking like a very good painting of Sisko’s face.

It still looks better than the original. I would 100% super crisp, vivid and sliiightly off DS9 over blurry, muddy DS9.

And yeah, it may not even take 5 years. Compare AI images and videos from just a year ago and the improvements are WILD.

I'm not on the AI hate club at all. I am absolutely jazzed up about AI. AI is going to be some absolutely amazing things. People will have to adjust, but it will be spectacular in the end.

 

[Tosk]

It still looks better than the original. I would 100% super crisp, vivid and sliiightly off DS9 over blurry, muddy DS9.

The above Sisko example is not "slightly off". It's a completely new image that only resembles the original.

 

[Qonundrum]

Sisko

Moire on the woman's headdress is not the most readily noticeable but there, regardless. Fuzzy definition yet neither looks like a blob of plastic. Sisko has defined pips...

SiskoGPT

A motion video succession of frames of that would be a hoot, as:

Sisko loses a pip, how nice. His uniform also has crushed detail as well as the wrong shade of red. The fabric wrinkles in the red are different too. Her headdress looks like it's floating off the screen, as well as being teal rather than blue - this was 1993, not 2023, when it came to palette wheel templates. The yellow plant behind Sisko looks completely different and what was possibly a basket handle is also completely removed. The fern-like plants in the background lost both accurate color tone AND definition, which is quite an accomplishment. Both characters look like plastic mannequins that also resembles waffle batter for some odd reason. Looks like a video game.

 

[Ray Hardgrit]

I wish my video games looked like that. But I'm glad my TV shows don't, the live action ones at least.

 

[Qonundrum]

The potential of generative upscaling and filling is unlimited. Artificial intelligence can already go far beyond source pixel interpolation by using extensive training on vast datasets to generate totally new information and can also very accurately convert 2D to 3D (I now watch all movies and shows in 3D using Moon Player's realtime stereoscopic conversion in my Vision Pro, and prerendered autoconversions can look even better), somewhat accurately colorize, regrade from standard to high dynamic range, interpolate frames, generate audio for silent video, generate video from audio, and even generate audio and video from text. In two or three decades, making a DVD look like native UHD could be possible and generative filling could be stable enough to reliably expand the aspect ratios of whole shows and movies. In the distant future, novels and scripts will be instantly convertible to flawless holodeck experiences.

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

Admittedly impressive! Not perfect, but really good for what the tool needs to do. For when the original material no longer exists, this is legitimately phenomenal. It would be fun to see a real remastered TNG 35mm frame at 1080p or 4K next to an upscaled original-edge VT frame (720x480i upscaled to 1080p or 4K equivalent) with Starlight. Obviously from the D1 archive tape and not from the DVD release as the amount of compression, on top of everything else, would be too much of an unfair comparison and most upscaled 480i items came from the original uncompressed source.

To view this content we will need your consent to set third party cookies.
For more detailed information, see our cookies page.

Accept third party cookies

Now that alone, without the cutup scenes and replacement music would be worth the purchase.

 

[Qonundrum]

I wish my video games looked like that. But I'm glad my TV shows don't, the live action ones at least.

Give it 5 years and an nVidia 7090 GPU or two of those in parallel processing on a Threadripper 27995WX Pro CPU or whatever will be out there then with 600 cores/1200 threads, plus 8675310GB of RAM (minus 1GB for the OS since you'll likely be running Linux on it), and perhaps it'll be that good. It's amazing how photorealistic things are already. And yet, so many people think Pac-Man from 1980 is good enough. Too funny. Amazed at how much as single 7995WX PRO CPU with a 5090 GPU costs right now.

 

[Tosk]

The yellow plant behind Sisko looks completely different

Everything looks different.

I'm not being hyperbolic, the "GPT" version is not an upscale, it's a completely new image that looks similar to the original.

There's not a single shared detail. Look at Rakal's headdress...it's not even remotely the same pattern. The AI drops her facial markings entirely and seemingly decided that she is Asian in appearance.

This is an advanced version of "Hey, look at this picture...now draw it from memory."

 

[Firebird]

The next-next generation of kids will be able to remould and redefine media as they say fit. AI is really incredible in terms of its potential as an editing tool.

Don’t like a scene? Rewrite it and let AI do the rest. How about giving SNW a TOS skin or vice-versa?

As an artist and a writer myself, the idea that someone could just take my work and run it through a soulless computer filter with no effort on their part and claim the output as their own is more than disquieting, and a touch disgusting to me.

 

[Richard S. Ta]

As an artist and a writer myself, the idea that someone could just take my work and run it through a soulless computer filter with no effort on their part and claim the output as their own is more than disquieting, and a touch disgusting to me.

Oh I agree. It is however the road that we are on.

 

[Tomalak]

The ultimate aim is to replace art with monetisable content that is owned by corporations, with no need for the involvement of pesky writers or actors who tend to go on strike.

 

[Firebird]

The ultimate aim is to replace art with monetisable content that is owned by corporations, with no need for the involvement of pesky writers or actors who tend to go on strike.

Yup. The actual creatives are irrelevant to the money men, who only see the end product and have no concept or care of the process it takes to get there.