meguIV: The Official Akiba-Online DVD Encoder (v1.0.1.1)

[Vitreous]

The ONLY reliable way to tell if a DVD video stream is interlaced is to apply SeparateFields() and framestep. Manually.

Which means if you have a DVD that's part progressive, part interlaced, the only way to get the best quality rip is to manually mark the interlaced sections then use a script like this:

# Progressive followed by interlaced followed by progressive again
prog = last
inter = last.QuickTGMC( Preset="Slow" ).SelectEven()
prog.trim(0,999) ++ inter.trim(1000,1999) ++ prog.trim(2000,2999)

You could also run a QTGMC InputType=2 on the progressive part if it has been badly deinterlaced
____

I've come across vids with a huge number of progressive->interlaced changes (>100), so the manual approach is too much effort. In that case you can automate the process. Works pretty well, but clearly it won't be perfect (needs the TIVTC and VInverse plugins):

# Auto run QTGMC on interlaced frames only, use VInverse to tidy up miscategorized frames
# Can't use SetMTMode or Distributor with this script, so a less efficient MT method is used
clip = last
ConditionalFilter( clip, \
    clip.VInverse(), \
    clip.MT( """QuickTGMC( Preset="Slow" )""", GetMTMode(), 32 ).SelectEven(), \
    "IsCombedTIVTC( cthresh=8 )", "=", "false" )

____

Both those scripts give same rate output. Can combine with motion interpolation on the progressive parts for double rate output

 

[astrayred]

Ok, so I have SeparateFields() after the loading the video. I load the script into VirtualDub and step manually. Each field shifts up and down between frames. Does this mean that it's interlaced?

Also, I have another request. I'm considering changing my CPU to a AMD Phenom II X6 1055T. If anyone else has this CPU, I would like to ask if you mind sharing the speed that this CPU achieves at QTGMC preset "Slow" and x264 @CRF 23 and Preset "Slower", for comparison with my current CPU. Thanks!

 

[Vitreous]

Each field shifts up and down between frames. Does this mean that it's interlaced?

No. That will always happen with SeparateFields. What you're looking for is whether there are duplicate frames.

If you have an progressive vid, then then when you step through with SeparateFields you'll see two frames the same, then two more the same etc. The only difference between these duplicates will be that slight shift up and down. If you have an interlaced vid, then every frame will be different (and also a slight shift up and down). It's easy to see this effect when there is motion - just something simple like someone moving their arm is enough to spot it.

I think it's easier to look for combing in the original stream without using SeparateFields. You first need to make sure your player isn't automatically deinterlacing. Then the combing is easy to spot where there is motion. The first picture on this post shows combing. Lighter combing may involve little jaggies of just 1 pixel, but zoomed in they're still easy to see.

Whichever method you use, just watch out when you're measuring the frame number. With SeparateFields you are looking at a 60fps stream, so when you take the frame number, you must halve it if you're output vid is going to be 30fps. Similarly, if you don't use SeparateFields then you're looking at 30fps, so if you're targeting 60fps then you need to double the frame number.

CRF 23

Can't help with your CPU query, but that CRF caught my attention. Is this for HD vids or are you squeezing the filesize?

 

[astrayred]

No. That will always happen with SeparateFields. What you're looking for is whether there are duplicate frames.

If you have an progressive vid, then then when you step through with SeparateFields you'll see two frames the same, then two more the same etc. The only difference between these duplicates will be that slight shift up and down. If you have an interlaced vid, then every frame will be different (and also a slight shift up and down). It's easy to see this effect when there is motion - just something simple like someone moving their arm is enough to spot it.

I think it's easier to look for combing in the original stream without using SeparateFields. You first need to make sure your player isn't automatically deinterlacing. Then the combing is easy to spot where there is motion. The first picture on this post shows combing. Lighter combing may involve little jaggies of just 1 pixel, but zoomed in they're still easy to see.

Whichever method you use, just watch out when you're measuring the frame number. With SeparateFields you are looking at a 60fps stream, so when you take the frame number, you must halve it if you're output vid is going to be 30fps. Similarly, if you don't use SeparateFields then you're looking at 30fps, so if you're targeting 60fps then you need to double the frame number.


Can't help with your CPU query, but that CRF caught my attention. Is this for HD vids or are you squeezing the filesize?

Thanks for the reply. My PC is currently encoding so I don't want to disturb it too much, but once it's done I will try what you have said.

Regarding the CRF, I'm squeezing the file size. I understand that most people will think it's too high (or too low), but early on when I was establishing my encoding methology, I did various comparisons between the lossless pre-rendering file that MeGUI produces and the end product @CRF23. Obviously while I would not say that they look exactly the same, I would also say the difference is not day and night. I realize that you use CRF 18.5, so 23 might seem absurdedly low to you...

In any case, what with HDD prices dropping like a stone, I should probably re-examine my CRF values...

Lastly, I would like to use 2-pass, but I'm worried about HDD wear and tear from writing 60+ GB of data everyday.

 

[Vitreous]

I realize that you use CRF 18.5

No, I usually use 19.5 for 30fps & 20.7 for 60fps. I take them up a little (~1) for complex or long vids, down a bit if I'm losing fine detail. Also higher values for HD (~22-23) - always use higher values for more spatial or temporal resolution.

You'll get more x264 grain and blur with a 23 setting. No problem with that if you've compared and are happy with the output. But I would be concerned that the improvements you gain with Slower over Slow might get lost in the x264 vagueness at 23.

Lastly, I would like to use 2-pass

Actually, I think 1-pass is a fairly good idea in some cases. 2-pass was first brought in MeguIV when we were having MT stability problems and it helped. Those stability issues are not so bad now. 2-pass reduces the amount of memory needed by the encoding process and speeds up the slowest scripts. But I've measured 1-pass as faster than 2-pass on some machines on the faster settings (even down to "Slow" in some cases). Those huge files are a harddrive burden, but they are great for additional processing (e.g. get a 30fps and 60fps rip from a single 60fps first pass). So several considerations...

 

[astrayred]

No, I usually use 19.5 for 30fps & 20.7 for 60fps. I take them up a little (~1) for complex or long vids, down a bit if I'm losing fine detail. Also higher values for HD (~22-23) - always use higher values for more spatial or temporal resolution.

You'll get more x264 grain and blur with a 23 setting. No problem with that if you've compared and are happy with the output. But I would be concerned that the improvements you gain with Slower over Slow might get lost in the x264 vagueness at 23.


Actually, I think 1-pass is a fairly good idea in some cases. 2-pass was first brought in MeguIV when we were having MT stability problems and it helped. Those stability issues are not so bad now. 2-pass reduces the amount of memory needed by the encoding process and speeds up the slowest scripts. But I've measured 1-pass as faster than 2-pass on some machines on the faster settings (even down to "Slow" in some cases). Those huge files are a harddrive burden, but they are great for additional processing (e.g. get a 30fps and 60fps rip from a single 60fps first pass). So several considerations...

Again we find ourselves going back to the main problem with video encoding. Balancing file size, speed and quality. It never ends, does it?

But I do get what you are saying. If I'm (or rather my computer) taking the time to do high-quality deinterlacing, I might as well do it a favour by capturing all that fine detail. Good point there...

Actually when I say 2-pass, I meant 2-pass bitrate encoding in x264, not pre-rendering and then encoding. At least, that's what I get from your reply. Also, with regards to 60fps vs 30fps, I never really do understand why anyone would encode at 30fps with TGMC... Throwing away temporal resolution like that...

 

[Vitreous]

Actually when I say 2-pass, I meant 2-pass bitrate encoding in x264, not pre-rendering and then encoding.

Ah, I see. The only advantage I can see to bitrate encoding is its predictability. A constant quality rip will look better than constant bitrate rip of the same size, because it has more flexibility in where to use the bits. But getting a constant quality rip to the size you want...

Also, with regards to 60fps vs 30fps, I never really do understand why anyone would encode at 30fps with TGMC... Throwing away temporal resolution like that...

Couldn't agree more. It's partly the misapprehension that encoding time and filesize will double, when it's more like 1.3x-1.5x. It's partly habit. And partly because temporal accuity varies widely.

 

[astrayred]

Ah, I see. The only advantage I can see to bitrate encoding is its predictability. A constant quality rip will look better than constant bitrate rip of the same size, because it has more flexibility in where to use the bits. But getting a constant quality rip to the size you want...


Couldn't agree more. It's partly the misapprehension that encoding time and filesize will double, when it's more like 1.3x-1.5x. It's partly habit. And partly because temporal accuity varies widely.

The difference between 60fps and 30fps is like night and day to me. Then again, I grew up playing plenty of FPS, so that might be why I notice these things.

Anyway, I have some unusual behavior WRT QTGMC to report.

I was in the process of applying QTGMC to some progressive 720p WMV videos for archival purposes. I noticed

QTMC(Preset="Very Fast", InputType=1) (Let's call this A for simplicity's sake)

is actually slightly slower than

QTGMC(Preset=Very Fast").SelectEven() (Let's call this B)

It would seem to me that B would be computationally more intensive as it involves NNEDI2? From reading the script I think that progressive input does not use NNEDI2...

 

[Vitreous]

I was in the process of applying QTGMC to some progressive 720p WMV videos for archival purposes. I noticed

QTMC(Preset="Very Fast", InputType=1) (Let's call this A for simplicity's sake)

is actually slightly slower than

QTGMC(Preset=Very Fast").SelectEven() (Let's call this B)

It would seem to me that B would be computationally more intensive as it involves NNEDI2? From reading the script I think that progressive input does not use NNEDI2...

I don't get this behaviour at all. Tested with 480p, 720p and 1080p - all go significantly faster with InputType=1. As you correctly note, that setting removes the need for the NNEDI2 call (no need to interpolate progressive input), which makes it faster for me.

So your result is fairly puzzling. The only idea I have is that maybe you have a low memory situation (HD is more prone to that) - you may be hitting the pagefile. The behaviour wrt speed might be different in that case. Also, I haven't done any cache analysis on QTGMC (in particular the AviSynth caching mechanism) - cache use may be relevant when memory is low. Neither of these explanations is especially convincing though...

 

[xater]

Actually when I say 2-pass, I meant 2-pass bitrate encoding in x264, not pre-rendering and then encoding. At least, that's what I get from your reply. Also, with regards to 60fps vs 30fps, I never really do understand why anyone would encode at 30fps with TGMC... Throwing away temporal resolution like that...

Couldn't agree more. It's partly the misapprehension that encoding time and filesize will double, when it's more like 1.3x-1.5x. It's partly habit. And partly because temporal accuity varies widely.

For me: my laptops won't play 60fps smoothly... :tea:

 

[Vitreous]

For me: my laptops won't play 60fps smoothly...

Of course, forgot about that - the best reason I guess.
____

Xater's input reminds me about something else - as I know he does rerips. People writing their own scripts should be wary of using DirectShowSource together with (Q)TGMC. It doesn't serve frames accurately on some vids, which can give strange jumpy or blurry motion when ripping. This is mostly likely to affect those doing rerips - I noticed it when using it to rerip an MKV, and have now seen it with WMVs as well.

Switching to DirectShowSource2 solved the problem for me. It's hidden away in Haali's Media Splitter. Install that, and copy the file "avss.dll" from "Program Files\Haali\MatroskaSplitter" (or similar) into your AviSynth plugins folder. Then you use it like this:

DSS2( "E:\MyVid.mkv") # Straight replacement for DirectShowSource

 

[astrayred]

Of course, forgot about that - the best reason I guess.
____

Xater's input reminds me about something else - as I know he does rerips. People writing their own scripts should be wary of using DirectShowSource together with (Q)TGMC. It doesn't serve frames accurately on some vids, which can give strange jumpy or blurry motion when ripping. This is mostly likely to affect those doing rerips - I noticed it when using it to rerip an MKV, and have now seen it with WMVs as well.

Switching to DirectShowSource2 solved the problem for me. It's hidden away in Haali's Media Splitter. Install that, and copy the file "avss.dll" from "Program Files\Haali\MatroskaSplitter" (or similar) into your AviSynth plugins folder. Then you use it like this:

DSS2( "E:\MyVid.mkv") # Straight replacement for DirectShowSource

What's your laptop specs? Normally I would suggest DXVA decoding, but I have a hunch your laptop might be too ancient for that. If that's the case, why not try the DIVX H.264 decoder? It's one of the fastest out there.

@Vitreous: DSS2, for some reason always output the video upside down for me. :/ I guess I could use FlipVertical(), but I'm worried about degrading video quality like that. Personally I use FFMS2 as a source.

Incidentally, WRT to my report about progressive mode being slower than interlaced mode, perhaps FFMS2 is the problem? I remember it being quite resource intensive...

 

[xater]

Currently, I am on a netbook Atom...

I actually used MeguIVit to do a rerip of a Bomb.tv wmv file... (I *hate* wmv) with my new Atom netbook...

I got around 2.70fps not using the edgeclean tool, and about 2.05 FPS (2.19 tops) using it, more or less an hour for 4 minutes, which is fine for me as I'd like to convert countless of small clips to mkv (saves around 25% for wmv, much more for some which are mpg!!).

Some of these old clips are really blocky... Vit, how would you say that QT works for it, or would it be better to use a different filter? uzzled:

Also, I just found that dss2 won't handle the sound... am I doing it wrong? (I used audiodub to get the audio with old directshowsource in other rips that required editing)

 

[Rollyco]

If you're just looking to change container from WMV to MKV (and not re-encode anything), you can do so with this: http://www.akiba-online.com/forum/showthread.php?p=752750&highlight=graphstudio#post752750

 

[Vitreous]

Some of these old clips are really blocky... Vit, how would you say that QT works for it, or would it be better to use a different filter?

You will get some deblocking because of the temporal blurring - it might not be too effective in stable or plain coloured areas. Serious blocking can affect the motion analysis. You could run a deblocking step *and then* a QT step although you may will lose some fine detail that way. You're using QT outside of it's area of intent so it's quite hard to say - I can only really offer the "try it and see" advice.

Also, I just found that dss2 won't handle the sound... am I doing it wrong?

DSS2 only handles video - you need to use AudioDub as well

 

[astrayred]

Vitreous:

I found this nifty little script that can be used to check whether a video is interlaced or progressive, or in what field order.

##############################################################
#    Allows you to Check the Field Order of your Source.     #
# Use frame advance in virtualdub to check for smooth motion #
##############################################################

function CheckFieldOrder(clip c) {
sourcevideo=c
TFF=sourcevideo.AssumeTFF().SeparateFields().subtitle("Top Field First, if motion is smooth use order=1")
BFF=sourcevideo.AssumeBFF().SeparateFields().subtitle("Bottom Field First, if motion is smooth use order=0")
StackVertical(TFF,BFF)
}

Also, after updating to the latest version of NNEDI3, QTGMC "Slow" preset, when using NNEDI3, returns an error that "NSize should be [0,4], even though checking NNEDI3's file, it says that a value of 6, as in your script, is a valid choice. What's happening?

 

[Vitreous]

I found this nifty little script that can be used to check whether a video is interlaced or progressive

I must be doing too much of this - I check by eye...

Also, after updating to the latest version of NNEDI3, QTGMC "Slow" preset, when using NNEDI3, returns an error that "NSize should be [0,4], even though checking NNEDI3's file, it says that a value of 6, as in your script, is a valid choice. What's happening?

I can't reproduce this behavior - works fine for me using NNEDI3 v0.9.1

 

[astrayred]

I must be doing too much of this - I check by eye...


I can't reproduce this behavior - works fine for me using NNEDI3 v0.9.1

Apologies, my mistake again. I thought I was using the latest version but actually was using the penultimate one. This is what happens when you have two machines doing QTGMC. :/

Anyway, a new version of NNEDI3 has been released, and testing by one of the board members indicate that it is actually faster than NNEDI2 itself. Also, the source code has been released which means compiling a 64-bit version will finally be possible. We might actually finally be able to use 64-bit QTGMC.

 

[Vitreous]

Yes, I've seen the new NNEDI3. Not had much time for coding, but when I do I may replace NNEDI2 with NNEDI3 as the default for the appropriate QTGMC presets (assuming it really is faster).

 

[astrayred]

Hey Vitreous. I had time to do some testing.

PC Specs

AMD Phenom II X4 955 @Stock speed, Undervolted @ 1.28V
4GB Corsair DDR2-800 RAM
Windows 7 Ultimate 64

Avisynth Script

SetMTMode(5,4)
DGDecode_mpeg2source("C:\Work\Test.d2v")
SetMTMode(2,4)
QTGMC(Preset="Slow",EDIMode="NNEDI3")#QTGMC(Preset="Slow") for NNEDI2 mode

Distributor()

Trim(0,1000)

Results:

tritical's NNEDI3
* Pass 1/1: Finished in 00:00:50.171 (19.95 FPS)
* Frames: 1001 (1001 keyframes)
* Size: 494.88 MB
NNEDI2
* Pass 1/1: Finished in 00:00:51.873 (19.30 FPS)
* Frames: 1001 (1001 keyframes)
* Size: 494.88 MB

I used avs2avi in null encoding mode, so as to eliminate the video encoder as a variable, isolating only NNEDI2/NNEDI3.

Let me know if there's anything flawed in my testing methodology and I will rerun it.