meguIV: The Official Akiba-Online DVD Encoder (v1.0.1.1)

[Rollyco]

The whole process is remarkably flaky. Sometimes there are random crashes or freezes, and sometimes MeGUI doesn't realize it and will wait forever.

Close all unnecessary processes like browsers, antivirus, defragmenter, etc. Do so before you start the encode and don't do anything while it's working. If you have overclocked your system, back down on the settings.

 

[Vitreous]

quite a few "Processes," and lots and lots of "Services."

Background disk access seems to be the main cause of crashes in MeguIV variants. That could be caused by Windows Defender, your virus checker doing a scan, the Indexing service, one of the .NET optimizers (mscorsvw.exe) etc. Research your machine and the web to see what you have running and then you could attempt to temporarily or permanently switch some things off and see if it helps.

Not really sure if it will help though. MeguIV and it's variants use multiple processors to get good encoding speeds, but as I've said before AviSynth is buggy when it comes to multithreading (mencoder might be partly to blame too). If you go single-threaded it's stable but much slower (but you can't do that in MeguIV directly anyway). 64-bit Win7 seems particularly sensitive to the problems, and despite my attempts to provide alternatives MeguIV may be unusable on 64-bit Win7 for some.

The only ideas I have (apart from switching off background tasks and crossing your fingers) for future development are:
- Wait for the next release of AviSynth (development hasn't stopped, but there have been no releases for years)
- Tweak MeguIV to use multiple single threaded workers
- Replace mencoder with ffmpeg for the 1st pass output and see if that improves things
- Make a 64-bit Megui with the actively updated unofficial AviSynth build from SEt (who seems to be the only productive developer)

Obviously none of these solve your immediate problem though. However, stability is something I will look at again for a new MeguIVit at some point as I've been having problems on 1080p 60fps material. I've been using the multiple single threaded approach but I don't think it's a long term solution (it's not so fast). I will most likely look at the mencoder replacement at some point and see if that helps. I'm not likely to make a 64-bit version any time soon.

 

[Vitreous]

Will this version even run on my Win7 if I install all the necessary programs and do the pointing?

Are you talking about the multi-single threaded method I said I was using? Unfortunately, that isn't a version of MeguIVit. I don't actually use MeguIV or MeguIVit, I work manually in MeGUI. MeguIV(it) were made to be convenient one-click rippers for the community - if you need to go beyond what they were designed to do then you need to learn how to do AviSynth scripting and how to use MeGui generally...

 

[anandneemish]

Background disk access seems to be the main cause of crashes in MeguIV variants

Hi Vitreous,
I wanted to chime in here on 201flyer's problems. His symptoms sound remarkably similar to my issues from several months ago. I want to report that since using your mods, overall encode speed is now much faster, and I no longer have any instability. In other words, whereas I had reported that disk access would reliably crash mencoder, that is no longer the case today.

In the last 10-15 encodes, I have not experienced a single crash ... even when one or more P2P clients are running, and I'm doing a massive file copy or actually playing a video.

Of the last 15 encodes, the most recent 5 were on MeguIVit 1.4. The earlier ones were with 1.3. So I just wanted to provide another data point that disk access may not be an issue.

Potentially relevant information regarding my rig:

Quad core Q9550 with 8GB RAm running W7 64b

As for hard disks, I have many disks. I minimize concurrent activity on a single disk so as not to disrupt MeguIVit. e.g.

Source ISO is on disk1, temp working directory is on disk 2, target output directory is on disk 3, P2P client 1's repository on disk 4, P2P client 2's repository on disk 5, file management functions (file move or copy) performed on any disk EXCEPT disks 1-3.

In this way, the bit traffic and disk controller loading is spread out.

HTH

 

[Vitreous]

I wanted to chime in here on 201flyer's problems. His symptoms sound remarkably similar to my issues from several months ago...

Thanks for that detailed info anandneemish. Confirms that Win7 64-bit in itself is not the problem and that disk access on other disks seems safe (I can confirm that). Spreading your disk load across several disks (not partitions) will provide a performance boost in any case.

However, I can reliably reproduce a crash with a HD encode running at the same time as a Share "convert to file" on the same disk, so maybe there is something with same disk access. Unfortunately multithreading bugs are notoriously random - their occurence depends on every last detail of your system and software.

Still, there's a clear difference between those who have near perfect reliability (I do aside from epic 1080p 60fps rips), and those who can barely make it through one SD rip. It's possible that the problems that some people are having are not to do with the encoding. For example, maybe there's a SATA driver bug on a popular mainboard under Win7 64-bit. Or a problem in one of the antivirus's realtime scanners. Etc. So keep everything up to date. If you're getting frequent errors, be suspicious of all resident software and try alternatives.

 

[anandneemish]

However, I can reliably reproduce a crash with a HD encode running at the same time as a Share "convert to file" on the same disk

My guess is that multiple data intensive processes operating off the same disk could be problematic. SATA drives under modern OSes try to be intelligent in how they queue data. This may cause timing problems for poorly written multi-threading software.

Even for processes that are well understood, you'll see hiccups. As an example, try moving two large files (e.g. >1GB movies) off the same disk to two other disks at the same time (not serially), and then try opening the disk with Explorer. It gets extremely laggy in an unpredictable way.

Anyways, hats off to you Vitreous. The encode process for me is now pretty fast and rock solid. (The only failure I've had in recent months was for for an 60fps encode. When I did successfully complete it, it did not appear that much better for me, so I quit doing it.)

 

[no__One]

To M.Vitreous.

So I did multiple tests on the latest version meguIVit_0.1.4: The Awesome Rip Machine by the only living GoD, I aptly named: His Excellency Vitreous. :exhausted:

Trier talk, here are my observations.
Regarding the scripts "CandyDolls" I haven't seen any problems.

One question: I wanted to try the script you propose for your encodes of ValensiyaS.

http://www.akiba-online.com/forum/showpost.php?p=857058&postcount=1

But I get this error (missing filter?)

Error message for your reference: FFT3DFilter: Can not load FFTW3.DLL

Scripts for ReRip, it becomes more difficult.
I failed to obtain 50/60fps Rip. How can I force these encodings? (I'm sure I checked and I had selected the right profile).
In addition, some encodings are very jerky when traveling cameras occurs but not present in original video. uzzled:

Last question, Is there any difference / advantage between MKV / MP4?

I thank you for taking the time to read and look forward to your answers will enlighten me to move forward. :study:

:bow-pray:

 

[Vitreous]

Can not load FFTW3.DLL

This is a nearly hidden requirement of FFT3DFilter:
- Download this zip file
- Get the FFTW3.DLL file from the zip and put it in your windows path:
ㅤㅤ- On XP or 32-bit Vista/Win7, put it in "Windows\System32"
ㅤㅤ- On 64-bit Vista/Win7, put it in "Windows\SysWOW64"
____

Scripts for ReRip, it becomes more difficult. I failed to obtain 50/60fps Rip

If you have a progressive source (non-interlaced), then you can only rip at the same framerate. If you have interlaced source then you shouldn't be using the rerip batch file. Look for the distinctive combing in the source to see if it's interlaced or not.
Edit: You can make double framerate using motion interpolation I suppose. Use the InterpolateFPSx2 script in this post. Add it to your QTGMC line:

QuickTGMC( Preset="Slow", InputType=1 ).InterpolateFPSx2()

____

In addition, some encodings are very jerky when traveling cameras occurs but not present in original video.

This is probably because the field parity is wrong on interlaced video. Again I note you shouldn't use the rerip batch files for interlaced material, you should use the others. You then need to edit the script before encoding. Add "ComplementParity" to the "DirectShowSource" line, for example:

DirectShowSource( file, convertfps=true ).ComplementParity()

 

[no__One]

Thanks a lot for all your time and your fast answers.

The source videos are definitely interlaced (= probably DVD ripped to DivX 5 708x576 25.00fps).
I think the orignal ripper hasn't or doesn't know how to de-interlace source. I see here and there the symptomatic artefacts as in your example...

I will try all your advice and report.

I do not know if you have already responded to this agonizing question, but that really bothers me.
Is there a particular reason to adopt MKV rather than MP4?? uzzled:

 

[Vitreous]

Is there a particular reason to adopt MKV rather than MP4

It makes no difference at all to the rip quality or performance. The differences are fairly minor and depend on what you intend to use the rip for:

MKV:
- Better tools (mkvmerge / mkvtoolnix are way more convenient for the regular encoder than anything I've found for mp4)
- More flexible (Can contain any kind of audio, video, subs, whatever - my recent blu-ray rips had the cover scans and an info file embedded in the rip. Underused feature though - I'll bet no-one noticed!)
- More robust (MP4 needs slight hacks for embedding some kind of content like chapters, get occasional problem MP4s - linked to poor tools I guess)

MP4:
- Wider device compatibility (if this is important to you, then MP4 is the way to go [for now...])

 

[Authorised-BMN]

mkvmerge

MKV:
- Better tools (mkvmerge / mkvtoolnix are way more convenient for the regular encoder than anything I've found for mp4)

Excellent answer as usual, which provides a segue for an issue that has been bugging me. I was able to successfully force keyframes at chapter marks in meguIV 1.0.0.0, but it doesn't work in the latest meguIV build (which means it doesn't work in meguIVit, of course). So, for example, if 1) the meguIV encoding has commercials at the beginning, 2) I try to cut them, 3) the cut is off-center by default (that is, one cut picks up a second or so of the main video, the adjacent cut picks up a second or so of the commercial), then I used to be able to enter the appropriate timecode in the chapter file, check the keyframe box, re-encode and, later, cut at exactly that spot. Not so any more, despite repeated attempts. Just to be clear, it does in fact insert the chapter mark, just not the keyframe -- or at least not a keyframe I can make a cut at with mkvmerge. I could upload some short music videos as examples, but I was hoping just by stating the issue that someone might go "ah ha!" and provide a solution.
erfectplan:

 

[Rollyco]

Forcing chapter marks at keyframes has never been accurate for me, which is why in meguIV it's disabled.

 

[Authorised-BMN]

Comes in handy sometimes

Forcing chapter marks at keyframes has never been accurate for me, which is why in meguIV it's disabled.

Thanks, Rollyco. I have found the ability to force the keyframes at chapter marks (not chapter marks at keyframes) handy from time to time. For example, see http://www.akiba-online.com/forum/showpost.php?p=810047&postcount=1784. The idea is to edit the chapter text file to create your own keyframes for splitting (as you know, since you know way more about this than I do). Is there another way to get a clean or precise split?

 

[Rollyco]

It's possible that the code in MeGUI for calculating timecodes is broken. I fixed one part that was responsible for chapter file creation, but I never looked at the chapter=keyframe functionality.

You could check the MeGUI log and see what x264.exe commandline is being used when the keyframe option is checked. It probably used the --qpfile option... and if so, you can check the frame numbers that MeGUI generates and see if they're off.

 

[anandneemish]

MKV:
- Better tools (mkvmerge / mkvtoolnix are way more convenient for the regular encoder than anything I've found for mp4)

BTW, I am using mkvmergeGUI to split mp4 files. Seems to work fine. The output is always MKV. No biggie though.

 

[Vitreous]

Prompted by the recent instabilities that people have been experiencing I've made an interim new point release for MeguIVit. I'd appreciate if some people could test this version for me because it's a major update internally.

Download MeguIVit 0.2.0 (beta)

In summary, the changes are:

• Replaced mencoder with ffmpeg for better stability (fewer crashes) but a small speed loss (reports please)
• Updated to latest tools and plugins for all round improvements in quality, speed and robustness
• My own rewrite of the MeGUI front end with some initial minor tweaks

Installation:
- Delete the Sandbox folder that is in the same location as your MeguIV
- Extract the attached zip in its place
- Run MeguIV as normal with the usual MeguIVit instructions

In detail:

• It uses the latest versions of all tools and plugins, including x264@r1745
• QuickTGMC has been updated to version 2.50. The main change is it now uses NNEDI3 by default for higher quality without a speed penalty. I'll update the QuickTGMC page later.
• I've replaced mencoder with ffmpeg for 1st pass encoding. This gives me excellent stability. I wasn't able to rip 1080p@60fps multithreaded using mencoder without near instant crashes. This revision can do it without problem. It's also more robust on the slower, complex SD scripts. Crashes will occur if you run out of memory though (many threads on complex/hi-res scripts). This might help those experiencing excessive crashes, although there may still be issues on some Win7 64-bit machines (feedback please). However, the downside is that encoding with ffmpeg is a little slower.
• I've used my own mod of MeGUI (based on version 3.5.21), which replaces Rollyco's tweaked version (sorry ). It has some extra minor changes:
  • The NTSC chapter points fix that was in Rollyco's original is still there. I've implemented the fix in full so the corrected chapter marks should permeate to all parts of the program, which might (or might not) help with the "Force Keyframes to Chapter Marks" problem
  • The other fix Rollyco made was the use of the "pure" aspect ratios like 4:3 or 16:9 rather than the slightly-off ITU variants. Again this update gives the same behaviour as before but now there is a setting in the Options dialog in case you want to use the ITU ratios anyway
  • Getting internet updates has been completely disabled at it's likely to break things (because this is an unofficial modified version)
  • The Chapter creator has a new "Menu Chapter" setting, which allows the injection of an extra menu chapter prior to the chapters actually found on the DVD/BD. Set the menu chapter time before loading the IFO in the Chapter Editor. This is only useful for those working manually (i.e. not one-click) and using this version as a replacement for MeGUI. That's a workable idea, all you have to do is copy all your plugins to

    Sandbox\meguIV\1.0.1.1\Virtual\MODIFIED\@SYSDRIVE@\meguIV\Avisynth 2.5\plugins

    and you get the advantage of the points above.
  • There might be other changes I've forgotten. I am developing some much more significant GUI changes, these are just the ones that are fully working at the moment
• I've tweaked the HD settings based on my recent HD ripping experience. Again relevant to manual encoders only, but I am working on a one-click BD version...

I've replaced pretty much everything in MeguIV now, but I'm still providing it as a Sandbox update just to take advantage of the convenient packaging system that MeguIV uses.

 

[Rollyco]

Cool beans. You should try and get your fixes into upstream, if you think they're clean enough.

 

[Vitreous]

Cool beans. You should try and get your fixes into upstream, if you think they're clean enough.

I don't really know what that means. Are you suggesting I commit the changes into the MeGUI trunk? I'd rather not become an active MeGUI developer.

 

[Rollyco]

You don't have to become a dev, take a look here:
http://sourceforge.net/tracker/?group_id=156112&atid=798478

What you do is create a sourceforge.net account, post a description of the problem, and a patch (against the SVN trunk) that fixes said problem. If the core developers agree with it they'll include it into trunk. Which means it will make it into the next stable release. I can help you with SVN or creating patches or whatever, if you need it.

 

[Vitreous]

Thanks for the suggestions but I'm really only interested in modding MeGUI unofficially for my own purposes. Some of my changes would be of no use to MeGUI generally and I definitely don't want to start maintaining multiple versions. In any case the MeGUI code is a mess because of too many piecemeal patches. It needs a clean-up not more patches. And I already have enough projects so I'd rather not get involved in this one.