I'll start with making a raw copy since you need to do that anyway to rip it.
You first need a software to remove the encryption and there are none that can do that for free and allow to rip it(the free ones only allow you to decrypt it so you can play it directly from the disc).
The best one by far is AnyDVDHD, but it's really not cheap(119 euro for lifetime updates. And you need the updates since the protection keep changing). The trial version is fully functional(but it resets your setting when you close it) and there are nice trial reseter software out there.
There's also DVDFab Blu-ray Copy who is even pricier(179USD$. I was under the impression it was cheaper somehow) and does a decent job, but not as good as anydvdhd from what I hear and it's slower.
Both of these will remove the encryption on the fly as soon as you insert a bluray, then you just need to rip it to the disc(both have an option to do just that).
Once you have it on your hard disc, you'll need some kind of encoding software to make a 6-8GB copy. I'm not exactly sure which one out there can take a bluray as an input since I do things manually with avisynth so I'll quickly explain using that.
You need to install avisynth(abviously) and add some filters to it that can decode bluray.
The best is dgdecnv but it only works if you have an nvidia graphic card(support for others should come in a near future) and is not free(20$ if I'm not mistaken). You run the software and open all the files that contain the movie in it and it will index then to make things go faster and also extract the audio which you will need later.
There's also ffms2 which does a pretty good job and is completely free.
Create a .avs script to load your bluray files, with dgdecnv it would simply look like this:
Code:
LoadPlugin("Path-to-DGDecodeNV.dll")
DGSource("Path-to-Dgdecnv-index-file-you-created-earlier")
Then you use x264 to encode that script into a smaller video file. It's a command line application, but all you need to do to run it is to open a command prompt, go to where x264.exe is and type something like this:
Code:
x264 --preset veryslow --crf 18 --output "output-file-with full-path" "input-.avs-script-with full-path"
The preset veryslow will give you the best quality(but it is very slow as the name says) and the CRF option is the actual quality number. 18, in my opinion, will give you quality almost as good as the original and the size should be close to what you want. If not, you can change it to up to(higher means lower quality) about 21 with good result.
Multithreading a 1080p bluray encode like that can be tricky since avisynth is only a 32bit software and will run out of memory pretty quick. People are working on an updated version of it, but it's not here yet so if you have CPU to spare and want things to go faster, you'll need to split the encode in multiple parts and join them together in the end. Without any filtering, it should be fast enough to use all of your CPU though, but if we're talking idol of JAV bluray, then you'll need to deinterlace with QTGMC and that will take forever(which is counted in days, not hours).
Once the encode is finished, You'll want to install mkvtoolnix, run mkvmerge and add your new video track and the previously extracted audio track and join them together.
You can now enjoy the final result.
It is a pretty complex operation if you want quality result so doing an exact step by step would take forever, but you should be able to fill in the blanks with a couple of google search and if you're stuck somewhere, I'll gladly help.