[How to] Test Encoded music files

[seadzz]

Question: I encoded several tracks in a few bitrates, how do I test to learn which sounds best?

Answer:

Doing a blind test between two encoded files or a wav file against a encode is not as easy as it sounds. Of course you can always encode two files and play them back to determine which sounds best. Unfortunately if you do that using your normal music player you can always tell which file is which by looking at the file name. The placebo effect comes into play if you know by looking which is the wav and which is the encode somewhat coloring the results of any test.

This placebo effect can only be ruled out by doing double blind testing and to do this right you need some specialized software. This thread will give you step-by-step instructions on how to configure a blind test referred to as ABX Testing and what you can expect to learn as a result of this testing.

There are several ABX Testing softwares on the internet and if you have one you like than by all means use that one. If not and you want a freeware version I would suggest you download a music player which contains an ABX Tester. One that comes to mind is Foobar2000 . It is a minimalist player that has plugins to enable ABX testing. I would point you to the Foobar2000 site and you can either download their “special” version that includes an ABX tester or download the normal version and then get the ABX Plugin . The plugin needs to be installed into the “components” folder using cut & paste or drop and drag after downloading.

After downloading and installing Foobar you will need to start the program. The main screen looks rather frugal but do not let that fool you. In fact you have a very powerful music player now that in addition to supporting every codec known to man has a vast collection of plugins available to do just about anything. Visit HA or the Foobar2000 and take a look at what is available.

In any event start the program and then load TWO tracks that you want to test. I would suggest you start with one WAV file and one encoded track. Drop and drag these two tracks into the playlist. Next highlight the two tacks by clicking on one (it will turn blue) and while holding down the “shift” key use the up\down arrow to highlight the other. Once both tracks are highlighted RIGHT click on the files which will open a dialog box. Click on the last listing in this box which is ABX two tracks. If all goes well you will see a “preparing files” green bar graph which should take a minute or so to make the files ready for testing. Next you will see the ABX Tool box open with this
Screenshot .

With the ABX screen shown what you will want to do is determine if track A is = to track X or track Y. In this test it is best to use the set start and set end to a ten second more or less piece of a given portion of the test track. The ABX tester will label the wav file and the encode either A or B at random and also randomly set up the Y or X. Your goal is to play back the file and try to determine if (X is A, Y is B) or (Y is A, X is B). This sounds easy but when you are dealing with high bit rate encodes it is more difficult than you would expect.

Results of the test are shown in the “Score” box which is some thing like this:

1/1 is you did pick the right files, 0/1 is a miss. To make anything meaningful out of this test it is best to do it several times until you get something like 5/7 which shows 5 times out of 7 attempts you successfully selected the wav from the encode with a very low probability you were guessing. 3/7 results would indicate you are guessing with 2/7 shows you can not tell the encode from the original file. This is where you want to be. You want to be as close to impossible as to determine which is the original source and which is the copy.

I spent several hours using this tool the other day and to my surprise my hearing was nowhere as good as I thought it was. I concluded that all modern codecs are very good at producing a passable facsimile of the original wav. It is only with very close scrutiny, with a great deal of effort on my part that I could distinguish 128 mp3LAME encodes from the original source. When using DT’s Command Line String and Preset Standard it is almost impossible to tell which is the encode and which is the wav. In the end I learned that I could with a very degree of confidence tell the encode from the wav but as soon as we passed 128 kbps telling them apart became more and more difficult. By the time we got to 170 and higher it turned into a quess more often than not.

Good luck to you, if you take the time and use this tool you should be much happier in your codec selection.

sdz