TRICK: Achieve Really Fast DB Startup (62.5% improvement!)

[Mr.Anderson]

I usually have a very good startup time with my H140 (that's my impression when I compare it to the startup-time survey conducted here at MisticRiver), because I often do the "folders first trick" and sort my files/folders in alphabetical order and have my collection of complete albums perfectly tagged. I usually have a startup time of:

previous startup time without DB: 9.7 sec.
previous startup time with DB: 1:28 min.

But today I achieved an improvement of 62.5%!

my new startup times are:

new startup time without DB: 10.0 sec. (because I loaded in more files since last test)
new startup time with DB: 33 sec.!

The size of my collection is:
4194 songs (mainly 192kpbs MP3 + some Ogg Q7 albums)
0575 folders
24 GB Music / 14 GB data (the data is in 3x4GB+1.5GB PGP-disk to avoid folders and fragmentation between data and music files. I have positioned the PGPs at the back of the disk)

It is already well-known that the number of folders have effect on the time spend on HDD-reading at startup, because all folders have to be scanned. Collecting the folders in the front of the HDD will improve this compared to the case where the folders are scattered all over the HDD. This trick will also effect the startup time without DB enabled. It is probably also a good idea to avoid fregmenting the "iRivNavi.iDB"-file because this file is read in at startup when DB-Scan is enabled.

So what was the trick? :P ... you ask

Well... this is completely new to me and I have not had time to investigate the exact reasons, but I can tell you that I did three things differently yesterday when I did the "folders first trick" with "xcopy" (the command prompt program).
My theory is that the structuring and sorting of the TAGs in lists are performed on startup when the DB is enabled. I especially have the "TITLE"-list under suspicion for being quite time-consuming to sort/structure.

The first thing I did different was that I had to convert all my TAG title from "##-title"->"title" without converting to filename first. So I used the MP3BookHelper suggested by "alienrider" (EDIT2: I have later on come to the conclusion that it should avoided to have the ##-title syntax in the TAG title field). While doing this conversion I selected that the new TAGs should be ID3v2.3 and I converted & removed some of my ID3v1 -> ID3v2.3.

Afterwards I used "xcopy" to make the "folders first trick" by first coying the folder structure by "xcopy . E: /T" (standing in my H140-backup dir - where E: is my H140-drev) and afterwards copied the musicfiles by "xcopy * E: /S". (credits go to ep0ch for the xcopy trick) (This should however not be the cause of the improved DB startup time because the resulting harddisk-layout was approximately the same as I achieved with another "folders first trick".) I verified with OO defrag that the file/folder layout in the clusters was acceptable.

Then I used Ewan's new TDT (3.0) to create my DB file. I don't know if he did any improvements in the construction of the DB file, but it is not mentioned in the TDT history so I don't think that is the reason neither.

So my guess is that the consistency of the TAG structure is of great importance for the startup time (I don't hope that's old news ). But maybe you guys and girls can find an even better explaination in the relationship between the TAG structures and DB startup time....


******************* NOTICE ******************
I know that this trick is not very specific yet, but I
hope that we through this thread, with your help,
can specify the trick to more concrete advices on
how to achieve better DB startup time. If we are
succesful in achieving that, the trick can be added
to the FAQ & "Tips & Trick"-thread.
******************** ******************** *****

You can use this http://www.cashcowmag.com/percentage_calc.html calculator to calculate your improvement..

If you receive positive improvements by using the DB advice and the "folder first trick" then post what things things you did and your collection info in terms of number of songs, folders and size. If possible, try to the describe the state of the collection in terms of structuring difference (i.e. I organized my collection from 48 genres into 28 genres and...etc.). Remember to time your DB startup-time before you start to optimize your collection and save a backup of the old "iRivNavi.iDB" file. Time from you press play until the player is ready and showing xxxx Songs, xxxx Folders.

[Fluidity]

I still don't understand what you mean by this:

The first thing I did different was that I had to convert all my TAG title from "##-title"->"title"

Simple ID3 Tags as I understand them have a track number field, artist field, Song name field, album name field, year field and depending on the verison of the ID3 Tag have other fields as well. I don't understand how how or why you appear to have the track number in the ID3 tags connected to the name, none of mine have this, its just in the track number field.

I once read somewhere that one of the ID3 tags is put at the start of the file and another of the types is put at the end of the file. I would have thought that only using the type of ID3 tag that is positioned at the start of the file would make DB startup a lot quicker as otherwise it would have to find the end of the file before it could read the tag. If all the info was at the start of the file it might deal with each file quicker. However I thought it was the v2 Tags that were written at the end of the file so maybe the opposite is true and its easier for it to search the end of the file. I generally have both v1 and v2.3 tags on all my mp3's so could easily wipe one type, it'd just be interesting to know if there is any rational between either type of tage being read quicker or perhaps it just being easier if all your tags are of the same type.

I also seem to have missed this folders first trick. From what you're saying, it appears that you need to copy all of your directories to an empty player and then after you've got those written to the start of the drive, you copy the information after them filling them but actually all the data they contain is written to the drive after the folder structure. Interesting and I might try it sometime soon but for now the copy of my music on my player is more uptodate and Tagged better than that on my PC so before I try this I'll have to delete the music on my PC to free up HD space, copy all my music from my player to my PC and then I can think about playing with my player setup. Still interesting nonetheless but I'd like to know why a bit better so I can try and optimize my setup before trying it.

Fluidity.

[Haku]

Ok I tried the "folders first trick" but only increased the bootup time from 26s to 20s, how the heck do you get yours to boot up in 9.7s? at 7s mine is spinning up the harddrive, I don't understand how you get such a low time unless you time the bootup differently from me.

I press the stopwatch start at the same time I press the Play button, then press the stopwatch again when the xxxx folder(s) xxxx song(s) is displayed.

So I just spent 77 minutes copying everything back onto my H140 to gain 6s bootup time?

[Mr.Anderson]

I don't understand how or why you appear to have the track number in the ID3 tags connected to the name, none of mine have this, its just in the track number field.

You asked this question yesterday in the "ID3-TitleTag(##-title)->ID3-TitleTag(title)"-thread http://www.misticriver.net/index.php...pic&t=3424. If it is not clear enough, then please ask again there instead.

I once read somewhere that one of the ID3 tags is put at the start of the file and another of the types is put at the end of the file.

Yes, the ID3v1 tags are positioned in the start of the mp3 and have fixed size. ID3v2 tags have variable size and are positioned in the end of the file. I was actually wondering if ID3v1 would speed up the startup even more, but I like the features of the ID3v2 so I have not tried it. If anyone have experience with a pure ID3v1 TAG collection, then please post your results.

I also seem to have missed this folders first trick. From what you're saying, it appears that you need to copy all of your directories to an empty player and then after you've got those written to the start of the drive, you copy the information after them filling them but actually all the data they contain is written to the drive after the folder structure.

Yes, that's about it. The folders first gets created and allocated at the first sector grouped. Then the music files (these are placed after the folders) + folders (which are already on the harddisk, so they will have no effect on the position on the harddisk). It is easy to see the harddisk layout with OO defrag. Unfortunately OO don't move folders eventhough they are no positioned at the startup of the harddisk.

The question is have much a well-structured TAG means for the DB startup-time and if similar TAG type or a special type (ID3v1 or ID3v2 or ID3v2.3) is to be prefered.

[Mr.Anderson]

Originally Posted by Haku

Ok I tried the "folders first trick" but only increased the bootup time from 26s to 20s, how the heck do you get yours to boot up in 9.7s? at 7s mine is spinning up the harddrive, I don't understand how you get such a low time unless you time the bootup differently from me.

I start the stopwatch when I press the Play button, then stop it again when the xxxx folder(s) xxxx song(s) is displayed.

I'm not the first to achieve around 10 sec. with the DB disabled. But please be more precise when you give information about startup timings. Information such your collection size, number of folders, type of music files etc. matters so it's hard to compare if this information is not present along with timing results.

So I just spent 77 minutes copying everything back onto my H140 to gain 6s bootup time?

I can not promise you that... but it sure may be worth a try if you did not try the "xcopy folder first trick" before - I'm quite sure that it will help.

[Haku]

I did the xcopy trick of copying the empty folders and then the files, there's 418 folders & 6004 songs, I don't use the DB stuff.

I don't understand how you can get less than 10 seconds bootup time when the first 5-6 seconds the drive hasn't even spun up yet?

[Fluidity]

Ok, so for some reason, I missed that thread today, it makes sense now. However, for the majority of people that shouldn't matter as I doubt many people have done that.

Basically the "trick" for the majority of people now runs as follows.

1) Remove all ID3v1 Tags converting them all to ID3V2.3.
2) Ensure Player is blanked (I don't know if deleting all files is sufficient or if it might also need to be formatted).
3) Copy directory structure from PC to player.
4) Copy files from PC to newly created folders (Avoiding fragmentation)
5) Use TDT to create the DB.

That sounds straight forward enough, but how well does it work? Does it offer any significant advantages over the defrag methods used by people in the past?

Fluidity.

[Mr.Anderson]

Originally Posted by Haku

I don't understand how you can get less than 10 seconds bootup time when the first 5-6 seconds the drive hasn't even spun up yet?

My drive spins after 6 sec. Read data in 4 sec. and shows the "Total Screen" at total 10 sec. It just reads the data f***ing fast

With the DB... it's 6 sec spin start, 10 sec. data has been read, sortings/structuring DB finished at 33 sec.

[Mr.Anderson]

Originally Posted by Fluidity

Ok, so for some reason, I missed that thread today, it makes sense now. However, for the majority of people that shouldn't matter as I doubt many people have done that..

Of course this was not the trick... it was just to explain why I was messing with all my TAGs. Sorry if that confused...

Originally Posted by Fluidity

Basically the "trick" for the majority of people now runs as follows.
1) Remove all ID3v1 Tags converting them all to ID3V2.3.
2) Ensure Player is blanked (I don't know if deleting all files is sufficient or if it might also need to be formatted).
3) Copy directory structure from PC to player.
4) Copy files from PC to newly created folders (Avoiding fragmentation)
5) Use TDT to create the DB.

Well, I don't think we can be that conclusive yet... we need more experience from other people who have tried homogenizing or removing special types of TAGs. We don't know precisely yet what cases the DB startup delay. But proposals like this is the way to go - we just need to refine it along the way.

Does it offer any significant advantages over the defrag methods used by people in the past?

Defrag is no good... it takes 3-4 hours and the results are not as good because it is hard to control the layout of the disk - many defrag programs can not do alphabetical defragmentation neither. I copied my 24GB in 1 hour 3 min. yesterday, spent 1 min on the xcopy folder structure copy and copied my music files back again in about an hour too.

[XavierGr]

Well we shall wait for more test to make a safe example.... though I will never use the DB!

[Mr.Anderson]

Originally Posted by XavierGr

Well we shall wait for more test to make a safe example.... though I will never use the DB!

Right now I just wonder, how much effect it had that I changed my title TAG from "##-title" -> "title". I think this fact may influence the TITLE sorting significantly because it's a list of 4100 elements... but will it really effect the sorting algorithm that much

I'm beginning to think that it does not have anything to do with the type homogenizing of my collection, because the DB information is saved in the iRivNavi.iDB and therefore the TAGs are not read at startup.

What I just find very strange, is that other users have much longer DB startup time while I now achieve 33 sec. when comparing similar collections (around 4100 songs)... I at least have not seen anyone who have faster DB startup-time.

It seems like, the trick is to have a "perfect" tagged collection without the "##-title" syntax in the TAG title field, because this results in extra sorting. So I sorry guys and girls... this is not just an easy one minut trick - you will have to do the hard work and start working with that semiauto-tagger

But do post your results here, if you discover any other tips/trick (and things to be avoided).

What I really don't understand is that this sorting is necessary every time the H1x0 starts up! Why can't the sorted lists just be saved on the harddrive after they have been sorted and just loaded at startup. The sorting would only be necessary if there had been made changes to the collection (after PC disconnection). I think that solution would be a lot faster - another job for iRiver

[Mr.Anderson]

Optimize Hxx0's Startup-Time: (Win98/Win2K/XP)

* Make a full backup of your H120/H140 on your PC.
* Delete all files... also hidden (Recycle Bin can be removed if you use the "TweakUI" tool - link: http://www.annoyances.org/exec/show/tweakui).
* Open a commando prompt START->Run->write "cmd" and press enter
* Go to your H1x0 backup drive (write "help cd" if you are lost here ) - Check out this tip BTW link: http://www.tech-recipes.com/windows_tips232.html
* Now copy your "folder structure" from your "H1x0 backup" to the your "H1x0-drive" (i.e. E: ) by

C:\H140-BACKUP> xcopy . E: /T

* Verify with a defragmentation program (i.e. "OO defrag") that your folders are positioned at the start of the harddrive.
* If not then try to delete the folders (maybe copy some random big files and delete them again) and do the xcopy folder structure copy again (FAT32 file positioning is strange)
* Copy all your files and music back to your H1x0 by

C:\H140-BACKUP> xcopy * E: /S

or use Total Commander (Windows Explorer does not copy in correct alphabetical order)

You now have a much faster startup time of your H1x0 when DB-scan is disabled. With the DB enabled the harddisk reading time is insignificant compared to sorting/structuring-time of the DB

Tips & advice on things to be avoided in relation to the harddisk:

* Try not to create many folders after this operation... it will increase startup time because the folders will be scattered.

* Use i.e. virtual harddisks (such as PGP-disk or similar) to avoid mixed music and data. This will decrease the likelihood of fragmentation and you can make all the folders you want on these virtual harddisks without effecting the structure on the Hxx0 FAT32 drive. Seek to position the virtual harddisk in the end of the harddisk if possible. I don't backup and delete these virtual harddisks when I do the "xcopy-trick".


Advice to improve DB Startup-Time:

The above trick only effect the harddisk read in time. I achieved an improvement from 1:28 min. to 0:33 min. with the DB enabled after I converted my TITLE TAG syntax from "##-title" to "title" on the entire collection of 4100 songs. I am not sure why I achieved this fast DB startup but I have a few advices which may help:

* Keep the TAGs in your collection as clean as possible. (1 artist = name, 1 album = name etc.)

* Limit the number of genres (I have 30 genres). The best is to have all the songs of one artist belonging to one genre.

* Avoid having the "##-title" syntax in the TITLE TAG (this is not the filename - the filename should be ##-title.mp3), because this syntax means extra sorting of especially the TITLE-list.

* Keep your iRivNavi.iDB defragmented (this will however not effect the sorting time when the harddisk has finished reading)


I don't know if it matters, but I have my collection in the following file/folder structure:
ROOT/ALBUMS/A-B-C-D/ARTIST/YEAR-ALBUM/##-TITLE.MP3

DISCLAIMER:
The relative effect of this trick and the optimization advice will be dependent on your current structuring of your harddisk drive and the tagging structure of your music files. There is no guarantee that you will achieve similar improvement of your startup times because our music collections naturally is very different.

[danielo]

I don't know how much time it took before, but it looks faster now.
Now with DB scan on, it boots in 34 sec.

The only thing I did is remove all ID1 v1 data, I only left the ID2 v2.3 data.

I have 3690 songs in about 350 folders... that makes 15.8 Go

[turdburglar]

I increased my DB boot time from 26 seconds to 16 just by getting rid of the ID3v1 tags.. I had both ID3v1 and ID3v2 and now it boots faster.

I'm thinking of trying the rest just to see how fast I can get it.

I have 2863 files in 270 folders - 11.4G

[Mike]

What's the best/fastest way to delete ID3v1 tags? I use The Godfather, here. And any reason not to do so? Thanks!

[Mr.Anderson]

Originally Posted by danielo

I don't know how much time it took before, but it looks faster now. Now with DB scan on, it boots in 34 sec.

Originally Posted by turdburlar

I increased (you mean decreased, right ) my DB boot time from 26 seconds to 16 just by getting rid of the ID3v1 tags.. I had both ID3v1 and ID3v2 and now it boots faster.

That is GREAT news guys!

Originally Posted by danielo

The only thing I did is remove all ID1 v1 data, I only left the ID2 v2.3 data.

Yes, I have been flip-flopping between if this has any effect at all on the DB startup. I came to the conclusion that the "##-title" syntax in the TAG title field was the main reason, because I can imagine that the sorting algorithm first sorted all the first characters '#', then the next '#' and last the '-'.... first here after the actually sorting begins in relation to the title. That made sense to me.

I removed around 500 ID3v1 TAG myself. But I am actually a little surpriced if it has an effect to remove ID3v1 TAG, because it is quite certain that the "iRivNavi.iDB" already contains the TAGs and thus they are not read at startup. I tried to load with my old "iRivNavi.iDB" but with new TAGs and then I again got the old 1:28 startup time with the DB.

Maybe Ewan knows something about the structure of the "iRivNavi.iDB" file since he is constructing it in TDT - he could maybe give some answers concerning the ID3v1 vs. pure ID3v2.3 TAG setup.

But let us see some more results (and remember to time your startup time before you try to optimize your collection )... this is becoming interesting


PS.

Originally Posted by Mike

What's the best/fastest way to delete ID3v1 tags? I use The Godfather, here. And any reason not to do so? Thanks!

I think that the tagger you use is irrelevant.... just use the tagger you know if it can do it easily. Remember that you should not just remove the ID3v1 - copy them first to ID3v2 if they are blank, then remove the ID3v1 afterwards. It is certain though that MP3BookHelper does the job very well... the problem it just that it is quite complicated. But post the name of tagger you used... maybe we can find a pattern, if a such exists.

[danielo]

The godfather is the easiest thing on earth to remove ID3v1 Tags!

Go in the option and in the "TAG" page, choose to save only in ID3v2.3 and to remove ID3v1...

Then you just have to select your H120 hard drive and Click on Scan, It will scan all your files, takes a minute, then just click on "UPDATE FILES" or something like that, and it will remove ALL ID3v1 data by itself, takes about 2 minutes...


Besides, I can't tell you about "xx - title" tags effects on speed, because I didn't use that, I had only "title".