|
|||||||
|
|||||||
| Home | Forums | Register | FAQ | Search | Today's Posts | Mark Forums Read |
|
Welcome to the misticriver forums. You are currently viewing our boards as a guest which gives you limited access to view most discussions and access our other features. By joining our free community you will have access to post topics, communicate privately with other members (PM), respond to polls, upload content and access many other special features. Registration is fast, simple and absolutely free so please, join our community today! If you have any problems with the registration process or your account login, please contact contact us. |
|
|
H10 Database SpecificationFrom iriver Wiki, your source for everything iriver - a part of MisticRiver.
[edit] Introductioniriver H10 media database, which is necessary for playback and navigation in MUSIC mode, consists of a number of files: H10DB.hdr, H10DB.dat, and H10DB_*.idx (typically H10DB_@DEV.idx, H10DB_FPTH.idx, H10DB_FNAM.idx, H10DB_FRMT.idx, H10DB_TIT2.idx, H10DB_TPE1.idx, H10DB_TALB.idx, H10DB_TCON.idx, H10DB_@DU3.idx, H10DB_@DU4.idx, H10DB_@DU5.idx, H10DB_TRCK.idx, H10DB_@DU1.idx, and H10DB_@DU2.idx). These files are generated by EasyH10, iriver Plus, or Windows Media Player 10 and stored in \System\DATA folder in H10 player's hard disc drive. Even H10 player updates media database during the playback to store dynamic information such as recent playback time, user ratings, and playback counts. H10DB.dat is a list of records that store music information. Each record has 24 (= 2 + 22) fields such as track title, artist, album, genre, year, genre, rating, recent play time, play count, etc. The MUSIC mode of iriver H10 player does not read music information from actual files but relies only on these records for displaying music information. When a user listens to the music with a player, the number of records increases because, whenever a music file is about to be played, H10 player duplicates the record for the file, adds the duplicated new record into the list, and inactivate the original record. Therefore, H10DB.dat may contain a number of inactive records. H10DB.hdr stores general information among H10 media database such as: the number of records in H10DB.dat; the number of inactive records; path to H10DB.dat; offsets to all records and fields in H10DB.dat (for quick access); and field descriptor that defines 22 fields in H10DB.dat record. The structure of H10DB.hdr slightly differs depending on the firmware type (UMS or MTP) and HDD capacity (5/6GB or 20GB). H10DB_*.idx (typically 14 files) store sorted indexes that point to H10DB.dat records. Each file has a different sorting order. For example, H10DB_FPTH.idx sorts records in alphabetical order of file paths; H10DB_TIT2.idx in alphabetical order of track titles; H10DB_TPE1.idx in alphabetical order of track artists; H10DB_@DU3.idx in numerical order of user ratings; H10DB_TRCK.idx in numerical order of track number, etc. These sorted indexes are used for displaying music tracks in different orders: e.g., alphabetical order of track title/artist/album; and numerical order of track number in an album. Since the H10 firmware seems to implement an efficient search method such as binary search, all indexes must be arranged in proper orders as the firmware assumes. If not, a player may lose sight of some music files, show duplicated entries, or show wrong information because of the inconsistency. This document describes the structure of H10 media database. Since the information is obtained from reverse engineering and analysis, it may not be correct. We have already developed a tool (EasyH10 CUI version with -D option) to dump the content of a database in a text. If you are interested, we suggest you to use it to view an actual database at hand. [edit] TypedefsThis document defines four types for explanation. Values in uint16_t, uint32_t, and ucs2_char_t are stored in little endian byte-order.
[edit] ConstantsThe document uses the following constants. #define H10DB_MAX_PATH 256 #if H10_MODEL == UMS /* H10 UMS models. */ #define H10DB_HDR_PADDING 1032 #elif H10_MODEL == MTP || H10_MODEL == MTP2 /* H10 MTP models. */ #define H10DB_HDR_PADDING 0 #endif #if H10_CAPACITY == 5000 || H10_CAPACITY == 6000 /* H10 5GB and 6GB models. */ #define H10DB_MAX_DAT_ENTRIES 4000 #elif H10_CAPACITY == 20000 /* H10 20GB models. */ #define H10DB_MAX_DAT_ENTRIES 8000 #endif [edit] H10DB.hdrH10DB.hdr stores general information among H10 media database such as: the number of records in H10DB.dat; the number of inactive records; path to H10DB.dat; offsets to all records and fields in H10DB.dat (for quick access); and field descriptor that defines 22 fields in H10DB.dat record. The structure of H10DB.hdr slightly differs depending on the firmware type (UMS or MTP) and HDD capacity (5/6GB or 20GB). [edit] Definitiontypedef struct {
uint32_t id;
uint32_t field_type;
uint32_t max_length;
uint32_t unknown5;
uint32_t unknown6;
uint32_t has_index;
uint32_t unknown7;
uint32_t unknown8;
ucs2_char_t idx_pathname[H10DB_MAX_PATH];
} h10db_fd_t;
typedef struct {
uint32_t unknown1;
uint32_t unknown2;
ucs2_char_t pathname_dat[H10DB_MAX_PATH];
uint32_t unknown3;
ucs2_char_t pathname_hdr[H10DB_MAX_PATH];
uint32_t unknown4;
uint32_t num_dat_records;
uint32_t num_dat_inactive_records;
uint32_t num_dat_fields;
h10db_fd_t fd[num_dat_fields];
uint32_t max_dat_field_offsets[num_dat_fields];
uint32_t dat_size;
#if H10_MODEL == MTP2
uint32_t unknown5;
#endif
uint8_t padding[H10DB_HDR_PADDING];
uint16_t dat_field_offset[num_dat_fields][H10DB_MAX_DAT_ENTRIES];
uint32_t dat_record_offset[H10DB_MAX_DAT_ENTRIES+1];
} h10db_hdr_t;
h10db_hdr_t hdr;
[edit] Semantics[edit] h10db_hdr_t
[edit] h10db_fd_t
The following code shows the actual field descriptor used in the international players. h10db_fd_t fd[] = {
{0x0000F001, 2, 4, 0, 0, 1, 0, 0, L"System\DATA\H10DB_@DEV.idx"},
{0x0000F002, 1, 128, 0, 0, 1, 0, 0, L"System\DATA\H10DB_FPTH.idx"},
{0x0000F003, 1, 128, 0, 0, 1, 0, 0, L"System\DATA\H10DB_FNAM.idx"},
{0x0000F00A, 2, 4, 0, 0, 1, 0, 0, L"System\DATA\H10DB_FRMT.idx"},
{0x0000002E, 1, 40, 0, 0, 1, 0, 0, L"System\DATA\H10DB_TIT2.idx"},
{0x0000003C, 1, 40, 0, 0, 1, 0, 0, L"System\DATA\H10DB_TPE1.idx"},
{0x0000001C, 1, 40, 0, 0, 1, 0, 0, L"System\DATA\H10DB_TALB.idx"},
{0x0000001F, 1, 20, 0, 0, 1, 0, 0, L"System\DATA\H10DB_TCON.idx"},
{0x0000E002, 2, 4, 0, 0, 1, 0, 0, L"System\DATA\H10DB_@DU3.idx"},
{0x0000E003, 2, 4, 0, 0, 1, 0, 0, L"System\DATA\H10DB_@DU4.idx"},
{0x0000E004, 2, 4, 0, 0, 1, 0, 0, L"System\DATA\H10DB_@DU5.idx"},
{0x0000E005, 2, 4, 0, 0, 0, 0, 0, L""},
{0x00000043, 2, 4, 0, 0, 1, 0, 0, L"System\DATA\H10DB_TRCK.idx"},
{0x0000004E, 2, 4, 0, 0, 0, 0, 0, L""},
{0x0000F009, 2, 4, 0, 0, 0, 0, 0, L""},
{0x0000F007, 2, 4, 0, 0, 0, 0, 0, L""},
{0x0000F006, 2, 4, 0, 0, 0, 0, 0, L""},
{0x0000F005, 2, 4, 0, 0, 0, 0, 0, L""},
{0x0000E000, 2, 4, 0, 0, 1, 0, 0, L"System\DATA\H10DB_@DU1.idx"},
{0x0000E001, 1, 40, 0, 0, 1, 0, 0, L"System\DATA\H10DB_@DU2.idx"},
{0x00000083, 2, 4, 0, 0, 0, 0, 0, L""},
{0x00000084, 1, 64, 0, 0, 0, 0, 0, L""},
};
[edit] H10DB.datH10DB.dat is a list of records that store music information. Each record has 24 (= 2 + 22) fields such as track title, artist, album, genre, year, genre, rating, recent play time, play count, etc. The MUSIC mode of iriver H10 player does not read music information from actual files but relies only on these records for displaying music information. When a user listens to the music with a player, the number of records increases because, whenever a music file is about to be played, H10 player duplicates the record for the file, adds the duplicated new record into the list, and inactivate the original record. Therefore, H10DB.dat may contain a number of inactive records. UCS-2 string in H10DB.dat has variable size (length) to save the disk space. The actual size (in bytes) should be calculated from hdr.dat_field_offset. For example, Given a record #i, the size of file_path (field #1) UCS-2 string field is calculated as the following:
Similarly, the size of title (field #4) UCS-2 string should be calculated as the following:
[edit] Definitiontypedef struct {
uint32_t status;
uint32_t unknown1;
uint32_t unknown2; /* field # 0 */
ucs2_char_t file_path[]; /* field # 1 */
ucs2_char_t file_name[]; /* field # 2 */
uint32_t media_type; /* field # 3 */
ucs2_char_t title[]; /* field # 4 */
ucs2_char_t artist[]; /* field # 5 */
ucs2_char_t album[]; /* field # 6 */
ucs2_char_t genre[]; /* field # 7 */
uint32_t rating; /* field # 8 */
uint32_t revision; /* field # 9 */
uint32_t recent_play; /* field #10 */
uint32_t unknown3; /* field #11 */
uint32_t number; /* field #12 */
uint32_t year; /* field #13 */
uint32_t filesize; /* field #14 */
uint32_t duration; /* field #15 */
uint32_t samplerate; /* field #16 */
uint32_t bitrate; /* field #17 */
uint32_t unknown4; /* field #18 */
ucs2_char_t unknown5[]; /* field #19 */
uint32_t unknown6; /* field #20 */
ucs2_char_t unknown7[]; /* field #21 */
};
h10db_dat_t dat[hdr.num_dat_records];
[edit] Semantics
[edit] H10DB_*.idxH10DB_*.idx (typically 14 files) store sorted indexes that point to H10DB.dat records. Each file has a different sorting order. For example, H10DB_FPTH.idx sorts records in alphabetical order of file paths; H10DB_TIT2.idx in alphabetical order of track titles; H10DB_TPE1.idx in alphabetical order of track artists; H10DB_@DU3.idx in numerical order of user ratings; H10DB_TRCK.idx in numerical order of track number, etc. These sorted indexes are used for displaying music tracks in different orders: e.g., alphabetical order of track title/artist/album; and numerical order of track number in an album. Since the H10 firmware seems to implement an efficient search method such as binary search, all indexes must be arranged in proper orders as the firmware assumes. If not, a player may lose sight of some music files, show duplicated entries, or show wrong information because of the inconsistency. The following table shows a list of index files and respective sorting order and check_value type obtained from H10 [5GB] International firmware 2.05. This table can be obtained from field descriptor in H10DB.hdr.
[edit] DefinitionEach H10DB_*.idx file consists of an array whose element is described as h10db_idx_t: typedef struct {
uint32_t status;
uint32_t entry_index;
uint32_t check_value;
} h10db_idx_t;
h10db_idx_t idx[hdr.num_dat_records];
[edit] Semantics
[edit] How to calculate CRC-32 valueCRC-32 value of UCS-2 string is calculated with the following parameters: #define POLYNOMIAL 0x04C11DB7 #define INITIAL_REMAINDER 0x00000000 #define FINAL_XOR_VALUE 0x00000000 #define REFLECT_DATA 1 #define REFLECT_REMAINDER 1 #define CHECK_VALUE 0x2DFD2D88 Before calculating a CRC value, all UCS-2 characters in a string must be lower-cased. [edit] How to lower-case UCS-2 charactersThere is an ambiguity for converting upper-case characters into lower-case characters in terms of the treatment of European characters (e.g., À and à Δ and δ ) As the result of our experiments, EasyH10 implements the following function to convert only US-ASCII characters. ucs2_char_t ucs2lower(ucs2_char_t ch)
{
/* iriver only converts what-is-called US-ASCII characters. */
if (!(ch & 0xFF80)) {
ch = tolower(ch);
}
return ch;
}
[edit] Note for sorting UCS-2 stringsThe following function (ucs2comp) implements a comparison function used for sorting UCS-2 string values. In addition to converting lower-case characters to upper-case, we must move empty string latter than any other values. #define COMP(a, b) ((a)>(b))-((a)<(b))
ucs2_char_t ucs2upper(ucs2_char_t ch)
{
/* iriver only converts what we call one-bytes characters. */
/* We must limit the range of case conversion. */
if (!(ch & 0xFF80)) {
ch = toupper(ch);
}
return ch;
}
int ucs2icmp(const ucs2_char_t* x, const ucs2_char_t* y)
{
ucs2_char_t a, b;
do {
a = ucs2upper(*x);
b = ucs2upper(*y);
if (!*x || !*y) {
break;
}
x++;
y++;
} while (a == b);
return COMP(a, b);
}
int ucs2comp(ucs2_char_t* x, ucs2_char_t* y)
{
/* It seems to be safer to move elements latter that have empty value. */
/* We must ignore case during sorting, or H10 fails to recognize music files. */
if (!x || !*x) {
return ((!y || !*y) ? 0 : 1);
} else {
return ((!y || !*y) ? -1 : ucs2icmp(x, y));
}
}
[edit] About this document[edit] AcknowledgementI thank Badger for the structure information of H10DB.hdr, Toby Corkindale for the experimental implementation of database construction written in Perl (iriver.pm), and Jevin for the CRC calculation in a MisticRiver thread, H10 database reverse ingeneering. [edit] LicenseThis document was initially written by Nyaochi, an author of EasyH10 software. It is released under GNU Free Documentation License (GFDL). Back to H10 |
