T10 MTP...for Linux ?

[TheFatElvis]

Ok, more digging, more info:
Media Transfer Protocol (MTP) is build off of Picture Transfer Protocol (PTP) which Microsoft also claims to own:
http://www.directionsonmicrosoft.com...4mpumsf_sb.htm
(whatever, not going to research that. Let 'em have it for now...)

gPhoto (available free under the gnu GLPL license) has been around for a while as a tool to transfer pictures between digital cameras and Unix-style systems and is built upon PTP. So more searching and I find this on sourceforge:
http://sourceforge.net/mailarchive/f...forum_id=10521

To quote: /* iRiver OGG and MP3 player. First MTP Device supported!

I'm not sure I have a spare machine with USB 2.0 laying around to build a Linux system on, but I will check and see (if anyone is currently using one, feel free to try as well.)

Not the UMS solution I *want* but perhaps good news for the non Windows crowd.

[orthoB]

I'm running an American T10-1G, firmware MTP-1.54-N-ENG using gphoto2 and
Fedora Core 4. It works well for transfering files, creating directories, and I updated
the firmware as well. The latest gphoto2 in CVS works best - it includes the iRiver T10
as a valid ptp2 device, so it "just works". There are also a couple of bugfixes that
help. I've created a patched source RPM of gphoto2 that incorportates the CVS ptp2
and another bugfix from CVS.

I ran with a lot of debugging on to see how it worked. Basically, just PTP2 will handle
the directory creation, subfolder creation, and file transfer or listings. It has the
feel of ftp. No doubt someone will build a nice directory sync app using this.

Next to try playlists ...

[TheFatElvis]

Outstanding to hear - thanks for the info. A guy here at work has fedora 2 (I think) so I may be able to give it a dry run. From the looks of things with gPhoto in the distro I may even be able to play around with the command line interface, and have the T10 connect by substituting in the correct USB id. More of an interesting exercise than a neccessity - I can always build off the current cvs state (pretty sure I saw the iRiver change in it...)

If it all works out, I'll do *something* at home (knoppix comes to mind - I don't want to rebuild twice for a dual boot - but if I can repartition my windows install, a 64bit Linux install might be fun to dink around with.)

As a side note: any tips to pass along for the firmware upgrade ? I'm assuming it will be obvious once I see the file layout on the T10, but maybe I'm being too hopefull.

[orthoB]

I just uploaded it into the main directory:
setenv LANG C
gphoto2 --port "usb:" --usbid "0x4102:0x1113=0x054 c:0x004e" --auto-detect -f /store_00010001 -u T10_MTP.HEX

This was using the FC4 version, without the T10 usbid's. With the CVS copy:
gphoto2 -f /store_00010001 -u "T10_MTP.HEX"

Then on the T10 go to the Settings/Advanced/UpgradeFirmware and say Yes. It finds and flashes the unit. This takes quite a bit of time - be patient. A fresh battery would be good (a marginal one definetely bad).

The root directory appears to be /store_00010001 on my device. From what I can see from the directory/name PTP numbering scheme, this is probably universal. A few other quick commands:

Make a directory:
gphoto2 -f /store_00010001 -m somedir
Recursive listing of directories:
gphoto2 -l
Recursive listing of directories & files:
gphoto2 -L
Recursive listing starting at some folder:
gphoto2 -f /store_00010001/somedir -L
Upload a file into a directory:
gphoto2 -f /store_00010001/somedir -u somefile.mp3
Grab a file from the T10:
gphoto2 -f /store_00010001/somedir -p somefile.ext
Grab all files from the T10 folder:
gphoto2 -f /store_00010001/somedir -P

Unfortunately there isn't a command to upload a group of files (yet). I use a bash script to take a list and upload. I also have a script to split the FM recordings into 15 min chunks (w/ 10 sec overlap) to make it easier to page thorugh a 2 hour program.

Have fun!

[TheFatElvis]

Outstanding, thank you orthoB ! I had gotten the T10 to connect using the gPhoto from Fedora 2 using some random camera alias. Turns out, it's the exact same usbid that you posted (scary, eh?) For some reason the CVS build barfed, and since it was a coworker's computer - I didn't pursue it. I'll repartiton my Win2k ntfs here at home and try it again - but for now I'm just too darn happy that the iRiver is playing nice(using Knoppix 4.0). I really, really, really didn't want to build a WinXP to talk to an .ogg player. Really. Thank you also for the command list. I must have missed -p in the man pages and had chalked it up to DRM.

On a side note - have you noticed the file WMPInfo.xml ? It didn't seem to have any interesting info when I pulled it over, yet it acts very particularly on the device. I selected a song from /music - it scrolled ID3 info, and was ready to play. Held down the center button, used the browser to find the .xml file, highlited it and pressed play. It sent me back to the song I had previously selected. I did it with a different song, it took me right back to it. Any chance playlists are .xml ? Anyone who's created a working playlist have any insight ?

Thanks again orthoB - great stuff.

[TheFatElvis]

Bah...scratch the .xml stuff. Looks like the device defaults to the last "playable" song whenever you try to play a file the T10 doesn't like (had some file corruption last night). Oh well, I'll keep at it.

[orthoB]

Originally Posted by TheFatElvis

Bah...scratch the .xml stuff. Looks like the device defaults to the last "playable" song whenever you try to play a file the T10 doesn't like (had some file corruption last night). Oh well, I'll keep at it.

I think the .xml file has/is the DRM component for the player. I think I read on an iRiver FAQ that .m3u files are supported. xmms (linux player) can generate & use .m3u files ... I'll try this out tonight and report back.

You'll also notice that when you list files (-L) that it shows the file type. When initally uploaded (while still connected) a .mp3 files shows up as "unknown" (can't remember the exact type). After unplugging the cord and the player browser starts up, subsequent listing queries will show the file as type "audio/basic". So the player itself inspects a file and sets/corrects the filetype in it's internal filesystem.

[orthoB]

Originally Posted by orthoB

I think I read on an iRiver FAQ that .m3u files are supported. xmms (linux player) can generate & use .m3u files ... I'll try this out tonight and report back.

Nope - .m3u files don't work. The docs indicate that there should be a "Playlists" directory at the top. Created that. .m3u, .M3U, with and without extended info weren't recognized by the player. Changing to DOS format (adds CR's) and using backslashes for directories (or unix slashes) didn't work. Putting the playlist in the directory with the music didn't work.

Reading some more, it appears that some sort of Window's Media Player playlist is used. I tried creating a .asx file, but that wasn't recognized either. I suspect that the playlist is in a .wpl format.

Can anyone confirm this or post a copy of a playlist file? Or at least the extension? I don't have a windows box to try this out on.

[Carlwill]

Anyone have any luck with just plugging in the device and being detected straight off the bat?

[orthoB]

Originally Posted by Carlwill

Anyone have any luck with just plugging in the device and being detected straight off the bat?

Only with the CVS gphoto2 version. I started with the Fedora Core 4 gphoto2-2.1.6-1.1 rpm, but having to use the --usbid flag to map it to an existing camera. But the sourceforge CVS version works. I've patched the gphoto2-2.1.6-1.1 source rpm with the CVS ptp2 implementation, and it "just works". The next upstream release of gphoto2 should incorporate all this, then all linux distro's should also pick it up in turn.

Well .. usually it just works. I'm looking into a bug where if the initial xfer directory size is a multiple of 256 bytes, gphoto2 fails to pull it over and then all the other ops fail. Record an additional file (radio or voice), making the directory larger, and it works. I've also patched gphoto2 to use 4096 rather than 512 byte xfers for directories, which gets rid of this bug. But I'm trying to determine if this is a gphoto2/usb bug, or an iRiver implementation bug.

[Carlwill]

Let us know what you find.

[TheFatElvis]

Hey orthoB - which function did you patch in the gPhoto2 build ? (it generally barfs out on me as a result of buffer overflow - most of the time the data file arrives just fine, sometimes I get corruption.)

Good(?) news tho - I commandered a WMP10 computer while the user was away at lunch . It appears the T10 creates a folder called "Playlists" on the device, and uses a file extention of ".pla" Just at a rough glance - it is a file of 512 byte chunks, the first being a device identification segment. Now, the chunks underneath are fairly straightforward looking: unicode path and filename on the device. It's the first two bytes that has me thrown off. It appears to be an interger value of the directory the file is in - shifted by 0x20. I'm just not super clear on how directories are named (WMP10 uses the ID3 tag info to create a directory structure on the T10) Chances are it is just a "count" i.e. the first directory created is 0x20 (or 0x21 depending on zero offset vs. one)

I'll cross my fingers and attatch the colorfully named Crapola.pla. The 512 data chunks are binary zero padded, so a hex editor is probably best. The order of things listed in the .pla file seems to be the order of files playing, but I don't know if the T10 modifies the playlist file to keep track of where it is in the file. Couple of notes: I'm not playing any files directly from /MUSIC (WMP10 stuck *everything* in a directory of some sort). Some files are under multiple directory levels. These are just some random mp3's I found I had on CD - I'd be more than happy to run a more controlled test to address specific concerns.

p.s. file is just a .gz that I renamed .gz.zip so the upload manager would play nice...

[orthoB]

Originally Posted by TheFatElvis

Hey orthoB - which function did you patch in the gPhoto2 build ? (it generally barfs out on me as a result of buffer overflow - most of the time the data file arrives just fine, sometimes I get corruption.)

I modified the .src.rpm, which takes the original release tarball and patches it prior to build. I added a patch to bring the ptp2 module up to CVS, and there's a 512 -> 4K xfer patch (I think I did that by hand, then just rebuilt the library and manually replaced the installed one). I've added the 4k patch to the build. Send a private post with your eMail and I can send the .src.rpm. It's about 3.8 Mbyte.

Originally Posted by TheFatElvis

Good(?) news tho - I commandered a WMP10 computer while the user was away at lunch . It appears the T10 creates a folder called "Playlists" on the device, and uses a file extention of ".pla" Just at a rough glance - it is a file of 512 byte chunks, the first being a device identification segment. Now, the chunks underneath are fairly straightforward looking: unicode path and filename on the device. It's the first two bytes that has me thrown off. It appears to be an interger value of the directory the file is in - shifted by 0x20. I'm just not super clear on how directories are named (WMP10 uses the ID3 tag info to create a directory structure on the T10) Chances are it is just a "count" i.e. the first directory created is 0x20 (or 0x21 depending on zero offset vs. one)

I'll cross my fingers and attatch the colorfully named Crapola.pla. The 512 data chunks are binary zero padded, so a hex editor is probably best. The order of things listed in the .pla file seems to be the order of files playing, but I don't know if the T10 modifies the playlist file to keep track of where it is in the file. Couple of notes: I'm not playing any files directly from /MUSIC (WMP10 stuck *everything* in a directory of some sort). Some files are under multiple directory levels. These are just some random mp3's I found I had on CD - I'd be more than happy to run a more controlled test to address specific concerns.

p.s. file is just a .gz that I renamed .gz.zip so the upload manager would play nice...

This is great! I kinda have playlists working. I made some python scripts to make them. It looks like the playlist format is pretty simple. It's a series of 512 (=0x200) bytes blocks. Everything is big-endian (network order) rather than Intel-host little-endian.

The first block is:

be_int_4 = N = # entries in the playlist
char[508] = "iriver MTP PLA\0"

The other N blocks have the format

be_int_2 = Unknown integer (album #?)
char[510] = big-endian utf-16 filename

This uses winDoze style names, with "\" as a directory limiter.

If the file is stored (using gphoto2) at
/store_0001001/Arrogant/17_Dog_Named_Bob.mp3
the corresonding entry is:
\Arrogant\17_Dog_Nam ed_Bob.mp3
Note that the root directory is removed.

I've created two python scripts (attached):
./pla2txt.py somefile.pla Dumps the contents of the playlist

./txt2pla.py mylist.txt mylist.pla Takes a unix list of files and creates a playlist

Then make the Playlist directory (if it doesn't exist):
gphoto2 -f /store_00010001 -m Playlists
Add the playlist to the directory:
gphoto2 -f /store_00010001/Playlists -u mylist.pla

This sort of works ... the list is present, it can be selected. It cycles through the files in order. But the titles are strangely front-truncated. It might have something to do with the unknown number, but I haven't found a correlation between the first number and the filename length. It does look like the album number, as you suggest. Maybe there's another database of info that lists metadata for all the cuts that this is referencing?

The attached archive is really a .tar.gz file. Unpack with:
tar xzf pla.zip

[TheFatElvis]

Originally Posted by orthoB

I modified the .src.rpm, which takes the original release tarball and patches it prior to build. I added a patch to bring the ptp2 module up to CVS, and there's a 512 -> 4K xfer patch (I think I did that by hand, then just rebuilt the library and manually replaced the installed one). I've added the 4k patch to the build. Send a private post with your eMail and I can send the .src.rpm. It's about 3.8 Mbyte.

Going to buy a spare SATA drive and try to mount a distro on it over the weekend. I get that up and working, I may just take you up on the rpm offer

Originally Posted by orthoB

This sort of works ... the list is present, it can be selected. It cycles through the files in order. But the titles are strangely front-truncated. It might have something to do with the unknown number, but I haven't found a correlation between the first number and the filename length. It does look like the album number, as you suggest. Maybe there's another database of info that lists metadata for all the cuts that this is referencing?

Bah, sometimes the answer is so close to your face, you can't see it. Try this - At the beginning of each 512 byte chunk, and counting from zero - which *unicode* byte does the song title start on ? Try it on the .pla I sent you, by George I think we've got it. (not a directory name, but since the directory name was the same for a large chunk of files, they all skipped the leading N bytes...) The mystery number seems to be the number of bytes to skip off the front of each block to find the song display title. No metadata required. /foreheadslap

[orthoB]

Originally Posted by TheFatElvis

Bah, sometimes the answer is so close to your face, you can't see it. Try this - At the beginning of each 512 byte chunk, and counting from zero - which *unicode* byte does the song title start on ? Try it on the .pla I sent you, by George I think we've got it. (not a directory name, but since the directory name was the same for a large chunk of files, they all skipped the leading N bytes...) The mystery number seems to be the number of bytes to skip off the front of each block to find the song display title. No metadata required.

Excellent! You've got it! Here are the corrected scripts. pla2txt.py now constructs a valid playlist that shows the file title.

Unpack the attachment (really a tarball) using: tar xzf pla.zip

The next thing would be to incorporate this stuff into a script that would sync the iRiver to a directory structure, and make it easy to create/edit playlists. Suggestions? I'm more of a command-line guy. When I'm forced to play video games against a GUI, the gui usually wins ...

For now I'm going to try to build a gphoto2 CVS rpm to see if the 512 byte xfer problem is something that exists in the latest development version. It'd be nice to get this fixed in the upstream gphoto2 (or the iRiver firmware).

Cheers!

[n0yd]

just an aside here, i dont have the equipment to test myself at the moment.

when you view the root directory on the player can you see any files with .sys suffix?

i was trying to force an MTP T30 to load UMS firmware and the problem seemed to be that i needed to change some data in a file in the root called T30*.sys (several different filenames for the different region/versions of the player). unfortunately when using explorer in windoze the contents of the directory were not visible, only its subdirectories . i'm assuming that the directory structure on T10 is the same as that on T30 ie. / (/store_00010001) has two subdirs named DATA & MUSIC.

[orthoB]

Originally Posted by n0yd

when you view the root directory on the player can you see any files with .sys suffix?

No .sys files. The only non-directory that shows up in my root is WMPInfo.xml. This is a xml file that sets "WMP DeviceID=" to something that looks like a registry key. All the other files are .mp3 or playlists.

[jackpot89]

umm

I just got FC4 running, and I am using the latest Gphoto2 from FC, and I get this error message using this command
gphoto2 --port "usb:" --usbid "0x4102:0x1113=0x054 c:0x004e" --auto-detect -f /store_00010001 -L

Model Port
----------------------------------------------------------

*** Error ***
Could not detect any camera
*** Error (-105: 'Unknown model') ***

For debugging messages, please use the --debug option.
Debugging messages may help finding a solution to your problem.
If you intend to send any error or debug messages to the gphoto
developer mailing list <gphoto-devel@lists.sourcefo rge.net>, please run
gphoto2 as follows:

env LANG=C gphoto2 --debug --port "usb:" --usbid "0x4102:0x1113=0x054 c:0x004e" --auto-detect -f "/store_00010001" -L

Please make sure there is sufficient quoting around the arguments.

Anyone know why?

[orthoB]

Originally Posted by jackpot89

umm

I just got FC4 running, and I am using the latest Gphoto2 from FC, and I get this error message using this command
gphoto2 --port "usb:" --usbid "0x4102:0x1113=0x054 c:0x004e" --auto-detect -f /store_00010001 -L

Anyone know why?

Try adding the arguement --camera "Sony PTP" to the command line. This should force it to use a PTP2 protocol.

It could be that your iRiver device has a different USB ID. Plug it in, and see what "/sbin/lsusb" shows you. This app is in the usbutils rpm. Change the 0x4102:0x1113 to whatever lsusb indicates the device ID is.

Hope this helps.

[jackpot89]

Originally Posted by orthoB

Try adding the arguement --camera "Sony PTP" to the command line. This should force it to use a PTP2 protocol.

It could be that your iRiver device has a different USB ID. Plug it in, and see what "/sbin/lsusb" shows you. This app is in the usbutils rpm. Change the 0x4102:0x1113 to whatever lsusb indicates the device ID is.

Hope this helps.

Dang...

Looks like I have to get a new cable, this computer at least doesn't seem to even see that the device is plugged it.

Off to test it on another computer, but I hav a feeling that somehow the cable is gone

=(

EDIT:
I found out its not the computer, because I plugged in a different USB device into computer, and it works... Might be cable, or device NO GOOD

[SonuvBob]

Hey all, I've been lurking here since I got a T10 for Christmas, and finally got around to opening the package yesterday. I've got gphoto2-2.1.99 (newest from CVS) set up on a Gentoo box, and it works great.

Originally Posted by orthoB