Challenge: figure out Amazon's crazy-ass "proprietary" MP3 tagging system -- UPDATED
Pete sez,
As an experiment, I recently purchased a couple of albums via Amazon's MP3 Downloads store. I figured since they've gone DRM-free and all the majors are on board, I ought to at least give them a try.So, who knows how this works? Can someone please reverse-engineer the Amazon tagging scheme and add notes to the comments? Let's get this figured out, folks. (Thanks, Pete!)Aside from the fact that full-album purchases *require* use of their pointless MP3 Downloader software (no Linux version, of course, so I had to fire up XP in a virtual machine), the experience wasn't so bad.
However, I use mpd (http://www.musicpd.org/) and mt-daapd (http://www.fireflymediaserver.org/) to share out my music on my local network, and both rely on background scans of ID3 tags. Unfortunately, neither service was able to pick up the ID3 tags on the mp3's I got from Amazon, even after I manually re-tagged all the tracks.
I contacted Amazon customer support, and got the following response:
"Indeed this is a little more of an abstract question. Unfortunately we are unable to release any information on how we run our ID3 tagging as it's proprietary. The only thing I can think to do is manually re-tag each Album/Track.
I hope you find this information helpful. Thanks for buying MP3 Music Downloads on Amazon.com."
Proprietary ID3 tagging? Seriously?
I've continued to fiddle with these tracks, so far to no avail. The weird thing is, local clients like Rhythmbox and Amarok (as well as my ID3 tag editor) have no problem seeing the tags.
Update: In the comments, Amazon MP3 sez,
I apologize in advance for the response from Amazon customer service. Don't know exactly where their info came from (they're usually totally on top of things), but we don't use "proprietary" anything. "Crazy-ass" sometimes, but not proprietary.We worked very hard to make sure the tag info we place in the MP3 files works everywhere (don't get me started on how different programs choose to interpret the "Ensemble" tag). Up till today we were doing pretty well; no complaints at all.
Obviously we've optimized the tags to work best with iTunes and, ahem, WMP, but they should work everywhere. If you've got any more detail around what might be confusing mpd and mt-daapd, post away. In the meantime, I'll look into it.
Update #2: Mystery solved! It was a file permissions bug:
Thanks to some help from an Amazon employee in the BoingBoing comments, we figured the initial customer service rep who told me "it's proprietary" was misinformed. This led me back to poking around on my system, at which point I discovered the mp3 files only had read permissions for my user account, when I made them readable for other users, both mpd and mt-daapd were able to find them.
the latest
latest episodes
I don't have a track I can try this out on, but see if any information comes up if you pipe it through the ID3 Processing Library. This will at least give a dirtier output -
http://jorgecardoso.org/processing/ID3/
It seems to me that a proprietary tagging system is DRM in a way. Isn't it? Can Amazon be tapped for false advertising?
Dang they must be using some of that one-click mojo on those mp3 tracks!
If you have to reverse engineer a product to make it functional, that's all well and geeky, but you're casting your vote for the wrong candidate. Your wallet is the ballot box. Demand a refund from Amazon.
I did exactly that from an Amazon merchant when I thought I downloaded an MP3 and instead received a proprietary DRM format -- and the company's loader wouldn't recognize my CDRW drive. They wanted to help me troubleshoot the problem. I instead asked for, and received, a refund. So then I bought a regular DRM-free CD from a different merchant and did as I legally pleased.
What I'm asking is that, if your solution to an inferior product is to reverse-engineer it into having basic functionality, where does that leave the rest of society? If the situation were inescapable, I'd say, more power to you. But the future is being written now and the more crappy digital products you buy, the more likely they are to be adopted across the board.
The geekerati shouldn't support greed / incompetence / stupidity just because they've written an escape plan for themselves.
Actually, you can download albums direct to Linux. I've done it with Ubuntu. It's a bit of a workaround involving wine, but it does work.
http://www.odonnellweb.com/?p=3956
Not to rain on the parade but... JESUS CHRIST. We're bitching and bitching for these crappy corporations to stop using DRM. And so finally Amazon puts together a DRM-free operation. And all I hear is complaints that "they use a *pointless* mp3 downloader software."
Ask for improvements--no demand improvements on the system. But give them props for taking a giant leap in the right direction.
- Eric Monse
The Amazon MP3 service has worked really well for me - DRM free, inexpensive and high quality (all my DL's have been at 320kbs). and all of the devices I use for playback have regognized the tags w/o any touchup and worked flawlessly (ROKU Soundbridge, zune, turtle beach audiotron, creative zen and ipod nano)
not that any of this helps pete, but i just wanted to offer up a pov from a satisfied customer. in fact, until the amazon store opened, I had NEVER purchased an online track. all of my music purchases had been plastic and vinyl til then. its nice to have a place with such a good selection and know that it will work with anything I play music on today and into the future
Can we step back for a moment and think about this? Amazon MP3 files load correctly in virtually all music players, including ones running on the original poster's system. The ID3 format can be easily read and edited from the MP3 files.
It's certainly possible that Amazon is watermarking the files and that the watermarks are in some way interfering with a bit of random software that tries to do something clever and off-spec, and it's also possible that the original emailer set up his software wrong, but it's not likely that Amazon is doing anything magical with the tags themselves.
"Proprietary MP3 tags" is nonsense, the Amazon support person who told you that didn't have a clue about what the company is doing.
Just think: you say yourself that the metadata show in some Open Source players, so it's obvious that there's nothing "proprietary" going on.
My bet: some unusual fields used or maybe they use ID3v2 tags and mt-daapd only understands v1. Indeed, the first two google hits for "mt-daapd id3v2" query indicate precisely that:
http://www.google.com/search?q=mt-daapd+id3v2
I had some tagging problems with Rhythmbox once - I could change the tag, but when I next loaded, things were right back to the original.
Turns out there's another tagging system, "APE" tags. I think they're written by the windows foobar 2000 music management software, could be wrong. Anyway, Rhythmbox seemed to be reading these tags fine, but writing any edits to the ID3.
I ended up just hacking the file in a hex editor, looking at the end of the files for the tags and killing the magic number of the other tag.
Maybe your problem is similar? Amazon might be writing tags in a bunch of different formats, but your tagging software is only modifying the ID3 section?
Both iTunes and the Windows shell have no trouble at all reading the track information from an Amazon purchased track. Neither does id3tool on a base ubuntu install.
It could depend on the file, I suppose. I tried this on the first song on The White Stripes "Conquest" EP, which I bought from Amazon a couple weeks ago.
They do embed a "Amazon.com Song ID" in the comment field. That would be trivial to strip.
Anyone try Tag&Rename or some similar program?
http://www.softpointer.com/tr.htm
If it works, I don't see what the problem is. Just retag your downloads as they recommend. If it is DRM-free, I don't see what the fuss is about.
A minor detail. Was the music bought for the sake of the music? If so, was it any good.
"and both rely on background scans of ID3 tags"
Does this mean it does not play or what? Not ONE person has mentioned the music. It is, after all, about the music.............
I haven't opened it in a Hex editor yet to see what tags they are using but on initial view they are using ID3v1 and v2 tags. Here is the info on the track I downloaded:
Artist A Perfect Circle
Title 3 Libras [Explicit]
Album Mer De Noms [Explicit]
Track 6/12
Disc 1/1
Genre Alternative Rock
Year 2000
Rating
Composer
Size 6.64 MB (17% of original, 1 to 1 compression)
Original Size 36.95 MB
Length 3 minutes 39 seconds
Channels 2 (stereo)
Sample Rate 44.1 KHz;
Sample Size 16 bit
Bit Rate 253 kbps
Encoder LAME 3.97
Encoder Settings Variable Bit Rate -V 0 (Extreme)
Audio Quality Very High (Lossy)
Contains Album Art, ID Tag [ID3v2.3 (ANSI) & ID3v1.1]
Channel Mapping Left, Right
File 06 - 3 Libras [Explicit]
Type Winamp media file [.mp3]
Everything looks good to me and shows up correctly in Winamp, iTunes, WMP 11 and my Sonos system.
If you are altering the data and not seeing the changes, my guess would be you're altering one of the ID3 tags and the reader is reading the other tag. I know how to remove ID3v1 tags in Winamp and iTunes, but not in other programs (I'm sure Google can save the day).
-Jason
I've purchased a few album, and the MP3s display and play fine in every player I use - Foobar on Windows (at work), Amarok on Ubuntu(at home), and Creative's software (on my Zen Vision:M). I've been EXTREMELY satisfied with Amazon thus far.
Frankly, I'm *sure* this is some issue with the ID3 versioning or perhaps the character set being used, given that the files play fine, and that every local player reads them fine (and yes, the music Intelligence's Deuteronomy is quite good).
My larger point was the default "We can't tell you, it's proprietary" answer from Amazon.
I definitely give Amazon credit for coming as far as they have, and I fully realize this is *not* an issue most users will run into.
Still fiddling with tag versioning...
Pete
I've had several linux based itunes impersonators throw fits with my mp3's that otherwise worked fine in winamp, itunes, and windows media player.
Perhaps there's more of an issue with some of the Ubuntu software?
Anyone know for sure that Amazon mp3s aren't watermarked with embedded purchaser info like iTunes DRM-free mp3s?
Don't like the idea of my ID embedded in my mp3s.
I've had no problems with tagging on any of the 100s of tracks I've downloaded through amazon. I do wish, though, that they named their files better -- starting with 01, 02, 03, instead of 1, 2, 3 and so on. That way they would be properly ordered when sorting by filename.
I wrote an ID3 library in Python that handles practically every tag that is in the ID3 specification(s), versions 2, 3, and almost all of 4.
If I was wont to download some silly MP3 and pay money for it, I'd figure out the problem. However, as I refuse to spend my money on degraded-quality products that cost the same amount as better quality at a B&M, I cannot.
I think it's worth noting that the Amazon support person didn't really say the tag is proprietary. They said "how we run our ID3 tagging" - I take that to mean that the process by which they tag the MP3's is proprietary. That seems more reasonable to me.. trade secrets and all.
Hopefully it's the ID3v1 versus v2 issue. Pete, did you check that out yet?
I'm waiting for ogg vorbis downloads!
<whistles while rocking on heels/>
<ducks/>
You can tell if they are watermarking per purchaser by buying the same track under two different accounts and seeing if they are different. IIRC, people have and found no differences.
It is very likely that these tracks are watermarked before they are given to Amazon, but this would not be a watermark that different per-user and so has no real privacy implications.
@dculberson:
So far, I've tried ID3v1 only, ID3v2 only, both v1 and v2, v2.3, v2.4, etc.
Isn't that kinda like saying "We have a proprietary XML system"; everything that defines ID3 can be easily found out, there are plenty of specs out there on it.
If it were in fact proprietary, it wouldn't be ID3.
I work at Amazon, and sit near tech support. I'll have to ask around about this tonight at work.
To Eric @ #7:
"Not to rain on the parade but... JESUS CHRIST. We're bitching and bitching for these crappy corporations to stop using DRM. And so finally Amazon puts together a DRM-free operation. And all I hear is complaints that "they use a *pointless* mp3 downloader software."
I haven't been bitching at them to stop using DRM. I've been bitching at them to stop trying to control what I listen to, how and where - they don't get a penny from me until that happens. This is a step in the right direction, but still I apparently can't purchase MP3s and listen to them on my MP3 player unless I have an OS and player that recognise the proprietary crap they're still trying to force on us.
Until this idiocy stops, I'm going to keep buying from *only* open a dn independent sources and not from the major labels. I know this isn't the RIAA's decision directly, but they're not going to get a cent until they decide to stop trying to control me (and yes, they can do that by licencing to more open retailers).
I'm an Amazon fanboi. Totally. I had never bought music online until Amazon offered it and I've had no issues with it (prior to that I bought music on CD...through Amazon). They play fine, the information shows up, burning them to CD and sticking it in my 3 year old car also works. Sounds to me like the original poster just managed to find a couple of poorly written software applications that get confused by newer tag formats. I don't see how Amazon is doing anything less than a Good Thing. Good for them, good for me, good for the music industry.
Now if you'll pardon me, I need to make a trip to the recycler to drop off all these stacked up Amazon boxes...
OK. Full disclosure: I work at Amazon. Plus I work on the MP3 development team.
I apologize in advance for the response from Amazon customer service. Don't know exactly where their info came from (they're usually totally on top of things), but we don't use "proprietary" anything. "Crazy-ass" sometimes, but not proprietary.
We worked very hard to make sure the tag info we place in the MP3 files works everywhere (don't get me started on how different programs choose to interpret the "Ensemble" tag). Up till today we were doing pretty well; no complaints at all.
Obviously we've optimized the tags to work best with iTunes and, ahem, WMP, but they should work everywhere. If you've got any more detail around what might be confusing mpd and mt-daapd, post away. In the meantime, I'll look into it.
I have purchased two MP3 tracks from Amazon (both recently) and have encountered no problems using them with mt-daapd 0.2.4 on a Linux host.
On examination with "id3v2 -l", both tracks have ID3v1 and ID3v2 tags which look normal, non-pathological and in no way sinister.
My first suspicion is that you've got a bad build of mt-daapd. Or, perhaps one of the libraries on which it depends is b0rked. (I'm using libid3tag 0.15.1b, FWIW.)
I have a Mr. Occam on the line, and he asks me to convey his suggestion that application bugginess may be a simpler and more likely explanation than proprietariness or crazy-assitude.
AmazonMP3 -- many thanks for turning up to clarify things! Much better response than the one from the CSR!
I've updated the post with your comment. Thanks again!
FWIW, I've bought one or two MP3s from Amazon and the tags have shown up as expected in Amarok. Sounds like a tag version issue; MP3 tags are really a mess.
Who the flip cares anyway?
"We are sorry... We could not process your order. The sale of MP3 Downloads is currently available only to US customers located in the 48 contiguous states, Alaska, Hawaii, and the District of Columbia."
Thanks Amazon, thanks for being retarded.
As the OP, let me add my thanks to Amazon as well, and repeat that, aside from this admittedly minor issue, I'm glad to see their extensive support for non-DRM music. My apologies if the initial "It's proprietary" response sent me barking up the wrong tree.
Here, roughly, is what I've seen thus far:
I'm baffled.
Update from us at Amazon:
I've tested this last night with about 500 files from our service, with mpd (not mt-daapd). No problems whatsoever. For reference, it was:
- Debian 4.0r2 (2.6.18-5-686)
- mpd 0.12.1
mpd was installed using standard apt-get install.
Perhaps there's really something funky with DownDB's particular setup of mpd/Linux? Has this problem been observed by anyone else?
Our files do contain both ID3v1 and v2 tags, but this should not be an issue.
I'm going to consider us off the hook on this...
@AMAZONMP3:
Agreed. Thanks for going the extra mile. I'll continue investigating my setup.
Would you believe it was file permissions?
I hang my head in shame.
I've not had any problems with ID3 tags, but I'm not the super user that most of you seem to be. All I do know is that I friggin love me some AmazonMP3 store - especially the fact that I can search directly from within my browser.
I'm no kind of coder, but I managed to put together a quick search plugin for Firefox that lets me search music in the mp3 store at will. Feel free to take it if you think it would be useful.
http://eleven21.com/blog/amazon-mp3-quick-search-plugin-firefox-ie7/