Meta Exotica

[qxb]

I've been looking into frameworks for extracting metadata from oldschool formats. Most notably perhaps mod, xm, s3m, it, etc. Not to mention sid of course. And all the others: ml, aon, bla, bla, etc. Basically I want to create a permanent index of all metadata of all files on my system: in particular all music and media files, most importantly retro music media files. All files will be hashed, possibly using multiple hash functions, and metadata should be extracted and injected into a generic database or XML file; maybe as simple as <x:nodes><x:node id="hash"><z:property>value</z:property></x:node></x:nodes>; maybe more structured with tables for particular file types. Windows Search Index appears it may be extensible in this regard but I'm not sure what formats one can find for it on the Web and I would prefer not to have to try and reverse engineer and implement all these formats myself. It would be best with a .Net framework for this with a nice collection of binary file parsers and metadata extractors or just extensions for Windows Search Index for these formats. Ideas, hints, tips?

There out to be an open repository of metadata for oldschool formats. A meta exotica, for the masses.

[analoq]

Novel idea, certainly.
You could poke thru the source code to libmikmod for parsing tracker files, it's hard to come across the specifications for those old formats but I remember finding some on textfiles.com's Programming->Formats section.

The HVSC already has a metadata collection, the STIL. In fact SIDPLAY for Mac includes a Spotlight plug-in, which I suppose would be the Windows equivalent of a Search Extension.

[qxb]

Novel idea, certainly.
You could poke thru the source code to libmikmod for parsing tracker files, it's hard to come across the specifications for those old formats but I remember finding some on textfiles.com's Programming->Formats section.

The HVSC already has a metadata collection, the STIL. In fact SIDPLAY for Mac includes a Spotlight plug-in, which I suppose would be the Windows equivalent of a Search Extension.

Yes the SID format is covered but I spent much of my youth listening to mod files and various softsynth exotic music formats like musicline (ml) and art of noise (aon).

I would imagine Spotlight have similar capabilities to Windows Search but am afraid I don't own a Mac and also am somewhat pessimistic about the pervasiveness of Windows Search extensions ("IFilters") for these formats. Haven't found that many IFilters on the Web so far and none for oldschool formats.

[analoq]

Hmm, I don't know of any open source code (MikMod, ModPlug, etc.) that can import AON or ML modules... what player do you use to even play them?

[qxb]

Hmm, I don't know of any open source code (MikMod, ModPlug, etc.) that can import AON or ML modules... what player do you use to even play them?

Well you can use an Amiga emulator or you can use Deliplayer. There is a PC port.

As there are not very many known ML modules in existence that particular format is not so important, it can be done manually.

[qxb]

I've started a codeplex project and started working on support for the mod format first. There are still many details to flesh out and many formats to support but at least I have documentation for the mod format and it is still a large milestone given how many mods exist. In unison with a metadata homepage with social networking features it will be possible to collaboratively edit and contribute extra metadata and annecdotes to these modules. I'll complete the mod support and then see if there's some community interest in assisting in this open "metadata preservation" project.

http://metaexotica.codeplex.com/

[analoq]

it will also become possible to create graphs and statistics showing sample-reuse which could be quite interesting and the oldschool scene was and is quite vibrant and many people reused samples: can can then see where cross-fertilization appears

This aspect of your project has the most potential to be interesting, I think. It would be very interesting to know what tunes out there ripped off samples from say, the module that came with ScreamTracker3: INSIDE.S3M

So good luck to you with that,
cheers.

[Ziphoid]

qxb: Have you had any look at http://exotica.org.uk for instance? BuZz, who runs that site, has done a lot of work dissassembling the HVSC/STIL and Modland collections and is very open to cooperate on many levels. Also, if you want more info about .ml (Musicline Editor) files, just howl to me, since I have the sources for it and was also part of creating it with my fellow groupmembers in Musicline. DeliPlayer for Windows actually supports playing .ml modules properly, since Peter Kunath and Florian Vorberger got the source of the replayer and rewrote it into C/C++.

// Ziph

[BuZz]

qxb:

One of the guys who helps with exotica has been after a database such as this for some time, and been asking me to code it, perhaps a collaboration would be a good idea?

I have parser code for a few formats, that extracts instrument data etc. written in perl. you are most welcome to look at it.

we also have files with details on many formats on our ftp (lots thanks to the hard work of laurent clevy)

im sure xtc has some ideas too, as he had it mostly planned out in his head for some years now, so its probably worth getting his feedback. might be worth posting on the exotica forum for this.

analoq:

you can sort of do this for some formats at the modland interface, by searching for instrument names that match. i once thought about check-summing all instruments from all modules and making a database showing the usage, but then I thought i was being a bit OCD with it and decided not to

[analoq]

analoq:
you can sort of do this for some formats at the modland interface, by searching for instrument names that match.

Come on now, if you're stealing other people's samples the least you're going to do is change their name

[BuZz]

well, true, but you can also search on sample sizes. of course this will bring back lots of wrong matches, but i did say "sort of"