I’d like to be able to read/parse these save files, but they don’t read very well on CED. I’ve had the best results on linux so far.
I want to create a way for users to access their Amiga calendar/tasks/contacts on the go and step one is reading the file!
The file is zipped up on my pCloud drive here: http://pc.cd/9Cy7
How can I read this file properly and with fidelity so I know everything is accounted for?
- You must login to post comments
Really, the simplest way of reading these files is by opening them with Digita Organiser, since I don’t know of any other software that can import those files. But that’s not an option for reading the files online or on a non-Amiga platform, so most likely some sort of interpreter will have to be written.
A quick look at the file looks like it’s simply an IFF file, which is great news because IFF is a very good standard format for all sorts of data, and is relatively easy to read. In fact, it was such a good format that, even though it was invented in 1985 for use on the Amiga, variations of it can still be found today in WAV, RIFF, AIFF and PNG file formats. It also looks like Digita didn’t use any sort of compression or encryption, which should mean it’s possible to build a simple interpreter on any platform.
IFF files use one or more chunks which hold data and are clearly defined by type and size, where type is defined by four characters and size is a 32-bit unsigned integer. These chunks are held in an overall container that also defines the type and size of the contained information. For example, a typical ILBM image will have the overall file type defined as “ILBM”, and which contains a chunk defined as BODY that contains the bitmap information. A BMHD chunk is also included that contains the bitmap header with information such as size and depth, and an optional CMAP chunk contains colour map, or palette information.
Anyone can define their own IFF chunk types, which is what Digita have done for their Organiser files. There is a registry of filetype identifiers available so that developers can avoid using chunks commonly used for other types of data. A useful description of the IFF filetype is also available on the AmigaOS Wiki. With this information, it should be possible to separate out all the chunks used in the Organiser file using relatively simple code. AmigaOS has routines built in for this purpose (iffparse.library), but other platforms will need a separate solution, either a 3rd party library or some simple file seeking routines.
Once the chunks are separated out, there are some bytes of binary information before the text of the individual item. This is most likely a date stamp, but could also include status flags etc. These parts will have to be reverse engineered by comparing known values to the bytes in the file, so it would be useful to have a number of similar entries with different dates to compare for example. The meanings of the individual bytes should be relatively obvious once they’re studied a little, though it might help to present them as decimal values to start.
A couple of points to bear in mind:
- IFF files are typically in big-endian format, so reading them on a little-endian platform such as a standard Intel-based PC or web server will mean byte order needs to be swapped. Chunk sizes and any other binary data (e.g. time stamps) will be read incorrectly otherwise.
- Not all the binary bytes in the chunks are guaranteed to be user data. Some could be checksum information or other internal Digita Organiser flags, so modifying the files could result in files that Organiser can no longer read. Make sure to back up any files that might be modified.
- Reading the file as you did under Linux is corrupting some of the bytes by possibly falling foul of the endian mismatch, and possibly trying to interpret them as Unicode. If you open the file in a hex editor you will see that the file starts with the bytes “FORM”, followed by 4 bytes of file size, then ORGP as the file format identifier.
Edit: After closer analysis of the file format, I can tell you the following:
- While it looks like a proper IFF format file, it does in fact break at least one of the rules of the format. For example, putting FORM containers within the main FORM container is a no-no. This is normally done using a “CAT ” type instead of a FORM type, but for some reason Digita have decided to do things this way.
- There are 3 sub FORM types defined:
- DIAR: This contains all the calendar and alarm entries;
- TASK: This contains all the task list entries;
- ADDR: This contains all the address book entries.
- Each FORM type has one or more sub-chunks:
- EVNT: An event entry in the DIAR form;
- ALRM: An alarm reminder in the DIAR form;
- REPT: A special chunk containing info on repeating DIAR form entries;
- TASK: A task entry in the TASK form;
- ADDR: An address book entry in the ADDR form.
- All chunks have a small amount of binary data at the start of the chunk with chunk-specific information.
- EVNT and TASK chunks have a variable amount of data – the first byte contains the length of this section, including the length byte.
- ALRM, REPT & ADDR chunks have a fixed length binary field, 3 bytes for ADDR and 4 bytes for ALRM & REPT chunks.
I have created a small quick n dirty PHP script for decoding the file’s information based on the above details. It doesn’t do much other than present the information of the chunk types listed, converts the \0 characters used to separate fields into line breaks for readability, and displays the binary data from the chunks as hex bytes. The code and the example file can be downloaded here. Note that it doesn’t attempt to decode any of the binary information, though I’m sure they will translate relatively easily to the times, dates and other details relevant to the file entries. For example, the ALRM chunk only has 4 bytes which could simply be a 32-bit timestamp.
- Thanks, that is a great launchpad into getting the files read into some kind of string format I can work with!
- If you post up another example file, I could write a quick n dirty PHP code segment for splitting the file into strings. An online IFF decoder might actually be a handy thing to have…
- That would be amazing! I know some PHP so I could make use of it :) Here’s a new link http://pc.cd/9Cy7 There are calendar entries, tasks and an addressbook entry which will hopefully illustrate enough fields. From there I hope to allow users to keep their organiser file in a dropbox/google drive (now available I think) and access online. Read only for starters :)
- Had a bit of time this evening so had a more detailed look at the example file. I’ve edited my answer with some additional information, and uploaded an example PHP script that decodes the file. More work would be needed on it to allow it to accept a file from any source since this has obvious security implications. I have the script and file on my own server at the moment for testing. It won’t be permanent, but will give you an idea of its output: http://www.robthenerd.com/test/iffdecode.php
- This… is incredible, thank you! All the heavy lifting is done here. I’ve transferred it to a PHP web server on an A1200 (http://www.bambi-amiga.co.uk/amigatoday/iffdecode.php) but looks like I’ll need to run it on a more standard/compatible installation which is easy to do on an Ubuntu system. From here I can create more thoroughly populated save files and test more and make changes if necessary. Final step will be to decide on how access the Amiga users save file with no more that a one-off set up on their part. Hopefully Dropbox can help there. Fantastic work Daedalus – thank you again!
- You must login to post comments
Please login first to submit.