The MPQKit and Riven X projects are ready for Leopard. You should have no problem building them using Xcode 3 for all supported architectures. If anything does go wrong, please let me know.
A man’s soul.
I’m late on releases, I know. I had to do a lot of infrastructure work and debugging when I got back from Apple to have reliable Leopard and Tiger build bots for MPQKit and Riven X. All that is now done, which means right now I’m focusing on Riven X and wrapping up MPQKit Beta 1. The fact I have a build infrastructure in place also means I will be introducing nightlies very soon, starting with Riven X.
Stay tuned. This will be an interesting year, I promise.
Just a quick word on what is currently going on.
- My internship at Apple is soon coming to an end. It was an absolutely fantastic experience, which I will surely talk about some more in future entries, NDA permitting :p
- There will be a new MPQKit release sometime by the end of August. It will likely be the last beta release before the release candidate for 1.0, which will happen sometime in September.
- I have started intensive work on Riven X again, and there is a lot more to come for that project as well. There will also be a new binary release by the end of August. I plan on finishing this project before next summer (for real this time).
- StarCraft II will be awesome.
- OpenGL 3 (codenamed Longs Peak) will go final sometime in September or October. This is fantastic news for OpenGL, as LP brings some much needed fresh air to the API, particularly in the realm of object model and data / state transfer efficiency. It will be a while before we see LP implementations ship to customers, but at least the spec will be out there.
Much more coming in a week.
MPQKit 1.0b2 contains an astounding amount of bug fixes and improvements. The major highlights include quad architecture support (PowerPC, i386, PowerPC 64 and EM64T), a new strong digital signature public key for Starcraft maps and a much better mechanism to represent files pending addition.
Download MPQKit 1.0b2. More information on the project is available on the MPQKit project page and on the MPQKit development wiki.
Update: mpqfs 0.3 is available.
MPQFS 0.2 is now available. This release can load external listfiles, which is useful when an archive’s internal listfile is incomplete, resulting in missing files or “unknown xyz” files at the root of the archive (MPQKit generates those filenames for files it can open but whose filename is unknown).
You can specify any number of listfiles using the -l or –listfile options when invoking mpqfsd directly. In addition, mpqfs.app has bundled listfiles for Blizzard games. To load those listfiles, use the File > Open… menu item and check the checkbox.
Update: mpqfs 0.3 is available.
MPQFS (or MoPaQ Filesystem) is a FUSE filesystem that allows one to mount a MoPaQ archive and use regular system tools such as the Finder or standard UNIX programs to access the content of the archive.
Before you can use MPQFS, you will need to install (or compile) MacFUSE, the Mac OS X FUSE implementation written by Amit Singh (author of the excellent Mac OS X Internals: A Systems Approach, the reference outside of Apple on Mac OS X’s architecture). MPQFS expects FUSE (specifically libfuse) to be installed in /usr/local/lib, so for those of you who have installed FUSE through MacPorts or Fink, you will need to either recompile MPQFS or create a symbolic link.
Version 0.1 is read-only (sorry!), but I fully intend on introducing write capabilities in the future. You can mount as many archives as you want, even the same one multiple times. You may even mount MPQ archives stored inside mounted MPQ archives, something MPQKit doesn’t even handle yet.
If you get a crash of find something that doesn’t work right, I have created a new MPQFS component on the MPQKit development wiki, so make sure to report those in.
Finally, it should be noted that inside the application bundle (in Contents/MacOS), you will find mpqfsd, the program that actually implements MPQFS. mpqfsd can be invoked from the command-line (even from non-graphical sessions for all you SSH people) and provides a rich command-line interface to let you play with mount options.
MPQKit 1.0b1 is the culmination of many months of development. It fully supports version 0 (the original format limited to less than 4 GB) and version 1 (or the extended format not limited to 4 GB) MoPaQ archives, all known compression methods, all known file metadata attributes, weak and strong signature verification, extensive error handling capabilities with NSError variants of most methods, and an undo operation method. Still missing from this release are a number of performance optimization, the rename operation and the test suite.
Indeed, the next big step towards MPQKit 1.0 is the construction of an extensive test suite to make sure MPQKit never regresses in functionality or performance and to ensure it remains compliant with the MoPaQ specification. The test suite will include a large number of operation sequences, detailed archive structural checks, and a collection of good and bad MoPaQ archives utilizing all aspects of the specification. It is my hope not only MPQKit, but all third-party MoPaQ libraries, will benefit from the MPQKit test suite.
I’ve also taken the liberty to clean up the MPQKit Subversion repository, which now sports the standard trunk, branches and tags structure. Obviously, there is now a 1.0b1 tag for those wanting to compile from source. Of course, you can also checkout the trunk.
Finally, the documentation has been updated to reflect all the changes in the code. Not everything is done yet, but all the important constants, structures and methods are accurately documented.
Download MPQKit 1.0b1. More information on the project is available on the MPQKit project page and on the MPQKit development wiki.
Update 2008-12-02: For people who want to browse and extract specific files from MPQ archives, I recommend checking out MPQFS, available from the main page’s sidebar.
It feels strange to release a 1.0, but I feel it’s time for this program to get out the door as a 1.0. There are still a few things that could be better, but overall it’s doing what it’s supposed to do.
MPQ Extractor is a small program that will extract the content of MPQ (MoPaQ) archives. Multiple archives can be opened and processed at the same time, and the extraction process can be cancelled at any moment. If the program is launched by double-clicking an archive or a selection of archives, it will automatically quit once it has extracted that set of archives (and any archive added in-between).
Here’s a screenshot of how the main window looks:
Requires Mac OS X 10.4 or later. Universal Binary, of course.
So, let’s start talking a little bit about the returns on investment (there has to be some, right?) from WWDC ‘06. Specifically, for the people following Riven X, the issue where looping movies with sound have this extremely annoying gap in the sound output when they loop.
For anyone keeping scores, you know Cyan seemed to have used, how to say, deficient audio encoders in Riven. Well, after chatting with some QuickTime engineers on Friday morning, we determined that the sound tracks in those movies are in fact too short, thus causing the gap during playback.
I had never previously considered that possibility, focussing instead on potential problems in my rendering code. After all, it was far more likely that I had screwed up rather than Cyan…
I haven’t done the DSP / analysis to determine if the track discrepancy is constant, proportional or random, but I suspect it will either be constant or related to the movie’s timescale (remember that QuickTime is not sample based but time based). Hopefully I should be able to address this issue sometime in October.