Sometimes, dirty hacks just do it
We used to have this problem with Apache FOP using loads of memory while processing documents containing too many SVG graphics. Until now, it was only an annoyance. We could let the rendering run over the lunch time or overnight. If we needed the output fast, the illustrator’s dual Xeon did the job without too much trouble. Simply allow 1.5 gig of memory to the virtual machine and the problem was solved.
That was until the amount of illustrations doubled and their complexity as well. It got to the point where the processing would die after eating up nearly 2 gigs of ram. The obvious solution was to buy more ram, the other one was to look deeper into FOP’s code to find the problem. Not having any time to waste, we obviously opted for both options: looking into the code until the additional ram arrived. Luckly, it didn’t take too long before the problem was solved. While Apache Batik, the SVG renderer, is actually eating up most of the memory, the problem was in the way FOP would handle the memory.
I used version 0.20.5, which is the current stable release. I could have tryed the new beta version, but since a large part of the code has been rewritten and XSL-FO is sensitive by nature, I didn’t feel like having to go back and play in the XML file to fix formatting issues. These memory issues are probably going to be fixed in this new release. The details of the fix will be given in the rest of the post.
This code hacking adventure really demontrated how useful HyperThreading could be. The main problem you have when a process dies of memory shortage after 20-40 minutes of processing is really that you actually have to wait that long to test if the fix actually worked. On a single CPU system, waiting that long could be considered a waste of time, since no more ressources are actually left to perform any work. But with HyperThreading, which for some unknown reason Dell turns off by default (which caused quite a waste of my time), it’s possible to actually code on some other projects while the document is rendering, without having any of the processes affected in speed… or at least… not in a significant way.
I’ll also have to take off all those bad things I said about Eclipse in the past. It’s quite a good environment to develop Java in, because Java is one of those languages that requires a complete IDE to be any usable. I could navigate in FOP’s 26K lines of code with ease. Thanks to that search function that scans for method signatures. I also found a nice metrics plugin that gives a bunch of useful information about the code. On the other hand, I was surprised to see Data Tools does not seem to work with MySQL (on stable release for Eclipse 3.1).
The fix to keep memory usage to an acceptable level in Apache FOP was actually quite simple. Most of it was about killing unused references, and while I’m not too certain it was really required, being very explicit to the virtual machine that cleaning up memory after image rendering could be a good moment. The fix is probably slowing down the processing as it totally removes any form of cache Apache FOP used. In my case, other than the bullet-like images, which are processed in less than half a second, all the images are only used once. But in the end, since less memory is used, there is less swapping and it’s probably faster anyway.
First step was to remove the reference caching in org.apache.fop.image.ImageFactory. The factory is called every time a graphic is encountered. It associates the URL to the image objects it creates in a HashMap. With a thousand graphics used a single time, it’s not that good, but it’s not that bad since it only keeps references to stream. The actual image processing is done at the end of the pageblocks. I used commented out every single line related to that HashMap.
Second step was to make sure the references to the SVG documents were killed after rendering. In org.apache.fop.render.pdf.PDFRenderer, I first made a modification to the renderSVGDocument method. I can’t seem to remember if I removed them afterwards, but I don’t think it had anything to do with the final results. You might want to kill a few references in there as well if the problem persists.
The modification that really made a difference was in org.apache.fop.image.ImageArea. All I did was set the image member variable to null and called System.gc() (I know, it’s not really clean) after the call to renderImageArea() in the render() method. Without the first modification, this would cause problems as there could be multiple references to the same image in the document, but I didn’t bother testing that option as the problem was solved.
The process that used to die after 2 gigs of memory usage went up from 250mb to 450mb over 40 minutes of processing, which I consider stable enough.