I made a second crucial memory leak discovery on Friday. Apparently, the default JSP compiler for Tomcat, Jasper, has a “bug” that doesn’t free memory with objects used in a ForEach or Set taglib (found via the always helpful webapp memory leak page).
Both of these tags are used judiciously on the patient dashboard and other pages throughout OpenMRS. As the day progressed, the 40+ users we have in Eldoret would gradually visit more and more pages with different data displayed. The foreach tags that looped over the encounters, obs, patients, etc, wouldn’t release those objects. Eventually, enough of our database would be kept in Tomcat’s memory to cause an Out of Memory (OOM) error — usually just manifesting as a confusing PermGen error in the log files.
The fix outlined on that tomcat bug report page is actually quite simple. We simply have to tell tomcat not to “pool” taglibs and magically the memory leak is gone. In <TOMCAT HOME>/conf/web.xml file, in the jsp servlet definition add the following element:
<init-param>
<param-name>enablePooling</param-name>
<param-value>false</param-value>
</init-param>
After changing this, OpenMRS has successfully been up for a full day of data entry for the first time in a few months!
If your installation of OpenMRS seems unstable or slows down after a few hours of use, I would suggest trying this fix. The memory/cpu trade-off that occurs from not having taglib pooling is definitely in your favor.
I’m beginning to feel like a paid blogger for yourkit, but I really must thank YourKit again for the their license donations. Their newest 7.0 release helped me track down the source of this latest bug with relative ease.
