I wrote this quick shell script to loop over the file and do a recursive grep for that key. The key is considered “used” if it exists in the current directory with quotes around it (single or double). This WILL NOT find keys if they are only used programmatically. (e.g. VAR + “.started”)

#!/bin/sh

# Loops over all the keys in the given messages.properties file
# and looks in the current directory for the string in quotes.
# Results are printed to stdout
#
# use: (from the “web” dir in openmrs)
# sh ./findunusedmessages.sh WEB-INF/messages.properties

propfile=$1
props=`awk -F= ‘{print $1}’ $propfile`
count=0
total=0

echo “These property keys were not found in the code base:”
for prop in $props
do
#echo “Looking for property: $prop”
if [ ! -z "$prop" ];
then
total=`expr $total + 1`
output=`grep –exclude-dir=”.svn” -re “[\'\"]$prop[\'\"]” *`
if [ -z "$output" ];
then
echo “$prop”
count=`expr $count + 1`
fi
fi
done

echo “done! Found $count possibly uneeded message keys in $propfile (out of $total total keys)”

Download the file here: findunusedmessages.sh

I changed the jdbc connection url string to set a session property that sets InnoDB as the default engine:

jdbc:mysql://localhost:3306/databasename?autoReconnect=true&sessionVariables=storage_engine=InnoDB

NOTE! Requires mysql jdbc connecter version 3.1.18 or greater. See http://dev.mysql.com/doc/refman/5.0/en/connector-j.html for documentation and then http://dev.mysql.com/downloads/connector/j/ to download.

The error message I was getting before upgrading the mysql jdbc connector was:

java.sql.SQLException: Server connection failure during transaction. Due to underlying exception: ‘java.sql.SQLException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ” at line 1′.

** BEGIN NESTED EXCEPTION **

java.sql.SQLException
MESSAGE: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ” at line 1

STACKTRACE:

java.sql.SQLException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ” at line 1
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:2921)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1570)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:1665)
at com.mysql.jdbc.Connection.execSQL(Connection.java:2972)
at com.mysql.jdbc.Connection.execSQL(Connection.java:2902)
at com.mysql.jdbc.Statement.executeUpdate(Statement.java:929)
at com.mysql.jdbc.Connection.setSessionVariables(Connection.java:5077)
at com.mysql.jdbc.Connection.initializePropsFromServer(Connection.java:3680)
at com.mysql.jdbc.Connection.createNewIO(Connection.java:2684)
at com.mysql.jdbc.Connection.<init>(Connection.java:1474)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:264)
at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:135)
at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:182)
at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:171)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:137)
at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1014)
at com.mchange.v2.resourcepool.BasicResourcePool.access$800(BasicResourcePool.java:32)
at com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask.run(BasicResourcePool.java:1810)
at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:547)

I was lucky that I have control over the server and could change the http/https to a simply proxy instead of redirecting.  If I didn’t, I would have had to catch the protocol change and refetch the new url.

It took me a while to find an answer for this because these keywords didn’t give many results on google: URLConnection HttpURLConnection openConnection() connect() getInputStream() Apache redirectMatch 301 302.

I really should have jumped to that last one first.  Googling for my second thought gave me nothing useful.  However, simply adding “eclipse” to my search term brought up the Metrics plugin as the first result!  The Metrics plugin counts all different types of things.  Lucky for me, “number of methods” just happens to be one of them.

To get the counts working for you, follow these easy steps:

1. Add http://metrics.sourceforge.net/update to eclipse as a plugin update site
2. Turn metrics on for your project:
   1. Right click on the root of your project in Package Explorer view
   2. Choose Properties–>Metrics–>Enable Metrics
3. Do an Eclipse clean rebuild
4. Click on the root of your project in the Package Explorer
5. Open the “Metrics” view

These are the relevant statistics from the OpenMRS project:

The second api/web are from the test package.  We have a lot of work to do to catch our total number of unit test methods up with our number of api methods.

I’ve also uploaded the entire OpenMRS metrics output.

Sometime last year Burke laid out his dream BIRT (or other reporting frameworks) connection. It entailed writing a custom jdbc connector that didn’t act on the database, but instead acted on our API. The “available tables” as the user would see it in BIRT would actually be a subset of our higher level Java objects.

In the session Mark discussed the new intercept-ability of the MySQL Connector/J (new in version 5.1ga of the connector). The relevant/interesting addition to the connector is the “statement interceptor”. We could use this to intercept the select calls from BIRT that look like “SELECT AGE, GENDER, WEIGHT, HEIGHT, LATEST CD4 FROM PATIENT WHERE DATE > 1/1/2008″. These calls could be turned into some LogicService calls on AGE, WEIGHT, LATEST CD4, etc to get the ResultSet instead of passing that (unusable) query to mysql. The LogicService’s resultSet could then be returned to BIRT for display as a data set. There isn’t any documentation on MySQL Forge on how this would work yet. Mark’s slides should show up as an option on the mysqlconf’s wiki soon though.

Side note: The jdbc driver’s version number does not match with the mysql version numbers. Version 5.1 of the driver can be used with the current 5.0 of MySQL.

Without that key piece of text, the service and its methods will still work. However, they’ll work in a readonly kind of way. All data written to the database will be rolled back when the body of work has completed. This little error manifested itself recently after our reporting code-a-thon. During the event we had to create a new API service, which is only done maybe a few times a year. The service didn’t get the @Transactional tag and so therefore all database editing failed that went through the ReportService. The unit tests set up for the object passed, but that is because a unit test happens all within a single transaction — the rollback is always done after the completion of the test anyway.

I added a note to http://openmrs.org/wiki/OpenMRS_API about the requirement, but I’m sure there will be an OpenMRS developer in the future struggling with their object not saving or updating in the database and be completely baffled by it.

Failed to read schema document 'http://www.springframework.org/schema/beans/spring-beans.xsd'

This is somewhat expected with the move from Spring 2.0 to 2.5. As of Spring Framework 2.5 the new url to use in the application context is

http://www.springframework.org/schema/beans/spring-beans-2.5.xsd

but even after making this change in the code and redeploying the webapp, Tomcat refused to start giving the same url (with just spring-beans.xsd).

Solution:

A Spring definition file (namely applicationContext-service.xml) was being cached in Tomcat’s work directory and not being replaced on webapp deploy for some reason. I uninstalled the webapp and then deleted all files under <tomcathome>/work/localhost. Upon installing openmrs and starting Tomcat again everything started up as expected.

No supported regular expression matcher found

I made sure I had jdk 1.6 installed. I tried upgrading to Ant 1.7 (from 1.6.5).

I ran an “ant -diagnostics”. It wasn’t showing the optional ant jars so I dropped all of the jars into the ant home lib directory. Now the jars were showing up but I was still getting the same error as before.

The solution ended up being that Ant lied to me. It was actually looking for the jars in /usr/share/java/ant (not in its reported /usr/share/ant). The puzzling thing is that Ant tells me it finds the jars when running “ant -diagnostics”.

I ended up installing the optional ant jars via yum. (or apt-get if you’re on a debian flavor) Yum didn’t have the explicit ant-optional package, so I just installed all ant-*.jar files. Voila! Successful building via ant is now possible.

A very huge find that came out of this was that we were “leaking” Tomcat sessions. In Eldoret we have about 30 data assistants right now that are filling in Infopath forms for about 8 hours a day. The average form takes about 5 minutes to fill out, so they are opening and closing Infopath fairly often. When opened, the taskpane on the side of Infopath is a mini IE browser. The first action taken by that taskpane browser is a stealing of the user’s session from their other web browser. The original session is left hanging around in Tomcat though. After 30 minutes of inactivity that session is dropped. Apparently, in our setup at least, either Tomcat or Apache has somewhere around a 250 limit on the number of sessions. With the DAs entering forms at a rapid pace, it would only take a few hours before the limit was reached — at which point Apache/Tomcat halted all traffic until some sessions were freed by timing out or Tomcat was restarted.

While trying to find the real cause of the problem, my first solution was to shorten the length of the session timeout. In <Tomcat Home>/conf/web.xml:

<session-config>
<session-timeout>15</session-timeout>
</session-config>

this obviously only lengthened the time between restarts.

The true solution came in the form of a beautiful one line fix in the formentry module: when initially loading the taskpane (before the session stealing happens) set the timeout of the current session to 10 seconds. Voila! Success! Our Eldoret server is now much more “dependable”!

Now, as far as the memory leak goes, I’d love to have some input from people who might have a better idea of what they’re doing. At times I felt as though I were just clicking randomly through the profiler looking for anything that had ‘org.openmrs…’ in the package name. If you have any pointers to give me or you have found a few memory leaks yourself, let me know!