MOHIE Planning

I'm headed to Jefferson City, to attend Missouri's HIE Planning Technical Architecture Committee tomorrow. This is an exciting chance to change healthcare in Missouri.

However, I'm worried that the group will be filled with lobbyists and vendors trying to get their piece of the pie. I would rather build something and give it away to Missouri and any other state which wants it. I think its time for an inclusive, open source HIE, based upon NHIN CONNECT.

Wish me luck

MySQL5.1 on Debian Installation

I've been having issues installing MySQL5.1 on Debian, for the NHIN Connect port to Linux.

First I tried downloading the tarball and following the installation instructions in their INSTALL-BINARY file. One thing which needs modification is substituting useradd for adduser.

The problem, though was the installation script

script/mysql_install_db

was failing, as it could not find files. After messing with this for some time, I decided to delete everything, start over, and use the package manager:

aptitude install mysql-server

However this installed an instance of mysql without a root or mysql user. I could never authenticate against it.

Back to the tar - I'm now working on script/mysql_install_db

root>cd /usr/local/mysql
root>./scripts/mysql_install_db

FATAL ERROR: Could not find errmsg.sys

The following directories were searched:

/usr/share/english
/usr/share/mysql/english

If you compiled from source, you need to run 'make install' to
copy the software into the correct location ready for operation.

If you are using a binary release, you must either be at the top
level of the extracted archive, or pass the --basedir option
pointing to that location.

Looks as though the "you must be either at the top level of the extracted archive" part does not work, so I'll specify basedir:

root>./scripts/mysql_install_db --basedir=.
Installing MySQL system tables...
091015 9:34:03 [ERROR] ./bin/mysqld: unknown option '--skip-bdb'
091015 9:34:03 [ERROR] Aborting

091015 9:34:03 [Warning] Forcing shutdown of 2 plugins
091015 9:34:03 [Note] ./bin/mysqld: Shutdown complete


Installation of system tables failed! Examine the logs in
/var/lib/mysql for more information.

You can try to start the mysqld daemon with:

shell> ./bin/mysqld --skip-grant &

and use the command line tool ./bin/mysql
to connect to the mysql database and look at the grant tables:

shell> ./bin/mysql -u root mysql
mysql> show tables

Try 'mysqld --help' if you have problems with paths. Using --log
gives you a log in /var/lib/mysql that may be helpful.

The latest information about MySQL is available on the web at
http://www.mysql.com/. Please consult the MySQL manual section
'Problems running mysql_install_db', and the manual section that
describes problems on your OS. Another information source are the
MySQL email archives available at http://lists.mysql.com/.

Please check all of the above before mailing us! And remember, if
you do mail us, you MUST use the ./scripts/mysqlbug script!

Ok, two oddities here.

1) I didn't specify --skip-bdb
2) /var/lib/mysql has no logs

The "Installation of mysql tables" message is from the script, the rest is from mysqld.

I modified the script to output its options and found these:

./bin/mysqld --language=./share/english --bootstrap --basedir=. --datadir=/var/lib/mysql --log-warnings=0 --loose-skip-innodb --loose-skip-ndbcluster --user=mysql --max_allowed_packet=8M --default-storage-engine=myisam --net_buffer_length=16K

So where is --skip-bdb coming from. After investigating, I found a global configuration file, my.cnf, which can specify options. It can be found in a number of places:

/etc/my.cnf
/etc/mysql/my.cnf
SYSCONFDIR/my.cnf
$MYSQL_HOME/my.cnf
defaults-extra-file
~/.my.cnf

Found one in /etc/mysql

#
# * BerkeleyDB
#
# Using BerkeleyDB is now discouraged as its support will cease in 5.1.12.
skip-bdb
#

So I commented out skip-bdb, and the script worked.

I finished out the installation instructions and started mysqld using

mysqld_safe --user=mysql

but it exited right away.

Nothing useful in the log, so I added the -log option:

091015 10:23:20 [Warning] The syntax '--log' is deprecated and will be removed in MySQL 7.0. Please use '--general_log'/'--general_log_file' instead.
091015 10:23:20 [ERROR] Can't find messagefile '/usr/share/mysql/english/errmsg.sys'

So I added a symbolic link to the missing dir

ln -s /usr/local/mysql/share /usr/share/mysql

restarted, and its running now. Wahoo!

Debian java confusion

On my Debian laptop, I've been having several java apps hang up on me and need to be killed. Primarily, these have been UI apps, like MagicDraw and UI installers, Like Metro 1.4, which I needed for NHIN Connect.

Looking into this, I found two versions of java installed, the 1.5 Gnu Compiler for java, and the 1.6.0 JDK from sun.

/usr/lib/jvm$ ls
java-1.5.0-gcj-4.3-1.5.0.0 java-6-sun java-6-sun-1.6.0.12 java-gcj

So which am I running?

/usr/lib/jvm$ which java
/usr/bin/java

Which is a symbolic link to /etc/alternatives/java. Looking further, looks as though /etc/alternatives is pointing to a mix of 1.5gcj/1.6jdk. This is somewhat less than ideal.

/etc/alternatives$ ls -lgo ja* | grep gcj | cut -c34-
jar -> /usr/lib/jvm/java-gcj/jre/bin/jar
jar.1.gz -> /usr/lib/jvm/java-gcj/man/man1/jar.1.gz
java -> /usr/lib/jvm/java-gcj/jre/bin/java

jeff@oci1412:/etc/alternatives$ ls -lgo ja* | grep sun | cut -c34-
jarsigner -> /usr/lib/jvm/java-6-sun/bin/jarsigner
jarsigner.1.gz -> /usr/lib/jvm/java-6-sun/man/man1/jarsigner.1.gz
javac -> /usr/lib/jvm/java-6-sun/bin/javac
javac.1.gz -> /usr/lib/jvm/java-6-sun/man/man1/javac.1.gz
javadoc -> /usr/lib/jvm/java-6-sun/bin/javadoc
javadoc.1.gz -> /usr/lib/jvm/java-6-sun/man/man1/javadoc.1.gz
javah -> /usr/lib/jvm/java-6-sun/bin/javah
javah.1.gz -> /usr/lib/jvm/java-6-sun/man/man1/javah.1.gz
javap -> /usr/lib/jvm/java-6-sun/bin/javap
javap.1.gz -> /usr/lib/jvm/java-6-sun/man/man1/javap.1.gz
java-rmi.cgi -> /usr/lib/jvm/java-6-sun/bin/java-rmi.cgi
java_vm -> /usr/lib/jvm/java-6-sun/jre/bin/java_vm
javaws -> /usr/lib/jvm/java-6-sun/jre/bin/javaws
javaws.1.gz -> /usr/lib/jvm/java-6-sun/jre/man/man1/javaws.1.gz

Easy enough to (attempt to) fix:
/etc/alternatives$ sudo rm jar
/etc/alternatives$ sudo ln -s /usr/lib/jvm/java-6-sun/bin/jar ./jar
/etc/alternatives$ sudo rm java
/etc/alternatives$ sudo ln -s /usr/lib/jvm/java-6-sun/bin/java ./java
/etc/alternatives$ sudo rm jar.1.gz
/etc/alternatives$ sudo ln -s /usr/lib/jvm/java-6-sun/man/man1/jar.1.gz

And verify:
/etc/alternatives$ ls -lgo ja* | grep gcj | cut -c34-

Problem Solved!

Patching SVN: Hunk Failed Happens

Hunks Are Failing...

I see everywhere how to patch working copies using the patch command, with an input of a file created using svn diff. As it goes:

svn diff > iamtheschmitzer.patch

then in a different working copy, apply the patch

patch -p0 < iamtheschmitzer.patch

Not so fast. Getting this error:

Hunk #1 FAILED at 2.

I submitted some rather large patches to Connect yesterday, and they appear to be useless. Every hunk is failing.

I looked into the problem, and patch with -p0 does not seem to work well. I did get patches created for -p1 to work properly.

What am I talking about? A unified diff file gives a patch to the file in question. Patch allows the user to specify how many levels of that directory structure to ignore, with the -p argument. -p0 hasn't worked for me (Debian).

So if your working copy is C:\Projects\NHINC\Current\Product

you should start in C:\Projects\NHINC\Current

and issue the command:

svn diff Product > GATEWAY-17.patch

and it should be applied with -p1, but still from within the working directory

patch -p1 < GATEWAY-17.patch

Finally, patch does not like DOS-style CRLF line endings. Both the patch file and the source file will need to be in unix format to work properly. This whole mess is scripted like this:

# assumes a -p1 patch
PATCH=$1

# patch does not like Windows newlines
echo "Transforming $PATCH to unix"
dos2unix $PATCH


# patch does not like Windows newlines
# modify the target files to unix
for file in `grep Index: $PATCH | sed 's/Index: [^/]*\///'`
do
  echo $file
  echo "Transforming $file to unix"
  dos2unix $file
done

echo "Applying patch file"
patch --verbose -p1 < $PATCH

# patch does not like Windows newlines
# modify the target files to back to Windows
for file in `grep Index: $PATCH | sed 's/Index: [^/]*\///'`
do
  echo "Transforming $file to dos"
  unix2dos $file
done

Patching Existing Patch Files

Oh lucky you. You have existing patch files, which you can't apply. It's easy to fix with a simple sed script. The file names must have one level of directory structure prepended to them.

Here is a sed script to do it for you:

s/^\(Index:\) /\1 ignoreme\//
s/^\(---\) /\1 ignoreme\//
s/^\(+++\) /\1 ignoreme\//

Save this in a file like fixpatch.sed and run this with

cat mybroken.patch | sed -f fixpatch.sed > myworking.patch

Then apply myworking.patch with patch -p1.

And if this post helped you, leave a comment, link to my blog, or thank @iamtheschmitzer on Twitter.

How NHIN Connect Ant Scripts Work

global build.xml task default
- depends on build
global build.xml task build
- calls do-to-projects(do-to-target => build)
global build.xml task do-to-projects
- parses xml file build.projects.xml
- for each /projects/project
- determines project directory by directory element
- invokes ant on project's build.xml for target build (or whatever do-to-target is set to)

project build.xml task build
- sets property target-to-call to jar
- calls call-target-in-isolation-if-out-of-date

build.lib.xml task call-target-in-isolation-if-out-of-date (imported from project)
- Creates xml file named temp.xml
- writes ant script to temp.xml
- invokes ant on temp.xml target temp

temp.xml target temp
- sets property project-to-build to .
- preserves property target-to-call
- sets property base to ./lib
- imports property file ./lib/nblibraries.properties
- imports property file ./nbproject/project.properties
- calls build-project-if-out-of-date

build.lib.xml target build-project-if-out-of-date
- Checks classpath
- If need to build, invokes ant on project's build.xml target jar (or target-to-call)

project's build-impl.xml target jar
- depends on compile

project's build-impl.xml target compile
- depends on -do-compile

project's build-impl.xml target -do-compile

Connect Code-A-Thon Photos

Add your Code-a-thon photos to flickr with tag of nhinc

Building the Connect Community

On August 27th, I attended the Connect Code-a-Thon in Washington D.C. Connect is an project originally designed to hook up 20 federal agencies to the NHIN, or National Health Information Network. The leadership recognized it had much broader applications, and so released the sources to the public earlier this year.

Like our other open-source products, my company, Object Computing, Inc. has been investing in Connect for a while now, building testing, providing feedback to the team and helping others in the community do the same. We were excited to hear about Code-a-Thon, but were lucky to get in. The 100 seats filled up fast.

One thing which happens frequently in open source products is a broadening of the solution to a larger problem set. Connect was fitting this model. While looking at the sources I realized it was currently tied to a specific directory. In this case it was C:\Projects\NHINC. Other paths were assumed as well. I found this to be unusable for the community, I proposed a project, called Path Neutrality, to allow the code to be checked out anywhere, deployed anywhere, tested anywhere. I posted my idea to the Connect forums and asked for those interested to respond.

I waited to hear from the community, but did not get a reply. The community manager, Brian Behlendorf, replied promptly that it was a well-written project and it made technical sense. Yet, the community was strangely silent as Code-a-Thon approached.

Another announcement about Code-a-Thon indicated there would be 10 tables - one for each of the project areas - including one for Path Neutrality. This was great support, but I still worried how embarrassing it would be to sit alone at that table. Still, the acceptance of this project, even into the agenda of the event, made me know, that community was important to the Connect team.

The day arrived, and I entered the Hubert H. Humphrey Health and Human Services Building which I called H5SB, met some of the staff and a few developers. I tried selling another developer who was interested in a Linux version on the idea of Path Neutrality. We found the table, and there were already two other people there. What a relief.

The event started and Brian introduced the topics at the various tables and asked people to indicate their interest. I was stunned that the table was, in fact popular. Path Neutrality, naturally, was critical to getting Connect running on Linux, and the Linux contingent had arrived. Also included at the Path Neutrality table were discussions on containers neutrality, database neutrality, and even EJB-independence. The Social Security Administration, who used Connect on Solaris, showed a lot of interest, as did Connect's Chief Architect.

We worked on some of the key issues surrounding path-neutrality and prototyped some possible solutions. Jason from the development team joined us for most of the day. There was a real sense of a can-do attitude in the community. It was great to be part of these passionate developers wanting to broaden the application of this product.

With all the buzz around our table, a reporter from GovHealthIT stopped by and we spoke for a while, and she wrote up the project in an article.

If you have an interest in helping out, check out the Connect project. This community is a very welcoming one. If you have Java web services experience or health IT domain knowledge, this open source project can use your help. Release 2.2 is coming soon, and quarterly Code-a-Thons are planned. California, Atlanta, and St. Louis are possible future sites. I hope I have the time to help keep momentum of Path Neutrality going.