Setting Priorities

Joel (on Software) Spolsky has a new article that's really a winner. It's a combination of why he does shrinkwrap software, how to set priorities (in life, in general), and a way to pick what to put in the next revision of your software.

Good stuff.

How To Do DB Testing (Part III, Mock Database)

We decided to use Dia to draw the ERD, and to use tedia2sql.pl (available at tigris.org) to pull the data out of the .dia file and generate DDL. In the case of a new table, I'll cut the new DDL out of the generated file and paste it into psql to actually create the table, then run a database diff utility that I wrote (that I already posted to this blog) to make sure what's in the DB is what's in the ERD. (That way, either I or Boss can make quick changes right to the dev DB, and I can make sure they get back to the ERD when things slow down.)

Another good thing about tedia2sql is that, since it's F/OSS, our resident Perl guru could (and did) modify it to spit out text that our internal tool needs, and that another tool (that I wrote) could generate Java classes off of the data model. Kind of like a poor man's Hibernate, perhaps. Once we had working data objects, it was a simple extension to make each of those objects implement a generated interface, and then generate mock objects that don't actually talk to the database, but rather expect to receive one of those Vectors I mentioned earlier.

So now any change to the database automatically updates the real Java objects, and the mock objects that implement the same interface, and a class factory to get a new one as needed. From there, writing Junit tests are the snap you'd expect.

How To Do DB Testing (Part II, Bulk Analyze)

So with all of the SQL expressions contained in query manager subclasses, the next step was to collect and run all of the queries at once. This had two big advantages.

First, all of the SQL could be exercised. Probably the biggest headache in writing SQL is that there's no real way to "compile" it, even to get a syntax sanity check, (without going into stored procedures and other DB-specific stuff). Our way, we have a hierarchy of classes that contain all of the SQL strings, and a TestAllQueries JUnit class that calls each method on each query manager class. Running one test runs all of the queries in the system. (It ignores the results; that's for other tests.)

Second, with a tiny tweak in how the methods are called, instead of running the query we can 'EXPLAIN ANALYZE' it, and run the output into a text file for further examination. One run, and I can see what queries are doing table scans, and what index that one query is using. Quite handy.

And with all of the application code not knowing anything about JDBC, but rather using an Iterator, we can put together a mock database fairly easily, just by instantiating and throwing HashMaps into a Vector, and returning the iterator() from it. Acquiring the database from a factory means that each db-using class is now easily testable without having to setup a separate database and worry about the attending headaches.

How To Do DB Testing (Part I, General Ideas)

Not that I expect this to be a tome of wisdom or anything, but I thought I might put into writing some of the wins we've had lately in being able to test our DB code. First, some background.

I got to design most of the DB layer of a web app. To prototype, I threw together a simple JDBC wrapper class to manage a connection and perform queries. This worked well for starters, but soon the number of queries got unmanageable. So we split out the connection stuff into a base class, and split the queries themselves into various derived classes. This is nice, as it means we can find all of the queries for the whole system in one easily spotted set of source file, and those source files have nothing else in them.

The other thing we did that works really well is to have the query manager class act like an Iterator. The big win here is that next() returns a HashMap<String,String> that contains the data from each row, and as a result, nobody anywhere in the whole system deals with JDBC classes outside of the query manager.

Here's the skeleton:

class QueryManager
{
   protected Iterator<HashMap<String,String>> executeQuery( String sql ) { ... }
}

class SpecificQueryManager extends QueryManager
{
   public Iterator<HashMap<String,String>> getPeople() { executeQuery( "select * from people where ..." ); }
}

Blogspot.com Stats

It's nice to get feedback on what you write in a blog. I mentioned getting comments; it's nice, too, to be able to look at your server logs and see who's reading what, and where they came from, and so on. But as my blogs are primarily hosted on Blogspot.com, and given that they don't share their server logs (harumph), what's a poor geek to do?

Well, images to the rescue. Turns out, you can put a link in your blog template to an image file that you host on your own server, and any time anyone in an actual browser (i.e. not a webcrawling 'bot) views your page, your image gets loaded, and you can see what page the reader was on at the time. Don't know why I didn't think of that sooner.

Now, if they'd just share my own stats with me...

Tomcat Deploy on Restart

It seems that Tomcat has a deployment problem. If you stop Tomcat, copy a new .war file to ./webapps, and start Tomcat, it won't pick up the new file and deploy it. I'm not certain about this, but it also seems that even if the file is placed there right when Tomcat finishes startup, it might not be picked up.

In my case, I was doing a full build and restart after I grabbed new files from CVS. A coworker suggested that the best way to do this would be to grab new, rebuild, deploy, let Tomcat catch up, and then restart Tomcat (especially if there are changes that would require a restart, such as a new JNI shared/dynamic library).