Dave's Code Blog

HP Laptop Headphone Fix

2011-07-05T18:09:00.000-04:00

Love my HP Pavilion DV6 laptop, and love Ubuntu running on it, but found an issue with my headphone, that plugging into the jack didn't turn off the speakers. Did some poking around, and found that adding:

options snd-hda-intel model=hp-dv5

to my /etc/modprobe.d/alsa-base.conf took care of it. There are other values for model that can solve issues with other machines. For reference, lspci reports:

00:1b.0 Audio device: Intel Corporation 5 Series/3400 Series Chipset High Definition Audio (rev 05)

for my audio card, and cat /proc/asound/card0/codec* | grep Codec reports:

Codec: IDT 92HD75B3X5
Codec: Intel IbexPeak HDMI

for my codec(s).

SQL Server 2000 notes

2010-11-23T15:53:00.002-05:00

A place for stuff that I don't want to forget. (There's stuff I only do with it once a year, and I'm perpetually looking this stuff back up.)

1) bcp.exe will let you dump a table at a time to a CSV file. DTS absolutely sucks for the same task, as it's a stinkin' wizard, that doesn't remember what you did last time, etc., etc., blah blah freakin' blah. You can at least script bcp with a .bat file, and have a record of what you did, for next time.

Ubuntu 10.10 Upgrade

2010-11-18T13:12:00.002-05:00

SpongeBob (my laptop): 10.04 -> 10.10

Done, no issues yet, love the new font.

Patrick (downstairs desktop): 9.10 -> 10.04 -> 10.10

9.10 to 10.4 went smoothly.

10.4 to 10.10 wouldn't bring up X. Poked around on forums, found "no screens found" message in /var/log/Xorg.0.log, moved xorg.conf out of /etc/X11, did "restart gdm" and it worked. Rebooted, and it worked.

MrCrabs (Son's laptop): 9.10 -> 10.04 -> 10.10

Done, no issues yet.

(work laptop): 10.04 -> 10.10

Screen started working again, wireless works on reboot (both were problems with 10.04). alsamixer still doesn't retail settings over a reboot (which it hasn't done since at least 9.10, maybe 9.04).

New problems:

1) Audio cuts out in just a few minutes of use.

2) CPU usage is up. Looking at top, appears to be processes called kslowd000 and kslowd001. Googled them, and apparently they're kernel daemons used for background processing (so as not to interfere with main CPU processing?). Looking further, this might be related to my GPU, as others with kslowd issues were also using an Intel 855GM.

Decided to reboot; didn't fix #2. Decided to boot into 2.6.32-25 (was running 2.6.35, which is apparently when the kslowd problems started, at least for Ted T'so on the kernel mailing list). Not only did kslowd stop showing up in top, but now problem #1, with the audit, seems to be fixed as well (quite possibly unrelated to #2, except by differing code between the two kernels).

So, for now, will stay in 2.6.32, as it's nice to not need to schlep MP3's over to my phone to listen to music at work. Will try out 2.6.36 when it comes down the pipe, probably.

Update: went up to 2.6.35-23-generic, and it seems to have improved both issues. Will keep this updated, but for now, I'm back up to 2.6.35.

Another update: while my audio and CPU usage issues were better, laptop was locked up with a blank screen for the past two mornings when I came into work. So I'm back to 2.6.32 for now.

(work desktop): 10.04 -> 10.10

Done, no issues yet. Well, except for that odd thing with my 2nd HD coming up randomly as /dev/sda and /dev/sdc. Fixed it by putting the drive's UUID in /etc/fstab.

Bind for Logging

2010-08-20T08:00:00.001-04:00

Talked to Rob and Nate, and both agree that to log internal web requests, setting up my own bind is the easy way to go, then set the router to set all internal machines to it via static dns on dhcp.

Installing was a snap (straight out of Synaptic), and it just worked, right out of the box. Logging wasn't turned on, but that took 5 mins. Don't need logrotate, as bind9 logging has that built in. Manually, I wrote a cut/sort/uniq/grep script, but I didn't have mail installed, (or the brainpower left at end of day), so the script will have to wait briefly.

I set the router's static DNS to this box, and by this morning, was getting requests from the XBox and Wii, as well as Wife's Macbook.

I added an entry to my crontab to generate the filtered list, putting it into ~/bin (for no real good reason). Given the size of it, I'll probably generate it nightly, but only mail it out weekly, and just poke around with adding things to the grep filter.

Installed first mailutils, then exim4, then mutt, then finally sendemail until the latter let me send email from the command line simply, without configuring an MTA. Configuring an MTA just isn't something I have the brainpower for tonight. Anyway, my ISP doesn't require authentication from within their network, so sendemail worked like a charm, after bzipping the file down, (apparently they have a size limit, or something?)

So crontab is setup to mail the log out weekly, and we'll see how it goes.

Kicking Ivy out of Eclipse

2010-08-19T08:00:00.002-04:00

To make Ivy-less operation easier, I:

Removed all of the jars from my development project in Eclipse
Created a directory (outside of the project)
Made symbolic links from that directory to my common jar path (i.e. "ln -s /usr/share/java/spring.jar ~/dist/project", etc.)
Added all of the jars to my Eclipse project in one operation (by clicking Add External JARs and highlighting all of the jars in that dir)
Scripted the creation of the symlinks, and added the script to SVN.

It might also be worth looking into how SVN handles symbolic links, which could make it feasible to add the symlinks into the projects as well.

2010-08-18T16:22:00.001-04:00

Fixing a hosed GRUB

Full details (and years of old fixes) at: http://ubuntuforums.org/archive/index.php/t-24113.html

Ubuntu Karmic:

make changes to partitions locations below
boot from cd/usb/network/yomomma
open terminal

type:

sudo mkdir /mnt/fixboot
sudo mount -t ext4 /dev/sda5 /mnt/fixboot
sudo mount -t proc non /mnt/fixboot/proc
sudo mount -o bind /dev /mnt/fixboot/dev
sudo chroot /mnt/fixboot /bin/bash
sudo grub-install /dev/sda
sudo update-grub
sudo umount /mnt/fixboot/dev
sudo umount /mnt/fixboot/proc
sudo umount /mnt/fixboot
reboot
:D

The first time around, it didn't find Windows, but after booting Ubuntu, ran update-grub, and it picked it up fine.

Rsync Notes

2010-08-17T08:00:00.001-04:00

Couldn't get rsync to work. Message was "connection refused", among other things. Realized that rsyncd wasn't running on other box. Realized that RSYNC_ENABLE in /etc/default/rsync wasn't set. Set that.

It then complained about a missing /etc/rsyncd.conf file. Did a locate for it, and found a sample one in /usr/share/doc/rsync/examples. Copied that into place, and tried again. rsyncd would start, and rsync on the other box then connected to it, but complained about a missing module (I think). So I copied lines that looked interesting out of the sample ("ftp") module, and created "home", which is what my old original command was looking for.

That got me to a point where it would connect to the module, but was unable to read any of the files. Seems I added uid/gid lines, but left them set to nobody/nogroup, which didn't work so well. Changed them to my user/group, and things worked fine.

Update: Nate suggests using ssh public keys and rsync -auv, instead of the rsync daemon.

Dia Hassles

2010-08-16T08:00:00.000-04:00

Upgraded Ubuntu to 9.10, and Dia got upgraded to 0.97. This update included a change to the file format for associations elements, (which breaks tedia2sql.pl, which is fully unsupported/orphaned at this point), and apparently changed the default font, which skewed all of our diagrams.

So I removed 0.97, and pulled down the 0.96 .debs from packages.ubuntu.com, and installed them, but then Ubuntu kept asking to upgrade them, since it saw 0.97 out there in Karmic. So, taking a page from my angel upgrade, I edited /var/lib/dpkg/status, and removed the three 0.96 dia entries from that, and now I have 0.96 installed, but Synaptic doesn't know about it, and doesn't ask to upgrade it. Hackish, but it appears to work.

Java and IBM 6400 Line Feeds

2009-12-30T10:41:00.005-05:00

I've got a client with several IBM 6400 line printers, and I started out printing to them with code that looks like this:

buf.append( stuff );
buf.append( "\n" );

So far, so good, until I print to a 2nd printer, which in some places double-spaces the printout, and in others, just gives line feeds. So I ask them to send me a printout of the printer config, and it shows that the printer is in Proprinter Emulation mode, which has an AUTO LF setting that is enabled. Okay, so I'm thinking I should be just sending carriage returns, and let it generate the line feeds.

No good. For whatever reason, when I just:

buf.append( stuff );
buf.append( "\r" );

to send just the CR, nothing comes out. Just to say I've tried all the combinations, I change the code to:

buf.append( stuff );
buf.append( "\r\n" );

expecting to get rid of my just-line-feed issue, but also expecting the whole report to be double-spaced, and to my surprise, it works.

My guess is that Java was doing something with the line-end characters, doing me a favor when I didn't want it to, and being explicit with \r\n turned that off? I don't know. And because both printers are some distance away from my office, I didn't bother to get the folks at the 1st printer to send me their config, so I'm still guessing that they have emulation set to something with AUTO LF disabled. Maybe next time I'm out there, I'll check...

(Oh, and yeah, looks like I just managed to squeak in my one post to this blog for the year. I've been making tech notes in Google Wave recently, so when that comes out of preview mode and integrates w/Blogger publicly, I should be reviving this blog somewhat...)

CPOSC '08

2008-10-20T22:13:00.002-04:00

Went yesterday to CPOSC, the Central Pennsylvania Open Source Conference. There were three talks going on at any one time, and I attended talks on: Asterisk, Blender, Android, Drupal, MozDev, jQuery and O/S project participation. All were at least good, and most were very good. I was already planning to get into JS/jQuery, and Asterisk, and now I'm thinking about Android as well (although I'm apparently one of the few attendees not standing in line for a TMobile G1).

Anyway, it was a day well spent, and I can't wait for CPOSC '09!

Javascript

2008-09-13T08:49:00.004-04:00

Okay, every now and then I get to be embarrassed. Many years ago (probably around 2001), I looked into Javascript, and only saw that it was a toy language, inconsistent between browsers (there were only two back then) and not worth learning. (IIRC, I fell in love with Python shortly thereafter.)

In my defense, a) there were only two browsers back then :), and b) JS was quite inconsistent between them (and possibly still is). But I've spent a few hours recently looking at JS again, and I'm seeing that it *IS* a real language, every bit as much as Python. And with add-ons like Firebug that provide a debugger and great visibility to what's going on in the browser, and jsUnit to provide unit testing for JS code, it's a full-fledged programming environment.

And while at work, we've done quite a bit with JS that's centered on AJAX and UI, we've more recently put together some prototype stuff that's looking a lot more like a full fledged application, which looks quite interesting, and as I dive in, I may well get to be involved in that.

Yup, I'm thinking it's time to shed some of my JS misconceptions, and join in the fun!

DWL-520 Wireless Woes

2008-09-01T14:30:00.009-04:00

Got a new machine that I'm trying to setup for MythTV. Decided to put a wireless NIC into it, and ran into install problems. Followed the directions at:

https://help.ubuntu.com/community/WifiDocs/Device/DWL-520vE1

then continued the process with the directions at:

http://ubuntuforums.org/showthread.php?t=643467
http://ubuntuforums.org/showthread.php?t=606861

and still didn't succeed. Figured I'd better make a note of it, in case I ever care to try again.

Java Woes

2008-06-27T12:55:00.002-04:00

Yes, I'm thinking back this day to the good old days of programming. Back to my C++ days. Back when, if a piece of code that you called didn't exist or wasn't available, it gave a nice, sensible link error and refused to create your executable.

Yes, I'm in an "I hate Java" moment.

We need to connect to a DB2 database. This particular database insists on encrypted connections, and because it's an IBM database, (and because they can, I suppose), DB2 insists on having client apps use THEIR encryption algorithms. Which either means using their Java runtime entirely, or messing with the /etc/java/security/java.security file in my Sun JRE install. Yup, red flags are waving, buzzers are going off. Danger, Will Robinson, danger!

So I replace the entries in java.security with IBM's classes. At this point, everything Java on my box stops working, complaining about not finding SHA-1 and/or MD5. I mean, Eclipse won't save my source files, for crying out loud. So I put the Sun stuff back, and add the IBM stuff at the end. A sample command line program that connects to DB2 now works (as does Eclipse, once again), but the same code in Tomcat 5.5 doesn't work. Gives error of "Prime size must be multiple of 64, and can only range from 512 to 1024".

After poking around a good bit further, it finally dawns on my boss, who was also troubleshooting this, that not only do we need to specify the classes in java.security, but we also need to make 6 new jarfiles available on the classpath for Tomcat (and presumably every other Java app that uses that runtime).

Which brings me back to my pining for the good old days. The days when, if I forgot a library, I got a message saying "Dude, you forgot a library", not something completely orthogonal like the above "Prime size ..." hooey.

I'm gonna go do something that makes sense, like C++. Grrrr...

Hardy Heron and My BCM4306

2008-06-26T16:02:00.002-04:00

It seems like every iteration of Ubuntu breaks my wireless, and every iteration requires something new to fix it. This iteration (Hardy Heron) requires the steps that can be found in this post on the Ubuntu Forums. I hate that it breaks; I love it that the forums (almost) always have the answer waiting for me when I get there.

Secure Wireless Proxy

2008-02-12T13:32:00.000-05:00

After reading a Lifehacker post a while back on more secure wireless browsing, I decided to setup an SSH Tunnel + SOCKS Proxy for Firefox, including (optionally) using FoxyProxy to make things easier.

The short of it is:

Use "ssh -D 9999 user@server" to establish a connection to a remote server.
Use FoxyProxy to point to a SOCKS proxy on localhost.

The long of it is that the "-D" option causes ssh to setup a proxy server on your local box that redirects your browser requests to the remote box, then from the remote box out to the 'net. All of your wireless http transmissions will be encrypted and hence unreadable by sneaky folks in the airport (or Panera's, or Starbucks, or wherever), and your browsing will be as secure as if you were sitting on the remote box. (Not that that's secure by default, but it's at least not being broadcast all over your surroundings.)

Using FoxyProxy in this case simply makes it easier to switch back and forth between proxied and direct connections if you need to. FoxyProxy sits on the bottom toolbar in Firefox, and with a couple of clicks will let you switch between proxy setups (including a "no proxy" setup, if desired).

SQL Server Timestamp Field

2007-08-10T14:53:00.001-04:00

It's been so long since I've had a decent Microsoft rant, I guess it must just be time. A while back, in a SQL Server database (don't laugh, the client insisted), I needed to have a timestamp for when a row is created. Silly me, I actually looked through the data type list, saw "timestamp", and thought, "Hey, that must be a timestamp!". Yeah, silly me.

The timestamp type? It isn't even a usable type. It gets set whenever the row gets updated, is binary data, and has nothing to do with dates or times. What I really wanted was a Datetime field with a default value of getdate(). Not a timestamp.

I'm so glad that the REST of my coding life, I get to be in Postgres. In Ubuntu. On Linux. Away from Microsoft. Harumph!

A Minor Rails Hiccup, With Solution

2007-08-02T16:10:00.001-04:00

As mentioned, I've been working through Agile Web Development With Rails, 2E. I got to p. 88, which talks about adding data via a migration. So I download the data for 003_add_test_data.rb, and run rake on it. Nothing. All of my rows get deleted, but no new ones get added. I add some print lines, which get displayed when rake runs, so I know the code is getting executed. But no data gets added. Harumph!

Well, let's back up a bit. A few pages prior, I added validation to the model. Pretty cool stuff, being able to throw in some regular expressions and have a pretty error message show up on the page. So I did a little extra-curricular playing around, and life was good.

After looking around a bit, I discovered that the data that the book wants me to add wasn't passing my validation tests. Okay, fine. So why wasn't rake showing any of the validation exceptions?

Well, a quick post to the Rails forum later, it turns out that create() won't throw exceptions, it just adds entries into the 'errors' list and continues. create!() will throw an exception with the same error text that the Rails app shows, and stops the migration. Problem solved.

Getting Rails Running on Ubuntu

2007-07-21T15:10:00.000-04:00

Time to check out the Rails hype. I borrowed Agile Web Development With Rails, 2nd Edition, from a coworker, and have started working my way through it. So far, so fun. The details:

I'm doing this under Ubuntu, so I figured I'd install via Synaptic. I grabbed just about everything I thought I might need, and started through the book. I got to p. 39, which has a couple of rake commands. These commands failed, and as part of troubleshooting, found https://help.ubuntu.com/community/RubyOnRails. Following its advice, I removed rails and gems from Synaptic, and ran #1, #2 "Install RubyGems via source (recommended)", then #3 and #4. I then tried the commands again, and they worked just fine.

Well, almost fine. When I got around to running gem update --system, I got the following:


Updating RubyGems...
Bulk updating Gem source index for: http://gems.rubyforge.org
Attempting remote update of rubygems-update
ERROR:  While executing gem ... (Gem::GemNotFoundException)
    Could not find rubygems-update (> 0) in any repository

On a lark, I ran the same command again, and it ran just fine. Go figure.

I then generated my own rails api docs with the rake doc:rails command, and was able to proceed.

So far, so fun.

Feisty + Tomcat5.5 HowTo

2007-07-13T10:28:00.000-04:00

I got the following off of the Ubuntu Forums, and include it here so I know where it is, for next time. (Yes, there's always a next time.) In particular, #2 solved my present issue of not being able to restart Tomcat via the /etc/init.d/ script.

Thanks to Emblem Parade for this:

Argh. So, if you tried installing tomcat5.5 from the repositories, you may have noticed that it's broken. You get a configuration error about JAVA_HOME not being set. Actually, there are three problems! Here's what you can do to get it to work:

1) Install

sudo aptitude install sun-java6-jdk tomcat5.5

(Note that you absolutely need the JDK, not just the JRE.)

2) Set Tomcat's default JAVA_HOME

sudo gedit /etc/default/tomcat5.5

Uncomment the JAVA_HOME line and set it to your JDK path. For Java 6 installed from the repositories, it's as so:

JAVA_HOME=/usr/lib/jvm/java-6-sun

3) Fix catalina.out

Unfortunately, it seems that Tomcat's log file is set to be a pipe, but Tomcat can't seem to start with it. We'll recreate it as a regular file with the same security settings:


cd /var/log/tomcat5.5/
sudo rm catalina.out
sudo touch catalina.out
sudo chown tomcat55:nogroup catalina.out
sudo chmod uo-wrx catalina.out

Tomcat should work now as a daemon. Start it like this:

sudo /etc/init.d/tomcat5.5 start

And point your browser at http://localhost:8180/

Hooray!

Been Running Ubuntu

2007-07-11T11:31:00.000-04:00

Since Novell drank the Microsoft Kool-Aid, we've been replacing our SuSE servers and desktops here at work with Ubuntu. Overall, it's been a very pleasant experience. I got Ubuntu 6.10 Edgy running on my work laptop and desktop, and on my home desktop as well. I then upgraded all three to 7.04 Feisty with no problems on any box except my 64-bit desktop, and that seemed to only be a problem with some X fonts.

I then had the hard drive crash on my home desktop, so I reinstalled 6.10 and immediately upgraded it to 7.04. (I was too lazy to download the 7.04 install CD.)

The only minor annoyance I had with upgrading to 7.04 the first time was that I clicked Go and went to sleep, only to find the next morning that it had stopped awaiting confirmation to something about 10 minutes in. The 2nd 6.10->7.04 upgrade didn't do that, so I'm guessing it was from something that I had installed. But if you're upgrading, you might want to babysit it, and not assume it can run unattended. (YMMV.)

Comparing Ubuntu with all other distros I've used (Gentoo, SuSE, Red Hat, and Slackware way back when), this is very sweet. I had been limping along with wireless on my laptop under SuSE. Under 6.10, I followed some info from the forums, and it just worked. (Admittedly, 7.04 has seen a minor step backwards in wireless, but it still works, and I just haven't taken the time to ask/troubleshoot it yet.)

Synaptic is much nicer than Yast. Not sure how to explain it, but it's just a much more comfortable experience. Gnome is also very nice. I was expecting to try to figure out how to get back to fluxbox, but after a few hotkey tweaks, I'm enjoying Gnome pretty well, and prefer it over KDE, which I used briefly when I installed SuSE.

Bottom line on Ubuntu: I wouldn't go back. Much nicer experience than I was having with SuSE.

Background Threads in Tomcat

2006-08-16T13:21:00.000-04:00

I have a task that I want to run once a minute. I have it coded as a command-line app, but getting that app to run as a Windows service is getting to be a pain. And, we also have an instance of Tomcat running on that server, so I started looking at how to get it to run within Tomcat.

This made me a bit nervous at first, as Tomcat does a lot of threaded stuff on its own, and I'd really rather not step on its toes, but after a bit of searching, I found that kicking off your own threads is a fine thing to do in Tomcat, but with a couple of caveats.

First, Tomcat lets apps ("contexts" in Tomcat lingo) be hotswapped, i.e. restarted or removed and replaced with new code, without stopping the VM. This means that you need to wire up your thread to stop when the app (sorry, "context") is stopped. You can write a servlet to watch over your thread (that can actually be the thread itself, too), and have Tomcat instantiate it, by configuring it in web.xml with "<load-on-startup />" included.

The servlet itself will extend HttpServlet (of course), and implement Runnable (if it's to have the thread code in it as well, which is handy. Then, when the destroy() method is called on your servlet, you tell your thread to stop itself (i.e. via a call to Thread.interrupt(), perhaps), as the context is going away.

I found a code sample at http://classes.eclab.byu.edu/462/demos/PrimeSearcher.java that helped, although that sample is coded to call Thread.stop(), which is now deprecated.

Anyway, I now have simple thread doing background processing, nicely cooperating with Tomcat.

SQL Server Default Values (or "Why Open Source Is (Still) Better")

2006-08-14T13:16:00.000-04:00

In getting our Java app to run on Windows, I'm forced to use SQL Server as a database. (Against my will, I might add.) I have an external application that will be adding rows to a table via ODBC, and I'm thinking it'd be nice to have SQL Server automatically generate a date value for when the row was written. In PostgreSQL, it'd be a default value in the DDL (I think; I've never needed it yet), but I wasn't sure what it'd be in SQL Server, so I tried timestamp.

That didn't look right to me, as the value in Enterprise Mangler displayed as '<binary>', rather than the actual date. Okay, let's try datetime with a default value. Well, that complained about something odd. I googled and found that timestamp *is* what I want, display problems notwithstanding.

Which leads me to today's punchline: when I switched from datetime w/default to timestamp, and tried to save my changes, EM complained about a default value not being appropriate for that datatype. Why, on earth, did EM not clear the default value when I switched datatypes?!

Okay, maybe I'm reading too much into this, but this is a problem you just don't see in open source programs. In the O/S world, the first decent coder to hit that problem would have pulled down the source, fixed the problem, and submitted a patch. The next version of the program would be fixed, perhaps in less than a month, depending on the release cycle. Heck, I was itching to do it, myself, except, whaddya know, I can't get at the code.

But I'll wager that this problem will never get fixed. It's not "important enough" (i.e. to the marketing team) to make it into a bug list, unless it annoys a SQL Server developer enough to make it happen "while he's doing other things" (i.e. without permission).

Harumph. When will I be free of this nonsense?

Windows Services and Network Drives

2006-08-11T15:56:00.000-04:00

Rather than turn this into a Windows rant (which it easily could become), I'll just skip to the punchline. After a few hours of trying to figure out why my File.exists() and File.isDirectory() calls don't work, now that I've moved the code into Tomcat, running as a service on Windows 2003 Server, I discovered that network drives are not available to services.

More specifically, I got the impression that network drives are mapped by a user for that user, and as such, a service, which doesn't have a user (?), doesn't get the drives that any (other) user specifies. More generally, it's another way that Windows is borken. (Oops, I wasn't going to rant.)

So I got to go back to my client and get him to move the app (whose data I was getting over their net) to the local machine, or figure out how to at least push the files to the local machine. Feh. (rant, rant, rant...)

Duplicate Methods in Python

2006-08-02T13:24:00.000-04:00

Stumbled across a little gotcha in Python. Python, when it loads a source file, executes the code as a command. For instance, assume the following code:


class Foo:
 def method1( self ):
  pass

It sees the class statement as a command, with two lines of text following as parameters. Similarly, it sees the def statement as a (sub)command with one line as a parameter. The def 'command' adds (or replaces) the method method1 in the current scope, in this case, class Foo.

Which leads me to my gotcha. The following code is valid, but probably not what you want:


class Foo:
 def method1( self ):
  print 'Yes'
 def method1( self ):
  pass

In this case, the first def statement will work, adding a method1 that prints to the class. The second one will work, too, replacing the existing method1 with one that does nothing.

Unfortunately for me, I had added some stubs to the bottom of my source file, then forgot that they were there, and added the real method higher up, and wondered why the method apparently wasn't being executed. Debugging a bit showed me that it was getting called by the calling code, but the call wasn't arriving in the method itself. The 2nd def had replaced the first with a method that did nothing.

Chalk one up to experience.

Postgres Speed, and Table and Index Sizes

2006-07-17T17:52:00.000-04:00

I've gotta start paying attention to my indexes in PostgreSQL.

I went to update one field across all rows in a big table I have. I did a test update on a local copy of the db, and it ran fine; I then ran it on our shared db, and it still wasn't finished after 3 hours. And other selects were waiting for it to finish, and so on.

After looking around, I noticed that the difference was that I didn't have any indexes on my local copy. And after reading a bit, I found that updating a row makes a copy of that row, freeing up the old space for more data (although not returning it to the O/S). Same general thing happens with index entries as well; the old entry becomes unused, and a new entry gets written.

As part of the conversation on IRC, I was told about contrib/dbsize, and installed it. I then found that my tables indexes took up 571MB from all the updating. I reindex, and it went down to 160MB. The table data took up 511MB; a vacuum full took that down to 190MB, but the index size went up to 338MB from the index activity. Another reindex, and the table/index sizes are 190MB/160MB, respectively.

I guess the main punchline to all this is to remove/replace the indexes on a file-wide update. I'm not doing this in production, just during table modifications, trying to bring a new column up-to-date, but it's something to be aware of.

Copying a File in Java

2006-06-16T16:17:00.000-04:00

I just wrote a little monitor program. It watches one directory, and when it finds a file, copies it to a 2nd dir, processes it, and deletes it. Using Java's File object, the whole thing was simple, except for the copying part. Any reason why the standard Java libraries don't have a file copy method?

With Google's help, I found one online at Java Almanac, which was almost as handy, but why, pray tell, ... oh, never mind. Now where'd I put my Ruby book...

Rails said: AdapterNotFound

2006-06-09T15:46:00.000-04:00

Oh yeah, as part of my playing with Ruby on Rails, when I went to run generate, I got an error message saying ActiveRecord::AdapterNotFound. Turned out I had put down postgres instead of postgresql. I'm still not sure where I should have seen that, (rather than the tutorial text, obviously).

First Tiny Steps in Rails

2006-06-01T15:05:00.000-04:00

Time to learn something about Ruby on Rails.

I started going through the tutorial, and had a minor hiccup that I thought I'd report.

We use PostgreSQL, so I wanted to use that as my db. This meant doing a gem install postgres to get the correct db adapter. Except I got many screens full of error messages.

At the top of these was:


postgres.c:18:22: error: libpq-fe.h: No such file or directory
postgres.c:19:70: error: libpq/libpq-fs.h: No such file or directory

so I went looking for how to tell it where the postgres include files are. Turns out that


gem install postgres -- --with-pgsql-include-dir=/usr/local/pgsql/include

is the simplest way, but you can also go into /usr/local/lib/ruby/gems/1.8/gems/postgres-0.7.1 and do this:


ruby extconf.rb --with-pgsql-include-dir=/usr/local/pgsql/include
make
make install

Learning Ruby

2006-05-24T14:26:00.000-04:00

So last week, and over the weekend, I've been playing around with Ruby. I'd like to learn a bit about Rails, having been given a very brief overview of rake, and having heard all of the buzz.

Well, I went away with Wife for the weekend (she had this conference/retreat thing to go to), an while I took my laptop, I didn't want to pay extra for a net connection in my hotel room. Yeah, I'm cheap. So I went looking for stuff to download to play/learn with. Alas, it seems that the Rails docs are either non-free PDFs or online in Wiki form, and not having time enough to wget what I needed, I went with plan B.

There are plain-Ruby PDFs out there, most notably Why's Poignant Guide to Ruby. The Poignant Guide reads like Douglas Adams having a fever dream, but the Ruby content is good, and seems quite appropriate for beginning programmers, moreso than just beginning Rubyists, but overall is a good read. (At present, I'm on p. 93, picking out the gems (yes, pun intended)).

Time for a Name Change

2006-05-23T12:16:00.000-04:00

Well, as posts have been fairly sporadic here since I started this blog, I decided to change my focus a bit, and with that comes a name change. This is now "Dave's Code Blog".

I started it when I was thinking that I'd be diving into Java more deeply than I had done in the past, (about a year and a half ago). After doing primarily Java coding over that time, I've found a couple of things out.

First, Java's a fine language. I mean that without sarcasm. It's fine for what we're doing with it here at work, and as long as you keep it simple, it does its job. As I mentioned a while back, the whole idea of Better, Faster, Lighter is a really good one, and has served us well. Even "lighter" tools like Hibernate were too heavy for what we're doing, and we left them by the wayside.

Second, my heart's not in Java. Not for fun. While it's fine to be employed doing Java work, and while it pays the bills nicely, my heart wants to be using a dynamic language. Like Python, which I know and love, or Ruby, which I started to explore last week, (and will be writing about more in the days to come).

Hence the name change. Ignore the "javadave" in the blog url, this will be where I log the fun things that I find in my coding.

And yes, my heart is still very much in Linux, in case you were wondering. Delightfully enough, Linux has faded into the background for me of late, being something that I just take for granted. No real surprises, no real frustrations, just enjoying the O/S Good Life.

Keeping track of versions of internal libraries

2006-04-25T13:07:00.000-04:00

We've got several applications, each in several versions. We've got several internal libraries, with their own versions. Because it was starting to get a bit harder to keep track of what version of an application requires what version of each library, we now have a solution.

Each library now has a class called Version. This class has one static void method called version_n_n_n() that does nothing, with the 'n's reflecting the version of the library. In the applications, a method early in the call chain (i.e. the constructor of an MVC controller) makes the calls.

When the library version gets bumped, we create a new branch in Subversion, and the method gets renamed. Any code that depends on a newer version of the library also gets updated to call the new method.

The big benefit here is that anyone who hasn't pulled down the new library versions, but gets the latest app code, gets immediate and obvious compile errors, and knows to get the updated libraries.

Setting up log4j

2006-04-07T12:00:00.000-04:00

I just got told by Boss that we need to make our logging more official. I can buy that. I suggest log4j as a de facto standard. He can buy that, and I'm off and running.

So I wander over to the log4j site, use SuSE's yast to install it, and pull up a source file with a few println() calls, to try it out on. I install the docs and the API javadocs, and pull up the docs front page. The first link is "short manual", which sounds promising.

Here's where the story bogs down, and I slip into rant mode.

I'll start with the punchline. I needed to add to my app's startup code:


BasicConfigurator.configure( new ConsoleAppender( new PatternLayout( "%d{ISO8601} %c%n%p: %m%n" ) ) );

In each source file, I needed to add the following member variable:


private static Logger logger = Logger.getLogger( MyClass.class );

And in place of each println() call, I needed to add the following:


logger.debug( "your message here" );

Actually, the call to logger.debug() could have been a call to logger.info(), logger.warn(), etc., depending on the nature of the message.

How long did it take you to read that? 30 seconds? Given that code, you now can go into the API docs and lookup PatternLayout, and see what that string means, or look up Logger and see what methods you call for what log levels. And while I still haven't mentioned much about what an Appender is, or how you control what log levels get output, (I'm not sure myself yet), you now have working logging, using log4j. Darn close to being as easy as those println() calls you're replacing.

We now return to the rant: How long did it take me to code this? About 20 minutes. The so-called "short manual", in my small Firefox font, takes up more than 22 full-screen pages. I glanced at it, saw bits and pieces of what I needed, poked around the API docs for a while, realized I needed to read short-man a bit more carefully, and gradually figured out what those three simple lines of code would be, to give log4j a whirl.

Why on earth did the so-called "short manual" not start off with three lines of code?! Why did somebody assume (not necessarily the short manual author, but rather the person who put the link to it at the top of the doc page) assume that I'd want to read 22+ small-fonted pages just to get started with log4j?

There was a moment, let me assure you, when, had I not suggested its use, and had I not been tasked with improving our logging, I would have said "println()'s are good enough", and stopped reading.

The key here is that I was in evaluation mode, having not yet bought into its use. 3 lines of code stuck in my app did convince me, once I divined what they were; handing them to me early would have been greatly appreciated, and sold me much quicker. One would assume others might glance around, see 22+ pages, and say without further ado, "println()'s are good enough", which would be a shame.

Later, I'm pretty sure I will want to know more about log4j, and the short manual will be quite useful. But at this moment, I just wanted to see what it'll do, and what I found was a bad way to show me that.

Okay, I've calmed down a bit.

Changing Gentoo's CHOST from i386 to i686

2006-04-05T14:25:00.000-04:00

I'm running Gentoo, kept pretty much up-to-date. I wanted to run distcc, to let my faster box help my slower box out on long compiles. This let me to want to change my CHOST variable on the slower box from i386-pc-linux-gnu to i686-pc-linux-gnu, to match my faster box. I got the impression at some point that I could have installed gcc as a cross-compiler, but changing CHOST sounded "easier" (or would at least let me end up with a less complicated config).

So I searched the forums, and found people talking about how to make such a change. Opinions fell into three camps: a) DON'T DO IT!, b) you can, but it's hard and not for the faint of heart, and c) I did it, no sweat. Having finished the process, apparently successfully, I'm writing up what it took. If you don't want to read the whole (quite long) thing, the summary is that I think I fall into (b). It wasn't that hard, but it was time consuming, and took a bit of troubleshooting. My full experience follows:

To start off with, I did what I would heartily suggest to you: BACK UP YOUR WORLD! Assume that your machine will end up fragged, and act accordingly. Actually, it probably won't mess with your data, but then actually, you should have a backup plan already in place. (There's two kinds of people: those who backup, and those who have never lost a hard drive.) There. I said it.

Before you start, there is a good thread to read on the forums. In particular, the three-post series in the middle of the page by amne, danielrendell and Bob P give a pretty clear idea of what it should take, and what I decided to do. To spell it out, my plan was to combine suggestions, and do this:


change CHOST
emerge glibc binutils gcc
emerge glibc binutils gcc
emerge -e system
emerge -e system
emerge -e world
emerge -e world

So off I went, to /etc/make.conf, and changed CHOST from i386 to i686. I kicked off my first emerge of the build tools. This ran fine. I kicked off my second tool emerge, and got an error running emerge, saying that Python couldn't find libstdc++.so.6. I "fixed" this by creating a symlink:

ln -s /usr/lib/gcc/i686-pc-linux-gnu/3.4.5/libstdc++.so.6 /usr/lib

Rerunning the 2nd tool emerge gave me a configure error on the glibc emerge, which referred me to config.log, which I found in /var/tmp/portage/glibc-2.3.5-r2/work/build-default-i686-pc-linux-gnu-linuxthreads. It said 'gcc-config error: Could not run/locate "gcc"'. So off I go, into debug mode.

The first thing I found was that PATH still had /usr/i386/gcc-bin/3.4.5 in it. (Keep in mind that when I say i386 or i686 from here on out, I probably mean i386-pc-linux-gnu or its i686 version). Looking in /etc, found i386 in profile.env, which tells you that it is generated by env-update. I ran env-update, but i386 was still there.

I grepped, and found i386 in several files in /etc/env.d/gcc, which led me to search for some way to reconfigure things, which led me to run:


gcc-config -P i686-pc-linux-gnu-3.4.5
source /etc/profile

This fixed PATH quite nicely, and the 2nd tools emerge worked.

On to the first system emerge. Started fine, but stopped at groff, with a message of:

'gcc-config error: Could not run/locate "i386-pc-linux-gnu-gcc"'

I went through a couple of cycles of trial-and-error: I removed /var/tmp/portage/groff* and reran; still had error. I checked /etc, no references to i386 anywhere.

I checked my environment, and found a couple of i386 mentions, in BASH_VERSINFO, HOSTTYPE and MACHTYPE. I bit of digging (and Googling) turned up the fact that MACHTYPE is a bash reserved variable. Did a solo emerge on bash, which fixed MACHTYPE, but a retry on groff still failed, even after removing /var/tmp/portage/groff* again.

I dug deeper, and found the Makefile in /var/tmp/portage/groff* with the i386 in it. I saw that imake was creating this, and used equery to find out where imake was coming from (assuming it was part of make, and that rebuilding make might fix things). No dice, 'equery belongs imake' reported x11-base/xorg-x11.

So now I'm confused. I also did an equery on groff, and found that it depends on X as well. Ah, but doing an 'equery uses groff' says that groff has an X use flag. Now we're getting somewhere. I created /etc/portage/package.use, and added this line to it: 'sys-apps/groff -X'. This tells emerge to remove the 'X' use flag just for groff. Ran emerge just on groff, and this time it worked.

Reran 1st system emerge, and it worked. Ran the 2nd system emerge, and it worked, too.

On to the first world emerge. Started fine, but gave me an md5 error on one of the docbook zip files. Removed that zipfile and emerged just docbook, but noticed that it was emerging a different version. Ah, the problem was on an old slot. Unmerged the entire old version, and started the 1st world emerge again.

This time, it stopped again, but past the docbook problem of the last try. But after a good bit of poking around, I couldn't see any messages indicating why. I had been writing messages out to one output file, and had seen to strangeness with the order that some messages would come out in, and decided to split up stdout and stderr. Did so, restarted 1st world emerge, and it worked just fine.

Given that the 1st one worked, I decided to setup distcc for the 2nd. Started 2nd world emerge, which worked fine until I got down to xephem, which gave an odd ld message: 'cannot find -llilxml'. I couldn't track it down, so I just tried rerunning xephem. That worked fine. Reran 2nd world emerge, which worked fine.

Took out the line in /etc/portage/package.use for groff (by removing the whole file, since that was the only line in it on my box), and re-emerged groff, successfully this time.

Whew! The world emerges took quite some time (days, on my 400mHz slow-box), but I ended up with a fully rebuilt system, and one running distcc, with a i686 CHOST.

All in all, it wasn't too much work, and I got much more familiar with slots, USE flags and equery, all of which are way cool and knowledge of them is quite helpful when maintaining a Gentoo box. If I had to do it again, I'd be willing to, but be aware that it took up a good bit of my spare time for a week or two to accomplish, I've been running Gentoo for a few years, and I am a developer myself, making poking through these problems less intimidating. YMMV.

Enjoy.

Circular FK References

2006-02-06T13:38:00.000-05:00

Circular foreign key references are evil. And they sneak in without being obvious, until you decide to do something big like reload a table.

In our case, a sale line points to stock, a stock item points to the order it came in on. So far, so good. Both FKs are nullable, as a sale line might not have been assigned to stock yet, and the stock item might be independent of an order still online.

But then the order line was referring to a sale that that line was ordered for. It was nullable, too, as not all orders are for written sales. Therefore, all three rows can be created w/o difficulty, then linked up, making it impossible to disconnect.

In our case, we solved it by reversing the direction of the third link. Now, the sale line refers to the order line that will satisfy that sale, if any. The links are then Sale->Stock->Order and Sale->Order, and no loop exists. Life is good once again.

But it snuck in there. Perhaps it would be amusing to write something to detect such loops, just in case...

Why I Hate Frameworks

2006-02-01T14:28:00.000-05:00

A coworker sent me a link to a very funny article and a rather nice ensuing discussion about the problems with frameworks.

Where I'm working now, we're pared down to a fairly simple level. Our presentation code is a port of a homegrown library; our database code (that I've mentioned in an earlier post) is homegrown and lightweight. We have one toe in Spring, pretty much just to handle dispatching of requests; I'm not planning to, but I could probably rewrite the part that we're using in a couple of days.

And we're happy, productive coders. Makes me feel good, as early on in our switch to Java, I lobbied hard for doing just what we needed, while having the option to go to J2EE/EJB/insert your buzzword later, if/when we need to.

How Not To Write a While Loop

2006-01-23T12:12:00.000-05:00

I was running a few long processes recently, and was getting tired of watching it in top. I wanted to run something that would tell me when it quit. I went looking, and found wait, but that only works on processes that are children of the current shell, which wouldn't work in my case.

So I figured I'd just write a bash one-liner. I tried while 1 as my base loop, but that didn't work. I thought (incorrectly) that I had tried while true already, and found that it didn't work (I hadn't), and got this flash of, well, non-genius:

while `yes`

Think about it. (Don't run it yourself, especially without reading the rest of this post!) yes outputs "y\n" in an endless loop. Wrapping a command in backticks runs that command, and puts its output inline for whatever command it's a part of. Do you see the disaster looming?

When I ran the loop, it started by running yes, saving its (infinite) output. This filled available memory. It then filled available swap. My memory docklet reported "MEM 96% SWP 99%", right before the system stopped responding completely. As a testament to the solidity of Linux, the non-responsive stuff only lasted a little while, after which the system responded to all of the Ctrl-C and "close window" clicks I had been sending it, and all of the offending processes (and a few non-offending ones) disappeared. No system crash.

Anyway, lesson learned, and I ended up writing my one-liner as follows:

while test -e /proc/nnnnn; do sleep 5; done; echo ^G

which waits for the process to go away, then beeps. Made much more sense than whatever I originally had in mind.

Setting Priorities

2005-10-21T13:54:00.000-04:00

Joel (on Software) Spolsky has a new article that's really a winner. It's a combination of why he does shrinkwrap software, how to set priorities (in life, in general), and a way to pick what to put in the next revision of your software.

Good stuff.

How To Do DB Testing (Part III, Mock Database)

2005-10-19T15:06:00.000-04:00

We decided to use Dia to draw the ERD, and to use tedia2sql.pl (available at tigris.org) to pull the data out of the .dia file and generate DDL. In the case of a new table, I'll cut the new DDL out of the generated file and paste it into psql to actually create the table, then run a database diff utility that I wrote (that I already posted to this blog) to make sure what's in the DB is what's in the ERD. (That way, either I or Boss can make quick changes right to the dev DB, and I can make sure they get back to the ERD when things slow down.)

Another good thing about tedia2sql is that, since it's F/OSS, our resident Perl guru could (and did) modify it to spit out text that our internal tool needs, and that another tool (that I wrote) could generate Java classes off of the data model. Kind of like a poor man's Hibernate, perhaps. Once we had working data objects, it was a simple extension to make each of those objects implement a generated interface, and then generate mock objects that don't actually talk to the database, but rather expect to receive one of those Vectors I mentioned earlier.

So now any change to the database automatically updates the real Java objects, and the mock objects that implement the same interface, and a class factory to get a new one as needed. From there, writing Junit tests are the snap you'd expect.

How To Do DB Testing (Part II, Bulk Analyze)

2005-10-18T10:33:00.000-04:00

So with all of the SQL expressions contained in query manager subclasses, the next step was to collect and run all of the queries at once. This had two big advantages.

First, all of the SQL could be exercised. Probably the biggest headache in writing SQL is that there's no real way to "compile" it, even to get a syntax sanity check, (without going into stored procedures and other DB-specific stuff). Our way, we have a hierarchy of classes that contain all of the SQL strings, and a TestAllQueries JUnit class that calls each method on each query manager class. Running one test runs all of the queries in the system. (It ignores the results; that's for other tests.)

Second, with a tiny tweak in how the methods are called, instead of running the query we can 'EXPLAIN ANALYZE' it, and run the output into a text file for further examination. One run, and I can see what queries are doing table scans, and what index that one query is using. Quite handy.

And with all of the application code not knowing anything about JDBC, but rather using an Iterator, we can put together a mock database fairly easily, just by instantiating and throwing HashMaps into a Vector, and returning the iterator() from it. Acquiring the database from a factory means that each db-using class is now easily testable without having to setup a separate database and worry about the attending headaches.

How To Do DB Testing (Part I, General Ideas)

2005-10-17T11:56:00.000-04:00

Not that I expect this to be a tome of wisdom or anything, but I thought I might put into writing some of the wins we've had lately in being able to test our DB code. First, some background.

I got to design most of the DB layer of a web app. To prototype, I threw together a simple JDBC wrapper class to manage a connection and perform queries. This worked well for starters, but soon the number of queries got unmanageable. So we split out the connection stuff into a base class, and split the queries themselves into various derived classes. This is nice, as it means we can find all of the queries for the whole system in one easily spotted set of source file, and those source files have nothing else in them.

The other thing we did that works really well is to have the query manager class act like an Iterator. The big win here is that next() returns a HashMap<String,String> that contains the data from each row, and as a result, nobody anywhere in the whole system deals with JDBC classes outside of the query manager.

Here's the skeleton:


class QueryManager
{
   protected Iterator<HashMap<String,String>> executeQuery( String sql ) { ... }
}

class SpecificQueryManager extends QueryManager
{
   public Iterator<HashMap<String,String>> getPeople() { executeQuery( "select * from people where ..." ); }
}

Blogspot.com Stats

2005-10-14T11:25:00.000-04:00

It's nice to get feedback on what you write in a blog. I mentioned getting comments; it's nice, too, to be able to look at your server logs and see who's reading what, and where they came from, and so on. But as my blogs are primarily hosted on Blogspot.com, and given that they don't share their server logs (harumph), what's a poor geek to do?

Well, images to the rescue. Turns out, you can put a link in your blog template to an image file that you host on your own server, and any time anyone in an actual browser (i.e. not a webcrawling 'bot) views your page, your image gets loaded, and you can see what page the reader was on at the time. Don't know why I didn't think of that sooner.

Now, if they'd just share my own stats with me...

Tomcat Deploy on Restart

2005-10-03T13:32:00.000-04:00

It seems that Tomcat has a deployment problem. If you stop Tomcat, copy a new .war file to ./webapps, and start Tomcat, it won't pick up the new file and deploy it. I'm not certain about this, but it also seems that even if the file is placed there right when Tomcat finishes startup, it might not be picked up.

In my case, I was doing a full build and restart after I grabbed new files from CVS. A coworker suggested that the best way to do this would be to grab new, rebuild, deploy, let Tomcat catch up, and then restart Tomcat (especially if there are changes that would require a restart, such as a new JNI shared/dynamic library).

More on File Handles

2005-08-31T08:45:00.000-04:00

In response to my "Too many open files in system" problem, I got the following email (posted here with permission, provided it's anonymous):

On our 2.6.9 appliance box we have been suffering the same problem running our application software. I tracked the issue down to the use of ps(1) in a system where /proc is changing quite often. It appears that repeated use of ps can cause the /proc/sys/fs/file-nr count to go up quite rapidly. In our case it was hitting its 200 k limit in just a few days!

The sort of command we were executing often was: count=`ps aux | grep "service" | grep -v grep | grep -v defunct | wc -l`

Things seem to be improved by doing the ps aux to a file, rather than a pipe. the problem seems worse the more output ps produces.

Note that the kernel doesnt increment file-nr too often - there is latency for performance reasons - so it tends to jump up in multiples of 25. When experimenting you have to be patient and wait a while to see if running a command repeatedly is causing a leak.

The problem is that ps itself doesnt appear to be leaking fds directly - it looks like its a "virtual" leak in the kernel, perhaps caused by ps's interaction with /proc in a system that has processes being created and killed very often.

We have worked round this issue by reducing our use of ps to a minimum. Now our application software only leaks a few hundred handles a day, rather than thousands a hour!

I have spent many hours on the net and havent found any kernel or ps patches for this. We will be upgrading to a later kernel at some point - hopefully that will fix the issue.

Executing Python Files

2005-08-25T14:42:00.000-04:00

This is a pesky little thing I just ran across. I was setting up some Python code on my Linux box, pulling it out of CVS on SourceForge, and did the obligatory chmod to make one of my tests runnable. But no matter what I tried, I couldn't run the darn thing.

Turns out, it has DOS-style line endings, (i.e. CRLF, instead of the CR that Unix/Linux likes). Once I used dos2unix to switch that, it worked fine. Apparently, the first-line hash-bang that tells the shell where to find the executable had bash all confused, and unable to find Python.

Denying the Worms

2005-08-24T13:56:00.000-04:00

I run Apache on my home box. As a result, I get the regular set of script kiddies coming in to see if they can break in. Checking for awstats.pl, phpBB, and the like.

But along with those, I've been getting regular requests for '/' from people on my subnet. It looks like a milder version of the logs we used to get when Code Red was at its peak. The distinguishing characteristic in the logs is that the user agent (browser) is blank.

Well, my root doc is 60k+, and having it hit up to a couple dozen times a day by worms is just a nuisance. So I started poking around in Apache to figure out how to refuse them. It wound up being a tiny bit trickier than I expected; here's what I did:


<IfModule mod_setenvif.c>
    ...
    BrowserMatch "^$" agent-deny
    ...
</IfModule>

...

<Directory /whatever>
    <IfModule mod_access.c>
        Order Allow,Deny
        Allow from all
        Deny from env=agent-deny
    </IfModule>
</Directory>

Getting to that point held a pitfall, though. In the logs, the user agent was displayed as "-". So my first shot at a BrowserMatch string was "-", but that matched anything with a dash, i.e. everything. I then tried "^-$" and "^-", but then nothing matched. On Freenode's #apache channel, bare-foot suggested that since the dash is displayed in place of an empty string, that perhaps "^$" would work, and it did.

Prior to getting on irc, I was trying to figure out how to display environment variables, and did manage to change my LogFormat to include "%{agent-deny}e", which indicated that my variable wasn't being set. I then changed the string to match my browser, and saw that it was being set, narrowing my problem down to the regex.

Anyway, now all the bots get is a 295-byte 403 response, and my world is a little bit nicer.

Java Generics

2005-08-01T16:49:00.000-04:00

Well, I got all of us in the office moved up to Eclipse 3.1, and have been fairly happy with it so far: a bit faster to start up, filters on the properties pages, etc. And support for JDK 5.0.

I've been interested in playing with generics since I heard Java was getting them. I had messed around with C++ generics, and liked them there, so once I got Eclipse running, I went to work correcting the type warnings that popped up in all of the Iterators and HashMaps we're using.

So I fixed everything in three of our smaller projects, and noticed that the remaining two had 1350+ warnings remaining. Hoo boy, what to do. I mentioned it to Boss, who leaned toward letting our junior guy, Junior, fix them at his leisure. So far, so good.

Then I looked at the code again. It was ugly. < and > everywhere, hideously long names, and after wandering through a few dozen source files, I had found nothing that made me think "hey, glad I put that there, or I wouldn't have found *THAT* bug". So I started thinking about ways to make it cleaner.

I Googled, and found a mention of subclassing, i.e. HashMap, which worked fine (we use HashMap<String,String> a *LOT*), and the code became a lot less ugly, and made further changes a lot easier to make (I called the new class HashMapSS; pretty easy change to that from HashMap). I played with that a bit, then took another look.

I got down to about 1300 warnings, and asked myself: What does it buy us to fix all of these? Initially, I was telling Junior and two other coworkers about generics, and just got blank stares. "Increase coding time and make the code ugly? For what?" With these new classes, we're just down to a minor increase in coding time, but again, for what?

In our case, we're not running into problems that adding these checks would have caught earlier. ClassCastExceptions and the like. As I'm the office code Nazi, I'm the one who would be most eager to code this way, and I'm sitting on the fence. Everyone else has turned off the warnings and continued merrily along.

I'll probably continue to code with the warnings on, and see how it goes, just because, but I'm not as excited about generics as I had expected to be. Perhaps the Python I'm doing in my spare time is rubbing off. Throw a few test cases at it, and all this static typing is more of a burden than an assist.

Apache Problem

2005-07-21T12:36:00.000-04:00

I'm reviving my blog on my home box. I'm using Blosxom, which throws index.html files into subdirectories of the main blog. I'm using Apache, which you can tell to look for index.html if the client requests a directory. So far, so good.

I set it all up at home. My server's machine name is 'moose'. (You'll see why this matters in a moment.) I test it out; it all works. I go to work, and try it out. The main page works, but when I click on a subdir link, it pulls up a different page, namely that of Mooseworld.com. What the heck?!

Okay, right away I notice the moose connection. When I put 'moose' into Firefox's URL, it does an autocomplete thing, and I'm guessing that some DNS magic sends such a term to Mooseworld.com. But why does such a thing happen?

I search my access_log, and find that any request for a plain directory (i.e. "/foodir") returns a 301. A 301 is a redirect. Seems that Apache was redirecting to a directory with a trailing slash (i.e. "/foodir/"), which would then match its pattern for trying the default index (index.html, in my case). Only it was sending back a full URL, in my case "http://moose/foodir/".

I looked around a bit, and found that apache2.conf had a ServerName directive that was set to "moose", and when I changed it to reflect the real server name, things worked.

I'm guessing that there should be a way (perhaps with mod_rewrite?) to not require a 301 return and subsequent 2nd page request for this to work, but that'll have to wait until I convert my blog.

"Nobody Cared" Problem

2005-07-20T11:32:00.000-04:00

I recently upgraded to a 2.6.11 kernel, and was getting the delightful "nobody cared" message shortly after booting up my laptop. This was preventing my combo NIC/serial PCMCIA card from working, (in particular, the NIC doesn't come up; I don't care about the serial card part). I've taken ACPI out of the kernel; several posts on the Gentoo forums pointed to ACPI as a possible culprit.

Under 2.4.x, the card appeared to run on IRQ 3; under 2.6.11, it was trying to go to IRQ 11. I started out trying to figure out how to get it back on IRQ 3, where it worked just fine.

I tried an edit to /etc/conf.d/pcmcia of 'PCIC_OPTS="irq_list=3,4,7"'. This put eth0 on IRQ3, but left ttyS1 on IRQ11. I also tried excluding IRQ 11 by adding "exclude irq 11" to my /etc/pcmcia/config.opts, which resulted in dmesg output that said that the NIC was at "io 0x300, irq 3", but below that "ttyS1 at I/O 0x2f8 (irq = 11) is a 16550A". And, I tried the "pci=routeirq" kernel boot option, to no avail.

I posted the above info to the Gentoo forums, and got a reply back from diastelo suggesting that I disable support for the serial device in the kernel. I had tried removing the 'serial' script from the boot runlevel, but that made no difference. But removing serial PCMCIA support from the kernel did the trick, and the NIC has worked fine ever since.

Out of disk space

2005-07-11T11:59:00.000-04:00

Time for a bit of embarrassment.

I was coding merrily away in my new Eclipse 3.1, and got an error saving a source file. This is instant cause for alarm; if it has problems saving, that's really, really bad. I missed the "More Info" button on the first dialog, and ran to a shell prompt to see if the file was really corrupt.

Shortly, though, I did something to generate a "disk full" message from the command line. Harumph! Last time I looked, I had 3 or 4GB free.

(I later went back to Eclipse, got the dialog again, and clicked "More Info" to get the "out of disk space" message. For short messages like that, IMO Eclipse should just display the message, and do away with the "More Info" button.)

So, let's go find the offending file. du is your friend, and showed that my home dir housed the culprit. But looking closer, it didn't show where. All of the files and subdirs that it showed were reasonably sized, and added up to about 4GB less than the total.

Well, it turns out that du doesn't show files starting with a period, just like ls. And like ls, it has a "-a" switch to turn on display of those dot files. Grrr!

The problem was runaway messages coming out of ALSA, being put into .X.err. Something odd about a file pointer being out of sync. Yes, I should have saved the message to troubleshoot, but instead, I just restarted and have been watching ever since for .X.err to be recreated. So far, it hasn't been. (I also should have started out by restarting alsasound, rather than sacrificing 72 days of uptime, but oh well.)

As an aside, I added a line to my crontab: 0 7 * * * test -f /home/me/.X.err && echo "It's back!" This way, I'll be alerted (via email) when it returns. Hopefully, it doesn't grow to 4GB in less than 24 hours.

Sigh. Now then, where was I?

More on file-nr, file-max

2005-06-08T15:15:00.000-04:00

Not to be satisfied with not understanding my "Too many open files in system" problem, I started digging into the Linux 2.6.8 kernel source code.

kernel/sysctl.c defines the /proc/sys/fs files. There's fs_table, an array of ctl_table structs, that defines file-nr and file-max, and establishes that they're pointers to the files_stat structure, i.e. files_stat holds the data that's displayed when you 'cat file-nr'. file-nr is R/O, file-max is R/W.

fs/file_table.c (and include/linux/fs.h) defines files_stat. As you'd expect, it's a struct that holds nr_files, nr_free_files and max_files members. max_files is what gets changed when you alter /proc/sys/fs/file-max.

As described in Documentation/filesystems/proc.txt, nr_free_files is always zero as of the 2.6 kernel. max_files starts out at 10% of memory, but as mentioned, can be changed via /proc/sys/fs/file-max. Examples exist of altering startup scripts to do this at boot time, overriding the default where needed.

fs/file_table.c has get_empty_filp() that does the magic check to see if nr_files > max_files, and if so (and you're not root), deny you the new file. Failing this way puts an info message "VFS: file-max limit reached" into the logs.

fs/file_table.c is the only place that alters nr_files. It gets incremented in filp_ctor() and decremented in filp_dtor. filp_ctor/filp_dtor, in turn, get set as function pointers in fs/dcache.c when the "filp" kmem cache (named filp_cachep) gets created. filp_cachep, in turn, is also only referenced in fs/file_table.c, by get_empty_filp() and file_free().

get_empty_filp() and file_free() call kmem_cache_alloc() and kmem_cache_free(), respectively. These functions live in mm/slab.c.

file_free() gets called largely only by put_filp(), also in fs/file_table.c. get_empty_filp() and put_filp() get called by a variety of places, including fs/open.c, fs/pipe.c, mm/shmem.c and net/socket.c.

"Too many open files in system"

2005-06-02T11:07:00.000-04:00

We got this message on one of our servers recently. I went to our sysadmin guy, who "fixed" it by raising the limit in /proc/sys/fs/file-max. So far, so good. Then we started looking at why we were running out, as that server's been fine for quite a while.

/proc/sys/fs/file-nr reports the number of file descriptors allocated by the system. We had over 200,000 descriptors allocated. /usr/bin/lsof lists open files per process, but showed less than 4000 in use.

I also found an article that explains this file in better detail. It mentioned that there'll usually be a difference between file-nr's and lsof's counts, but it implies that lsof's should be higher, since lsof shows net connections, pipes, maps, etc.

I started shutting down processes, in hopes that one was holding leaked descriptors, but nothing helped. We finally rebooted, and file-nr's allocated count came back nice and small, less than lsof's count, as expected.

If anyone has an explanation for this, I'd love to hear it.

"Could not open relation" in PostgreSQL

2005-06-01T16:31:00.000-04:00

A coworker was getting "ERROR: could not open relation with OID nnnnnnn" in PostgreSQL. I poked around, and indeed, the query he was issuing was trying to use an index that I had dropped the day before. Shutting down psql and restarting it fixed it.

It seemed odd that psql would apparently cache a query plan, rather than letting the server handle it. Unless the problem was in the connection, and not psql, and recycling psql just recycled the connection, which was probably the case, now that I think about it.

Getting the diff between two Postgres databases

2005-05-20T13:23:00.000-04:00

These days, I'm pretty much the de facto DBA at our company. We're using PostgreSQL, which I've also been enjoying.

I keep a diagram (in Dia) with our ERD, and generate the DDL using a utility called tedia2sql.pl that we've modifed a slight bit for our own evil purposes. But from time to time, I've needed to get in quick and alter the database, and wanted a way to make sure that the changes I made (that Boss makes, and who am I to tell him not to) make it back into the ERD.

I went searching for a diff utility, and didn't find one, so I rolled my own. Pretty simple, in fact. What I did was use the generated DDL to create an empty database on my own box, then use pg_dump to get PG to give me the DDL out of that empty one, and out of the development database. From there, diff tells me what I need to know.

I even automated it:


dropdb   -h localhost dbfoo
createdb -h localhost dbfoo
psql     -h localhost -f generated-erd.sql dbfoo
pg_dump -s -h localhost dbfoo > localhost-schema.sql 
pg_dump -s -h devbox    dbfoo > devbox-schema.sql 
diff devbox-schema.sql localhost-schema.sql > diff.txt

Simple, fairly quick, and keeps me in sync. I can then keep the .dia and .sql files in CVS, and get diffs between versions that way as well. The mnemonic I use in reading diff.txt is that 'c' indicates a change (obviously) between the two schemas, 'a' means add to devbox, and 'd' means add to the ERD.

Enjoy.

Eclipse, KDE and X

2005-04-13T09:57:00.000-04:00

I'm didn't bother digging deep enough in this to find a solution, but here's the problem I was seeing:

Over time, I would run out of memory. Swap space would fill up, my box would crawl, and I'd have to restart things. To be precise, the first time it happened, I started shutting things down, assuming that I'd get my memory back. Turns out, I shut down all my apps, and still I was mostly hosed. I looked at 'top', and my X process was taking up close to 1GB of physical and swap space. Logging out and back in took care of it.

When I first started my new job, this wasn't an issue, but I wasn't aware of what I was doing differently. I suspected Eclipse, but wasn't sure. I went through a few cycles of this, each taking between a few days and a week or so. Then I went a couple of weeks with no real problems, and started thinking about what I was doing. The biggest change I had made was doing CVS work from the command line, rather than from within Eclipse.

So I went back to Eclipse, and started poking around in the CVS perspective. Sure enough, when I did a file compare, my X process would grow by 5MB, and never shrink back. I sat there and chewed up 150MB in about 20 minutes of back-and-forth clicking. It seemed like a resource leak moreso than a memory leak, since it wasn't the Eclipse process that grew.

Others in the office were seeing this as well, so I sent out an email. One coworker sent back that he wasn't having this problem, despite doing all his CVSing from within Eclipse. The common denominator between those of us who were seeing this problem was that we were all using KDE and running Eclipse. (Others using KDE but not Eclipse didn't seem to have such problems either.)

The coworker who wasn't having problems was using windowmaker for his window manager. I sat and watched him go through all of the gyrations I did, but X just sat there.

So I switched to Fluxbox. (I use that at home, and love the lightweightness of it.) Since then, I'm back to doing about half my CVSing in Eclipse, (and half on the cmdline), and my memory use has been fairly stable.

I suspect that Eclipse and KDE have bugs that hold hands in this instance. Again, I haven't really investigated this, or searched Eclipse or KDE forums, or anything, but wanted to at least record my findings before I get back to work.

Turning On Remote X Access

2005-03-29T16:00:00.000-05:00

By default, KDE (on SuSE, anyway) turns off remote Xserver access. This means that you can't run an X app on another box (say, hopper) and have the display window show up on your own.

To enable this, edit:

/opt/kde3/share/config/kdm/Xservers

and change at least the first line to not have the "-nolisten tcp" option, then log out of X (back to kdm).

You'll still need to point the remote box to yours for display, i.e. "export DISPLAY=yourbox.foo.com:0.0" before running your app.

This is pretty simple, so you may wonder why I'm mentioning it. Seems that there's a file /etc/X11/xinit/xserverrc with a line in it that mentions a variable called $DISPLAYMANAGER_XSERVER_TCP_PORT_6000_OPEN. This is set in /etc/sysconfig/displaymanager, which is sourced right above it. So I set this var in /etc/sysconfig/displaymanager, and it had no effect. I eventually grepped for "-nolisten tcp", which appeared to be the culprit, and found /etc/X11/xdm/Xservers, but changing that did nothing, either. Nor did the file /etc/opt/kde/share/config/kdm/Xservers, which was just a symlink back to /etc/X11/xdm/Xservers.

As always, Google was my friend, and I found a post that pointed me to the right file, so I'm posting this, in case it might be of some use to a fellow traveler.

Eclipse Not Finding My TestCase

2005-03-09T15:58:00.000-05:00

I wrestled with Eclipse yesterday, regarding a JUnit TestCase. I had always just added a class derived from TestCase, implemented my test methods, and Eclipse has always just run it. I have a Run task that runs all test cases in the project, and I've never had to do anything special. Until today. (Well, sort of.)

Today, I added a new TestCase-derived class. And because I try to write (at least) one test method for each method in the tested class, I copied my actual methods over to the test class, and renamed the methods to start with 'test'. (Yes, I know I should have tested first. Sorry.) I then added an implementation, and ran it. And it passed.

Hmm. My tests usually don't run right the first time, and frequently I sabotage them to make sure they don't, so I know they're being run. I check Eclipse's run window, and sure enough, they're not being run. I go into the Run dialog, and I can't even specify the class in the single-test option.

After much wrestling and wrangling, it turns out that my flaw was in copying the methods over. Seems I forgot to take the parms out. Seems that Eclipse saw no void methods w/no parms starting w/'test', and assumed that this wasn't a TestCase. And wouldn't even show it.

Now, if it sounds like I'm a little miffed at Eclipse, I suppose I am. But what *should* Eclipse have done? Ideally, I suppose a failure could have come up, saying "cannot run anything in this TestCase". Or at least show the class in the Run-single mode, and complain when I try to save it that there aren't any methods with the required signature.

Anyway, now I know. And I'll go back to writing the test cases first. I promise. <sulk>

SSH Tunnelling

2005-02-25T15:50:00.000-05:00

I just setup a SSH tunnel from my box to a distant server. It worked fine, with one minor problem.

When I first tried the tunnel, I would get timeouts on the client, and this message on the server session:

channel 3: open failed: connect failed: Connection timed out

What I didn't realize is that in the command ssh -TCN -L 1234:baz.bar.com:2345 foo@baz.bar.com, the first 'baz.bar.com' is relative to the host machine, not the client. As a result, the command connected to the server, but the port forwarding didn't work because the server didn't know about baz.bar.com. Using ssh -TCN -L 1234:localhost:2345 foo@bar.com did the trick, since the server did know about localhost relative to itself, and the client process connected immediately.

Gentoo Reload, Part 2

2005-02-18T14:10:00.000-05:00

My Gentoo reload continues fairly well, with a minor hiccup.

I mentioned that the network didn't come back up after my first reboot into the installed system. I found an old post (c. 2002) that mentioned removing certain kernel options; this is outdated info, as removing those options gave an error message saying, in essence, "put them back in".

The actual error message was "cardmgr[nnnnn]: no pcmcia driver in /proc/devices". Loading the 'ds' module (modprobe ds) took care of that, and gave me messages in the system log (/var/log/messages or /var/log/everything/current, depending on which logger you're using) complaining about what module wasn't found. In my case, I had to turn on 3c574_cs and serial_cs, and it worked fine. The only annoyance was that because I had been using genkernel, (which compiles many, many modules), each recompile took several hours. (In my case, several 10-minute evenings, as I made changes, then did the make dep && make clean bzImage ... dance and let it run overnight.)

I'm still left with a question and a curiosity. The question is why the difference between the install and the first boot? (The answer is probably that the installer is a LiveCD, and the installed system is not, but I'm going to ask on the forums to know for sure.)

The curiosity is the LiveCD stuff. There's this post that talks about rolling your own custom Gentoo LiveCD. That just sounds way cool; I could have all my system stuff read-only on the CD, mount drives from other machines over the network for writeable files, and never touch the hard-drive on the local box. (Not sure why I'd want to, but that's not the point, is it.)

Strangest Darn Thing (PHP, Apache and a Firefox bug)

2005-02-16T15:16:00.000-05:00

I've been running Postgres, and was told of a nifty web-based tool called phpPgAdmin that makes administering PG databases easier, so I tried to install it.

Somewhere early on in my attempts to verify my install, I started seeing this problem when I would try to access the app (i.e. http://localhost/phpPgAdmin/index.php). It would try to make me download the index.php page, and if I clicked okay on the dialog (that said "you have chosen to open ..."), it would save the page to my homedir. Not what I wanted.

So I fiddled, and I futzed, and I tried just about every config combo I could think of. Nothing worked. I reinstalled PHP. Nope. I copied index.php to dave.php, and that worked fine, but the original index.php didn't. (Permissions were identical between dave.php and index.php.)

I Googled several times, and finally decided to put in "you have chosen to open" as part of my search term. This took me to a page that gave me the idea to try replacing the current index.php with a very simple one, (basically just a Hello, World page). Nope, still asked me to download, and when I saved it, it gave me ... the old page?! Now wait a minute! Okay, let's restart Apache. Try again ... still the old page! Now come on...

Okay, let's shut down Apache. Same thing: "you have chosen to open ..."! So Firefox wasn't even going to the server for this. Restart Firefox, try again. Same thing. Yup, somebody's wedged.

I finally, rather than save the file, tried to get Firefox to open it. It sat there, paused; I clicked Cancel and Retry, and it finally said something about not being able to contact the server.

Now, then. Let's start Apache, and try again. Now it works.

Best I can tell, Firefox must've gotten wedged caching this file way early on. Not sure if Apache was complicitous or not, (and PHP and phpPgAdmin seem innocent enough). Grrr.

Email From A Coworker

2005-02-15T10:15:00.000-05:00


Subject: obligatory linux gloating

Uptime on xxxxxx

10:17am  up 648 days,  1:07,  2 users,  load average: 1.91, 1.72, 1.40

Go ahead, LAX, boot every month or so. We're here, when you're ready.

Upgrading SuSE

2005-02-07T11:38:00.000-05:00

This is old news from me at this point, as I performed this upgrade a while back, and just forgot to blog it, but I'm mentioning it, as it continues to nibble at my hind parts from time to time.

I upgraded my SuSE install. Apparently, I went from 9.1 Server to 9.2 Workstation, or somesuch, and it removed a bunch of stuff. Like PostgreSQL and Ant, among other things. And when I went back to install those two things off the 9.1 disks, it thought I wanted to downgrade everything. Harumph.

It would have been nice to have had YaST mention that I was doing this. Techically, it said that it was removing outdated packages, but during an upgrade, that's what's supposed to happen, IMO. I didn't make the connection that those particular packages wouldn't be reinstalled, and it wasn't until much later, when I couldn't even find PostgreSQL in the package list, that I realized I had switched products. (In my defense, Boss just handed me the disks and said "here, dude, upgrade your box".)

I "fixed" things by pointing YaST at a full online 9.2 mirror, at which point it had the full list of packages to choose from.

Then today, I reinstalled ant. Okay, fine. But from within Eclipse, ant found javah just fine; from the command line, it didn't. Solved it by installing SuSE's ant-nodeps package. Eclipse includes everything ant needs, apparently, and doesn't expose it to the command line version.

Postgres and bigint

2005-02-02T16:13:00.000-05:00

We're putting together a database design. In this design, we decided to specify all of our primary keys as bigint, as we're going to make them unique across the system, and we'll (hopefully) have more than 2^32 rows system-wide.

Somewhat unrelated to this, Boss came up with a question about foreign keys, wondering what the performance would be on them if we wanted to do ON DELETE CASCADE between two large tables. So I went to investigate.

My first try, to delete a row on an ON DELETE RESTRICT table that still had fk referrers, went quite fast. Seems I picked a key that was at the top of both tables. Fortunately, I thought to try a key at the bottom of both, and sure enough, it was doing a table scan.

I checked my indexes, and added one, on the fk column. Seems like it would be reasonable to have the db add one as part of the fk constraint, but maybe not.

But even with indexes, it still did a full table scan. I poked around, and found that even a pk search was doing a table scan. After a good deal more poking, I Googled, and found that indexing bigints is buggy under PostgreSQL 7.mumble (i.e. the indexes won't get used, ever), but that under 8.0 works fine. Fortunately, one of our newer servers was already running 8.0, so I tried it, and voila, worked fine.

So now I'm moving my box up to 8.0 as well.

(Minor note: I didn't track down the actual source of the problem, but was just satisfied with moving up to 8.0. Just so you know.)

Gentoo Reload

2005-02-01T10:28:00.000-05:00

Just for grins, I decided to reload my old laptop with Gentoo.

For those following along at home, I had put (and praised) SuSE 9.1 on this laptop, but it didn't detect my PCMCIA NIC, and the video card itself had been fritzing, making X less than usable, and displaying the root console with huge borders (i.e. using only 1/4 of the screen). I'm now wanting to run the laptop essentially as a server, with VNC handing the display, and letting another box do the actual I/O.

So I downloaded the universal CD via BitTorrent (in part to be able to seed it when I was finished), and popped it in. It detected all my stuff, including the PCMCIA NIC, and had no graphics problems (although it didn't run X, either). All went swimmingly, until I rebooted, at which point it didn't find PCMCIA, and the root console was small again. At present, I'm not all that interested in an extended troubleshooting session, especially as the install CD worked flawlessly.

But it also makes me curious what they use for the installer, and whether or not I could just use that. I glanced at Knoppix, as I have this vague feeling they do something similar, but I'm thinking that I'll go play in the Gentoo forums for a while and see if I can get any pointers.

More as it unfolds.

String vs StringBuffer

2005-02-01T10:06:00.000-05:00

On a lark, while a particularly long data load was going on, I wrote some code to try out timing the differences between doing string concatenation with a StringBuffer, versus just writing out String math. The different in timing was surprising.

Of course, no entry like this would be complete without quoting Hoare and Knuth, that Premature optimization is the root of all evil. If String math is clearer, it should be used until there is a clear reason to do otherwise, IMNSHO. There, I said it.

Having said it, I wrote this:


String bigLongString = "cranky ";

bigLongString += "poodle ";

for ( int i = 0; i < 100; ++i )

	bigLongString += "frank ";

bigLongString += 25;

bigLongString += " times.  Hoorah!";

return bigLongString;

and ran it 10,000 times. On my machine, it took around 14.2 seconds to run. I changed it thusly:


StringBuffer sb = new StringBuffer();

sb.append( "cranky " );

sb.append( "poodle " );

for ( int i = 0; i < 100; ++i )

	sb.append( "frank " );

sb.append( 25 );

sb.append( " times.  Hoorah!" );

return sb.toString();

and it took around 0.92 seconds to run 10,000 times. I changed it again to reuse the same StringBuffer (via sb.delete( 0, sb.length() );), and it took around 0.5 seconds to run 10,000 times.

That's a pretty stark difference in times, and pretty surprising, too, that reusing the StringBuffer would halve the execution time again as well.

(For the curious, "Cranky Poodle" is a parody of "Yankee Doodle", as can be found in the book Take Me Out of the Bathtub, by Alan Katz, and is a hilarious book.)

Generating Code

2005-01-26T09:15:00.000-05:00

In conjunction with tedia2sql, I also wrote some Python code to take those custom files that our hacked version of tedia2sql generates, and use them to generate Java data objects. In this case, the data objects have getters and setters, and 'getFooAsString()' getters, and are derived from a base class that has insert, update and delete methods to call our internal database layer (that sits on top of PostgreSQL). That's a mouthful, but the upshot is that I enter the stuff into Dia, save the diagram, run one script, and can start writing database client code.

Automating away tedious stuff is awesome. (Or put another way: it's great to be lazy.)

tedia2sql

2005-01-26T09:06:00.000-05:00

We're up to designing our database. We're porting an existing database from Filepro (?) to PostgreSQL, and wanted to have a diagram of our ERD. After a brief bit of poking around, we found tedia2sql, a Perl program that takes a diagram out of Dia and generates DDL for you. It's Open Source, of course, so we were able to have one of our crack Perl coders (i.e. not me) modify it to generate some files used by an internal program. Just what the doctor ordered.

It's interesting, too, in that it uses Dia's class diagram widgets, (which are more functional than its database widgets). But the docs are quite clear, and it was very easy to put into Dia the objects that tedia2sql needed, and the PostgreSQL DDL worked just fine. (tedia2sql also supports MySQL, Sybase, Oracle, DB2, and InnoDB.) Another nice thing tedia2sql does is automatically generate join tables, if you specify a n:n relationship between two tables. Very helpful.

PostgreSQL Docs

2005-01-07T13:42:00.000-05:00

One other PostgreSQL tidbit: I'm pretty impressed with the docs. They're fairly thorough, fairly readable, and aimed at someone without lots of pg experience. (In fact, they're aimed at someone without much db experience.) Good job, folks.

PostgreSQL Performance

2005-01-07T13:41:00.000-05:00

Today I started poking around in PostgreSQL, doing some performance testing. I loaded it up with a million rows in one table, and a half-million in another, and started doing some selects to see how fast it would run. And played around with 'explain' to see what pg thinks it should do.

The first interesting thing I saw is that at some point, it'll switch from doing an index scan to do a sequential scan (i.e. a full table scan). In my case, I'm simulating 5 years worth of data, and doing a query across a date range, and seeing results come back in several seconds. 'explain' shows that if the query is over more than a week or so, it'll do a full table scan rather than use the index.

So I look at my data, and realize that I'm generating the dates randomly. In reality, the dates will be in physical order on the disk in roughly date order (i.e. written as they're created). So I redid my data, reran the explain, and now even a full year range uses the index. Neat stuff.

Digging deeper, I find that 'analyze' generates statistics on each field in each table, and put them in the pg_stats table. One of the columns is called 'correlation', which indicates if like data can be expected to be found close together or not. Changing the load order on the table changed the correlation value from almost 0 to almost 1, meaning the pg figured if it needed rows with similar values from that table, it could expect to find them close together on disk, meaning that an index scan would deal with data that in most cases would already be in memory.

So when my dates were random, correlation was 0, meaning that if (for example) the table had 15000 pages on disk, and I wanted 15000 rows (which pg could also estimate), then it would be reading all of the pages anyway, and would choose a sequential scan. When correlation is 1, 15000 rows might live in 500 pages, meaning reading the index and those 500 pages would be much quicker than a table scan, so the index scan would be chosen. (Hope this makes sense.)

Google Likes Me

2005-01-05T16:44:00.001-05:00

As part of my khubd thing here, I googled for khubd, found 19,800 pages, with my recent blog entry at the top! I wondered if Google put it at the top knowing that it was mine, but I searched from a coworker's machine, and got me at the top as well. Now, this blog has very few readers at present (at least, judging by what I see on bloglines.com), so I'm wondering if I'm getting some kind of karma from being on blogspot, or something. Anyway, I find this quite curious.

khubd, Round 2

2005-01-05T16:44:00.000-05:00

Plugged in MP3 player, khubd hung again. Googled for a while, and found nothing that jumped out at me to try. A couple of people reported having this problem, and confirmed its association with plugging in a USB device.

Servlet Mini-Annoyance

2004-12-28T11:33:00.000-05:00

Was just putting together a test servlet, and was trying to get some values from a form. Spent about 30 minutes trying to figure out what I was doing wrong; in the end, I was calling getAttribute() when I should have been calling getParameter(). I still don't know what attributes are; in my case, they're coming up empty (i.e. getAttributeNames() returns an empty enumeration). Ah well.

Moving NB Project to Eclipse

2004-12-20T11:31:00.000-05:00

Having accomplished the task of getting my build.xml to work under NetBeans, I need to go the other way, and get the same thing working under Eclipse. So I copy the NB folder over, and try to create an Eclipse project from it.

First thing Eclipse does is try to create src/com/foo/Hello packages, and gives errors on all files. That's not what I want. I try to delete the files from the project. Whoops, that actually deletes the files. I copy them back over.

Let's try importing. Hmm. That complains that it's trying to import into the same place that it's importing from. Let's create a new project. Okay, that imports okay, but doesn't place the files into a ./src folder. Scrap that idea.

Let's copy the project from scratch again. Still gives me the src/com/foo/Hello packages. Maybe we can move things around. I create a ./src source folder. It warns me about the overlap, but then fixes the packages, and puts things where I want them. But there are compile errors. Ah, just things missing from the classpath. Fixed that, and life is good. I can run my JUnit tests, and I can run build.xml.

Much easier going from NB to Eclipse than vice versa, for this particular test, at least.

First Look at NetBeans 4.0 Final Release

2004-12-17T13:58:00.000-05:00

Back to NetBeans. I have a project with a custom Ant script that I want to get working under NetBeans. I do the New/FreeForm Project thing, and get to the part that asks for the source folder. I specify 'com/foo/hello', and it loads both .java and .class files (at present, I leave both in the same place, which is probably bad practice, but works fine under Eclipse, and Ant doesn't mind either). Loading both seems bad, so I delete the .class files, then open a source file. It marks an 'implements' line as being an error, even though the interface I'm implementing is in the same package (and same directory). Something's wrong; I remove the project.

2nd attempt: specify the top-level path as the source folder. This time, no source files show up, and I can't find anywhere to add or import an existing file. It won't let me add the existing package, (which makes sense, as it already exists, but I was hoping it'd see it and use it, or something). I remove the project.

3rd attempt: RTFM. The docs say to specify the top part of the path, which in this case is './com'. Huh? How counterintuitive is that?! Okay, I'll play along. I figure I'll come back to the classpath thing, and just click Finish. Nothing happens. Ah, on the status bar it's giving me an error message: "java.lang.reflect.InvocationTargetException". Cancel, try again, same thing. Restart, it won't even go past the project location page. Restart, try again, now it works. Odd.

Well, sort of works, as the project list shows a 'com' folder, with 'foo.hello' and 'foo.hello.client'; I would have expected to see 'com.foo.hello' under the project name ("HelloWorld", in this case). I expand 'foo.hello', and see both my .java files and my .class files. Also not what I expected; maybe under the Files tab, but not on the Project tab. I delete the .class files.

Then I run the build, (i.e. run the 'main' target of my build.xml, which builds the whole thing). It complains about not finding the 'cc' task; it's in a jar in ~/.ant/lib. Ant from the cmdline finds it fine, but this is okay. Adding the jar to my project classpath does nothing; ah, it wants it under the additional classpath under Options/Ant Settings. Makes sense. The project compiles, and deploys. Of course, the .class files are back as part of my project. Hrm.

Sidebar: In trying to add cpptasks.jar to the Ant buildpath, NB's "Add JAR File" dialog didn't pick up my ~/.ant dir, making me type it in by hand. And not just ".ant" and hit Enter, either, as that closes the dialog; I had to navigate to "~" and type ".ant/lib/cpptasks.jar". Kind of defeats the purpose of having a dialog.

Next odd thing: Hover over a .java filename under the Project tab, and it says 'Invalid package declaration: "com.foo.hello"'. Probably because it's separated 'com' and 'foo.hello' as two separate things. Probably wants me to have a ./src dir, and com/foo/hello under that. Let's see ... okay, that project gives me a ./src dir, with com.foo.hello and .client packages underneath. Grr. Making me do things your way isn't very friendly.

So I need to alter build.xml to get at my source in ./src. Fixed my javac task, so now things compile okay. Fixed my javah task, but now javah gives java.lang.reflect.InvocationTargetException. Well, actually it also said it couldn't find my class; silly me, I pointed it at ./src, while it wanted ./build. So why on earth does it say "Error starting javah", when javah started fine, but just couldn't find the classfile?!

Okay, now my build.xml runs fine. Interestingly, an empty build (i.e. I just ran it, so there's nothing to do) runs in 2 seconds under NB and 6 seconds from the command line. I guess NB is using the existing JVM. Repeated clean builds take 6-10 seconds in NB, while cmdline takes 15-16 seconds. (Of course, Eclipse recompiles .java behind the scenes, which is what I usually want to build...)

Bottom line is that I got NB to work, but I'm barely whelmed. The next task will be to work in it for a few days and see if it grows on me. My boss is attracted to it, and we both would like to make an informed decision about it, or at least verify that one project can be developed by folks split between IDE's.

.desktop Files and KMenuEdit

2004-12-17T10:36:00.000-05:00

In an attempt to properly evaluate the IDE that we'll be using, I pulled down the final release of NetBeans 4.0, and installed it. I then went to load it, which in my case meant going to the KDE menu (on my SuSE box). I found the menu item for NetBeans, but assumed that it was still pointing to my old install, 4.0b2.

So I went to SuSE's Control Center, and tried to remember where they put the config tool for changing the menu. It's hidden under the 2nd or 3rd tab under Panels (which only makes sense if you really strain hard; in this area, Windows has it nice, just right-click on the menu item to edit it). Anyway, I then click the Edit K Menu button, which brings up KMenuEdit.

Only KMenuEdit doesn't show any entries under Development/IDE, which is where I have both NetBeans and Eclipse. Both show up on the KMenu, but neither do under KMenuEdit. I search the KDE bug database, and it appears that this is a SuSE bug, already fixed in a later version. (Mighty nice to be able to search a product's bug database.)

So where does KDE store this info? Apparently in .desktop files. I found netbeans.desktop in /usr/share/applications, updated to reflect the newly installed app. I'm guessing that the installer took care of that, pleasantly enough. The Eclipse one, that I added manually, was in ~/.local/share/applications/Eclipse.desktop.

So, it turned out that I needn't have gone to this trouble, but I figured I'd post this in case others have this difficulty, or are new to KDE and are curious.

ssh Without a Password

2004-12-16T16:01:00.000-05:00

Okay, I have access to another box. I can ssh into it, but it requires a password, so I copy id_dsa.pub to server, put it in a .ssh dir, rename it to authorized_keys, but ssh still requires password; ssh -vvv says:

debug1: Offering public key: /home/djaquay/.ssh/id_dsa

debug3: send_pubkey_test

debug2: we sent a publickey packet, wait for reply

then a delay, then:

debug1: Authentications that can continue: publickey,gssapi-with-mic,password

debug2: we did not send a packet, disable method

debug3: authmethod_lookup password

debug3: remaining preferred: ,password

debug3: authmethod_is_enabled password

debug1: Next authentication method: password

I'm not sure what that "debug2: we did not send a packet" means, especially as the earlier debug2 says "we sent", (past tense). I've verified the authorized_keys in ~/.ssh, and it looks fine. I vaguely remember something like this happening a while back, but don't recall how i solved it (should have blogged it then; will blog it now).

I changed sshd_config on host to only use protocol 2, and uncommented the 'PubkeyAuthentication yes' and 'AuthorizedKeysFile .ssh/authorized_keys' lines, to no avail.

Odd thing: doing '/etc/init.d/sshd restart' doesn't drop my current connection; ah, it forks the process for each connection, so it restarts the main process, making new connections off the restarted one. I drop my own connection after doing each restart.

Talked to my network guy after suspecting that my box didn't have a proper name or DNS entry; he fixed that, and now there's no delay before being asked for my password.

Finally broke down and Googled for that "we did not send a packet" message, and voila (as they say in France), up turned a message on the SecurityFocus forum saying that my .ssh directory and authorized_keys file must both not be group or world writeable; both were, and once fixed, no password was required.

System.setOut()

2004-12-16T10:13:00.000-05:00

This post talks about using System.setOut() to change where System.out sends its output. Useful, as the article mentions, when you don't really want everything going to stdout.

Bootchart and Visualization

2004-12-16T08:47:00.000-05:00

From Slashdot recently comes this article on Linux boot performance visualization. In addition to being somewhat interested in boot time optimization, I was quite intrigued by the images that bootchart puts out.

I'm fascinated by the power of visualization, especially as it relates (or possibly might relate) to coding. From simple stuff like editors that collapse method bodies to the use of UML and beyond, it's something that I just have this gut feel could really be powerful in comprehending a system and making informed, meaningful changes to it.

(OTOH, I could also just be fascinated by small, shiny objects...)

Eclipse and the Mouse Scroll Wheel

2004-12-10T13:05:00.000-05:00

In getting back accustomed to using Eclipse, I was playing around with the mouse wheel. Seems that when I scroll with it, the page moves by half of the page height, which is counter to how just about every other app uses the scroll wheel. I Googled for a solution, but found none, and went to the Eclipse IRC channel on Freenode and got an answer.

Seems that Eclipse on Linux uses GTK+, and the version of GTK+ I have (2.2.4) is what's making Eclipse do that. Apparently it's not something that Eclipse controls, and that has been fixed in more recent versions of GTK+. Oh well, guess I won't be using the scroll wheel for now.

Loads of Fun

2004-12-10T11:01:00.000-05:00

Well, I've been busy, and haven't posted as things have been going fairly swimmingly. So let's see, what have I done?

I got a test servlet and test JSP page running under Tomcat, and a simple object coded, using Eclipse. (Eclipse is way cool, BTW. More on that later.)

I got Spring's MVC running and forwarding a request to that JSP as a view. Spring is way cool as well.

I also got Spring to expose that simple object via SOAP, using Axis. One little oddity there: I had setup Spring to handle requests to Axis under a ./axis path; Axis puts a default page at that location with links to request the WSDL files, but the links assume that you're running under a ./services path. I tried them out first thing, and they broke, leading me to think something more serious was wrong. I switched my web.xml servlet-mapping to put the Axis servlet under ./services, and all was well.

I then got an object talking to PostgreSQL via their JDBC drivers. Actually, jdbc.postgres.org was down, but I managed to get fairly current drivers from someone off of the IRC channel on Freenode.

Lastly, I got some data objects coded up for my sample database, using JNI to make calls into a company-internal library that we have. It's been a while since I've done any C coding, but it brought back some mostly-good memories. (Although I'm not giving up Java to go back, that's for sure.)

khubd in "uninterruptible sleep"

2004-12-03T16:57:00.000-05:00

Curious thing. My load average is hovering around 1.0, but nothing is taking up any CPU time. Makes sense, sort of, as the load average is (I believe) the average number of processes active, and not a measure of CPU usage. So I look at top, and see that no processes are in run state.

But there is one, khubd, that's in D state. The top manual reports that this is "uninterruptable sleep". I Google (of course) for khubd, and find that it's a kernel thread, and a suggestion that removing usbcore will kill it. Curiously enough, I'm not able to automount my USB MP3 player (it worked on my old machine, last week, but not on this one). Probably not a coincidence.

So I go to unload usbcore. usbcore is used by three other modules: usb_storage, usbserial, and uhci_hcd. I remove usbserial; no problem. I rmmod uhci_hcd, and now I have two processes in D mode, the new one being rmmod. Can't kill or kill -9 rmmod; the kernel is serious about the uninterruptible thing. Hrm. Time to reboot?

Bounced my machine, khubd came back, but is now in S mode, or just sleeping. I suppose the next thing to try is plugging in my MP3 player again, and see if that's what wedged khubd.

Moving up to Firefox

2004-12-02T16:42:00.000-05:00

I've been using Mozilla for a while now. Probably since version 1.1, IIRC. I've been interested in Firefox since it was announced, but not interested enough to give it a try. Well, coming into a new job, I decided to try it out. Peer pressure, and all that.

I love it.

I had been thinking all along that it really wouldn't be that much of a change from Mozilla, but I was wrong. It's got much nicer fonts, for starters. Much easier on the eyes.

The search bar is really cool; Ctrl-K takes you to a search field that defaults to searching Google, but has several other search engines as well, including Amazon and eBay. And you can add more. There are hundreds (!) of search engine plugins that you can get from Mozilla.org. I added Wikipedia and SourceForge; you can get CDDB or IMDB, Creative Commons, and a huge number of specialty sites. Nifty.

It's a small download, too. Less than 5MB. It does this by not having many of the standard plugins like Flash and Shockwave, but it makes getting them a snap, if you want to. (I don't. At least not yet.)

Of course, it still has tabbed browsing and popup blocking, but now popups are blocked by default, which is nice. The preferences are leaner as well, and better organized. Much more like OS X than Windows. (Mr. Bill hasn't innovated his way back up to Apple yet, apparently.)

Finding on the page is really neat; Ctrl-F opens an unobtrusive little find bar at the bottom of the page, which stays there until you close it by pressing Esc. Beats the Ctrl-F, type-a-term, Cancel, Ctrl-G dance I used to do.

Anyway, Firefox is a really nice browser, and really worth the trivial amount of time it took to install it. I can really see why all of the early adopters have been switched to this for months, and why IE is losing percentage points so fast. Way, way cool.

Wrestling with Axis

2004-11-29T11:23:00.000-05:00

More on Axis. Started playing with the examples, and got 1 and 2 working just fine. That .jws thing is pretty cool.

Then I got to example3. I got the MyService.java and Client.java files to compile fine, then ran AdminClient on deploy.wsdd to deploy them. Got a "401 Unauthorized" back in an AxisFault, and went looking for how to setup admin security. Poked around a bit, and found not much on AdminClient. Started looking at AdminServlet, which said not much, and AxisServlet, which has a WSDL that just exposes an AdminService method, but says nothing about how to configure it on the back end.

I had seen mention of server-config.wsdd, but when I looked for it, I didn't find it. Eventually, I grepped the docs for mention of it, and found in reference.html that Axis will create it if it doesn't exist, and will add to it when you deploy things. So I created it, and added a globalconfiguration section and an adminPassword parameter. The 401 went away, replaced instead by a "500 Internal Server Error" and a voluminous stack trace.

Looking at the exception output, it said "Invalid WSDD element 'globalConfiguration' (wanted 'deployment')", which contradicts reference.html. Ah, (oops), it wants that globalConfiguration wrapped in a deployment tag. (Googled and found an example server-config.wsdd, and mirrored the relevant parts of that; did I mention that I love Google?)

So now my AxisFault says "Server.NoService", and "The AXIS engine could not find a target service to invoke! targetService is null" when I run AdminClient. Seems that if server-config.wsdd is missing, AdminService (and Version) get deployed, but if server-config.wsdd is present, only those services listed therein get deployed. Okay, fine, how do I deploy AdminService manually? No wsdd file ships with Axis, it seems (apart from the ones in the samples dir).

I then reread the install instructions, that say that "the client" (presumably AdminClient) has a default password, so I decide to go look at the source and find out what the default password is (on the server) so I can provide it to the client; I assume that it's out of sync. In the process, (of looking through the Axis CVS), I find a server-config.wsdd that has reasonable-looking entries for AdminService (and Version). Plopped them into my server-config.wsdd, kicked Axis via Tomcat, and I'm back to a 401. Grrr.

After looking at the Admin.java source in CVS, I decide to at least try enabling the remote access, just in case. Sure enough, I make that change, and life is fine; "AdminClient list" returns the blob of XML expected. Harumph. I'm doing this all on one box, so why didn't it see that I'm local?!

Okay, back to what I had been doing. I run the example3 deploy.wsdd, and it accepts it. Running AxisServlet lists MyService. Running Client says "The AXIS engine could not find a target service to invoke! targetService is null". This is what I got before when Axis wasn't running anything, but now, if I do a "AdminClient list", it shows MyService, with allowedMethods of "*" and a correct classname.

Now, oddly enough, I went looking for the .class file under Tomcat, under WEB-INF/classes, and found an old version, that I'm guessing shipped with Axis and that I had copied over when I installed Axis under Tomcat. So I replace that with the one I just compiled. I undeployed, and the file remained. So is this where the file gets deployed? And do I have to place the .class file(s) myself, or does the deploy.wsdd and AdminClient take care of that?

I removed MyService.class, undeployed, and deployed, and can find no MyService.class anywhere under tomcat/webapps. So I copied it there. Same error. Un/redeployed (via AdminClient and the wsdd files). Same error.

If I go to the MyService URL, Axis says "Hi there, this is an AXIS service!", while if I go to another, invalid one, Axis says "No service is available at this URL". So I'm guessing that Axis at least knows that it should know about the service. But from Client, hitting either MyService or a known invalid one both generate the "targetService is null" message. Kicked Axis, then Tomcat. Same error. Removed every copy of MyService.class I could find. Same error, and yet the un/redeploy worked fine. How *does* this work?

Ran happiness page again, installed mail.jar (just for grins), and kicked Axis. Ran AxisServlet page, and got message saying "Fault - Could not find class for the service named: samples.userguide.example3.MyService. Hint: you may need to copy your class files/tree into the right location (which depends on the servlet system you are using)." Nice of them to confirm that.

Copied MyService.class file into Tomcat's webapps/axis/WEB-INF/classes/samples/userguide/example3. Reloaded AxisServlet, and the "Fault" message went away. Still get the "targetService is null" message, but at least I can see that I need to place the class files myself, and that I'm doing so in the right place. Time to post to the board, methinks.

Or just Google for "targetService". Found a message from Steve Maring (thanks, Steve) saying that you can't just pull down an incomplete version of server-config.wsdd, or use deploy.wsdd in its place. Instead, extract it from axis.jar, and modify it to include what you need. Did so, re-enabled remote admin, kicked Axis, redeployed the example, and it worked. All's well that ends well, I suppose, but somebody deserves a "grrrr"!

Tomcat Manager

2004-11-23T16:59:00.000-05:00

Okay, Tomcat has a Manager (which can be discovered via RTFM) that can redeploy a webapp without bouncing the whole thing. RTFM. It's your friend.

Spring and Axis

2004-11-23T15:35:00.000-05:00

Started looking into Spring. Inversion of Control (IoC) and Dependency Injection are pretty cool things. (See Martin Fowler's article for more info on that.) Spring uses IoC/DI to remove dependencies from your objects, turning them into POJO's, which in turn makes them much easier to test. And Spring makes using web services (via JAX-RPC, anyway) a much easier task.

So I'm now setting up Axis. You plop it into Tomcat, then run a test page to see if all's well. It wasn't. It complained: "Error: could not find class javax.activation.DataHandler from file activation.jar". I have a few different versions of activation.jar on my box; I copied the one that came with Spring into axis/WEB-INF/lib, but Tomcat didn't see it without a restart. (There's probably a better way to get it to see it; I'll look into that shortly...)

With activation.jar in place, Axis' test page is all happy, so I tried out a couple of the samples, and got NullPointerExceptions. Poked around on Google a bit, and it seems that Axis 1.1 doesn't play nice with JDK 1.5. Grabbed Axis 1.2RC2, installed it, replaced it in Tomcat, added activation.jar, got the happy page working, tried the samples, and viola (as they say in France), I get proper responses.

SuSE and my Muvo

2004-11-18T11:22:00.000-05:00

I had played around under Gentoo with my old USB keydrive, and got it to do a directory listing, but not read or write. It was still on my list of things to pursue at some point. So it was with a wee bit of trepidation that I plugged in my Creative Labs Muvo NX MP3 player into my work box, (running SuSE 9.1). See, I really, really needed my .emacs file from home to be able to work like I am accustomed to.

Well, nothing to see here, folks. I plugged it in, heard some HD chatter, and got a dialog asking me if I wanted to open a KDE app when new media is detected. It found the drive, mounted it under /media, and popped up a file browser. Copied off the desired files, pruned a few podcasts that I listened to on the way in to work this morning, and I'm good to go. Even better, I can now work in Emacs like I like.

I'm liking SuSE more and more.

Bruce Eckel on Thinking in a Different Language

2004-11-17T13:22:00.000-05:00

GatorJUG has (or had, I can't find the archives at the moment) a podcast of Bruce Eckel talking about programming languages. The point he made (or at least the one I was most interested in; he talked about a couple of things) was that you think differently in different languages.

One example he gave was about dynamically loading a class at runtime. In C++, it's possible (he said), but not something you're likely to even think about doing. In Java, it's quite easy, making it something that you'll probably run across at some point. And while you can dynamically create classes at runtime in Java, in Python it's a very simple exercise. Using each language gives you a different insight on solving problems, and mixing languages can be a real boon to solve a given problem, given the strengths of each.

He also mentioned the type differences among C++, Java and Python, and how loosening up type restrictions greatly changes the nature of solving some problems.

He also mentioned the applicability of this idea to natural langauges as well. This is something I've been greatly curious about, and is one of the reasons I'd love to learn a 2nd language. I'd love to see just how different expressing given ideas or thoughts are between two languages, and whether thinking in another language would change those ideas, somehow. (Maybe next month...)

Ant, Java and SuSE Mixup

2004-11-17T13:20:00.000-05:00

My "new", current, (temporary) work machine is an older box with 190MB RAM, which isn't quite up to the task of running Netbeans. (NB takes up more than that by itself.) So I've dropped back to coding in Emacs, and running the NB project Ant script (and Ant) from the command line. (I should get a better machine by next week.)

Which brings us back to the question of how to setup the taskdef classpath without using the NB dialogs. It looks like NB wires stuff up before calling Ant during builds, but doesn't make it apparent what gets wired or to what values.

I ended up using properties for the XDoclet jars, chaining them together into another property for the classpath required for a single taskdef, and putting that property into the taskdef's classpath attribute.

Then it wouldn't compile. My setup was thus: Java 1.4.2 was already installed on the machine, I then installed 1.5 manually (i.e. not via SuSE's mechanism, as they didn't have 1.5, at least in YaST), and then installed Ant via YaST. The Ant script (in /usr/bin/ant), for some reason, was assuming that I was using a 1.3 JDK, and when run, ant said "Unable to locate tools.jar. Expected to find it in /usr/lib/SunJava2-1.4.2/lib/tools.jar". Took me a while, but I found the line in /usr/bin/ant that referred to 1.3, changed it to 1.5, then looked at the setJava script, and created the proper .conf file in /etc/java. And voila, no runs, no drips, no errors.

More on Ant Paths

2004-11-11T13:51:00.000-05:00

Tried running build.xml from the command line, and got errors regarding XDoclet. Makes sense; I mentioned earlier that I added the XDoclet jars in the Additional Classpath entry under Ant settings in Netbeans, which means that they're not listed in the build.xml itself. Gotta change that.

Odd XDoclet Error

2004-11-11T13:46:00.000-05:00

Tried to get XDoclet to generate a .wsdl file; got error saying something about an ambiguous subtask definition for deploymentdescriptor. Couldn't figure out what to do about it, and Googling revealed nothing useful. I had two targets, one each for wseedoclet and ejbdoclet. I combined them, and it worked.

I decided to blog this, so I moved wseedoclet back to its own target to get it to generate the error message again, and it worked. Go figure. Even after doing a clean and build, etc.

Netbeans, Ant and XDoclet

2004-11-09T17:47:00.000-05:00

Today, I played around with getting NetBeans 4.0 Beta 2, Ant 1.6.2 and XDoclet 1.2.2 to talk together and play nice. (NB 4.2b2 comes w/Ant 1.6.2 built in; XDoclet is separate.) The goal was to get JBoss into the mix as well, but that'll probably have to wait until tomorrow.

I created a simple project (Hello, from the Mastering EJB 2.0 book). NB shows the build.xml file under the Files tab (and not on the Project tab, where I was first looking for it). build.xml expands to show all targets; most of the targets live under nbproject/build-impl.xml, and several are empty and can be overridden in build.xml. (The overridables are listed in a comment.)

For background, Ant build files set up chains of dependent targets. NB's default is default, which depends on dist and javadoc. dist in turn depends on init, compile, pre-dist, do-dist and post-dist, which it most likely runs in that order; each of these depend on others, and so on.

So I added various tags to my simple bean file. At the class level, I added an @ejb.bean tag, with name and type attrs. jndi-name and local-jndi-name attrs also look useful. I also added an @ejb.interface-method tag to my one business method.

In build.xml, I added an ejbdoclet target, with an embedded task of the same name, depending on init. The destdir attr looked most useful, and I added an excludedtags and verbose attr as well. I then added remoteinterface, localinterface, homeinterface and localhomeinterface elements (all empty), and a fileset that listed my one source file, explicitly.

I'm pretty sure there's a better way to list my source files. I tried **/*.java, but with my destdir putting the generated files into my source dir (which may be a problem), compiling a 2nd time gives me interfaces for my interfaces, which is wrong. So for the moment, I'm listing each file separately, and will be looking for a better way. Like having a "generated" directory that's separate, or something. Suggestions welcome.

Then I troubleshot it. First off, it couldn't find the ejbdoclet task. Played around with the compile classpath, to no avail. Found that I could add an Ant classpath element under the taskdef, but that felt unwieldy, especially as I had 5-6 jars eventually to add. Was about to separate it out as an Ant typedef, then thought that since Ant is a proper part of NB, there should be a config option. There was, under Tools/Options, Building, Ant Settings, Additional Classpath. Added several files there (xdoclet, xdoclet-ejb-module, xjavadoc, commons-logging and -collections, and jboss-j2ee.jar), and life was good.

I was also getting empty bean interfaces. Seems I forgot to put the @ejb.interface-method on the method, but adding the tag fixed that.

One thing made it slightly harder than it should have been. I had intended to start out with a very simple stateless session bean, and mimic xdoclet's sample. It turns out that xdoclet's sample stateless session bean was a sample of how *NOT* to let xdoclet generate the interfaces for you. Bad choice on my part. Ah well.

Shared Library Woes

2004-11-03T21:10:00.000-05:00

Pulled down Sun's J2EE SDK, and ran the installer, which couldn't find libstdc++-libc6.2-2.so.3. Found what I thought was the Gentoo package, but all were masked. Set ~x86 in ACCEPT_KEYWORDS in /etc/make.conf, emerged, and tried the installer again; no luck. Googled, and found that the file is part of the libstdc++2.10-glibc2.2 package; did emerge search, but didn't find it. qpkg -f libstdc++-libc6.2-2.so.3 turned up nothing. Googled again, found libstdc++2.10-glibc2.2 in a .deb file on packages.debian.org.

Okay, now how do I pry apart a .deb file on a Gentoo system? Wikipedia knew; it's an ar file, that contains .tar.gz files. No sweat, extracted the file, the installer didn't find it; remembered to run ldconfig; didn't find it; looked for the file; didn't find it. what!?

Re-extracted it, looked closer, and saw that it's just a link; extracted the link target, copied that as the link name, ran ldconfig, ran the installer, found it; whew.

I couldn't imagine that Gentoo didn't have this somewhere, so went back and searched the forums. Didn't find it. Googled for it, saying 'libstdc++-libc6.2-2.so.3 site:gentoo.org'. Found it, it's in lib_compat. emerged that, and life's good.

PostgreSQL == Good

2004-11-02T17:02:00.000-05:00

Despite having finished the emerging that I thought might have been causing problems, Eclipse continues to crash (I will pursue this later).

I worked through the PostgreSQL tutorial and docs, and got things running on my machine. I'm liking PostgreSQL as well; table inheritance looks like a neat thing. I'm sure there are times when I wish I'd had this. And I assume you can do things like abstract base tables as well, which I certainly wish I could have had elsewhere.

emerging libgnomecanvas stopped with a complaint about not finding index.sgml. Again, I searched the Gentoo forums; finding a message saying to re-emerge gtk-doc; no luck. A followup said to try USE="-doc" emerge, which worked. Same thing happened with libgnome and libgnomeui, same solution.

Missing gconf-2.0.pc during emerge

2004-11-02T13:47:00.000-05:00

During an emerge of gnome-vfs, got error message of 'Library dependencies ... not met'. Turned out that gconf-2.0.pc was missing. Searched Gentoo forums, and found a suggestion to copy in existing gconf-2.0.pc; didn't exactly like that, as I wasn't sure how appropriate it was for my system.

So I untarred /usr/portage/distfiles/GConf-2.6.2.tar.bz2, and ran ./configure, which created a gconf-2.0.pc; copied that to /usr/lib/pkgconfig, and re-emerged gnome-vfs. No luck, got report that header file was missing. I examined gconf-2.0.pc, did a locate for missing header file, and hence saw what needed to be fixed in gconf-2.0.pc, (prefix was pointing to /usr/local, and should have pointed to /usr). Re-emerged gnome-vfs, which worked fine this time.

Still not sure why a) the wrong value was generated by configure, and b) the file wasn't placed where it should have been. But it works, and I can continue.

Started in on Netbeans; Eclipse Trouble

2004-11-01T18:05:00.000-05:00

Got Netbeans 3.6 running, and started playing with it. Worked through the QuickStart and Using guide. It looks fine, but feels a little sluggish compared to what I remember Eclipse being. But A) I'm also doing emerges (compiles) at the same time, and B) I'm probably not on as fast a box as I was when I used Eclipse. I pulled down Eclipse 3.0.1, and will compare apples to apples tomorrow.

I do like Netbeans' Import Management Tool. Nice amount of control over how you specify and manage your imports. And I wish I'd had a GUI editor when I was doing that Swing stuff.

As a footnote, Eclipse was crashing on me, but again, I was mid-compile, and the .so's that Eclipse listed on the dump were on both sides of my compile, so I'll wait until I'm not messing with stuff before trying to troubleshoot.

Getting D-Link DWL-520 Wireless Working

2004-10-31T20:58:00.000-05:00

Spent an hour or two yesterday getting my D-Link DWL-520 wireless NIC working under Gentoo. I found a Howto at this site, which worked great, except it didn't mention what iwconfig commands I needed, saying only that "[t]he sequencing of commands will vary by network". Googling for "dwl-520 iwconfig" took me to the hostap mailing list, and the following commands:


ifconfig wlan0 up

iwconfig wlan0 mode managed

iwconfig wlan0 essid "your-essid-here"

ifconfig wlan0 192.168.1.100

ifconfig wlan0 up

... and now I'm online. Whee!

First Post!

2004-10-31T19:05:00.000-05:00

First post! Oh, wait, this is my blog.

Anyway, I'm getting back to the Java world, and wanted to have a place to put all of my Java (and possibly Linux) experience. I figure I'll keep this for my techie work stuff, and my other blog for all of my wisdom. (!?)