LSB init scripts with a java daemon in Ubuntu

I have several java-based services that someone wanted to run as daemons. I would have preferred running them in Tomcat, but they didn't want to administer that. So I thought I would create an init script in /etc/init.d, and use that to stop/start the service and to check its status via service --status-all. If you follow the documentation on this the Debian people recommend using their init functions in /lib/lsb/init-functions. The ones I don't like are start-stop-daemon and the use of pid files in /var/run. Firstly, for a java service run as java -jar MyProg.jar ...options... & this doesn't fit into the mould of an executable daemon with arguments, since the program name is just "java" and the actual "daemon" is a long path name with even longer classpath variables etc. Also I don't see the point of the pid file. Its only purpose is to test if the daemon is running. But you can do that with:

PID=`ps aux | grep $DAEMON | grep -v grep | awk '{print $2}'`

That gives you the process ID so long as you have a long enough name for the daemon. So if $PID is empty the daemon is not running, and according to the standard I am supposed to call exit 3 or exit 0 if it was running. However service --status-all ignores this return code. Oh yes. After hours of hacking I found a script that did work because it called log_success_msg in each of the two cases (see below). And with that, it works. Presumably because this reads the script status code. Without those lines service --status-all reports that the service "isn't running" no matter what.

In the following sample code just replace my daemon, and all the paths etc with yours. The meat of the script is generic, although it doesn't support reload.

#!/bin/sh # # /etc/init.d/bhlpages -- startup script for the BHL pages service # # ### BEGIN INIT INFO # Provides: bhlpages # Required-Start: mongodb $remote_fs $syslog # Required-Stop: mongodb $remote_fs $syslog # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: Start BHL Pages. # Description: Start the BHL page description service for TILT ### END INIT INFO # Exit if the package is not installed [ -e "/usr/local/bin/bhlpages/BHLPages.jar" ] || exit 5 PATH=/sbin:/usr/sbin:/bin:/usr/bin DESC="BHL page description service for TILT" NAME=bhlpages DAEMON=/usr/local/bin/bhlpages/BHLPages.jar # LibPath is a simple java program that prints the java runtime libpath # which is only needed so we can add our install location to it for JNI LIBPATH=`java -cp /usr/local/bin/bhlpages/ LibPath`:/usr/local/lib # Load the standard init functions . /lib/lsb/init-functions # make a classpath out of a directory of jars getjarpaths() { JARPATH="" for f in $1/*.jar do JARPATH="$JARPATH:$f" done echo $JARPATH return } do_start() { JARPATHS=`getjarpaths /usr/local/bin/bhlpages/lib` pkill -f $DAEMON java -Djava.library.path=$LIBPATH -cp .$JARPATHS:/usr/local/bin/bhlpages/BHLPages.jar bhl.pages.JettyServer -w 8084 >> /var/log/bhlpages.log 2>&1 & } do_stop() { pkill -f $DAEMON } ### main logic ### case "$1" in start) log_daemon_msg "Starting $NAME" do_start log_end_msg 0 ;; stop) PID=`ps aux | grep $DAEMON | grep -v grep | awk '{print $2}'` if [ -n $PID ]; then log_daemon_msg "Stopping $NAME" do_stop else log_daemon_msg "$NAME not running" fi log_end_msg 0 ;; restart) # Restart the daemon. $0 stop && sleep 2 && $0 start ;; status) PID=`ps aux | grep $DAEMON | grep -v grep | awk '{print $2}'` if [ -z $PID ]; then log_success_msg "$NAME is not running" exit 3 else log_success_msg "$NAME is running" exit 0 fi ;; *) log_action_msg "Usage: $0 {start|stop|restart}" exit 1 ;; esac exit 0

Reading an AJAX response gradually

I wanted to display a progress bar on a Webpage using Javascript/jQuery while a time-consuming process on the server was taking place. It wasn't uploading or downloading significant amounts of data, but it did take time. For this reason I couldn't use the progress events in Ajax, or their jQuery implementation. People said it couldn't be done, that it exposed "the limitations of the HTTP protocol itself". In fact it has nothing to do with HTTP, but with TCP. When I make an Ajax call the client first establishes a TCP connection using the SYN,SYN+ACK,ACK exchange. Then the server sends data back to the client until it is finished and then sends a FIN packet, which the client acknowledges, to signify "end of flow". So if we provide a callback that gets called on "success" it will wait for the end of flow and not report any data meanwhile. But that doesn't mean that data is not available. At the socket level in Java I can call something like "myStream.available()" to see if there is data to be read, and in Ajax we can test the ready state to see if it is 3 (not 4). If the server is writing data out gradually, in my case the percentage of process completion, and flushing at the ends of lines, then data will be available for the onreadystate function. Here's an example. I provided a button to make the Ajax call, whose id is "rebuild". My service is at "/search/build":

jQuery("#rebuild").click( function() {
    var readSoFar = 0;
    client = new XMLHttpRequest();
    client.open("GET", "http://"+window.location.hostname+"/search/build");
    client.send();
    // Track the state changes of the request
    client.onreadystatechange = function(){
        // Ready state 3 means that data is ready 
        if(client.readyState == 3){
            if(client.status ==200) {
                var len = client.responseText.length-readSoFar;
                console.log(client.responseText.substr(readSoFar,len));
                readSoFar = client.responseText.length;
            }
        }
    };
});

This prints out the text received from the server at the same rate that it was sent. It doesn't appear to be possible to do this in jQuery, because there is no "onreadystatechange" field in jQuery's jqXHR object, so I have used raw Javascript instead. This text can then be used to implement a progress bar.

Cross domain requests in tomcat

I had a strange problem in Tomcat. Everything worked fine when running my webapp on localhost, but when I uploaded it to the server on xxxx.net the call to connect with the Mongo database failed with a mysterious timeout error when opening a socket. But all seemed fine. I could access mongo through the commandline. I could even copy exactly the same instructions in Java and run a local test app that connected to the database flawlessly. The problem was too subtle for me. My test app set up the database connection via localhost, but the webapp used the current host name, which was xxxx.net. So this became a cross-domain request and was refused. All I had to do was change my openConnection call in the Java driver to use localhost and all was fine. A genuine 'gotcha!'

Here's the code that screws up: (you need the mongo driver jar in the classpath)

import java.util.*; import com.mongodb.*; public class Test { public static void main(String args[]) { try { // change "localhost" to myhost.com (your server domain) // to fail. localhost actually works MongoClient mongoClient = new MongoClient( "localhost", 27017 ); // substitute the names of your database and collection here DB db = mongoClient.getDB("mydatabase"); DBCollection coll = db.getCollection( "mycollection" ); if ( coll == null ) throw new Exception( "Unknown collection mycollection" ); BasicDBObject keys = new BasicDBObject(); keys.put( "myfield", 1 ); DBCursor cursor = coll.find( new BasicDBObject(), keys ); if ( cursor.length() > 0 ) { Iterator<DBObject> iter = cursor.iterator(); int i = 0; while ( iter.hasNext() ) // substitute the name of some key field here System.out.println( (String)iter.next().get( "myfield" )); } else throw new Exception( "no docs in collection mycollection" ); } catch ( Exception e ) { e.printStackTrace(System.out); } } }

Submit form in jQuery with file input and stay on the same page

You can use Ajax to send the contents of a form to the server asynchronously, so you can stay on the same page. What no one explains is how to do that if you have a file input control. The reason is that neither javascript nor jQuery has access to the contents of the file input control, just the file name. So to upload a form that contains a file input control you have to use ordinary submit, and let the browser do the file upload for you. So all you need is a regular form like:

<form action="/myserver.com/form_processing" 
    enctype="multipart/form-data" type="post">
<input type="file" name="file_element"></file>
<input type="submit"></input>
<input id="source" name="source" type="hidden" value=""></input>
</form>

Then somewhere in your javascript for the page add the source url to the hidden "source" input:

$( document ).ready(function() {
    $("#source").val(window.location.href);
});

Now on the server, in the "form_processing" script or servlet, execute a redirect to the address of the original form. In java all you need to do is:

response.sendRedirect(source);

To set the value of "source" you will have to process the multi-part request, which will require a library like Apache commons fileupload, or in php you just read $_POST, and use http_redirect to do the actual redirect. This will submit the form and it will flash, but you'll land back where you came from. No need to jump through hoops or reinvent the wheel.

Reading output

If the url you redirected to produces output you want to display you have to use a different technique. Don't use redirection on the server. Instead, just add a target="myiframe" to the form element. Then add an <iframe name="myiframe"> to the page to receive the output from the submitted-to URL. The output should be a regular HTML page. You can even add a stylesheet to it.

Ubuntu on Gigabyte P15F V2

I tried to install Ubuntu 14.04 on a Gigabyte 15F V2, i74710MQ, 8G DDR3, 1TB HD.I know that this model is sold in France with Ubuntu pre-installed. In Australia I had no such luck. After checking the warranty I decided to replace the 1TB HD with a 128GB SSD. So if it goes in for repairs I can just reinsert the original HD and pretend that I have been using Windows all along. It comes with this massive 2 year warranty, but with the proviso that if you don't use it as laid out in the user guide this will void the entire warranty. With that grain of salt swallowed I proceeded with installation. In order to get it to boot into Ubuntu 14.04 from a DVD I had first to disable the UEFI. This is a simple matter after pressing F2 during startup, and turning the UEFI into "legacy" mode. Once installed, I noticed that the wifi card didn't work. So I installed directly via an ethernet cable to my router. To get the wifi card working I had to install the new driver. (I cloned the repository, then built and installed it.) Following that I had a pretty much perfect installation. I would recommend this model if you want a Linux laptop and don't mind paying the Microsoft tax. My only gripe so far is that although the motherboard is by Gigabyte they seem to have used a Clevo shell for the laptop. And my last Clevo developed several faults just 8 months into its lifespan. But this model is more robust, if also heavier. Battery life is nowhere near that suggested elsewhere (4 hours??). I'm getting around 2 hours on a new battery, and the charger is on the big side: 120W and about 7.5x15cm. The question is: what other good Linux laptops are out there for this price? My main gripe is the lack of a backlit keyboard, which adds about $5 to the keyboard price. I really miss it when working in the dark.

On the positive side the trackpad is good, and the buttons are tolerable. The screen is very bright. In fact I have to keep having to turn it down. The keyboard is responsive and robust. It's powerful cpu-wise, and if you use the 1TB HD, also capacious.

Addendum 1: the trackpad buttons get stuck because the case gets caught in a raised position. If that happens just push the front edge of the case near the trackpad buttons down until it clicks back.

Addendum 2: The RTL8723BE driver in Ubuntu 14.04 doesn't handle bluetooth, so forget about buying bluetooth headphones for this baby. In the end, frustrated at having to use cabled earphones, I swapped it for an Intel 7260 I just had lying around (as you do), which is well supported.

Addendum 3: For Ubuntu system 15.10 I used the "live" install from a usb drive. I had many errors when booting from the device, but that USB worked perfectly in another laptop. The problem seemed to be the hardware, since it complained about the ePCI bus having a fault, and that I should check the "cabling" (?). So I removed all drives including the DVD (there's a little screw you remove in the case holding it then you can pull it out) and any auxiliary drives you installed. But in my case it required like 10 reboots before it would even run the installer. I don't know why, but every time it got a little bit further. Then you must do two things once you finally have ubuntu running: 1) run sudo apt-get update in the terminal, then run software update and 2) install the NVIDIA graphics driver "tested and proprietary" in Software Updates->Additional Drivers. Don't use the default X.Org X Server - or you will get random screen freezes. Once I did all that everything worked perfectly, but what a hell it was to get there.

Addendum 4: After 1 year three months the case is holding together. The screen hasn't fallen off, although there are a few groans in the chassis they are likely due to my falling over on concrete with the laptop in my backpack. Screws pop out now and again so it is good to have a screwdriver handy. Other than that the "structural integrity" of this cyber-space-craft is at about 70%. Not bad given how much I use it every day.

Addendum 5: The DVD drive is iffy. It works but it is overly sensitive to dvds that have some damage. The playback stops too readily after encountering an error, rather than than trying very hard to skip over damaged portions. The same DVDs in my cheap Toshiba however, usually play just fine.

Find the greatest value in a list less than a value

Binary search must be one of the most useful and simplest of algorithms. It allows you to find an element in a sorted list that is either there or not. But what if you want to put an element in the list, one position after the next less or equal element? I would have thought that to be a common enough requirement, but I can't find any implementations on the Web that make any sense, or which are more efficient that just iterating through the list and noting the last less-than-or-equal-to-value.

In my case I had to find the scroll position in a HTML div, which was marked at intervals by "page-breaks". Each page-break marked the start of a new page and I wanted to find out which page the display was on. To do that I kept a list of where each page-break began in pixels down the div, and then I needed to look up quickly the highest value in the list, the page-break, that was less than or equal to the given scroll position. Here's what I came up with in Java. Adapt as you see fit to other languages.

int getIndex( long[] list, long value ) { int top = 0; int bot = list.length-1; int mid=0; while ( top <= bot ) { mid = (top+bot)/2; // NB integer arithmetic if ( value < list[mid] ) { if ( mid == 0 ) // value < than first item return -1; else bot = mid-1; } else // value >= list[mid] { if ( mid == list.length-1 ) // value is >= last item break; else if ( value >= list[mid+1] ) top = mid+1; else // list[mid] must be biggest <= value break; } } return mid; }

Distance between two polygons in Java

I looked everywhere for a nice, clean, simple, easy to understand algorithm that would compute the minimum distance between two polygons. I had no luck, but I did gain an understanding of the problem. In a nutshell the minimal distance between two polygons P and Q is the minimum of

1. the distances between each of the vertices of P and the vertices of Q,
2. the distances between the vertices of P and the edges of Q,
3. the distances between the vertices of Q and the edges of P.

It is quite possible for the two closest vertices to be further apart than the two closest edges. And no, contrary to my usual practice, this code is far from optimal. It is O(N²) not O(log n+log m), which is possible. But you figure it out. I just don't have the time, and none of the papers I read had any sample code.

import java.awt.Polygon; import java.awt.Point; import java.awt.geom.PathIterator; import java.util.ArrayList; public class Utils { /** * The distance between two points squared * @param v the first point * @param w the second point * @return the distance between v and w squared */ static float dist2( Point v, Point w ) { int xDiff = v.x - w.x; int yDiff = v.y - w.y; return xDiff*xDiff + yDiff*yDiff; } /** * Distance from a point to a line-segment squared * @param p the point * @param v the 1st point of the segment * @param w the 2nd point of the segment * @return the distance as a float */ static float distToSegmentSquared( Point p, Point v, Point w) { float l2 = dist2(v, w); if (l2 == 0.0f) return dist2(p, v); float t = ((p.x-v.x)*(w.x-v.x)+(p.y-v.y)*(w.y-v.y))/l2; if (t < 0) return dist2(p, v); if (t > 1) return dist2(p, w); Point q = new Point(v.x+Math.round(t*(w.x-v.x)),v.y +Math.round(t*(w.y-v.y))); return dist2( p, q ); } /** * Compute the distance from a point to a line segment * @param p the point * @param v first point of the line * @param w second point of the line * @return the distance as a float */ public static float distanceToSegment( Point p, Point v, Point w ) { float squared = distToSegmentSquared(p,v,w); return (float)Math.sqrt(squared); } /** * Get the minimal separation between two polygons * @param P the first polygon * @param Q the second polygon * @return the smallest distance between segments or vertices of P, Q */ public static int distanceBetween( Polygon P, Polygon Q ) { float minDist = Float.MAX_VALUE; ArrayList<Point> points1 = polygonToPoints(P); ArrayList<Point> points2 = polygonToPoints(Q); Point last1=null,last2=null; for ( int i=0;i<points1.size();i++ ) { Point p1 = points1.get(i); for ( int j=0;j<points2.size();j++ ) { Point p2 = points2.get(j); int x = Math.abs(p1.x-p2.x); int y = Math.abs(p1.y-p2.y); // distance between vertices float dist = Math.round(Math.hypot(x,y)); if ( dist < minDist ) minDist = dist; // distance between p1 and a segment of Q if ( last2 != null ) { float fDist2 = distanceToSegment( p1, last2, p2 ); if ( fDist2 < minDist ) minDist = fDist2; } // distance between p2 and a segment of P if ( last1 != null ) { float fDist1 = distanceToSegment( p2, last1, p1 ); if ( fDist1 < minDist ) minDist = fDist1; } last2 = p2; } last1 = p1; } return Math.round(minDist); } /** * Convert a polygon to an array of points * @param pg the poly * @return an arraylist of type Point */ public static ArrayList<Point> polygonToPoints( Polygon pg ) { ArrayList<Point> points = new ArrayList<>(); PathIterator iter = pg.getPathIterator(null); float[] coords = new float[6]; while ( !iter.isDone()) { int step = iter.currentSegment(coords); switch ( step ) { case PathIterator.SEG_CLOSE: case PathIterator.SEG_LINETO: case PathIterator.SEG_MOVETO: points.add( new Point(Math.round(coords[0]), Math.round(coords[1])) ); break; default: break; } iter.next(); } return points; } public static void main(String[] args ) { if ( args.length==2 ) { String[] coords1 = args[0].split(","); String[] coords2 = args[1].split(","); Polygon P = new Polygon(); Polygon Q = new Polygon(); for ( int i=0;i<coords1.length;i+=2 ) { P.addPoint( Integer.parseInt(coords1[i]), Integer.parseInt(coords1[i+1])); } for ( int i=0;i<coords2.length;i+=2 ) { Q.addPoint( Integer.parseInt(coords2[i]), Integer.parseInt(coords2[i+1])); } float dist = distanceBetween( P, Q ); System.out.println("dist="+dist); } else System.out.println("usage: java Utils <x1,y1,x2,y2...> <x1,y1,x2,y2...>"); } }