Wednesday, December 24, 2014

Cross domain requests in tomcat

I had a strange problem in Tomcat. Everything worked fine when running my webapp on localhost, but when I uploaded it to the server on the call to connect with the Mongo database failed with a mysterious timeout error when opening a socket. But all seemed fine. I could access mongo through the commandline. I could even copy exactly the same instructions in Java and run a local test app that connected to the database flawlessly. The problem was too subtle for me. My test app set up the database connection via localhost, but the webapp used the current host name, which was So this became a cross-domain request and was refused. All I had to do was change my openConnection call in the Java driver to use localhost and all was fine. A genuine 'gotcha!'

Here's the code that screws up: (you need the mongo driver jar in the classpath)

Monday, November 24, 2014

Submit form in jQuery with file input and stay on the same page

You can use Ajax to send the contents of a form to the server asynchronously, so you can stay on the same page. What no one explains is how to do that if you have a file input control. The reason is that neither javascript nor jQuery has access to the contents of the file input control, just the file name. So to upload a form that contains a file input control you have to use ordinary submit, and let the browser do the file upload for you. So all you need is a regular form like:

<form action="/" 
    enctype="multipart/form-data" type="post">
<input type="file" name="file_element"></file>
<input type="submit"></input>
<input id="source" name="source" type="hidden" value=""></input>

Then somewhere in your javascript for the page add the source url to the hidden "source" input:

$( document ).ready(function() {

Now on the server, in the "form_processing" script or servlet, execute a redirect to the address of the original form. In java all you need to do is:


To set the value of "source" you will have to process the multi-part request, which will require a library like Apache commons fileupload, or in php you just read $_POST, and use http_redirect to do the actual redirect. This will submit the form and it will flash, but you'll land back where you came from. No need to jump through hoops or reinvent the wheel.

Reading output

If the url you redirected to produces output you want to display you have to use a different technique. Don't use redirection on the server. Instead, just add a target="myiframe" to the form element. Then add an <iframe name="myiframe"> to the page to receive the output from the submitted-to URL. The output should be a regular HTML page. You can even add a stylesheet to it.

Monday, November 17, 2014

Ubuntu on Gigabyte P15F V2

I tried to install Ubuntu 14.04 on a Gigabyte 15F V2, i74710MQ, 8G DDR3, 1TB HD.I know that this model is sold in France with Ubuntu pre-installed. In Australia I had no such luck. After checking the warranty I decided to replace the 1TB HD with a 128GB SSD. So if it goes in for repairs I can just reinsert the original HD and pretend that I have been using Windows all along. It comes with this massive 2 year warranty, but with the proviso that if you don't use it as laid out in the user guide this will void the entire warranty. With that grain of salt swallowed I proceeded with installation. In order to get it to boot into Ubuntu 14.04 from a DVD I had first to disable the UEFI. This is a simple matter after pressing F2 during startup, and turning the UEFI into "legacy" mode. Once installed, I noticed that the wifi card didn't work. So I installed directly via an ethernet cable to my router. To get the wifi card working I had to install the new driver. (I cloned the repository, then built and installed it.) Following that I had a pretty much perfect installation. I would recommend this model if you want a Linux laptop and don't mind paying the Microsoft tax. My only gripe so far is that although the motherboard is by Gigabyte they seem to have used a Clevo shell for the laptop. And my last Clevo developed several faults just 8 months into its lifespan. But this model is more robust, if also heavier. Battery life is nowhere near that suggested elsewhere (4 hours??). I'm getting around 2 hours on a new battery, and the charger is on the big side: 120W and about 7.5x15cm. The question is: what other good Linux laptops are out there for this price? My main gripe is the lack of a backlit keyboard, which adds about $5 to the keyboard price. I really miss it when working in the dark.

On the positive side the trackpad is good, and the buttons are tolerable. The screen is very bright. In fact I have to keep having to turn it down. The keyboard is responsive and robust. It's powerful cpu-wise, and if you use the 1TB HD, also capacious.

Addendum 1: the trackpad buttons get stuck because the case gets caught in a raised position. If that happens just push the front edge of the case near the trackpad buttons down until it clicks back.

Addendum 2: The RTL8723BE driver in Ubuntu 14.04 doesn't handle bluetooth, so forget about buying bluetooth headphones for this baby. In the end, frustrated at having to use cabled earphones, I swapped it for an Intel 7260 I just had lying around (as you do), which is well supported.

Addendum 3: For Ubuntu system 15.10 I used the "live" install from a usb drive. I had many errors when booting from the device, but that USB worked perfectly in another laptop. The problem seemed to be the hardware, since it complained about the ePCI bus having a fault, and that I should check the "cabling" (?). So I removed all drives including the DVD (there's a little screw you remove in the case holding it then you can pull it out) and any auxiliary drives you installed. But in my case it required like 10 reboots before it would even run the installer. I don't know why, but every time it got a little bit further. Then you must do two things once you finally have ubuntu running: 1) run sudo apt-get update in the terminal, then run software update and 2) install the NVIDIA graphics driver "tested and proprietary" in Software Updates->Additional Drivers. Don't use the default X.Org X Server - or you will get random screen freezes. Once I did all that everything worked perfectly, but what a hell it was to get there.

Addendum 4: After 1 year three months the case is holding together. The screen hasn't fallen off, although there are a few groans in the chassis they are likely due to my falling over on concrete with the laptop in my backpack. Screws pop out now and again so it is good to have a screwdriver handy. Other than that the "structural integrity" of this cyber-space-craft is at about 70%. Not bad given how much I use it every day.

Addendum 5: The DVD drive is iffy. It works but it is overly sensitive to dvds that have some damage. The playback stops too readily after encountering an error, rather than than trying very hard to skip over damaged portions. The same DVDs in my cheap Toshiba however, usually play just fine.

Monday, August 18, 2014

Find the greatest value in a list less than a value

Binary search must be one of the most useful and simplest of algorithms. It allows you to find an element in a sorted list that is either there or not. But what if you want to put an element in the list, one position after the next less or equal element? I would have thought that to be a common enough requirement, but I can't find any implementations on the Web that make any sense, or which are more efficient that just iterating through the list and noting the last less-than-or-equal-to-value.

In my case I had to find the scroll position in a HTML div, which was marked at intervals by "page-breaks". Each page-break marked the start of a new page and I wanted to find out which page the display was on. To do that I kept a list of where each page-break began in pixels down the div, and then I needed to look up quickly the highest value in the list, the page-break, that was less than or equal to the given scroll position. Here's what I came up with in Java. Adapt as you see fit to other languages.

Sunday, July 6, 2014

Distance between two polygons in Java

I looked everywhere for a nice, clean, simple, easy to understand algorithm that would compute the minimum distance between two polygons. I had no luck, but I did gain an understanding of the problem. In a nutshell the minimal distance between two polygons P and Q is the minimum of

  1. the distances between each of the vertices of P and the vertices of Q,
  2. the distances between the vertices of P and the edges of Q,
  3. the distances between the vertices of Q and the edges of P.

It is quite possible for the two closest vertices to be further apart than the two closest edges. And no, contrary to my usual practice, this code is far from optimal. It is O(N2) not O(log n+log m), which is possible. But you figure it out. I just don't have the time, and none of the papers I read had any sample code.

Wednesday, June 11, 2014

Writing a module for an entire page in Drupal

If like me you like to modularise your programs you might like to create a page of your website that corresponds to a single module. I currently have modules for biography, browse and experiments. What doesn't satisfy me about Drupal is the fact that "content management" usually means writing stories in HTML and displaying them on pages. I see a CMS as a shell within which to develop interactive content with more bells and whistles. Hence I tend to write one module for each major page.

First there is the problem that Drupal considers modules as extensions to every page, and will load your javascript and css throughout the site. One way to avoid this is not to add them to the file. Don't write stylesheets[all][] = css/my_module.css, or scripts[all][] = js/my_module.js, or that will get added on every page. Instead, load them dynamically whenever the module gets loaded, and then control when that happens. In your my_module.module file define the following hook:

So only when the url contains the string "my_module" (please substitute the name of your module) will the js and css get loaded.

The next problem is how to make it call the module so that the url will actually display that page. The secret is another callback in mymodule.module:

So now the path is a page, and any attempt to access it calls my_module_all, which can do anything you like and should just return some html. That, I believe, is all there is to it.

Tuesday, March 11, 2014

Scroll to selection using rangy

Rangy, like ierange before it, is great for reliably selecting text programmatically across browsers. It uses the W3C Range model and applies it to all modern browsers, greatly simplifying a task that is not covered in jQuery. But, what rangy does not do is provide a way to scroll to a selection. In cases where the selected text is off-screen, it is necessary to scroll the document down so the user can see it. A selection does not have a bounding box in rangy, so there is no way to tell where it is. The only way to get that information is to enclose it in a node, and then position that node in the centre of the screen. Provided that the node-wrapping is possible this method will suffice:

function scrollToSelection( sel )
    var node = document.createElement("span");
    var scrollDist = node.offsetTop 
        - (window.innerHeight+node.offsetHeight)/2;
    if ( scrollDist > 0 )

If the selection covers parts of various elements you could wrap each separate text-node in a <span>, then compute the bounding box of the entire thing. But for my purposes the above suffices.

Tuesday, March 4, 2014

Drupal add module javascript for a specific page

If like me you prefer to split your interface into modules, and to call each of those modules on separate pages, then one problem you are likely to encounter is the use of jQuery init functions ($(document).ready(function() etc.) in each of those modules. For example, I had separate javascript for a biography page, and for the homepage. Each called its own jQuery init function, and each was incompatible with the other. When loading the biography page I looked for a certain element and then embedded the timeline and biography data fetched from the server into it. This allowed me to leverage existing javascript libraries and to reuse interface code. The drawback, though, is that in Drupal all modules are loaded for each page, and if they contain any javascript it will be executed on page load. Likewise any css files specified in the module's .info file will also be loaded, even though they are not required. Not only is this very inefficient, but it will lead to clashes, as each javascript init function looks for its specific features and fails. So to make it work better and faster you must detect the current page and only load css stylesheets and javascript functions as required. Something like this, in the mymodule.module file will work:

This way both the javascript and css only get loaded when needed, so long as you don't use module names that can occur elsewhere in request uris.

So the moral of the story is: "don't load javascript or css files via a .info file, because they will get loaded on each and every page of your Drupal installation".

Friday, February 28, 2014

Adding Search and navigation by year to TimelineJS

TimelineJS is easily the coolest timeline on the planet. It has a wonderful slider and excellent animations and design. But, and this is a big but, it lacks a proper API for programmers. Making it do anything except the original uses envisaged by its designer is a pain. For example, I wanted to search the timeline. We use it for a timeline of poems, initially 720, but later this will grow to 2,200. And without search you can't find when he wrote what. Nobody is going to flick through that many entries. (And yes it does work well with 720 entries - the only problem is the time it takes to download, not render, since we don't use video or even graphics much). So I wanted to add a search. It turns out that the only practical way to do this that I could find was to add a function to the timeline.js code itself. So I added a search button, and bound my function to it:

function buildNavigation() {
    VMM.bindEvent("#search_button", doTimelineSearch);
    VMM.bindEvent("#year_dropdown", goToYear);

I also have a text box called "search_expr" to hold the search text and error messages. First I tried searching the HTML itself using jQuery but the browser caches most of the elements, so I had to use the copy of the original JSON object that TimelineJS uses to construct the slides. It was still hanging around in config.source. I added some code to the NAVIGATION section of timeline.js:

First I search from the current slide+1 to the end, then from the start to the current slide. If it is only on the current slide or not found, I write "not found" in the search box. Pretty simple. No doubt it could be written more elegantly but it was easier to debug like this, and I'm not very good at Javascript. But, hey, it works. Here it is, live. And yes it requires jQuery.

After thinking about it for a bit I added a goToYear function, which allows the user to quickly jump to an interesting section of the timeline.

Tuesday, February 25, 2014

Internationalizing Drupal 7

Recently I had to create a bilingual website in Drupal 7.26. I found information here and there but no comprehensive, up to date and detailed roadmap to say: "do this and it will all work". Here's my attempt to set the record straight.

  1. First you have to install the internationalization modules: Locale, Localization Update, Internationalization (comprising Block Languages, Content translation, Field translation, Internationalization, Language Icons, Menu Translation, Multilingual Content, Multilingual Select, Path Translation, String Translation, Synchronize translations, Taxonomy Translation, Translation redirect, Translation sets, Variable Translation, should all be enabled), Variable (comprising Variable, Variable Realm, Variable Store, should all be enabled)
  2. Now go to the Configuration page
    1. In Regional and Language click on Languages and make sure your language is the default and that they are all enabled. On Icons you can set the icon size.
    2. In Multilingual settings->Selection select "Select nodes by language" and "Select taxonomy terms by language". In the Variables tab select "Default front page". In Strings set Translatable formats to Plain text and HTML and set the source language to English.
    3. In Translate Interface load the language files of all the languages you want to support. You can get these from
    4. In System->Site Information choose the home page for each language by first selecting the language using the language switcher and then specify a Default Front Page for it. If you generate content from nodes you don't need to do this.
  3. Now enter the administration pages and take the tabs one by one:
    1. In Structure->Menus select "Translate and Localize". Menu items with language will allow translations. Menu items without language will be localized." in the multilingual options for each menu you want to appear in multiple languages. Save. Now go back and edit each menu item of that menu that needs translating. Click the tab "Translate" and the "Add translation" to provide a translation for the missing language(s). Save.
    2. In Blocks sets the Language Switcher to appear somewhere on the page - in the header or in Primary. Now configure it so that it shows all the languages you want in "Visibility Settings" or the menu won't appear at all. In Content Types set it to appear in all. Similarly make it appear for all roles. In Footer make it translatable and select all desired languages for display. Set the correct footer text for your current language (check the URL in the browser path). If you want to translate it, switch to another language and return to Structure->Blocks->Footer. Now hit the translate button, add a translation and supply the text. Save.
    3. In Content Types: Basic page, set Language to "Hidden" or you will get a report on the language at the foot of each page.
  4. If like me you have generated content in the footer, for example, to say "Last updated on..." you can add some code to the footer (or wherever) like this:
    global $language;
    $lang_name = $language->language;
    if ( isset($node) && isset($node->changed) )
        if ( $lang_name == 'it' )
            print "Ultimo aggiornamento ".format_date($node->changed);
            print "Last updated ".format_date($node->changed);

For each page in the source language you will have to "add a translation" and have it saved under the target language. I use Google Translate, which often does a passing good job. But really you need a native speaker to go over it. Now when you switch languages the menus should swap, the custom text should swap and the content will swap. You may have to do some of these things in a different order, since they are all interdependent. Good luck, and I hope I haven't left anything out. If I have, then Google it!

Sunday, February 16, 2014

TimelineJS configured dynamically with a string

TimelineJS is probably the prettiest and most reliable timeline software that works over the Web. Unfortunately there are some limitations: first that it only handles about 100 entries, although we regularly use much more than that and its performance is OK. The big problem is getting it to read from a string returned by a server. The documentation is unclear on this, so here is an attempt to spell out the technique with an example (lacking in the software distribution).

TimelineJS is configured via a JSON config object. Once timeline_config is set to this structure and TimelineJS is loaded, the config is read and the timeline gets set up automatically. To invoke it dynamically you have to specify a javascript object as input instead of a file name for the source config property. To create this JS object I put the following in a <script> element in the <head> of my HTML. For this to work you must previously have loaded both the jQuery and "../build/js/storyjs-embed.js" scripts:

function httpGet(theUrl) {
    var xmlHttp = null;
    xmlHttp = new XMLHttpRequest(); "GET", theUrl, false );
    if ( xmlHttp.readyState==1 )
        xmlHttp.send( null );
    return xmlHttp.responseText;
$(document).ready(function() {
var dataObject = httpGet("http://localhost/json/timeline/");
if ( dataObject != null )
    var jsObject = eval("("+dataObject+")");
       type:       'timeline',
       width:      '100%',
       height:     '100%',
       source:     jsObject,
       embed_id:   'my-timeline'

This does a http GET via an AJAX call to get the JSON, objectify it using eval, and then call createStoryJS, passing in the JSON config object. The JSON just follows the format given in the documentation. In our case we didn't want the "asset" property so we just left it out. It seems to work OK. This way you can create the timeline after the page has loaded. All it needs is a <div id="my-timeline"> in the body of the HTML for it to work.

Tuesday, February 4, 2014

Ubuntu -- disabling Recently Used in file chooser

When designing gtk someone thought it would be a great idea to help the user by providing a list of recently accessed files, and make selecting from them the default. That way you can open one of those, instead of searching for it in the file hierarchy, and so save time. But this doesn't work for a number of reasons. First, the user can't see which version of the file is being offered, just its name. Not knowing where it is located means that when it drops out of the Recently Used list you will have to search for it everywhere. I may have dozens of files with the same name on my disk. That's why we have folders: to distinguish them. Since humans like to organize information hierarchically folders tell us immediately where to find it. Not having that crutch by offering an immediate candidate actually does the user a disservice. In order to get out of the "Recently Used" list you have to click away every single time. It drives me and quite a few other people nuts. I doubt that even the designer of this feature actually uses it.

So in Ubuntu there are two possible strategies for fixing the problem:

  1. Disable "Recently Used" altogether
  2. Make the default not be Recently Used but Home.

1. looks the most attractive. Unfortunately "Recently Used" is hard-wired into the open file dialog. Short of patching gtk you can't remove it. And I wouldn't recommend patching it, even though you can find one here. Next time you update your system the fix will disappear, if you don't break your current version, which won't be the same as the one assumed by the patch. So, forget this.

2. This is an easy configuration setting. Pity that the designers of gtk didn't also provide a "recentlyused=disabled" option, but whatever. In home .config/gtk-2.0/gtkfilechooser.ini change StartUpMode=recent to StartUpMode=cwd, and then your current working directory or home will be where your heart is. By the way I use Ubuntu 13.10, so this works on the latest system.

If you want to simply prevent files from being tracked in recently used just disable it:

rm ~/.local/share/zeitgeist/recently-used.xbel
touch ~/.local/share/zeitgeist/recently-used.xbel
sudo chattr +i ~/.local/share/zeitgeist/recently-used.xbel

or as someone else suggested, instead of touch-ing and chattr-ing it, just make it a directory:

mkdir ~/.local/share/zeitgeist/recently-used.xbel

Monday, January 27, 2014

Short review of Clevo W550EU

I bought myself a new laptop recently - a Clevo W550EU. For $1000 I got an Ivy bridge i7@2.4GHz, 8GB, 128GB SSD, the latest Intel wireless card and a 1080p screen. A lot of bangs for my buck. I installed Ubuntu 13.10, wiping out the 13.04 they gave me. After playing with it for a few days, here are my likes and dislikes:

  • Likes
    • The screen is fantastic. Very bright. I love the crispness and the pixel space for programming with an IDE. This is an absolute must have if you want to program on a laptop.
    • The trackpad is sensitive and responsive, although it keeps firing off right menu clicks whenever I two-finger scroll in NetBeans. But not in other programs, so this seems to be a NetBeans only quirk.
    • The battery is good, giving me 4 hours with normal usage on a charge. (Quoted as eight but who believes that?)
    • It's light and fairly well constructed, and looks nice. The top deck seems to be metal, but the rest is definitely plastic, but a nice kind of moulded plastic rather than faux metal with paint. The surface seems to be cast to look like brushed metal and so won't wear off.
  • Dislikes
    • The keyboard is tinny. I can mitigate that by putting it on some cloth or felt, but it's a bit annoying. It's not too bad if I type slowly, though.
    • The case does not sit quite flat. I guess this is due to it being made of plastic and thin at that. Putting it on cloth helps though.
    • It's easy to mishit the F4 "sleep" key when you want to adjust the volume. It's sandwiched between Mute and Lower volume. In the middle of a movie that screws VLC up big time.
    • It came with the network card turned off with the F11 key. I spent more than an hour trying to figure our why my fancy wireless card wouldn't work.

So that's it so far. No worse than most laptops you buy. So I don't regret buying a non-Apple, non-Microsoft computer and avoiding to pay the OS-tax. Penguins of the world, unite!

Addendum: After only 8 months it developed the following faults: 1) the right hinge broke. It still opened and closed but the screen backing started to split off and eventually broke away entirely, so I had to stick it back together with tape. 2) the screen developed a hotspot on the bottom right hand quadrant. Annoying but not fatal. 3) The power supply cable frayed, although I treated it well. 4) The space bar on the keyboard malfunctioned. This was really annoying. In the end I bought another computer because I couldn't be without it for the time it would take to repair.

Addendum: I got most of the above repairs fixed for $140, except for the space bar which still doesn't work, although they claimed it does. One month later I tripped over the cable and the laptop fell about 50cm onto padded carpet. After that it wouldn't charge, and when it ran out of charge it was effectively dead. I presume it will need a new motherboard so it is useless now. So I would NOT recommend this laptop - it's too fragile.