Login to or logout from Drupal in Java

I wanted to post and update a lot of biographical data to Drupal, but I could only do so by first logging in – manually. How tedious. So I thought that it would a good general tool to be able to programmatically login and logout of Drupal without having to go through the GUI each time. But how to do it?

The method I settled on was to use Java's HttpURLConnection class. I would open a http connection to Drupal, send it my username and password, get back a cookie and then send my stuff. When I had finished I could then programmatically logoff. Cool. But it turned out to be a bit more difficult than I had imagined. i used POST because I couldn't make GET work properly, and all applications seemed to use POST. Some of the parameters are probably not necessary, but I added them all from the packets I captured in Wireshark, and at least this way it works:

import java.net.URL; import java.net.URLConnection; import java.net.HttpURLConnection; import java.io.OutputStream; import java.util.Random; /** * Login/logout of Drupal programmatically * @author desmond */ public class DrupalLogin { String boundary; public DrupalLogin() { boundary = "----------------------------"+randomHex(12); } String randomHex( int size ) { String str = "0123456789abcdef"; Random r = new Random(System.currentTimeMillis()); StringBuilder sb = new StringBuilder(); for ( int i=0;i<size;i++ ) { int index = r.nextInt(16); char c = str.charAt(index); sb.append( c ); } return sb.toString(); } void encodeParam( StringBuilder sb, String key, String value, boolean last ) { sb.append("Content-Disposition: form-data; name=\""); sb.append( key ); sb.append( "\"\r\n\r\n" ); sb.append( value ); sb.append( "\r\n--"); sb.append( boundary ); if ( last ) sb.append("--"); else sb.append("\r\n"); } String buildLoginRequest( String user, String password ) { StringBuilder sb = new StringBuilder(); sb.append("--"); sb.append(boundary); sb.append("\r\n"); encodeParam( sb, "name", user, false ); encodeParam( sb, "pass", password, false ); encodeParam( sb, "form_id", "user_login", false ); encodeParam( sb, "op", "Log in", true ); return sb.toString(); } String login( String host, String user, String password ) throws Exception { String cookie = null; URL login = new URL("http://"+host+"/user/login"); URLConnection conn = login.openConnection(); HttpURLConnection hconn = (HttpURLConnection) conn; hconn.setDoOutput( true ); hconn.setRequestProperty("Accept","*/*"); hconn.setRequestProperty("Host",host); hconn.setRequestProperty("Expect","100-continue"); String body = buildLoginRequest( user, password); hconn.setRequestProperty( "Content-length", Integer.toString(body.getBytes().length) ); hconn.setRequestProperty("Connection","keep-alive"); hconn.setRequestProperty("Content-Type", "multipart/form-data; boundary="+boundary); // without this Drupal redirects and we lose the cookie hconn.setInstanceFollowRedirects(false); OutputStream output = hconn.getOutputStream(); output.write( body.getBytes() ); output.write("\r\n\r\n".getBytes()); output.flush(); output.close(); String headerName=null; for (int i=1; (headerName = conn.getHeaderFieldKey(i))!=null; i++) { if (headerName.equals("Set-Cookie")) { System.out.println("logged in!"); cookie = conn.getHeaderField(i); } } if ( cookie ==null) System.out.println("failed to login. error=" +hconn.getResponseCode()); return cookie; } public void logout( String host, String cookie ) throws Exception { URL logout = new URL("http://"+host+"/user/logout"); URLConnection newConn = logout.openConnection(); HttpURLConnection hconn2 = (HttpURLConnection) newConn; hconn2.setRequestProperty("Host",host); hconn2.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9," +"*/*;q=0.8"); hconn2.setRequestProperty("Accept-Language", "en-US,en;q=0.5"); hconn2.setRequestProperty("Accept-Encoding", "gzip, deflate"); hconn2.setRequestProperty("Referer", "http://"+host+"/user"); hconn2.setRequestProperty("Cookie", "has_js=1; "+cookie); hconn2.setRequestProperty("Connection","keep-alive"); hconn2.connect(); int code = hconn2.getResponseCode(); if ( code ==200 ) System.out.println("Logged out!"); else System.out.println("Not logged out. code="+code); } /** * Just a test of Drupal login * @param args the command line arguments */ public static void main(String[] args) { try { DrupalLogin dl = new DrupalLogin(); String cookie = dl.login( "mydrupalhost.com", "anonymous", "secret" ); if ( cookie != null ) dl.logout( "mydrupalhost.com", cookie ); } catch ( Exception e ) { e.printStackTrace(System.out); } } }

Mouseenter, mouseout on tablets

My website looked cool with its mouseenter and mouseleave events being fired as the cursor sped across the screen, until I tried it on an iPad. Mouseenter was simulated by tapping, and a double-tap opened the link like it was supposed to, but the mouseout event wasn't firing at all, so that tapping on another link didn't clear the state of the first link. So the program logic was broken. How to fix this I had no idea. Googling it just came up with opinions saying that basically there was nothing you could do. Come on! There's always something you can do. Mouseenter works fine via tap. Then all you have to do is refactor your code so that it doesn't need mouseout to work. So on mouseenter just test if there is already a highlighted link and if so, do the mouseout stuff. But keep the mouseout function because mice are still numerous in this laptop/tablet/phone world in which we all live.

Also remember that fingers are a lot less sharp than pointy cursors. Things to click on need to be of a good size. Tablet users are a bit like babies: they need big fat bightly coloured things to touch.

More Drupal trickery

CMSes are not rocket science but they are complex systems, and learning your way around a new one can be a bit tiresome. So I decided to record a few fixes that took me hours to get right in case anyone else has the same problems. Also I can go back to this page if I forget.

Getting rid of those pesky node titles

If, like me, you like a simplistic home page that is mostly graphical, you don't want a title taking up vertical space and spoiling your carefully crafted design. Urgh! Although some themes seem to allow you to turn off titles, if you don't have that luxury you can still turn them off by setting the link inside the h1 or h2 element to "display: none" in your theme's .css file. I noticed that the exact element enclosing the node titles changes even within the space of a few weeks, so be careful. On my localhost installation the following worked:

.title { display:none }

But on the server a slightly different Drupal version or configuration forced me to change this to:

h2 a { display:none }

And even that is a bit dodgy, since it hides all links inside h2 elements. But refine it if you will. I used the 'Inspector' in Firefox Dev tools to find out what display properties the element in question had, then I looked for those properties in the theme's .css files, and if they weren't there I wrote a new rule. Easy stuff, but it's satisfying to get it working.

Just the story title appearing, not the content

This trivial problem had me going for a couple of hours. All it was was the 'teaser only' setting in Structure->Content Types->Appearance->ManageDisplay->Custom Display Setttings. Wow, they really bury this stuff in the bowels. I just set it to "Full content" and I got my content back on the home page. Whew!

Drupal on Postgres

I decided to use Postgres because someone else was already using Mysql on the server and I didn't want to be accused of screwing up their database. Also, I didn't know the Mysql password :-). Half the advice on this topic is just rubbish. People suggest tinkering with things that Drupal does for you automatically. Like adding a $db_url line to settings.php. What utter rubbish -- the installer does that for you. All you have to do is ensure that you have pdo_pgsql and pgsql installed. (You can check this by putting a file containing just "<?php phpinfo();" and then accessing it over the Web, to get the server configuration. You should have two sections with exactly those titles). Once everything is installed correctly Postgres comes up as a database option when you install. Easy-peasy.

Tooltips that are not painfully slow

If you use image-maps and areas to highlight portions of an image it is nice to tell the users what's in store if they click on that link. You can add a tooltip using the title attribute on <area> but all the browsers display it only after about 3 dreary seconds, by which time the user's mouse has moved on. No doubt that prevents irritating little popups as the mouse skims over the page looking for something, but in this case it is just plain annoying. So a custom tooltip is called for. I found plenty of advice and implementations on the Web but only a few that worked with <area>s. In the end, not wanting to download a monstrous library that didn't work the way I wanted I just concocted my own. (I should have done that in the first place).

You start with a simple definition of a tooltip. If you feel interested you can add a pointy arrow or whatever, but I was feeling a bit lazy:

div.tooltip
{
    background-color: #FFFFCC;
    width: auto;
    height: 18px;
    position: absolute;
    padding: 5px;
    -moz-border-radius: 6px;
    -webkit-border-radius: 6px;
    border-radius: 6px;
    font: 15px times,serif;
    box-shadow: 4px 4px 4px #202020;
}

That gives you a boring tooltip-like box with a drop-shadow and flexible content. The text of the tooltip can then be specified in the area's alt attribute. The real trick was placing it on the screen in just the right place: centred under the area in question. The problem is that image-maps don't belong to the browser's 'box-model'. Areas can be regions and are placed within the image-map as only imagemaps know how. So browsers ignore them. If you ask for the position of an image-map jQuery will give you the bottom-right corner of the image, and the size of the map as 0, 0. The same goes for the areas themselves. Luckily you can parse the coords attribute of the areas and work out their x and y offsets and their height and width pretty easily. Then all you need is to find the offset (top, left) of the grandparent of the area to find out where to place the tooltip. The meat of it is:

function doNothing()
{
var text = $('.tooltip').text();
}
function initTooltips()
{
$('area').each(function()
{
    $(this).mouseenter(function() 
    {
        var desc = $(this).attr("alt");
        $(this).parent().parent().append( 
            "<div class=\"tooltip\">"+desc+"</div>" );
        var tooltip = $('.tooltip');
        if ( tooltip.size()>1 )
            tooltip.first().remove();
        /* allow new element to get a size */
        setTimeout(doNothing,0);
        var tooltip = $('.tooltip').last();
        var coords = $(this).attr("coords");
        var values = coords.split(",");
        var bot = 0;
        var left = 10000;
        var right = 0;
        for ( var i=0;i<values.length;i+=2 )
        {
            var x = parseInt(values[i]);
            var y = parseInt(values[i+1]);
            if ( bot < y )
                bot = y;
            if ( left > x )
                left = x;
            if ( x > right )
                right = x; 
        }
        var awidth = right-left;
        var offset = new Array();
        var ppoffset = tooltip.parent().offset();
        var ttwidth = tooltip.width();
        offset.top = ppoffset.top+bot+5;
        offset.left = (ppoffset.left + left + (awidth/2))-(ttwidth/2);  
        tooltip.offset( offset );
    });
    $(this).mouseout(function()
    {
        $('.tooltip').remove();
    });
});
}

You need to call the doNothing function to give the renderer a time-slice to calculate the dimensions of your new div. Otherwise it can end up returning a width of 0. Now just call the initTooltips function in your jQuery ready function and it will be awesome.

A different home page

If you want to have your home page use a different template from the rest of your site I'd say you're quite sane. After all, who would be mad enough to want it to be the same? The easiest way to do this is to create a page--front.tpl.php file inside the templates folder of your chosen theme. Then hack it until it looks right. That particular template file will then be used instead of the regular page.tpl.php file just for the home page. A victory for simplistic home pages everywhere.

Ugly "read more" link on home page

If you publish a page to the front page Drupal assumes that it is only a teaser. So it adds a "Read more" link, which takes you back to the home page. This is really a bug, but you can get around it by making that page the default home page in Configuration->System->Site Information. Just type in your node URL for the basic page on the home page and hey presto: no more annoying "Read more" link. (This also gives the node-titles the .title .class, which makes it easier to hide them). Sometimes I get the impression that the authors of Drupal have created something even they don't fully understand.

Variable number of module configuration parameters in Drupal's hook_form

Drupal lets you define in a module a form for editing its configuration parameters but the documentation doesn't seem to explain how to do a variable number of parameters. For example, let's say you wanted to supply some fixed quotes to cycle through like a news ticker, but you wanted to let the user specify not only which quotes but how many^*. But Drupal seems to require you to specify a fixed number of configuration parameters for your module.

There is an easy way around this. You can define a parameter like quote_ticker_numquotes, and set it to 6 by default:

function quote_ticker_form($form, &$form_state)
{
    $n = variable_get('quote_ticker_numquotes',6);
    if ( $n > 32 )
        $n = 32;
    elseif ( $n < 1 )
        $n = 1;
    $form['quote_ticker_numquotes'] = array(
        '#type' => 'textfield',
        '#title' => t('Number of quotes (1 to 32)'),
        '#default_value' => $n,
        '#size' => 2,
        '#maxlength' => 2,
        '#required' => TRUE
    );
    ...
}

Now, having set the number of desired quotes to $n it is a simple matter to create that many fields:

...
    for ( $i=1;$i<=$n;$i++ )
    {
        $form['quote_ticker_'.ordinal($i)] = array(
        '#type' => 'textfield',
        '#title' => t('Quote '.$i),
        '#default_value' => variable_get('quote_ticker_'.ordinal($i),
            ordinal($i).' quote <a href="">more...</a>'),
        '#size' => 64,
        '#maxlength' => 128,
        '#required' => TRUE
        );
    }
    return system_settings_form($form);
}

'ordinal' is just a simple function to turn a number into its ordinal name. Now, whenever the user changes the number of quotes the number of fields for defining quotes will expand or contract accordingly.

^* The reason I decided to write a 'quote-ticker' was because news-tickers pull data from stories (nodes) whereas I wanted the quotes to be essentially fixed and to point to arbitrary URLs not just to Drupal pages.

Java garbage collection on Linux

Recently we had a problem with our Java web service. It ran a computationally expensive and memory intensive program that often ran out of memory for no apparent reason. Even though the files it was dealing with were around 70K each and it had 1GB of heap it ran and ran for 5 minutes and then failed with an out of memory error. So I ran the same program with exactly the same JVM settings on my MacOSX laptop with the same data and hey presto it ran flawlessly in one minute. Then I ran JProfiler on the two instances of the program - on OSX and Linux (Ubuntu 12.04). The JVM 1.6 settings were -Xmx1024m -Xss8m in both cases. Here is the OSX memory profile:

And here is the Linux memory profile during execution of the same program:

What seems to be going on here is that the garbage collector isn't being called often enough on Linux. And I know that the JVMs are not exactly the same but I did install the Oracle JVM, not the default OpenJDK on Linux but it made no difference. On the Oracle site they suggest trying the -Xincgc setting (incremental garbage collector). I added this to the JVM on Linux and magically the memory profile looked exactly like the one on OSX. It seems that those Apple guys aren't so dumb after all when it comes to tuning Java and their setup simply works better - for me at least. But you can get the same thing on Linux by adding one setting to your java invocation.