Testing websites (headless)

I covered using Selenium to test website in previous posts (starting with this one which covers install and first test). Using a full browser ensures real world testing and can be done interactively. Using a full web browser does come with a performance penalty and may make CD integration tricky.

Partially for these reasons, PhantomJS was developed. It is based on WebKit and as well as offering a JavaScript API it can be controlled using Selenium.

Installation is easy. Download the windows zip file from the downloads page. There are no dependencies, you just need the phantomjs.exe file from the bin folder. Move this to a directory that exists on your path, like you did for the other webdrivers. You can then use this as apposed to one of the other drivers simply by calling the PhantomJS webdriver with

driver = webdriver.PhantomJS()

All of the Selenium examples will work with this one change.

Running JavaScript

I’ve covered running PowerShell scripts using the powershell.exe command in this previous post. The same principle will allow you to run any script engine. For example, you can run JavaScript as long as you have node installed. Assuming you add node to the path variable (the default) there are no additional options required, just specify the script to run as the argument.

I’ll also use this opportunity to save a couple of lines. One of the many great things about Python is it gives you the source code to standard library functions. This is a great way to improve your Python ability, looking at how problems were solved in the library. I want to see if I can improve on the try catch block so I need to know what subprocess.check_output actually does.

I can see the code for check_output by opening subprocess.py from the lib directory. Here we can see that the main work of check_output is done by just three lines, initial the Popen object, then call its communicate and poll methods. We can take the 6 lines of the try catch block and replace them with these 3 giving us the following.

import subprocess, tempfile, sys, os

def js(command):
    commandline = ['node.exe']
    with tempfile.NamedTemporaryFile(suffix=".js",delete=False) as f:
    process = subprocess.Popen(commandline,stdout=subprocess.PIPE)
    result, unused_err = process.communicate()
    exitcode = process.poll()
    return exitcode , result

retcode, retval = js("console.log('Hello Python from JavaScript (node.js)';\n")
print("Exit code: %d\nReturned: %s&" % (retcode, retval))

You can make the same changes to replace the try catch block on the PowerShell post from a few weeks back.

Your own web Python IDE

A few years back I was wondering about editing Python in the cloud when I first got a Chromebook. At the time web based IDEs were only just starting to go mainstream; they have flourished since. How much better would it be if you could set up your own server? Writing a code editor in JavaScript is a huge undertaking but there are tools out there to do this and once you have the code the rest should be straight forward.

I wanted to try out Ace code editor for a few reasons; it is mature, supports Python (along with every other language I’m bothered about), is customizable, extensible, is licensed under a BSD licence, has an API and seemed easy to set up. So I set about creating a test rig so I could investigate the code editor with. Just getting the code editor to display without an absolute window position proved tricky (it’s always the little things that take the time) but eventually I discovered the div tag needed a height property to work the way I wanted it to.

With the editor now displaying, I really wanted to get my rig to run the code and display the result so I knew everything was working. This prompted the executing code post a couple of days ago (now you know the project I was hinting at). With a working execute function that returned the output as a string I just needed to get the text out of Ace using the method provided and back to the server.

I originally tried to do this without using any other JavaScript libraries but soon realised this was adding a lot of unnecessary and distracting script to the web page so went back to using the ever useful jQuery library to deal with the post and the updating of the results div. Surprisingly it worked on the first go.

The code (in BitBucket due to the length) uses all of the executing arbitrary code lines from the previous post but with the test now replaced with a bottle web server and a 35 line single page web app. I mixed CSS and JavaScript in with the HTML to simplify the server side code not because I recommend this or would do this in reality.

Running the code should give you an editor similar to the one below.