Saturday, 19 September 2015

In search of the perfect URL

For the most part, I really don't care about URLs, at least those on public-facing websites. I mean, when do you actually type a full URL beyond the domain name? Not very often. On some browsers and mobile phones, the URL is shortened or hidden, and that's fine too.

There are those who spend a lot of time worrying about URLs for the purposes of Search Engine Optimisation (SEO). The URL should contain the right keywords, imply the correct hierarchy etc.

They're very welcome to try, although I'm not sure what difference it makes. I don't imagine that the engineers at Google (note: other search engines are available) have spent the last eighteen years crafting a search algorithm that radically alters its understanding of content based on an easily manipulatable address, much like I don't expect banks to predict my credit rating using the numbers on my credit card. However, in common almost everybody else, I don't know this for sure, and so the business imperative to mess around with URLs remains.

The particular problem with URLs is that they are keys on the web, that massive distributed database of which we each control only a tiny portion. Changing URLs breaks links from parts of the web outside of our control. Redirects can be set up for each change, which creates extra work, and extra complexity. Software engineers hate complexity, because therein await the bugs.

Anyway, I was sent a link the other day that's been doing the rounds on the Internet. It was an article from the lovely news site Quartz.

Being utterly inconsistent on that day with my initial assertion in this post, I spent longer reading the URL than the article (the girl was probably bored and disconnected from wifi, let's try not to panic). I was interested that they put a numeric ID in the URL (which my current organisation is moving mountains to remove).  I made an experiment, and changed the latter part of the URL:

Hey! It still works. A redirect takes you back to the original URL. They've opted for a permanent 301 redirect, but otherwise they could still change the original, provided the ID remained intact.

This is the sort of solution that I really like. The SEO folks can fiddle with the URL until the cows come home, the engineers have the luxury of a straightforward rule, and the user never sees a broken link. Is this simple structure enough to keep everybody happy?

Sunday, 2 August 2015

The Surprising Vocabulary of the Computer Programmer

A thought popped into my mind one morning, then festered. Does writing software help with other forms of writing? “Obviously not!” seemed the reasonable reaction. Writing software has nothing to do with writing creatively. 

You could say that the rudimentary vocabulary of a computer scientist barely qualifies one of our number to make a good reader, let alone a writer. Look at those famous incantations we utter on a daily basis, ingrained into muscle memory, always at our fingertips: if, then, for, end. No, there’s no poetry there, just a repetitive succession of Germanic conjunction and command.

Those words. I’ve known them a long time. They are the first few keywords I stuttered into a keyboard, fingers on one hand, grasping “An Introduction to Programming in QBasic” in another. These few commands have passed into public understanding of computation. They are the alphabet of programmers. But the alphabet alone does not a book write. In common with any other craft, these fundamentals provide no more than the platform upon which arrays of grander abstractions stand.

And those ideas, that is where the vocabulary of the computer scientist begins to accelerate. The little Saxon conjunctions end and the Latin and Greek lexicon begins. We have wonderful words at our daily disposal: our lives are filled with encryptions, protocols, and transformations to name a few. Regressions and recursions. Interfaces and domains, caches and closures.

But these are all dry, academic words, you say. You’re right. There’s little to be done with them, they do not compile into meaningful or even beautiful prose, at least the sorts of prose that non computer scientists could enjoy.

Perhaps they can?

The thing is, we’ve borrowed almost all of these words from the rest of language in order to describe the abstract casts of our ideas, we see many words from a different view. Your concrete words: fork, or loop and hash mean something quite distinct to us, the same for threads and shells and branches. You could say we have a different perspective on the English language.

Perhaps we can concatenate words together in novel patterns.

Then refactor and reorder them to fanciful abstractions.

We know of maps and hashes

Of dashes and doubled forward slashes.

We’re compilers and composers

Sometimes even transposers.

Of abstract thoughts and n-dimensionality

Far from the immuted laws of our reality.

And day by day our expertise

Extends the mysterious and exotic:

The agents, daemons, and foreign keys.

Not to mention advanced robotics.

We build each day from thinning air

Instant worlds that live upon

The clouded hosts of software billionaires.

Wednesday, 15 July 2015

The perfect API that nobody uses

We all know the one about the tree falling in the woods. Does it make a sound? The question is not quite as trivial as it first appears because sound is a perception that only makes sense when there is a being present to hear it. Otherwise it is just a vibration, just as any unseen image is just a snapshot of flying photons.

It reminded me of a situation at work where one team built an internal API to process textual content and return useful tags. In order for it to be used, some amendments were requested to permit cross origin communication from a front end client. The builders of the API were unwilling to amend their API to accommodate this, and the web support team was equally unhappy about building a proxy.

So the API went unused. I wondered idly to myself whether it then still qualified as an application programming interface, if there was nothing for it to interface with.

Whatever the answer, it makes sense to find a balance between doing things correctly and the pragmatism to get something used.

Sunday, 31 May 2015

Why programming is sometimes like painting behind a towel rail.

Standing in the shower this morning, I decided suddenly that the colour on the walls was completely wrong.

I had painted the bathroom in a cool, blue colour two years earlier and thought nothing further of it since until, trapped inside by typical bank holiday weather, I’d been painting another room in our flat. A mellow yellow. I was in the mood for yellow. 

My mind idly turned to refactoring the room. Yeah, a warmer colour would do much better beside the wood, would complement it. There was, after all, plenty of time for an indoor job, with bank holiday rain outside lashing down the window panes.

A hurdle presented itself in the form of the chrome heater that keeps the towels warm (or might do, if we ever turned it on). The bars sit a few centimeters proud of the wall, enough to slip a towel through, but barely room for a paintbrush to apply a new colour behind it. 

It occurred to me that it would be complicated and fiddly. In fact, much like making a refactor to a piece of software.

It would definitely be possible to use a mini roller, and get a reasonable way in from each side. You could probably poke a brush between the bars and have a go at the middle. You’d need to use a few techniques but I could probably get it done. You could probably do the whole thing without even turning the heater off.

And for sure, it would be easy to miss something, that would be noticed and remedied later. The wall refactor would probably have bugs

Thinking about it, the way to do it properly would be to remove the heater from the wall completely, then repaint, then replace the heater. 

As a software programmer, that approach appeals because of its simplicity. There are just three things to do, and each of them is well-defined. The alternative, working around a big obstacle piece-by-piece, using various different techniques might take less time at first, but could ultimately take longer in the end. Taking it off the wall may require some up front effort, but on the other hand, would definitely give the best finish, and the least chance of missing something.

Monday, 28 July 2014

Coolant Crisis - From agile teamwork to lone-wolf game development

TL;DR: Made a game. On my own. Here it is

My day-job is developing web applications, usually as part of a team. I've done it for a while. Long enough to notice that the project or company may change, but the work is often the same. I challenged myself to do something different. I made a game. For mobiles. Entirely on my own. Here's what happened.

The Challenge

I'd never made a mobile game before. I didn't want it to show. I challenged myself to make something that was:
  1. Not a direct clone of another idea.
  2. A decent piece of engineering that doesn't look like "my first game."
  3. Enjoyable by people people who are not my friends.
Hoping to emulate the great tradition of classic games created entirely by one person I opted to do absolutely everything myself - with all the freedoms and restrictions that entails.

The Game

I needed ideas. I wrote some down. They were mostly puzzlers like the ones I enjoyed as a kid. My favourite idea was titled "Space Junk". In charge of a giant space magnet, you'd suck up pieces of flying flotsam before they'd burn up in the atmosphere. When three-of-a-kind match they'd disappear.

I gave a quick-and-dirty version to my sister to test. Pieces of flotsam fell from the top, the magnet grabbed them at the bottom. My sister practically cried with boredom.

It is quite easy to come up with an idea; it's hard to make one that's fun.

The game was dull because there was no drama. It was just a chore. I changed the junk into nuclear fuel rods, which explode when left outside of a coolant tank - which slowly leaks. Coolant Crisis! was born!

With an idea in my head, I moved on to the design..

The Design

..That's a lie actually. Like most programmers I jumped in to the code, then reworked everything several times. Working in a team, somebody would probably have nailed all the designs and UX first. Let's pretend I did..

Making something that looked "proper" required a lot of time. My first version was definitively 8-bit, but the nuclear connection drew me toward the Cold War, the 1960s, and something more analogue.

I put on my designer hat and sat down for the first of several long nights with Photoshop.

The evolution of the game graphics - the initial concept, the first version and the reworked current version. 

The Code

Although I said I'd do everything myself, I didn't include building my own mobile phone with home-made transistors and a USB wind farm. By the same token I opted for a good game framework..

I settled quickly on LibGDX, and I'm glad I did. The game is written in Java from whence it can be made to run on Android, Desktop, and can also be coerced onto iOS and Javascript.

LibGDX handles many of the scenarios that a novice mobile developer wouldn't think of - such as pausing the game when a phone call is received. You can build a simple game within a couple of hours.

There are a few differences for a web developer embarking on game development. It is hard to implement a model-view pattern. Instead it's more typical to work with independent actors that have their own act() and draw() methods, which slot nicely into LibGDX's scene graph.

As a web developer, there's usually little concern about perturbing memory, either on the server or client. However, to keep a game running smoothly, you need to keep the memory stable - avoiding new instances in the render loop, and relying on object pools, or (gasp!) reusable static members.

You also have to learn that unlike the browser environment, OpenGL gives you very few visuals for free (except for that ubiquitous teapot) - everything else you have to make yourself. For the most part in a 2D game it is just a case of placing bitmaps on a screen. The water needed a custom OpenGL shader. Other effects; explosions and bubbles were produced with LibGDX's particle system.

Developing the essence of the game didn't take very long and I won't dwell on it here. I created a simple model to manage the pile of blocks, created my "actors" - blocks, water, explosion effects, and they came together relatively quickly. But the game wasn't finished..

The Screens

There's a surprising amount of work to be done on all those other screens that I never thought much about as a player. Depending on the game, you might need quite a few. Apart from the mandatory menu screen, there's a level chooser, game won and lost screens, as well as an instruction manual. Subsequently there are screens for settings, stats, sharing on social media, rating the app, and so forth. 

All these screens are largely bespoke; with few exceptions games seldom use standard UI widgets, which brutally break that sense of fiction that players enjoy. For a web developer, it is a bit of a culture shock to find yourself without CSS, or even any usable fonts. However, such a blank slate almost demands you to come up with something different, which was kind of exciting.

Here's the result of many hours spent trawling the web for inspiration, toiling in Photoshop, then finally coding. It was a lot of work:

There is a lot of work beyond the actual game screen. The buttons and controls are very loosely based on the IBM 1401 Digital Computer, first produced in 1959.  The headings are based on Dymo embossed labels.

The Testing

A few friends tested an alpha version and gave me tons of feedback.

Whereas a web programmer usually deals with nice snapshots of data at given instants, the time element inherent in animation creates many transient bugs. What happens when other blocks are falling or in the process of being destroyed? Although the total development to that point had taken about a month of evenings, it was easily another month before I was confident that everything had been fixed.

While my testers tested, I spent many enjoyable hours in GarageBand, preparing some theme music for the game.

When a friend of mine told me that he had been playing Coolant Crisis! into the night I knew that something was going right. Time for a production release!

The First 100 Users

I built it. They didn't come.
From this point on, things became less easy. The era of "build it and they shall come" - if that ever existed - certainly didn't apply. The Google Play store currently hosts about 1.3 million apps. My little app became the newest fish in a very big sea.

Usually distribution is very much somebody else's problem. In this case, it was still all up to me.

I checked to see how my game ranked for certain searches. It was disheartening. It sucked. I wrote a good description, but came nowhere near the top 100 for the majority of queries I imagined players might request. Coolant Crisis! even ranked #15 in a search for its own name.

It came as no surprise then that organic growth did not materialise. I decided to jump-start the game with some paid promotion. I used AppBrain's pay-per-install advertising service and got a hundred users playing the game within a few days. But there was a problem...

The UX Fail

.. there was a discrepancy between my loyal testers - who played to the end - and my unknown installers  - who typically didn't get past the first level.

But why couldn't they pass Level 1 when it was so easy? Loyalty aside, my hunch was that they didn't understand the controls - which required double tapping. I had already unconsciously acknowledged the problem; the manual explained double tapping IN CAPS. Explaining why a message needed caps in an developer planning meeting would probably have resulted in a change of heart much sooner..

I remember the good days, when the games I bought came in a box, often with a manual, which I would read on the way home.  Those days have passed - certainly for mobile apps. There is no interval between the shop and powering up the PC at home - the app is in your hands in seconds. People don't spend time reading your words.

I tried everything I could to make double-tapping ever more obvious. I made the user manual interactive, then added custom help message if no double-taps were detected. It didn't make any difference.

I realised why a bunch of mobile games use clunky up-down-left-right buttons - there's no room for doubt.

In the end I did the sensible thing and changed the control method completely. I eliminated double taps and added little markers indicating where blocks could be placed. People started to play past the first level.

As to how things will pan out now, it's too early to tell. A trickle of people are finding the game, and it has been played in unlikely places all around the world. I'm into marketing now, and it's an eye opener. In a couple of months I will post an update on what happened next in the life of Coolant Crisis.

The End?

There is something great about having created a finished product from scratch, even better one that you can hold in your hands (kinda) and has downloaded entirely out of your own brain.

Working in a team is usually great - and you can achieve so much more; yet there's supreme satisfaction in building something all of your own. There's lots to do; a solo indie developer has to wear a lot of different hats - designer, programmer, tester, composer, marketer, trilby. I learned a lot.

But is it finished? A personal labour of love is difficult to let go. Especially when there are so many parts that aren't perfect. Maybe in a few months I'll have a new obsession, but for now I've got some new ideas to try out.

So did I meet any of my objectives? Judge for yourself:
Click here to download Coolant Crisis from the Google Play Store

Monday, 27 January 2014

Sizing images for a responsive website, oh and retina too.

One of the issues with responsive sites is mixing the fluid width elements with fixed sized ones. I came across this problem recently, while adding a news feature (containing images) to a responsive site.

Here is my attempt at a no-compromises approach, where the image would always look great, regardless of the size of the screen.

Basic Implementation

Here's a summary of the basic idea:

The Server Resizes Images on the fly
This is key to making it work, and is not so frightening as I thought. As I'm currently in the .NET world, I came across a library called ImageResizer which is intended to do exactly this. The library claims to be almost as fast as the underlying GDI+ graphics operations, and can spit out your image in any shape or size you want, with nice cropping options.

Here's the basic idea. Create a service / web form which generates the image on a fly by resizing it and sending the result through the HTTP response. Here's an example in C#, but the principle is the same in Java.

var filename = Request("filename");
        var resizeSettings = ResizeSettings() {
            MaxWidth = Request("width"),
            MaxHeight = 261,
            Mode = FitMode.Crop
        Response.ContentType = "image/jpeg";
        Response.Cache.SetMaxAge(New TimeSpan(7, 0, 0, 0));
        GetImageService().GenerateThumbnails(filename, resizeSettings, Response.OutputStream);

GetImageService() simply wraps image resizer and maps the filename to the image's actual location.

The Client decides the width of the image
The client needs to ask for an image in the right size. The image filename is generated dynamically by the Javascript. When the article model loads, the image filename is also returned. The client then adds parameters asking for the correct width.

I'm using Knockout JS on the front end, so its easy to encapsulate all this within a custom binding. Here's a simple example:

ko.bindingHandlers.dynamicImage = {
        init: function () {

        update: function (element, valueAccessor) {
            var filename = ko.utils.unwrapObservable(valueAccessor()),
                width = Math.min(jQuery("main").width() || 600),
                height = defaultHeight,
                src = "article-image.aspx?" + jQuery.param({
                    filename: filename,
                    width: width,
                    height: height

                .css("background-image", "url(" + src + ")")
The knockout markup looks a bit like this:

  <div class="article-image" data-bind="attr: { title: caption }, dynamicImage: filename">
    <span class="attribution" data-bind="text: attribution"></span>
  <figcaption data-bind="text: caption"></figcaption>

I'm using a background image within a <div>, rather than an <img> tag for a reason, see notes at the end.

A little CSS ensures the image appears as a correctly sized placeholder while the image loads.

.article-image {
    padding-top: 50%; /* 2:1 ratio */
    width: 100%;
    background-color: $image-placeholder-background-color;

The no-repeat will stop the image tiling if the user maximises or otherwise increases the size of their browser window.

News article on a desktop browser

On mobile devices


Supporting High Pixel Density "Retina" Screens
It's not just iPhones anymore. Most android have high density screens to varying degrees, and my lovely Mac Pro has one too, so it seems worthwhile to get this right. Fortunately the generate-on-the-fly approach makes it very easy.

I modified the server script to accept another parameter "retina". If this is set to "true" then the requested width and height are doubled.

if (retina) {
   width *= 2
   height *= 2
   quality = 50

In addition, as people have noticed, you can also dramatically ramp down the JPEG quality setting on retina images. In my script it takes it down to 50, although you can probably go lower. This keeps the retina images on a par (or sometimes even smaller than) the 1:1 image sizes.

Front End
The image filename builder needs to send the retina request. It needs to know if it is a retina device. You can do this in various ways using Modernizr queries, but since the image change is fairly harmless in modern browsers, I think you can get away with:

var retina = window.devicePixelRatio && window.devicePixelRatio > 1;
src = "article-image.aspx?" + jQuery.param({
                    filename: filename,
                    width: width,
                    height: height,
                    retina: retina

                .css("background-image", "url(" + src + ")")
                .css("background-size", width + "px " + height + "px");

Then it's just a case of ensuring the background size is maintained with an additional CSS property.

The advantage of this approach is that the browser only needs to download a single image.

Giving the server a break
ImageResizer comes with various options for caching generated images, although you could roll your own too. Asking for images at arbitrary widths, however, may reduce cache hits though. Here are some suggestions:

  • Give your article a max width. I went for 600px, which in any case is about as wide as an article can be before readability starts to suffer. This limits the number of sizes that can be requested, and also allows you to resize images to 1200px upon upload, as this is the largest (retina) size that can be requested.
  • Round up your width requests to the nearest 50px, reducing the number of possible size requests. By placing your image inside a div with background-image, the browser will automatically crop and center the image if it is a little larger.

Sunday, 8 September 2013

A £100 Solar Light Challenge

There comes a time in every young man's life when he becomes, in fact, not so young anymore. At this point instinct takes over and before long the fellow has furnished himself with a shed, just as did generations before. Once all obvious enhancements anticipated by the shed manufacturer have been installed, the time comes to pimp the shed further.  The generation of electricity, a precursor to WiFi and therefore happiness, is a strong contender. Time for a challenge..

That it is possible install a solar powered lighting system for less than £100, without using a kit.

(The kit constraint was easy, because most of them cost more than £100 anyway)

Literature Survey
Now, it has been a long while since I've done anything electrical, but a brief amount of research suggested I'd need the following:

  • A solar panel, obviously
  • Battery to store all the lovely power
  • A solar controller
  • Light switch
  • A high power LED
In the interests of efficiency, I decided not to trouble myself further with any theories of how these components might work together.

Method I
Although tempted to buy something enormous that would either improve or catastrophically degrade the structure of the shed roof, I opted for a 40W 12V solar panel on eBay (£41.99). It's a decent size, about the area of an iMac screen. It seemed like it ought to charge a few batteries.

I bought 10x super 2400mah AA batteries for that purpose (£16.95). Then a weatherproof switch (£6.20), and the solar controller for £8.95 (Total £74.09). The solar controller takes the current from the panel and charges the battery, as well as providing output for the lights. As you can see, it is fairly obvious what you're supposed to do with it:

The solar controller. Looks professional doesn't it? Good UX.
With the big ticket items bought, but with only about £25 left from the budget I headed off to Maplin electronics, where I hadn't been since school. How the UK high street is able to support a store which, as far as I can see, is simply an emporium of wonderful toys for people like me, I do not know. I am, nonetheless, very grateful for its continued existence and spent an endorphiny hour browsing the interesting things. I bought some more stuff:
  • A pack to hold the batteries together, £1.50
  • A hobby box for the battery. Unnecessary but it looked professional. £2.99
  • A couple of blocks of strip terminals for making hassle-free connections, £2.18
Remembering from tedious GCSE physics experiments that thicker cable has less resistance, and not wanting to squander the precious juice from the sun, I also bought:
  • 5m electric cable, conveniently red and black £6.95
Now, my little 12V system would struggle to illuminate even the most efficient of domestic energy saving lights, so I read somewhere that the best thing to do is to build your own bulb from an LED light.

This turned out to be a mistake.

Unawares, I went over to the components desk (surely the most awesome part of any Maplin store?) and bought the brightest looking 0.5W LED I could find (£3.09), a heat sink to stick it on (£1.50), and an LED lens (£1.49) to make it more light bulby. 

Total cost so far £93.79

Being a software developer who likes modular development (and not wanting to spend the next few hours standing on a chair), I cut out a piece of MDF to fit into a space above the door, and laid out
the pieces on top. The piece would slot onto the wall later.

An issue was identified after connecting the batteries to the controller. The controller display shows the battery charge level continuously with three LEDs, ironically draining the battery. Although appreciating that measurement naturally changes the state of things, I didn't want Heisenberg to leave me in the dark either. The best thing to do is to add a switch (£2.49) to be able to disconnect the battery from the controller when not needed.

At this point OCD took over and I decided to run all the cables round the back of the panel so the front would give the impression of competence by the installer and might even add valuable equity to the house.

Connecting all the cables together was straightforward. The next exciting stage was constructing the light bulb. This involved sticking the LED onto to the heat sink (the requirement of which should have been a warning), then soldering on a cable on either side, then sticking on the lens with a hot glue gun.

I hadn't soldered anything for years. Now I can pretend I have a robot helper.

The finished bulb. Professional but doomed.

A fan of unit testing, I decided to test out the light before attaching the solar panel (the batteries were already fully charged). The LED lit up brighter than the sun for an instant, then dispatched itself to the great LED socket in the sky. I didn't think very hard about this, and ran out to buy a bigger 1W bulb. (£4.09).

This took my budget to £100.37

Close enough, I thought. And it would have been if the 1W bulb hadn't blown in exactly the same inevitable way as the 0.5W bulb. I should have paid more attention in the tedious physics lessons.

It turns out that 10 x 2400mAh batteries at 1.2V can readily produce a lot more than 1W for an instant. What you need, I learned, is a 2W LED driver, which manages the current to the LED, which changes continuously as the LED operates. To be safe, I bought a 3W LED (£5.99) to match. Now everything seemed to work OK, albeit at £15.35 over budget.

No point crying over blown LEDs, so I moved on to the exciting and manly phase of installing the solar panel on the roof. I drilled a decent hole in the roof, just under the top layer of roofing felt (still watertight, hah), then attached a baton for the panel to rest on, allowing gravity to take care of its adhesion to the roof (a decision to be regretted as soon as the wind picks up, no doubt). And here it is:

And what's this? The controller lights up like a Christmas tree, the charge light also illuminated. What epic success.

The smugness subsided when I tried out the light properly. Ouch. After two minutes the light and heat sink was too hot to touch. Afraid it would burn out (perhaps taking the tinder dry shed with it) I switched it off. Not many of my planned shed projects will take less than two minutes. In any case, I don't want my precious solar juice wasted heating the back of the light.

Method II
The budget in tatters, I returned to eBay and found an exciting alternative:

The car lights arrive. 4 LEDs in each, with connectors and adjustible brackets. Bargain.
With the £100 challenge failed, I decided to try and make it look as professional as possible and the car lights and spotlight turned out to be just the ticket. But first it's time to massacre the spot light..

Turns out there's simply a latch in there you can pop out with a screwdriver, but hacking it out is its own reward.

By chance the car LED lights were a perfect size, and with another connection and a bit of hot glue I had a low energy adjustible spotlight that looked like, well, a proper light.

Compared with the high power LED, the light is rather clinical in nature. But it burns as cold as it looks and seems to last as long as I need it for. The batteries are sufficient to run both of them at the same time with no dimming. 

Success at last.

So there you have it. A working solar system, admittedly for more than £100. But, I console myself, if I hadn't wasted time and money on the high power LEDs light dead end, it would have been possible to do the whole lot for just about under £100. Well, £105.17. Still not quite within the budget, but on a good day on eBay..

Nonetheless, the trial and error is much more fun than deciphering instructions on a kit. Part of the fun, as with software, is to be able to say "I made that." Then, returning six months later, cringing inwardly, and think "What on earth was I thinking when I did that?"