Hallways user testing: the crow on the shoulder.


Every software-product developer should be familiar with the milestone-book "Don’t make me think" from Steve Krug, and probably lots of us read it.

I did, and I agreed!

This happened some years ago…

Nowadays my days runs faster, and it may happen that old precious hints get lost.

Some days ago, I called Massimo, an expert .NET developer,  for testing our new product BugsVoice, but secretly to push it to develop, for free, an example of .NET page for catching errors (please don’t tell him).

He sat down in front of me (see our space here )  and started to enroll in BugsVoice.

I was peeping him over the monitors barrier watching  his face struggling, and then I

remembered Steve’ rule,  that actually is the all-month tips…IUCN_054

So I moved around the table and I sat behind him, really close, looking over the shoulder , to see both monitor and keyboard, like a crow on the shoulder…

… and the magic happens!

I discovered in a half hour session that some parts of our application are hard to understand or completely unclear.

We have already used, with satisfaction, professional testing service like UserTesting for our products; they usually provide you a video file recording user voice and screen interaction, but seeing a “human” (are programmers human?)  face give you an emotional feedback that you will miss in video files.

Some hints to get the best from user testing:

  1. give the tester a small overview of your product. Do not explain your product in detail, give only the context. Prepare a short story board: I gave my tester the following tasks: enroll, choose an error template, configure your server, collect some errors.
  2. invite the tester not to be shy and talk while playing with your app. Explain that it is not a competition, and that you are testing the application not the tester 🙂
  3. resist to interfere. Do not help your tester and take notes when the tester stops. Eventually hearten him and ask what’s the matter.
  4. look at the mouse movements. If the pointer goes around the screen without a goal, probably there is something too small, in a bad place, or too far. Take notes!
  5. look at the tester’s face. When you see self-satisfaction (seems that he/she is understanding something) ask what he/she thinks that feature is. Take notes!
  6. let the tester play whith you app discovering features. Take note of the path the tester follows to reach the goal. Is he using different/unplanned ways? Probably there is something to be done on the interface!
  7. look at the clock. Usually a “real” user does not give you more than few minutes before giving up. It is valuable to reach at least a rewarding step in the first minutes.
  8. at the session end ask the tester an overall impression of your application. Ask what he feels about each phase (enroll, step 1 etc.) and take notes
  9. then ask the tester some suggestions to refine your app. Take notes, but then throw them away, you already have a lots of  better feedback.

In our case a thirty minute session gave me five A4 pages full of  interesting notes.

What I wonder about user testing is that very often what  misleads the user is a wrong message, a color in a title, a misplaced button, a hidden something; all things that you can fix in minutes and usually every programmer tends to underestimate. Of course sometimes you may get in serious trouble when you miss a core point; this is why it is crucial to start user testing very early. I know that it is sometimes hard to do, how to test an application if a core functionality is missing?

We started user testing quite late with BugsVoice alpha (internally), and we are continuing with the public beta.

Give yourself the opportunity of a day as a crow on the shoulder, you won’t regret it.

XSS war: a Java HTML sanitizer

Yesterday I was testing one of our forthcoming online services, in order to check XSS (Cross Site Scripting) robustness.

XSS and CSRF (Cross Site Request Forgery) are the two scary “black beasts” of online services in good company with scalability,  but while even nerd-programmers think about scalability, almost nobody takes care of XSS and CSRF, at least until the first attack 🙂

The XSS becomes a serious  problem when you allow your users to add contents on your site/service; typical situations are blogs, forums, online chats etc.: when you are about to “print” user contributions you must do it conscientiously, you may found some “easter egg”.

Of course if you limit user contributions to plain text you will solve the problem in minutes, by just encoding every html tag (the idea is to change every “>” in “&gt;” and “<” in “&lt;” and so on). But things quickly get harder when you try to accept some basic html tags.

In our case, Patapage is a service that allows to add comments, threads and more options to existing  web sites, it is tailored to appealing interfaces so it is a MUST to allow users to insert a wide set of html tags. We have found on stackOverflow.com a good balance from admitted tags and refused ones; as usual Jeff  Atwood is prodigal of precious hints, but in our case while his set is fine for “patacomments” it is too restrictive for the scope of “patacontents”.

As you can imagine, I’ve found a lots of holes, so going upstream as usual and not giving a damn on, probably, the best Jeff’ suggestion (“do not re-write you own implementation”, an advice which he too is not following) I started writing code basing my implementation on these articles.

Another aspect of the problem is the conservation of layout: users can break the page by inserting unclosed or misplaced tags. If you hope that these flaws do not impact on security you are on the wrong path. A well planned css attack can, for instance, layer your application for click stealing or simply “switch” two buttons of your application with funny(?) results.

Of course I’ve looked around to find something matching my needs, but I found only two kind of approaches:

The first approach isvery basic focused on removing “< s c r i p t>” tags from your html; obviously this is a silly approach, you can inject js code without a “script” tag: you have a bunch of ways to do this by using dom events (onclick, onload etc.).
The second approach is to parse HTML to understand exactly what is happening in the code in order to remove un-allowed tags; this approach is logically the right one, you cannot expect to do better. Your code analyses the HTML, extracts the tags, and then you walk down the tree dropping out unwanted ones. Unluckily this approach requires a HTML parser library really “strong” with respect to malicious code, and usually these libraries are built for a different scope. Another aspect is that parser, lexer and walker are quite complicated pieces of code, so it is not a joke to test them completely. I’ve tested a couple of parsers with unsatisfying results.

This is why we wrote our own sanitizer by hand. Our approach is to remove unwanted tags and properties without testing HTML correctness in deep.

First step is to tokenize the code: a token could be one of : tag start (), comment (), tag content (blah blah), a tag closing ().

For instance <p style=”color:red” align=”center”>test</p> generates three tokens:

  1. <p style=”color:red” align=”center”>
  2. test
  3. </p>

The tokenize method looks for <…> pairs or comments, and that its fine for our scope of restricting accepted tags: if a <…> pair is badly closed the tag will be html-encoded.

Having the tokens list, we will test every single token whether it is acceptable; again, we do not perform tag matching at first, so for us <b>test</i> its fine, we are working on security, not on syntax – we’ll also fix that afterward, as it is easy in any case to add a tag-pair counter to close at the end unclosed tags, which you actually find done in our code.

We loop for every single token and we test it with regular expressions. The flow is:

  1. if token is a comment discard it.
  2. if token is a start tag (<p style=”color:red” > ) extract the tag (p) and attributes (style=”color:red” align=”center”)
    1. if tag is forbidden it will be removed
    2. if tag is allowed we will extract every attribute performing a check
      1. check “href” and “src” for admitted tags (a, img, embed only) and check url validity (only http or https)
      2. check “style” attribute looking for “url(…)” parameter, and eventually discarding it
      3. remove every “on…” attribute – e.g. onClick, onLoad, …
      4. encode attribute’ value for unknown ones
      5. push the tag on the stack of open tags
    3. else the tag is unknown and will be removed
  3. if the token is an end tag (</p> ) extract the tag (p) and check if the corresponding tag is already open. Eventually close those that are still open.
  4. else it is not a tag and we will encode it

In order to avoid js injection on user inserted URLs we will accept tag only if the “href” attribute is there and points to a valid URL; we test url correctness with Apache UrlValidator, and this will cut out every “javascript:” tries. Same approach for the and tags.

Finished my hard work coding the shelter, I give the happy news to our design department that the sanitizer was done, and after a first minute of excitement Matteo (http://pupunzi.open-lab.com) told me that they have three different usages of user input: for displaying an HTML page on the front office, for displaying a textual abstract in lists in the backoffice, and of course for storing contents on the database.

So a sanitizer needs three different outputs, html-encoded with tag, text-only without tag, and the “original” version for the database. This is why the latest version of the sanitizer returns “.html”, “.text”  and “.val”. Why you should store “.val” instead of the original input or “.html”?  Because the original input may be “dangerous”, and may mislead  the user in believing that all tags are allowed. The encoded value is not suitable in case of subsequent modification because of  double encodings (e.g. “>” –> “>” –> “>” and so on). On the other side “.val” removes only forbidden tags maintaining all other user oddities (strange tags, comments, etc.).

We have set-up a public playground for testing our sanitization code: http://patapage.com/applications/pataPage/site/test/testSanitize.jsp. This page allows you to input a text and by pressing “test” your input will be printed (sanitized) on the page.

Source of our sanitizer are released under MIT license (i.e. free as free beer, just keep the attribution); see the complete code here.

These tags will be accepted, others will be encoded, and printed. If you like challenges try  to inject a js in your text and, for instance, get an alert. Tell us about your victories, if any.

BTW, I’ve tested my code using XSS Me plugin for Firefox, and it passed all (about 15o) tests 🙂

30 years riding IT – part2

Year 1982, “mathematics 1st” second lesson: I meet Massimo, the red haired boy that will share my professional destiny for next 15 years.

Massimo owned an amazing programmable pocket calculator Hewlett Packard 41-CV.

“Calculator” is definitively a too modern term to define this strange tool that in order to do (1 + 2)*3 require you input 1 enter 2 enter + 3 *; this is called RPN (reverse polish notation). But the most amazing features of hp41cv where the HUGE memory (2233 bytes!), the capability of remember something like 300 steps of code and last but not least an alphanumeric display. Ok, alpha not so much, but surely numeric!



So, after a Christmas selling Hi-Fi in my father’s shop I got enough money to by my first programmable device (second hand, but in good shape).

My HP41cv was fantastic, with its “crunching” keybord.

With this archaic device I‘ve been initiated to the art of “quasi-computer” programming.


…to be continued

Berlusconi’s error codes

Writing javascript rules, used to process incoming bugs in BugsVoice, I’ve found some unusual http error codes (from wikipedia) that fit perfectly with Italy’s prime minister :

1) about criticism versus President of Republic:

error 405: Method Not Allowed. “A request was made of a resource using a request method not supported …”

2) about criticism versus high court’s judges:

error 406: Not Acceptable. “The requested resource is only capable of generating content not acceptable according to…”

3) about conflict of interest:

error 409: Conflit. “Indicates that the request could not be processed because of conflict in …”

4) about refund from Berlusconi’s Fininvest to De Benedettis’ CIR known as “Lodo Mondadori” (750.000.000 €):

error 413: Request Entity Too Large. “The request is larger than the server is willing or able to process.”

5) about “La Repubblica” newspaper:

error 415: Unsupported Media Type.

Finally, what doI think about the Italian’ prime minister?

error 417: Expectation Failed!