Creative Synthesis

Great (and content packed) talk by Nick Cawthon, University of Sydney, Australia.

Abstract:Through results gathered from a large-scale online survey, this paper empirically investigates the assessment of aesthetic in 11 common data visualization techniques. Visualizations represented in this study were generated from an identical hierarchical dataset and visually normalized to avoid unwanted implications of default application parameters or personal preferences. Results from subjective participant response shows data visualizations that portray non-orthogonal, organic qualities, receive higher aesthetic rankings. Provided rationale further correlate these qualities with animate attributes of motion, growth and evolution, positively affecting the perception of their beauty.

Qualities of Perceived Aesthetic in Data Visualization

It should be noted that (as far as I can tell) the paper only captures an earlier version of the study. Quite likely (and this is evident in the Alt.Chi reviews) there was some question about only studying the reported aesthetic quality, removed from questions about usability or fitness for task. This has obviously changed. There is now a task or two associated with each visualization. This allows not only a study in usability but also how actual experience with a visualization affects perceived beauty with it. Quite possibly a frustrating visualization will be seen as less beautiful than a slightly less aesthetically pleasing but more useful one. An interesting hypothesis.

One interesting thing about this study was their use of a very analog rating control. The control is a slider, with ugly and beautiful on either extreme. The subject is asked to move the slider back and forth to indicate their preference for one of the extremes (and the degree of that preference) but they are never given the digital values. This isn’t an entirely novel approach but it is very distinct from the continuing prevalence of the 1-5 radio ranking on surveys you see in the web. This is followed up later with a nice bit of ajax style interaction where the subject is asked to place visual screenshots of the representations on a preference line. This is a nice bit of survey design, mixing objective rankings with subjective ones in order to determine real preference.

I did find the survey to be overly long. This is not to say that it was a long survey but for a web survey with no compensation it was a little too much. By the time the subject gets to the free response (text input) question I can’t imagine they feel much like saying a lot. The authors did mention during their talk that they had a high amount of drop off. One of the advantages of a web survey is the numbers and the large number of respondents helps cover for this - producing enough full survey results to get meaningful data despite the high drop off.

The authors mentioned their use of timing during the survey questions. They used this capability to ignore survey results with large lag times where (presumably) someone left the survey for a long period of time before returning to it. This would skew the data related to the efficiency of a visualization for answering a particular question. This is a difficult problem because you do want someone to complete the survey in one more or less controlled session. The internet isn’t like a usability of psychology lab though - things do happen. I’m afraid I’m guilty of taking a phone call during my completion of the survey. I’m not sure if I was away long enough to get thrown out. It is this kind of situation, however, that suggest the need for a more interactive and flexible web experiment methodology

As they survey itself goes, its design makes some very nice use of modern ajax style interactions. In addition to the nice input controls mentioned above the designers also use some nice ajax transitions to help avoid the problem of an overwhelming number of pages. Page transitions still occur but additional questions are also loaded in when answering questions that still relate to a particular task or visualization. It helps to promote a nice logical grouping and a better user experience.

Well that’s enough about an interesting experiment and methodology. If you’d like to actually see the results, I suggest the paper or Take the survey for yourself. My personal favorite, the sunburst, does perform pretty well.