The Seapine View

Automating UI tests has the down side of needing to be able to control the mouse and the keyboard as well as needing to be able to see the application running on the screen. This can cause trouble if you need to use the computer for other things or want to keep the workstation secure while the tests are running. Luckily, there are tools available to allow you to automate in a virtual environment and still be able to use or lock the hosting computer.

Continue reading…

Automating UI tests has the down side of needing to be able to control the mouse and the keyboard as well as needing to be able to see the application running on the screen. This can cause trouble if you need to use the computer for other things or want to keep the workstation secure while the tests are running. Luckily, there are tools available to allow you to automate in a virtual environment and still be able to use or lock the hosting computer.

To setup an example virtual environment, I installed Microsoft Virtual PC. It’s available from Microsoft as a free download, all you need is a way to install an operating system that you want to use to test on it. The operating system doesn’t even need to be the same as your host computer. In my tests, I used Windows Vista to run a Virtual PC with Windows XP SP2.

I installed the virtual machine additions by using the “Install or Update Virtual Machine Additions” item in the “Actions” menu. This allows you to resize the virtual PC in the same manner as you resize other windows on your desktop. It also allows the mouse to cross into and out of the virtual PC window seamlessly. This mouse feature will cause the automation problems however, as we need to allow the virtual PC to completely control the mouse if we want it to test the UI. We can turn off the mouse capture by going to “Settings” in the “Edit” menu. From the dialog, select “Mouse” and uncheck the box labeled “Use pointer integration”. Note that this can only be done while the virtual machine is running.

Within the virtual machine, I would also recommend turning off any screen savers and setting the power options to “Always On” so that you can be certain that your automated tests will be able to run uninterrupted.

Now that the virtual machine is setup, you can start your automated UI tests within the environment. After the test has started, pressing the right-hand Alt key will release the mouse from being locked in the virtual machine and allow you to continue to use the host PC as normal. This includes locking the host machine, or even running another copy of the virtual machine to run multiple tests at the same time.

I review a lot of UI design lately, and this is the thought that most often comes to mind:

“A designer knows he has achieved perfection not when there is nothing left to add, but when there is nothing left to take away.”
    ~Antoine de Saint-Exupery

A quick SAT-style analogy:
“Our software is bug-free.” : QA tested ::
      “Our software is easy-to-use.” : Usability tested

My brain is full…

WPF Internals

The WPF Internals class was less about WPF and more about what made it work. Not quite what I expected, but interesting and useful regardless.

Dependency Properties are like normal C# properties, but with some special and interesting ideas that make them work for WPF. Unlike properties, they: can have a default value, do not use a backing store if one isn’t needed (i.e. if the property is set to a default value, it won’t bother allocating memory for the property), can inherit values from parent classes (e.g. if the form’s background color is red, and a button on the form is default, it will inherit the red), and can have change notification events setup for them.

“Commands” are single actions that can be attached to menu items, toolbars, etc. so that a single place can be marked enabled/disabled and cause a single action to fire when triggered. Almost another page directly from Qt’s QActions book.

Data Binding using the Dependency Property events creates simple action-reaction lines. Sort of like how Qt handles signals and slots.

Advanced UI Design Strategies

Jim Hobart had a great session on strategies for UI design. Here’s some of the better bullet points I picked up:

• Rules vs. Strategy: Rules (like UI guidelines) are like rules to any other game. You may know all of the rules on how to play basketball, but that won’t let you win the game. You need a strategy to succeed.
• Complex systems are still usable, there are some things that have to have a learning curve because of their complexity. Are you building a kazoo or a violin?
• If you ask a user “What fields can I get rid of?” they will always claim to need all of the fields. Instead of asking what fields to remove, ask the user what they’re trying to accomplish from the page.
• For an average web user, mousing over causing a highlight is a good way to indicate interaction is possible. After staying over for 1200ms-1300ms (gathered from testing), a popup of more information can be shown.
• Delays in the UI cause a user to lose their concentration. After about 2 seconds, users will lose track of what they were trying to do.
• People don’t read.
• UI designs for probability. (What do most users do?)
  Engineers design for possibility. (What might happen?)
• Better interfaces usually have a high ratio of content to the tools necessary to navigate that content.
• Error messages make a user feel dumb, whenever possible try to interpret whatever the user inputs and use that.

Some good UI classes today, likely best explained by the slides. Jeff Patton also gave a good analogy for UX testing. Looking at a parked car, can you tell how fast it can go? Sure, with a bit of knowledge about cars, you can make an educated guess, but it’s still just a guess. Usability is like that, you can look at a product and guess how well a user can use it to accomplish a task, but you’ll never really know unless you see them try it.

He also made mention of Donald Norman’s Emotional Design which had some good points to it, and I’ll have to read that book next. In it, it’s mentioned that perceived ugly things are labeled by users as “difficult to use”. Users will even report bugs in the software and use those to claim that it’s of poor quality. A “pretty” UI will still get users to ask for fixes, but they’ll be much more forgiving of them.

Jared M. Spool also gave a good talk on what causes software to be labeled “intuitive” and steps to take to get there. A lot of his talk was theory and results of his testing, with some nice visuals on the slides. Unfortunately, the slides are not yet available.

Just a few things of interest today:

On the UI, Vista Command Links are really a quite good alternative to annoying message boxes. Unfortunately, this implementation of them is Vista-only, making them near useless.

WPF seems like a page borrowed from Qt’s book. XML describes the UI, C# (or other .NET language of choice) controls what the UI is able to do. But they go beyond that and let you customize things like colors, gradients, shapes, animations, fonts, etc.

Microsoft Virtual PC is now available for free. You could even download a 30-day eval VHD of Vista if you wanted to run that.

In C# (and .NET in general), pinvoke.net offers the C#/VB method signatures needed to call into native Win32 (and other) APIs. Handy reference if you’re in to that sort of thing.

I thought I’d jot down a few notes of some of the cooler things I see while at SD West.

C# 3.0 Method Extensions

File as: Better code organization.

In C# 2.0 you might do something like this:
public static class MyStringUtils
{
public static string PrependDate(string str, DateTime date)
{
string strOut = date.ToShortDateString();
strOut += ": ";
strOut += str;
return strOut;
}
}

// And use it:
string s = "Try this.";
string s2 = MyStringUtils.PrependDate(s, DateTime.Now);

It works, but you end up with little utility classes when you want to do something specific with a string that the developers of the string class didn’t think of or didn’t need. All developers need to know where your special class is and to use it when they want to do this action.

C# 3.0 lets you make a method an extension on some other class. The above example would be done like this:
public static class MyStringUtils
{
public static string PrependDate(this string str, DateTime date)
{
string strOut = date.ToShortDateString();
strOut += ": ";
strOut += str;
return strOut;
}
}

// And use it:
string s = "Try this.";
string s2 = s.PrependDate(DateTime.Now);

The code differences are relatively small, but the implications are large. Essentially we’re saying we wish this built-in class had this method, and now we have a way to do that without dealing with knowing to use a specialty side class or creating a new inherited string class. They even conveniently are available in Intellisense:

C# 3.0 Lambda Expressions

File as: Shortcuts to easy, readable code.

I thought foreach was a great thing in C#:
foreach (string str in strList)
{
Console.WriteLine(str);
}
It at least eliminated that nastiness of a temporary int variable that counted up through the string list until it hit the count of number of strings in the list.

C# 2.0 had anonymous delegates to make it a little shorter:
strList.ForEach(delegate(string str) { Console.WriteLine(str); });but the syntax is a bit confusing, and therefore rarely used.

C# 3.0 makes it short and neat:
strList.ForEach((str) => Console.WriteLine(str));

There were many other things in C# 3.0 that I found useful, but these were my top two picks.

The long-awaited conclusion…

Part 3: Post-Test

Our usability testing regarding recording seemed inconclusive. Both the old way of recording and the new way were intuitive enough for our testers to breeze right by them. In the end, we ended up with a kind of hybrid approach that seemed to please both the developers and the testers. It turned out something like this:

This organization and wording of the menu items spell out the actions they cause as clearly as possible to (hopefully) avoid any confusion.

As for the grid view, we made what changes we could before release to address some of the more troubling issues our test users had with it. Even more changes are in store for the next one and we plan to look at some even larger-scale changes for future releases. Many of the changes would not have been as high a priority or looked at as closely without actually seeing a user struggle with them. It’s difficult to ascertain usability from a list of defects marked “UI”.

A summary of post-test lessons learned:

• Sometimes testing won’t reveal what you want it to. Either perform a varied test, or go with what data you did get.
• Reviewing the video can reveal even more than your initial notes. Be sure to check it.

If anyone tries to convince you that the second kid is easier than the first, they have it backwards. So, here’s a belated usability test sequel.

Part 2: In-Test

Our first tester sat down with the testing goals, made a new QA Wizard Pro script, and just started recording. Our whole setup to look for troubles while trying to record were bypassed in less than a minute. He was well underway to completing the entire first goal in just a few minutes. We kind of looked at each other. “That means it’s intuitive, right?” Shrugs for a response.

Upon hitting the grid view of the script however, there was some trouble. It was nothing we either expected or were looking for, and I frantically started writing down as many observations as my hand would allow. We knew that there were some minor issues in this area, but the extent of how those affect actual users is lost until you see it in person. We had a hard time to keep from offering help, but we held our tongues. Eventually he got the script how he expected it.

Script > Run. Script runs, all looks good, first goal accomplished. Or so we thought.

Our tester saw it run, it flew by quick and seemed to end, but he had no proof. The script run ended without error, but also without any other message. A good 20 minutes later after re-running the script in many different ways and beginning to consult the help files, we decided that we had to intervene to keep the process moving. This was a surprising find for us, we always had found QA Wizard Pro playback to be quick and easy. The reports were always available and ready if you wanted to save the results.

So again we gathered a large amount of great information, and again it was not what we had set out to test. Record in the new fashion, the way that all the developers hated, seemed to work naturally in the test. So, we decided that we would run a second test to get a data point with the record set up the old way.

Our second tester took a little time starting up and looking over things, but took to recording just as quickly as the first. It seems that both the old organization of recording and the new organization were just as intuitive.

Watching the second tester work with the script in the grid view was mostly an exercise in us nodding our heads. All the troubles we saw from the first tester were reinforced by the second one, just in case we missed it. A few other informative finds later, and the second test was complete.

A summary of in-test lessons learned:

• You’ll want to help the testers along, resist. Unless the tester gets to a point where they are not likely to make any further progress.
• You may go in expecting to see data in a particular area. You probably will come out with data from lots of other “known good” areas.

So what did we decide to do with record and the other results? Refresh often for part 3. Hopefully with a smaller posting delta than the gap from part 1 to part 2.