How often do we apply blink testing?

How often do we apply blink testing?

I apply blink testing any time I can arrange to be confronted with a blizzard of data: comparing screens (I glance at a million pixels, then at another million pixels, and in an instant I see the tiny difference between them), scrolling through huge log files, or watching an extremely rapid process take place. Anything that seems overwhelming to take in triggers me to consider a blink test. By the way, when I say “blink test” I’m talking mostly about a blink oracle.

When presented with an application to test, what triggers (if any) lead you to choose model-based testing? or do you always think through state transitions?

Everyone, always, is doing model-based testing if that term means “testing according to a model.” The only possible exception to that would be testing by accident. As soon as you test purposefully, that means you already have a model in mind.

Automatically generating tests according to a specified model is the narrower definition of model-based testing that people like Google’s Harry Robinson prefer. What triggers that for me is whenever I want to meticulously cover a test space that I can conveniently describe with a tractable handful of variables (or if I see a regular and simple notation to describe those tests, regardless of the number of variables). I then write a little program in Perl to generate the test ideas. I may also try to automate those tests. For instance, I wrote a program to generate tests for the example I used in my talk. It produced 152 state transition cases, each consisting of a start state, three actions, and an expected end state. Like this: TRANSITION SEQUENCES: 1 launching -> (finished launching) stop start -> running 2 launching -> (finished launching) stop reset -> resetted 3 launching -> (finished launching) stop stop -> stopped 4 launching -> (finished launching) reset start -> running 5 launching -> (finished launching) reset reset -> resetted ..... 148 stopped -> stop reset start -> running 149 stopped -> stop reset reset -> resetted 150 stopped -> stop stop start -> running 151 stopped -> stop stop reset -> resetted 152 stopped -> stop stop stop -> stopped


How do you suggest that you apply this technique with a complex application, something like MS Word for instance.


I’m not sure what technique you are referring to. If you are talking about using state models to describe a system, then it’s interesting that you ask that, because I asked Harry Robinson that same question (it was even about applying it to Word) after seeing an otherwise fascinating talk about state-based testing that he gave, years ago. As I recall, he wasn’t ready to answer that question. But I am. My answer is: ask Harry. Seriously, he has actually worked with model-based testing tools at Microsoft, in the years since I challenged him on this point.

I have another answer, too. I don’t apply automated model-based testing to entire applications, I apply that method opportunistically. So, if I were testing Word, I would be looking for features of Word that seemed especially like tractable state machines, or were tractable (meaning not too many variables and complications) to test via some other kind of model. Otherwise, what I am always doing is developing models in my mind (we call it learning) and using those models to test any given product, no matter how complex it is.

Have you come across any tools for automated Model based testing?

I use Perl. I’m sure there are other kinds of tools, but I haven’t used them.

What’s too much modeling?

Too much formal modeling is when you give it more time than it is worth, or when many other interesting things don’t get done because you are obsessed with the formalisms and the cool tools. Too much attention to one kind of model will starve attention for other kinds of models that also have testing value.
Technology Notes

The technology of Webinars is still a bit immature. I used GoToWebinar, and I think the price/value arrangement is pretty good. WebEx has better features, but it’s much more expensive.

I had to wear two headsets, one to record my voice and talk to the CAMUG auditorium via Skype, the other to talk on the webinar conference line. The CAMUG people (in an auditorium) and the online people were not able to hear each other. I wasn’t able to hear the online people, but they could type questions to me. I wish the audio was all somehow integrated online. I would have wanted to record the CAMUG audio, too.

Animations happening on my screen were not smoothly displayed to the audience, but at least they were displayed.