Wednesday, August 30, 2006

TMap Next book

New TMap book "TMap Next"

Near the end of last year the start was given for a new version of TMap. Based on experiences, ideas and a lot of change requests from testers around the world we (my co-authors are Leo van der Aalst, Bart Broekman and Michiel Vroon) started writing. Now, 8 months later, we look back on a highly effective period with a revised and enriched method and book as a result. The results of the first reviews are very hopeful and give us the idea we did our work well.
Initially, we aimed for a Dutch publication first, but this has changed. The aim is now to launch the English book at the Eurostar conference early December (, the Dutch book will follow soon after. See below for a quick overview of the contents of the book. Essentials of the book and method are:

  • Based on a business-driven test management (BDTM) approach, allowing the customer to control the test process based on rational and economic considerations.
  • Gives a full description of the test process.
  • Contains a complete ‘toolbox’, i.e. technique descriptions, checklists, procedures, and so on.
  • Adaptive, suitable for all testing situations in most development environments (new systems, maintenance, waterfall / iterative / agile development, custom-made or software packages, and so on).

There are a number of reasons for this completely revised version:

  • Time didn’t stand still, a large number of people have asked for revising the method.
  • At present, it’s clear for most organisations that testing is necessary. So, instead of the discussion on why to test, the discussions are more about how long, how much and what to test.
  • In the current version testing is described as a stand-alone process in waterfall development of new information systems. The current field of play in IT is much larger: more maintenance instead of development of new systems, many implementations of standard software (like SAP) and iterative and agile development besides waterfall. Although the method has grown with current practices, this is not the case for the book. In the new book testing is more seen as an integral part of the larger whole.
  • In many organisations testing is organised as a separate department instead of purely as a project activity. Many kinds of test organisations are possible, from expertise centers up to complete test factories, each with certain advantages and disadvantages. In current testing literature this is given little attention.
  • Besides the theoretical part, readers were looking for the practical ‘how’ in TMap. In the new book a lot of tips and examples are given.
  • And perhaps the most important reason: testing should much more be seen as an economic activity within IT. Time and costs, but also the benefits and risks, have to be made clear to the customer. With this information he or she can control testing in a rational and businesslike way, finding the right balance between required time and money on the one hand and the benefits on the other: insight in quality and risks, confidence in the product and project (tracking) information. This part of TMap is called BDTM, Business Driven Test Management, and will be the guiding principle of the method.

TMap offers the tester and test manager a guide to deliver results to the customer within his/her context.

The change in method does not only has consequences for the book. Also trainings, certification programs, websites and such need to be adapted. We will keep you informed!

Overview book
General part
The first part of the book contains an introduction to TMap and to testing, the what and why. The essentials of TMap are explained.

This part deals with the various test processes, including a number of test supporting processes. The relations between these processes are also explained. Processes described are master test planning, acceptance testing, system testing and development testing. In the processes, special attention is given to their relation with the rest of the system development process, to BDTM and to managing the tests. To the phases Planning, Control, Preparation, Specification, Execution, Completion a new phase has been added: Setting up and Maintaining the Infrastructure. Within the phases, a better distinction is made between activities of the tester and the test manager.
Besides these primary test processes, a number of supporting processes is dealt with. These are in the following areas:

  • Organisation
    Organizing a permanent test organisation, from a test expertise center to a test factory, outsourcing of testing;
  • Test environments
    Requirements of the test environments, processes to maintain the environments;
  • Test tools
    Kinds of test tools, implementing a test tool;
  • Test professionals
    Hiring, test functions, training and career paths.

The last part of the book contains so-called component chapters that are used within a number of the above processes, like techniques and procedures.
Some examples of components are:

  • Product risk analysis (PRA)
    The steps for conducting a successful PRA are explained: choosing participants, defining the approach, organising the PRA, classifying risks, preparing a session/interview, collecting and analyzing risks, checking completeness;
  • Test types
    Discussed test types are for testing regression, usability, performance, portability/compatibility and information security;
  • Estimation techniques
    Several techniques for estimating the test effort are given, including Test Point Analysis;
  • Test design techniques
    Starting with an explanation of the concept of test design and test coverage, a number of basic techniques (9x) giving certain coverage are described, followed by a starter set of useful test design techniques (12x).

Based on many practical experiences, the book has been revised and enriched, containing many tips, examples and in-depth explanations.

Thursday, August 17, 2006

Is there anything left for us to write in Testing?

Hi Reader,

I have often stumbled upon Cem Kaner's Articles page and it is very recently that I realized, most of the articles have been written and published.

For every problem, you encounter in testing, people like James Bach, Cem Kaner, Jack Falk, Hung Nguyen, Bret Pettichord.....( I could have missed your name too, apologies for that), have written and published articles but "What are we writing nowadays?". For everything else, there is Jerry Weinberg .

I, being naive, did get disappointed but something struck me recently, with which, I now have confidence, to write or continue writing articles about testing.

_ Is there anything left for us to write in Testing? _

Myself being in India, luckily, gave me an opportunity to become optimistic on the scope left in writing articles in testing.

You might be interested to know as to what was one of them, which pulled the trgigger in me to become optimistic about writing and here it is for you.

I graduated, as an engineer, from one of the engineering schools in India and I was able to recollect that for every subject we had to study, we usually referred to two books, one a foreign author and the other an Indian. It is not that we wanted to show patriotism but just that for things that we could not grasp from the foreign author book there was a simpler version written by an Indian author, who understood the audience/students and put things in a layman fashion. Fortunately or unfortunately, they too are a part of the testing community and there should be articles tailored to their understanding levels.

I was able to recollect this concept and this did give rise to my optimism of continuing writing. Before you conclude anything, it is my responsibility to let you know that the above situation prevailing in India or might be in other countries is not because the foreign authors write it in a way that is too complicated for students here to understand but just that there are lot of factors that influences a student to refer to a particular book. One such is psychology, if a person has come across a book authored by some foreigner and due to his own naivety, was unable to grasp then he/she brands all foreign authors book as something written in Greek or Latin, despite the book being very simple to understand.

I also need to mention that the command over the language English is very important, to understand the simplicity of the written book/document.

Did I fail, by making you think that all Indian authors are the ones who abstract work from the ones who originally published the work ?

Not all, but some, yes. There are many genuine writers and I must appreciate their work in this context. Those who write the abstract versions of the original book mostly are the ones who write for commercial or fame, provided they do not give due credits to the original authors. ( As a tester, if I write such abstracted versions of original, I would be mentioning the limitation of my work in terms of the effectiveness in conveying the topic in detail or its usefulness)

It is time for me to make you think of "What can we write?" apart from one such I have mentioned in detail above -
  1. Extensions of research work of published articles.
  2. New experiments, its results, matching or defying with published articles.
  3. Testing, itself, is a game of perspectives, hence, each one's perspective.
  4. Applying the existing research to any non software field/domain.
  5. The problems that did not exist during the days the experts wrote articles and proposed solutions.
  6. Mistakes you have committed and the learning you have had from it.
  7. A new skill that a tester needs, which was not discussed earlier by any of the experts.
  8. Case studies of a project that you have been in, which you are authorized to write and publish.
  9. The kind of change you did to testing to suit new/different business needs.
  10. Things that have baffled you as a tester.
  11. Re-writing an article in native language, giving due credits to the original author.
  12. Lots more... The only limitation is your imagination - ( sorry, I do not know who quoted this)

_ End of _ Is there anything left for us to write in Testing? _

"What can be written, itself, has turned out to be a writing"

Thanks and Regards,

Pradeep Soundararajan

Tester Tested !

Note : In this post, I am representing those upcoming testers who are experimenting, trying to come out of naivety. Seniors excuse if it had not made sense to you.

Wednesday, August 02, 2006

Testing for Accessibility

This is an article from the August edition of the EuroSTAR Newsletter - STARTester, from Ruth Loebl. You can view the complete newsletter by clicking here and don't forget to subscribe to receive future issues.

Testing software for accessibility involves little more than imagination and common sense, but you have to pick the right standards, and then get to know some users.

• Lots of disabled people use computers, even people whom you might at first assume could not possibly use one.
• Many disabled people are not "disabled". My mother simply can't see as well as she used to, and has a bit of arthritis in her hands.
• Research commissioned by Microsoft indicated that in the United States, 60% (101.4 million) of adults from 18 to 64 years old "are likely or very likely to benefit from the use of accessible technology due to difficulties and impairments that may impact computer use.

"Even excluding people who are 65 or over, that's more than half the population – this isn't a niche market. So there ought to be a demand for accessible interfaces, although it's sometimes hard to detect. I'm encouraged by the improving legal situation" – check out the Code of Practice on the Disability Equality Duty in the Disability Discrimination Act 2005, example in para 3.46.

So how do all these disabled people use computers? For most, the answer is: in the same way that non-disabled people use computers, with a standard keyboard, mouse and screen.
As an example, one of the most important features of Windows is the ability to change the colour scheme and system fonts. While some of us just like a bit of variety in the colours we look at on the screen all day, for quite a few people choosing the right font and colour scheme is what enables them to read the screen at all. A few pre-set font and colour schemes are offered through the Accessibility Wizard (Programs, Accessories, Accessibility). More can be achieved through the Control Panel (Display, Appearance tab, Advanced button).

Accessibility testing should highlight when systems interfere with or disable these features that are provided through the operating system. It would be most annoying if your choice of colour scheme were ignored by a system that you have to use. All too often, oh dear, it is.

Access technology
Access technology is used where the effect of an impairment is such that an intermediary tool is needed to enable someone to use a computer.

• Partially sighted people who can't get by with an alternative font or colour scheme often use screen magnification software, which enlarges some or all of the screen contents, and provides other powerful features such as image smoothing and colour manipulation.
• People who have a reading impairment, poor literacy or dyslexia can use 'text-to-speech' software, where text highlighted with the mouse is spoken out loud. Sometimes, voice input is useful too.
• People who have a problem with their hands or arms will need adjustments or alternatives to the standard keyboard and mouse. This might simply be different hardware (a one-handed keyboard, a trackball, joystick or mouse pad) or a full speech recognition system.
• People who are blind use a standard keyboard but cannot operate a mouse. Speech output software conveys the contents of the screen, sometimes complemented by electronic braille output on hardware called a braille display. A full keyboard interface without reliance on the mouse is essential for effective access.
These access technologies are very powerful, but are often helpless when faced with really poorly designed software.

Software testing
As testers, you may want to know more about disability and access technology, and to meet some disabled people – I would certainly encourage this. For effective accessibility testing, it is important to involve as many different types of real users as possible, with different abilities, background, experience and so on. We often criticise software and web designers who don’t include people with disabilities in their testing processes.
Real user testing does demand a functional interface, but at this late stage it may be too late to change some aspects of the underlying design without compromising the viability of the whole software development project.

So as well as getting to know your users, we advocate the use of 'inclusive design' standards and guidelines at the earliest stages of interface design. The testable statements focus on the software itself, in isolation, independent of any particular user. They are intended to minimise barriers to both accessibility and usability, and to address many of the requirements of disabled users.

Standards and guidelines
Which standards and guidelines are most applicable for testing software? Three suggestions are below, and for our latest thinking on software and web accessibility, visit the RNIB technology site and follow the link to Software Accessibility. This information is due to be refreshed in early August.

ISO 9241-171 (formerly ISO/TS 16071)
The full title is "Ergonomics of human-system interaction – Guidance on software accessibility". It is in final draft, but as an internationally recognised standard written by professional standards-makers, we hope it will become a reference point alongside the guidelines and checklists that exist.
Within RNIB, we have adopted ISO 9241-171 as the basis for our software acceptance procedures, but it is actually too wide-ranging to implement in its entirety! For each software development project so far, we have had to extract a more workable subset of the full range of standards, tailored to the particular development and delivery platforms for that project.

IBM software accessibility checklist
The IBM Checklists are available for various technologies, including software in general as well as Web, Java and hardware. Each key point is explained clearly, and some information about implementation and testing is also given. They are much more usable and user-friendly than ISO, and free, but less comprehensive.

Section 508
Section 508 is US legislation to ensure that “electronic and information technologies” which Federal agencies develop, procure, maintain, or use, conform to standards designed to provide comparable access for people with and without disabilities. If you want to sell to Federal government in the US, these are the standards to apply. They don't include enough to make your system fully accessible, though.

Some messages to end with
• Accessibility is ultimately subjective, like usability. Effective testing depends on having a wide variety of users to test products in the later stages of the design process.
• Inclusive design is more objective, and applies to the software itself, independent of users. It can be tested from the earliest stages of a software design project.
• Inclusive design does not stifle creativity: good design for people with disabilities results in good design for all.

In my presentation at EuroSTAR 2006, you'll have the chance to see access technology in action, and some examples of accessibility standards and testing in the real world. See you there!

"Ruth Loebl has been with the Royal National Institute of the Blind (RNIB) for 13 years, working in the area of sight loss and technology."

Emotional Testing - Testing From the Heart of the Business

This is an article from the August edition of the EuroSTAR Newsletter - STARTester, from Ian Londesbrough. You can view the complete newsletter by clicking here and don't forget to subscribe to receive future issues.

In the last few years, testing and quality assurance have started to shake off the preconceptions of geekiness and started to carry more gravitas within organisations. Seemingly always the poor relation to development in the IS profession, testing is finally making it onto the agenda for board rooms, businesses and IS communities. But to claim that testing is "sexy" or "cutting edge" would be to overstate its appeal - if this is the case, how can the technology industry demonstrate its importance to the business decision makers?

Testing, like many other practices in the IT industry, has long been the preserve of logical, structured, “left brain” thinkers - the tech heads. In truth, many of the people who drive business are the creative forces, people who are more “right brain” in their thought processes. Therefore if testing is going to take up its rightful place in driving the development of projects that actually deliver the expected results, then something has to change – it has to be made more appealing to the creative forces within business.

Consider for a moment the difference between left brain and right brain thinking and the types of thought processes they engage in:

Left Brain
Looks at parts

Right Brain
Looks at wholes

There is no doubt that testing professionals need all the “left brain” attributes, but in addition to those, they need to move beyond the robotic, structured motions of traditional testing - both the left and the right side of the brain need to be utilised. This would enable testing professionals to start thinking about business from the more strategic and creative perspectives. It would also help them to gain a firmer foothold in the boardroom, allowing them to communicate the importance and criticality of testing to the right audience.

As we know, testing is required for a very diverse range of products and systems. For example, to test a console game which is designed to stimulate emotion requires both emotional and subjective decisions, as well as random and intuitive assessment of risk. These elements require a mental attitude different to that of the traditional testing professional.

The testing industry not only needs to adopt a “whole brain” approach when testing, it needs to utilise the right side in order to engage with the board and demonstrate the added value and business benefits of testing. Jargon such as “stakeholder buy-in”, “board level sponsorship”, “grass roots support”, is forever being bandied about, but how can the testing and technology industries really engage directors, business users, and staff and make a connection which delivers the value and benefit they expect?

To engage with any of their audiences, senior decision makers in particular, testing professionals need to have a firm understanding of the business requirements. The job of a professional tester is to fulfil the requirements of all stakeholders: from the client and their board through to partners and the tester’s own employer. By fulfilling the requirements, testing is not only seen in a favourable light, but it becomes a “must have” for any project.

In order to engage with their audiences effectively, professional testers need to be connected, from an emotional standpoint, to the business and understand on a cerebral level what the business is trying to achieve. When responding to situations the right side of the brain is the reactive, emotional side – think adrenaline rushes, increasing heart rates – the left side of the brain has a more considered, logical approach. Therefore testers can harness this cognitive process to ensure they immerse themselves in the business, the requirements and the risks and dangers it faces and then use their logical, analytical abilities to come up with testing and quality assurance strategies to meet the business’s requirements.

Traditionally, testers are not always vocal about their work and the positive benefits they are bringing to the business. Using this more emotional approach, throughout delivery of the project, testers need to constantly re-engage and maintain a positive relationship with their customers by communicating and demonstrating how testing has removed the risk from the project and produced success and profit.

The connection between an emotive approach and business and technology issues in a testing environment becomes clear when assessing the success that testing professionals achieve in delivering positive outcomes for the business. Testers who are trapped in the old school thought process of testing for testing’s sake and approaching it from a box-ticking, operational perspective, fail to engage effectively with the business and thus achieve a lesser degree of success. For example they may not understand the importance of capturing information that demonstrates the value testing is bringing to the business.

Using an emotive approach to testing will also enable professional testers to bring the discipline to life – using real life examples of projects that have failed due to shortfalls in the testing and quality assurance procedures will be much more effective than the traditional “death by powerpoint” approach.
If testers can engage the business and technology industry on an emotional level, about the added value and business benefits of testing, then testing will be able to move forward. Once the testing industry embraces the “whole brain” approach, they can assume a leadership role, guiding clients through projects safely eliminating the risk and enjoying much more success than you could possibly have from a traditional, logical and structured approach.

For all professional testers convinced of how critical their role is to the success or failure of IT projects, they need to engage the business at the ideas stage, drill down effectively to determine the requirements from the project and really get to the nub of what organisations and IS communities want. In doing this, disaster can be averted and businesses will start to realise the true benefits of successful projects.

Graduating from Warwick University in 1988, Ian started his career in IT as a Junior Programmer with Barclays Bank. Ian then moved to ICI, which became Zeneca and then Astra Zeneca. Ian has also worked for PA Consulting and prior to joining IS Integration he was the Testing & Release Manager for RWE Shared Services IS (serving npower and Thames Water).