The Captain of Your Special Teams……The Performance Test Lead!

This is an article from the September edition of the EuroSTAR Newsletter - STARTester, written by Scott Barber, Perfectplus Inc, USA. You can view the complete newsletter by clicking here and don't forget to subscribe to receive future issues.

You are familiar with the “Software Development as a Sports Team” analogy, right?
The project manager equates to the coach, lead developer to offensive team captain, test lead to the defensive team captain – where the entire team views the development process as collaborative and each member of the team is driven to produce his or her best work in order to achieve the team's common goal of delivering a “winning” application.

Typically, this is as far as the model goes, but it doesn't account for some important members of the team – the specialists.
There are a variety of specialists that may be a part of your team: security experts, network engineers, configuration managers and performance testers, to name a few.

If we look to American Football, we find a structure to enhance our model to accommodate these team members.

In American Football, there is a third group known as the special teams. The special teams consist of the kicking teams, kick return teams and other groups dedicated to special plays. Historically, coaches would populate these teams with non-starting players to keep the starters from getting excessively tired or injured during the game and so that the starters could remain focused on their primary positions during practice.

Recently, however, coaches have started fielding their best players, sometimes known as “game breakers,” on the special teams to improve their chances of winning games. These players have become more than just specialists; they have become expert generalists who can contribute to the game in a variety of roles and positions.

The captain of the special teams is often a senior player with both exceptional leadership skills and the ability to play a variety of positions on the field. These are the players coaches put in the game in critical situations when they feel the team needs a big play or a shift in momentum. They are the players that make the crowd cheer and inspire the rest of the team to redouble their efforts simply by taking the field. Much like the recent shift in football where coaches look to top players to populate the special teams, project managers have started looking for experienced, senior individuals who are expert specialists and established generalists for their special roles.

On a software development team, this unique individual equates to the performance test lead... minus the fanfare. On the most effective development teams I've ever been a part of, the performance test lead is someone with leadership abilities, strong generalist skills, and a unique and critical specialty.

So what makes the performance tester so unique? On top of their specialization as a performance tester, these individuals tend to be competent and have experience in a wide variety of roles enabling them to effectively contribute to virtually any aspect of the team. Let's take a brief look at all the different roles a performance tester assumes at various points during a project.

Business Analyst – Before performance testers can begin conducting effective tests, they must understand how users are going to interact with the system under test, what tasks they are going to be trying to accomplish, what their state of mind is likely to be while interacting with the system, and what their performance expectations are. Additionally, to establish relevant performance goals or requirements, the performance tester must also determine what the user's tolerances are and how competing applications are performing. Most performance testing literature implies that this information is simply available from the existing business analysts, but experience says that it is rarely available and when it is available it is poorly formed or simply wrong because very few business analysts have any training in this area.

Systems Analyst – Performance testing is not a black box activity. An effective performance testing strategy has to take into account not only the system as a whole but also the logical, physical, network and software architectures of the system both in test and in production. While this information is generally available, it rarely exists in a consolidated form, and as it turns out, it is often the case that the performance tester ends up being the single person on the team who understands the system from the greatest number of perspectives and has the best grasp on how all of these perspectives interact with one another.

Usability Analyst – When the application finally goes into production, there is really only one aspect of performance that matters: customer satisfaction. And the only way to determine customer satisfaction is to get the customer to use the system. The challenge in determining a customer's satisfaction with performance is that customers often know neither how to quantify performance nor how to distinguish between poor performance and an inefficient interface. Worse, very few organizations have dedicated usability teams, leaving the performance testers on their own to design and conduct these studies.

Test Strategist, Test Designer, Test Developer, Test Manager, Functional Tester, etc.
Typically, the team is just that, a team of people with individual roles and expertise who work together to effectively test the system. Most often, the performance test team is a team of one, so the performance tester has no choice but to be competent at all of the various test team roles. Since there is so little training available that is specific to performance testing, most practicing performance testers were initially trained in functional, systems or even unit testing and have since adapted those skills and techniques to performance testing. Frequently, performance testers were either systems or functional testers prior to becoming performance testers, or have served in those roles after becoming a performance tester.

Programmers – Developing performance tests is far from point and click or record and playback. In order to accurately simulate actual users, it is almost always necessary for performance testers to write elements of at least somewhat complex code. It is frequently necessary for performance testers to be able to read, understand, and interpret the developer's code, and, not infrequently they find themselves developing their own “test harness” simply to enable the possibility of load generation.
Performance testers often write their own utilities to help them parse through the huge volumes of data they collect, to generate test data, to reset their test environments, or to collect performance related metrics on remote machines. Performance testers may not always be senior programmers, but they certainly aren't afraid of code.

There are other roles performance testers play and reasons why the lead performance tester frequently turns out to be that game breaker who equates to the captain of your “software development special teams”, but I've come to the end of my allotted space. I guess you'll just have to attend my keynote at Eurostar to hear the rest of the story. I hope to see you there!