Below is an article from Thorkil Sonne telling the amazing story of Specialisterne, his company and how he hopes to reverse the terms of normality. It's a great story & I hope you enjoy it! The article was published in the first edition of the New EuroSTAR newsletter.
Clcik here to view the entire EuroSTAR newsltter and to subscribe to future monthly editions.
A fragile world
6 years ago my youngest son was diagnosed with autism (Autism Spectrum Disorder). I was given a chance to get a glimpse of the fragile world of autism.
My son frequently impresses me with his skills derived from strong memory, attention to detail, motivation for tasks, correctness in communication and learning ability.
But he also worries me, as he does not comply with the social requirements in society. As an IT-professional with 15 years background, I can see that his skills are potentially valuable to the corporate sector. After 3 years as president of a local branch of Autism Denmark, I can see that my son will only have a very small chance to use his special skills in a job situation.
Someone has to make a difference - so I quit my job as CTO to establish a company for persons with valuable skills and who are non-compliant with the social requirements of most companies.
Specialisterne
To get a job in most ‘normal’ companies, you have to comply with social requirements like: flexibility, social skills, team player, humour with flair for irony and a high stress threshold. If you do not match the social requirements – then you are left out.
At Specialisterne we reverse the terms of ‘normality’.
We define ‘normality’ as ‘whatever the majority decides it to be’. In our company the employees are the majority and thereby set the standards of ‘normality’.
We expect our candidates to stick to planning, schedules and agreements, require special working conditions, are strong individuals, have attention to detail and concentrate on doing what they are good at.
By being accepted as ‘normal’ at Specialisterne our employees build up their self esteem and put their full efforts into providing unparalleled quality in testing for corporate customers.
The customers
The customers of Specialisterne are small, large and global companies with high test standards. We perform any kind of test, where domain knowledge is not a requirement.
For different customers we perform tasks like:
* test management
* establish or improve test documentation
* static, dynamic, beta and system tests
Our customers will have ISEB test certified project managers as single point of contact to make sure, that it is easy for the customer to get access to the special skills of our employees with no risk to the customer.
Our customers are positively surprised by the skills of our employees and present us with high scores on the satisfaction evaluation report and we frequently see comments like: “The Specialisterne employees have impressed us with their ability to maintain attention to detail throughout the entire test, giving us a thorough result.”
Perspectives
The Specialisterne concept is a success in Denmark where so far 27 employees with the mild form of autism, Aspergers Syndrome, have found a niche to perform tests for corporate customers.
The needs for motivated testers are the same internationally – and so are the needs for alternatives for persons who do not fit into standards of ‘normality’.
I intend to reuse the experiences gained at Specialisterne to spread the concept internationally and build bridges between the corporate sector and the potential of specialist resources worldwide.
The roll out will take place where Specialisterne can join efforts with large and global companies to benefit from adding a new brick to the puzzle of setting up Test Dream Teams.
EuroSTAR 2006
At EuroSTAR 2005 I had the privilege to present the paper ‘Adding Autism Competencies to Testing’. I told about our vision, skills and experiences so far. The presentation was awarded ‘Best Presentation’.
I am very happy for the invitation to do a key-note presentation at EuroSTAR 2006.
I will present to you the Specialisterne concept, experiences, and references – and discuss the perspectives of how you can benefit from the experiences gained at Specialisterne.
Best regards
Thorkil Sonne
Founder and CEO
thso@specialisterne.dk
www.specialisterne.dk/english
Tuesday, April 18, 2006
It’s About People!
This is an article from Jens Pas, the EuroSTAR programme chair for 2006 which was published in the first edition of the EuroSTAR Newsletter. It is entitled 'It's about people' and pays special attention to the importance of people issues within the testing profession.
You can view the entire EuroSTAR newsletter by clicking here
My biggest surprise when reading all the submissions was that so many people really sent in proposals very closely related the theme of the conference “Assembling the Dream Team”. Many of you have a lot of experience to share when we talk about people issues. Thank you for providing us with all this real life experience. You allow us to offer our delegates a very interesting and, most important, “human” conference.
Aside from the submissions, I also experienced what people can be and can do. As I already said, 20 reviewers and a Program Committee have voluntarily processed the submissions. In addition, many have provided us with great ideas we might want to incorporate into the conference to make it even more interesting, fun and innovative. Combine the lively EuroSTAR-community with the professional team of Qualtech, who manage this event, and you have a highly energizing cocktail. Everybody is very motivated to make this conference ‘the’ event of the year for the European Testing Community. It is this testing community that has given me the energy to try to distill a program that, I hope, you will appreciate. It’s a luxury to be able to work under these pleasant conditions.
If we look more closely at the topics people are proposing, you’ll notice a lot is about communication and the ‘issues’ this generates. It seems that, as opposed to the clear and well defined TCP/IP standards computers use to communicate, we, humans have a lot of trouble in understanding each other. When a computer has a communication hick-up, such as packet-collision, it simply sends the information again. When humans collide…well, they collide… and that’s about it. Communication stops.
On top of that, we seem to live in different worlds. IT professionals have their language, business people use another. Developers even have a different dialect it seems, compared to testers. So, we not only have the challenge of managing a communication process, we have compatibility issues.
I guess everyone has experienced what I describe above. This is confirmed by the vast amount of testimonials that came in with the presentation proposals. As I read through all these papers I saw that many people have found their own way of dealing with this challenge. Some use discussion techniques, such as the Six Thinking Hats of De Bono. We will address the use of the Hats for testing at the conference. Others perform a team analysis to improve communication and co-operation. And again others discuss the infrastructure that we need to be able to optimally work together. They propose frameworks for value setting of the team allowing organisations to assemble a team that has a compatible set of working values. There are even standards and certificates which can be deployed in your organisation to improve the interaction between people. All this will be discussed in Manchester.
Each of these solutions contributes to making a group of people a team that generate value. 1+ 1 = 3 as they say.
In the next edition of this newsletter, I look forward to being able to announce the official conference programme. Names will appear and I hope you’ll like them. After all, it’s all about people.
Let me conclude this first article by repeating a metaphor I used in Copenhagen, last year, when I invited you to come to Manchester:
Take two glasses of water, half filled. Pour the water of one glass into the other glass. You now have one full glass of water. The beauty is that you cannot see anymore which water came from which glass. They are stirred, not shaken, into a transparent homogeneous drink. Wouldn’t it be great if our software team would be like this glass of water, a Dream Team assembling great software.
I look forward meeting you in Manchester.
Best Regards
Jens
Read the entire EuroSTAR Newsletter by clicking here
You can view the entire EuroSTAR newsletter by clicking here
My biggest surprise when reading all the submissions was that so many people really sent in proposals very closely related the theme of the conference “Assembling the Dream Team”. Many of you have a lot of experience to share when we talk about people issues. Thank you for providing us with all this real life experience. You allow us to offer our delegates a very interesting and, most important, “human” conference.
Aside from the submissions, I also experienced what people can be and can do. As I already said, 20 reviewers and a Program Committee have voluntarily processed the submissions. In addition, many have provided us with great ideas we might want to incorporate into the conference to make it even more interesting, fun and innovative. Combine the lively EuroSTAR-community with the professional team of Qualtech, who manage this event, and you have a highly energizing cocktail. Everybody is very motivated to make this conference ‘the’ event of the year for the European Testing Community. It is this testing community that has given me the energy to try to distill a program that, I hope, you will appreciate. It’s a luxury to be able to work under these pleasant conditions.
If we look more closely at the topics people are proposing, you’ll notice a lot is about communication and the ‘issues’ this generates. It seems that, as opposed to the clear and well defined TCP/IP standards computers use to communicate, we, humans have a lot of trouble in understanding each other. When a computer has a communication hick-up, such as packet-collision, it simply sends the information again. When humans collide…well, they collide… and that’s about it. Communication stops.
On top of that, we seem to live in different worlds. IT professionals have their language, business people use another. Developers even have a different dialect it seems, compared to testers. So, we not only have the challenge of managing a communication process, we have compatibility issues.
I guess everyone has experienced what I describe above. This is confirmed by the vast amount of testimonials that came in with the presentation proposals. As I read through all these papers I saw that many people have found their own way of dealing with this challenge. Some use discussion techniques, such as the Six Thinking Hats of De Bono. We will address the use of the Hats for testing at the conference. Others perform a team analysis to improve communication and co-operation. And again others discuss the infrastructure that we need to be able to optimally work together. They propose frameworks for value setting of the team allowing organisations to assemble a team that has a compatible set of working values. There are even standards and certificates which can be deployed in your organisation to improve the interaction between people. All this will be discussed in Manchester.
Each of these solutions contributes to making a group of people a team that generate value. 1+ 1 = 3 as they say.
In the next edition of this newsletter, I look forward to being able to announce the official conference programme. Names will appear and I hope you’ll like them. After all, it’s all about people.
Let me conclude this first article by repeating a metaphor I used in Copenhagen, last year, when I invited you to come to Manchester:
Take two glasses of water, half filled. Pour the water of one glass into the other glass. You now have one full glass of water. The beauty is that you cannot see anymore which water came from which glass. They are stirred, not shaken, into a transparent homogeneous drink. Wouldn’t it be great if our software team would be like this glass of water, a Dream Team assembling great software.
I look forward meeting you in Manchester.
Best Regards
Jens
Read the entire EuroSTAR Newsletter by clicking here
Ready to Ship? – Using Release Readiness Metrics
This is an article from John Fodeh which was published in the first edition of the new EuroSTAR newsletter. This article is an excerpt from John’s 2005 Tutorial in Copenhagen entitled "Establishing an Effective Test Metrics Programme". This tutorial sold out in advance of the conference.
To view this article and all other articles, view the EuroSTAR newsletter here
Why Measure?
A metric is a measure. "Measurement is the process by which numbers or symbols are assigned to attributes of entities in the real world in such a way to describe them according to clearly defined rules" (Fenton, Pfleeger). As you know, metrics play a crucial role in the advancement of all sciences. This should also include computer science. Tom DeMarco stated, "You cannot control what you cannot measure". In software development and testing we use metrics to:
* Understand: Gain insight into the applied processes and identify relationships. We use measurements to establish a baseline and goals for future assessment.
* Assess: Determine status with respect to plans and goals, and then monitor progress.
* Predict: Detect bottlenecks, early warnings of problems and determine what tradeoffs can be made.
* Improve: Identify root causes and learn from historical data.
* Communicate: Visualize status and trends, and share the knowledge with the rest of the team.
What Metrics Are Needed
Some organisations select a single or a few isolated "golden" metrics to collect and monitor. However, most metrics are not useful when used separately. For example, the number of defects found during testing has limited value until it is combined with other data such as the severity, type and root cause of the defects, the number of defects fixed, the type and effort of testing, the number of defects found by the end-users, etc.
On the other hand, having too many metrics to collect, analyse and report can be infeasible and problematic. There are numerous aspects in software development and testing that can be measured and it can be tempting to do so. However, spamming your team with tons of metrics is certainly not the answer (and will probably not be appreciated as well).
Instead of spending your time on a multitude of metrics that you or someone else might find useful sometime in the future, your time is much better spent doing more testing. The solution is a suite of key metrics. This set of metrics should be simple and balanced and help you track progress toward your target, i.e. release readiness.In this respect, applying a metric paradigm such as Goal/Question/Metric (GQM) can be very effective. GQM provides a formalised framework for developing a metrics programme, prompting you to think; "what information do I need?" instead of "what is easy to measure?" In other words, to derive your set of metrics you need to know what you need to know.
The following describes some basic metrics that you can use and include in your set of release readiness metrics.
Test progress
Test progress metrics include information concerning the status with respect to the plans, e.g. how many of the planned tests have been run? What is the pass rate of the tests? Throughout development this information will show the current status and help detect bottlenecks and early warnings of trouble, e.g. if we have only completed a fraction of the planned tests and the pass rate is low, is it still possible to finish on time with the available resources? When approaching release this metric will speak for itself; can we release the system with the current test completion status?
Obtained coverage
Coverage metrics hold vital information regarding the thoroughness and completeness of the testing. Coverage can be expressed in terms of requirements, code or risks and provide the means for quantifying the portion of the requirements/code/risks that is exercised by the applied testing. A low value reveals an insufficient test effort and the risk of potential latent defects. This metric is typically used in conjunction with the test progress metrics and can reveal details not seen from the progress data. An example could be the situations where many tests have been completed while critical requirements or high risk areas remain untested.
System quality factors
Data about system quality factors contain information on different aspects of the product, such as functionality, performance, reliability, installability and conformance to standards (you might also consider using ISO 9126). I prefer to show the test completeness of these quality factors combined with information about the defect density. In this way I can monitor progress in the different areas and detect if some deserve special attention.
Found defects
Defect metrics are usually collected by applying the appropriate queries in the defect reports database. The metrics typically include the total number of defects reported and categorised in open and closed (fixed and verified) defects, sorted according to their severity.
This data delivers a snap shot of the system state at release time, making it possible to take into account the risk and consequence of releasing the system e.g. If the data reveals a large number of open high-severity or non-verified defects, then it clearly shows that releasing the system at this moment is a high-risk decision. By depicting the found defects over time (or test effort) it is possible to create a defect trend. A defect trend is a graph showing the accumulated number of reported defects as a function of test effort (expressed in terms of test days).
This graph is typically S-shaped. When testing is started, the defect-finding rate is low, as the functionality of the software is restricted to few areas and because showstoppers might prevent testing in some areas. The defect-finding rate increases with the addition of new functionality and the correction of already found defects. As the software matures, the defect-finding rate starts decreasing, as it becomes harder to find new defects. Ultimately, the graph flattens. Finding further defects at this stage requires a huge test effort and shows that the software is possibly ready for release (or that the limitation of the applied testing technique has been reached).
Monitoring the status on the S-curve helps to determine when to stop testing, i.e. is the curve starting to flatten?
It is even possible to extrapolate the graph, providing a predictive evolution of the defect-finding rate and a means of estimating the number of unknown defects.
Customer feedback
User feedback during development is of major importance. During testing we normally verify that the system performs as specified, i.e. "are we building the system right?" The user evaluation data helps you answer the question: "are we building the right system?"
Summary
Successful test management involves making complex decisions. These decisions need to be based on solid, quantitative data and well-calculated risk. Using a simple set of metrics you can get a snapshot of the system state and quality that is useful throughout the entire development process. However, in the closing stages, possessing the right metrics has a tremendous value, in particular when you need to find the proper timing for release.
To view the EuroSTAR Newletter, click here
To view this article and all other articles, view the EuroSTAR newsletter here
Why Measure?
A metric is a measure. "Measurement is the process by which numbers or symbols are assigned to attributes of entities in the real world in such a way to describe them according to clearly defined rules" (Fenton, Pfleeger). As you know, metrics play a crucial role in the advancement of all sciences. This should also include computer science. Tom DeMarco stated, "You cannot control what you cannot measure". In software development and testing we use metrics to:
* Understand: Gain insight into the applied processes and identify relationships. We use measurements to establish a baseline and goals for future assessment.
* Assess: Determine status with respect to plans and goals, and then monitor progress.
* Predict: Detect bottlenecks, early warnings of problems and determine what tradeoffs can be made.
* Improve: Identify root causes and learn from historical data.
* Communicate: Visualize status and trends, and share the knowledge with the rest of the team.
What Metrics Are Needed
Some organisations select a single or a few isolated "golden" metrics to collect and monitor. However, most metrics are not useful when used separately. For example, the number of defects found during testing has limited value until it is combined with other data such as the severity, type and root cause of the defects, the number of defects fixed, the type and effort of testing, the number of defects found by the end-users, etc.
On the other hand, having too many metrics to collect, analyse and report can be infeasible and problematic. There are numerous aspects in software development and testing that can be measured and it can be tempting to do so. However, spamming your team with tons of metrics is certainly not the answer (and will probably not be appreciated as well).
Instead of spending your time on a multitude of metrics that you or someone else might find useful sometime in the future, your time is much better spent doing more testing. The solution is a suite of key metrics. This set of metrics should be simple and balanced and help you track progress toward your target, i.e. release readiness.In this respect, applying a metric paradigm such as Goal/Question/Metric (GQM) can be very effective. GQM provides a formalised framework for developing a metrics programme, prompting you to think; "what information do I need?" instead of "what is easy to measure?" In other words, to derive your set of metrics you need to know what you need to know.
The following describes some basic metrics that you can use and include in your set of release readiness metrics.
Test progress
Test progress metrics include information concerning the status with respect to the plans, e.g. how many of the planned tests have been run? What is the pass rate of the tests? Throughout development this information will show the current status and help detect bottlenecks and early warnings of trouble, e.g. if we have only completed a fraction of the planned tests and the pass rate is low, is it still possible to finish on time with the available resources? When approaching release this metric will speak for itself; can we release the system with the current test completion status?
Obtained coverage
Coverage metrics hold vital information regarding the thoroughness and completeness of the testing. Coverage can be expressed in terms of requirements, code or risks and provide the means for quantifying the portion of the requirements/code/risks that is exercised by the applied testing. A low value reveals an insufficient test effort and the risk of potential latent defects. This metric is typically used in conjunction with the test progress metrics and can reveal details not seen from the progress data. An example could be the situations where many tests have been completed while critical requirements or high risk areas remain untested.
System quality factors
Data about system quality factors contain information on different aspects of the product, such as functionality, performance, reliability, installability and conformance to standards (you might also consider using ISO 9126). I prefer to show the test completeness of these quality factors combined with information about the defect density. In this way I can monitor progress in the different areas and detect if some deserve special attention.
Found defects
Defect metrics are usually collected by applying the appropriate queries in the defect reports database. The metrics typically include the total number of defects reported and categorised in open and closed (fixed and verified) defects, sorted according to their severity.
This data delivers a snap shot of the system state at release time, making it possible to take into account the risk and consequence of releasing the system e.g. If the data reveals a large number of open high-severity or non-verified defects, then it clearly shows that releasing the system at this moment is a high-risk decision. By depicting the found defects over time (or test effort) it is possible to create a defect trend. A defect trend is a graph showing the accumulated number of reported defects as a function of test effort (expressed in terms of test days).
This graph is typically S-shaped. When testing is started, the defect-finding rate is low, as the functionality of the software is restricted to few areas and because showstoppers might prevent testing in some areas. The defect-finding rate increases with the addition of new functionality and the correction of already found defects. As the software matures, the defect-finding rate starts decreasing, as it becomes harder to find new defects. Ultimately, the graph flattens. Finding further defects at this stage requires a huge test effort and shows that the software is possibly ready for release (or that the limitation of the applied testing technique has been reached).
Monitoring the status on the S-curve helps to determine when to stop testing, i.e. is the curve starting to flatten?
It is even possible to extrapolate the graph, providing a predictive evolution of the defect-finding rate and a means of estimating the number of unknown defects.
Customer feedback
User feedback during development is of major importance. During testing we normally verify that the system performs as specified, i.e. "are we building the system right?" The user evaluation data helps you answer the question: "are we building the right system?"
Summary
Successful test management involves making complex decisions. These decisions need to be based on solid, quantitative data and well-calculated risk. Using a simple set of metrics you can get a snapshot of the system state and quality that is useful throughout the entire development process. However, in the closing stages, possessing the right metrics has a tremendous value, in particular when you need to find the proper timing for release.
To view the EuroSTAR Newletter, click here
Tuesday, April 11, 2006
The EuroSTAR Newsletter
As you may know, EuroSTAR have begun to send a monthly newsletter to tester's throughout Europe. This all new Software Test focused newsletter will contain a variety of articles from some of the industry's leading visionaries in Software Testing.
This months's articles include "Ready to Ship? - Using Release Readiness Metrics" from John Fodeh, and "Specially Motivated Test Resource" from Thorkil Sonne, winner of the best presentation award at EuroSTAR 2005 who tells a remarkable story. There are also articles from the EuroSTAR 2006 programme chair, Jens Pas "It's all about people" and further information on the Free EuroSTAR mini events taking place in the UK & Holland in early May.
To receive the 1st edition of the EuroSTAR newsletter and further monthly editions, Sign up to the EuroSTAR Newsletter
This months's articles include "Ready to Ship? - Using Release Readiness Metrics" from John Fodeh, and "Specially Motivated Test Resource" from Thorkil Sonne, winner of the best presentation award at EuroSTAR 2005 who tells a remarkable story. There are also articles from the EuroSTAR 2006 programme chair, Jens Pas "It's all about people" and further information on the Free EuroSTAR mini events taking place in the UK & Holland in early May.
To receive the 1st edition of the EuroSTAR newsletter and further monthly editions, Sign up to the EuroSTAR Newsletter
Monday, April 10, 2006
Testing "Big Picture"?
Hi all!
First let me introduce myself: my name is Erkki Pöyhönen (or Poyhonen for those with keyboards without funny dotted characters :-) and I come from
I really enjoyed some of the posts in here, especially the one about the maturity. For most of the past decade I've been concerned about testing competence. That's why I'd like to share my recent quest to the testing big picture.
In large companies test teams or units grow an internal testing culture. It is a natural way to manage complexity: based on our experiences we learn self-evident ideas, what is important, what is possible, and so on. This means that after a while two organisations in different contexts (differing business situations, working with different technologies or producing different products) have developed different sets of axioms -- "self-evidents".
I suppose this really is natural and good for individuals and teams in general. The hard part comes from having to co-operate with other organisations or adapting to the changing conditions. If testers are not aware of their culture or their context, they might totally devaluate everything coming from a person from different organisation. This can be seen in attitudes into literature, training or cross-organisation co-operation; it is easy to dismiss quite valid outside information saying simply "that was not relevant for us; it is made for another organisations; we are special". BTW: good, welded teams share that elite feeling, so feeling special is not bad as such. It becomes a problem only when it prevents us to learn from others for being different in a vague way.
As discussed not so long ago in software-testing mailing list (a wonderful email list of the context-driven testing school with rather high S/N ratio; somewhat slanted to agile thinking) it might be more useful to have 5 times one year experience of different organisations than having a 5-year experience working in similar projects. I agree. And I'd add that working for both product and project organisations is useful, and working both in industrial setting and high-tech IT industry.
Working with one product year, year out (like in Telecoms used to be quite common) allows both testers and developers to optimise their approach and practices for efficiency. But for example changing into another technology can be a disaster. Or a test team might work parallel in tens of different projects for many customers in varied businesses. Of course the focus would be more into adaptability and learning, but achieving repeatable practices or sustained organisational improvement might be harder.
My newest project is building a repository of generic testing information that is useful for induction, competence development and process improvement. Building such a beast for any certain context is simpler: find the relevant sources and then assume this is good for us to know. Trying to be useful for a wider audience means: identifying context behind all sources and material, highlighting the assumed context-dependencies and sourcing the generic ideas from context-specific material.
Suddenly many common words are not so common any longer. Example: a test plan is for many "a reviewed text document, based on a template and that nobody reads". For others it can be a wall in the team meeting room. Or if the main constraint is delivery date it can be very unproductive to assume that quality is the driving factor.
Good news is that we need not fight over certain testing terms with religious intensity, if we identify we come from different contexts. It becomes more important to learn the "one step up" definition - what does this thing really mean, and not focus on how does it look like in different organisations, or "what do we ideologically hope this should mean for everybody".
It is easier to look for material supporting my current view than take the small extra effort to learn from sources of other contexts. But sticking to a strict "one testing" viewpoint takes energy, is counter-productive and prevents us from developing our industry forward. Widening my horizons and listening to (and getting to know) people from different industries has exactly been the biggest pay-back for me from attending testing conferences. And where else to do this better than in EuroSTAR again next December?
Subscribe to:
Posts (Atom)
