Oh, not again… How many times do I have to go through a basic process of application release and acceptance? Yet, here it comes again. Lack of understanding of software testing basics, misunderstanding of the roles severity and priority, bizarre views of the acceptance process. It seems like déjà vu all over again ;) And so far I’ve seen it with it with every small offshore s/w development shop that uses waterfall or similar type of process. So with no more ado let me just state these ABCs of the application release and acceptance…
To accept or not to accept the software should not be a philosophical question. As a matter of fact it should be as subjective as reasonably possible. To stop feelings and egos getting in our way we need to start with setting up a process of release and acceptance. For example, something as simple as that:
Release & Acceptance Process
- Offshore team delivers the software change package (SCP) to my secure FTP site. The SCP must include properly named and versioned build, configuration files, metadata, resources, and release notes.
- My release team takes the SCP and installs it on User Acceptance Testing environment. If installation is successful release team notifies offshore team. Project Manager and QA/UAT team. If install fails the release team notifies offshore team and Project Manager.
- QA/UAT team performs testing according to test plan / strategy that is based on the nature of changes included in SCP. If the system passes Acceptance Criteria QA/UAT team notifies offshore team, PM, Release Team, and Product Manager and clears the SCP to be promoted to staging/production. If the system fails to passe Acceptance Criteria QA/UAT team notifies offshore team, PM, and requests a replacement SCP.
What is the key here is to make sure that Acceptance criteria are clearly defined and communicated across teams and organizations.
Here is an example, SCP is considered to pass acceptance criteria if
- No S1 or S2 issues are discovered AND,
- No S3-4/P1-2 issues are discovered AND,
- No more than 10 S3/P3-P5 issues are discovered AND,
- No more than 20 S4/P3 issues are discovered AND,
- No more than 20 S5/P1-2 issues are discovered.
Now it’s time to look at those illusive concepts: Severity and Priority.
Software Defect Severity Definition
Defect Severity is a classification of software defect (bug) that indicates the degree of impact on the customer and/or the quality of software. Defect severity is typically set by QA engineer after the defect is confirmed.
For example, Severity Types, in application to Production, could be defined as follows:
|S1||Critical||The defect affects critical functionality or critical data. It does not have a workaround. Example: Unsuccessful installation, complete failure of a feature.|
|S2||Major||The defect affects major functionality or major data. It has a workaround but is not obvious and is difficult. Example: A feature is not functional from one module but the task is doable if 10 complicated indirect steps are followed in another modules.|
|S3||Minor||The defect affects minor functionality or non-critical data. It has an easy workaround. Example: A minor feature that is not functional in one module but the same task is easily doable from another module.|
|S4||Trivial||The defect does not affect functionality or data. It does not even need a workaround. It does not impact productivity or efficiency. It is merely an inconvenience. Example: Petty layout discrepancies, spelling/grammatical errors.|
|S5||RFE||Reserved for Request For Enhancement (RFE) and other Feature Requests.|
Software Defect / RFE Priority Definition
Defect / RFE Severity is a classification of software defect (bug) or Request For Enhancement (RFE) that indicates the degree of the importance or urgency of fixing a defect or implementing the RFE. Though priority may be initially set by the QA, it is usually finalized by the Project/Product Manager.
For example, Priority Types, in application to Production, could be defined as follows:
|P1||Crucial||Must be addressed asap. Drop everything to get it done.||Hot fix or Next Patch|
|P2||Urgent||Should be addressed swiftly, can wait a couple days.||Hot fix or Next Patch or Minor / Maintenance Release|
|P3||High||Should be addressed in reasonable timely manner, can wait a few weeks.||Next or the following Minor / Maintenance Release.|
|P4||Medium||Not urgent but should be addressed in reasonable time, can wait a few months.||Next Major Release.|
|P5||Low||May or may not be addressed.||No commitment|
And just in case since we are talking about it let’s add example of definition of software releases.
Software Release Definition
Release Types are defined as follows:
OK, that good enough for today…