business strategy
DESCRIPTION
TRANSCRIPT
![Page 1: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/1.jpg)
1
Tsuneo Yamaura
Hitachi Software Engineering Co., Ltd.
T-Shirts vs. Kimono Approachin
Software Quality Assurance
Jan. 16, 2006
AOSS Training@Bankgkok
![Page 2: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/2.jpg)
2
Some QA Statistics at Hitachi Software:
Percentage of Bug Detection in Software Phases code unit subsystem system inspection after releaseinspection debugging debugging debugging by QA dept. (at user site)
21.5% 35.1% 28.0% 6.1% 9.3%
Only 2 bugs out of 10,000 emerge at customer’s site.
10 engineers work for 12 months to make 100KLOC and 1,000 bugs.
0.2 bugs emerge at customers ’ sites
0.02%
![Page 3: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/3.jpg)
3
What Do We Do for Quality Assurance ?
(1) Developers design, write test cases, and run test case.
(2) Statistical bug data is collected throughout product life cycle.
(3) A project manager analyzes the data to control the project.
(4) Independent QA engineers test products.
There is no magic. We simply employ traditional methods.
![Page 4: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/4.jpg)
4
(1) Developers design, write, and run test cases
a. When developers have completed coding, they start to design
test cases.
b. Test cases are documented. Advantages of the written test
cases are:
- Designing test cases requires that the specification be examined
from different vew points. Many critical bugs can be detected.
- You can re-execute the same test cases.
- You can « test » the quakity of the test cases.
- The execution of the test cases can be passed on to another
person, perhaps somebody who was added to the project to
recover a delay and does not understand the program very well.
c. Automatic test execution (test suites) is desired.
![Page 5: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/5.jpg)
5
(1) Developers design, write, and run test cases (cont.)
Test case design guideline
a. approximately 1 test case per 5 - 15 LOC
Language processor: 1 test case per 8-12LOC
Online system : 1 test case per 5-10 LOC
Batch system : 1 test case per 10-15 LOC
b. normal cases : 60% or less,
abnormal cases : 15% or more,
boundary cases : 10% or more,
enviromental cases: 15% or more
![Page 6: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/6.jpg)
6
(1) Developers design, write, and run test cases (cont.) Example of matrix-based test cases
test case ID AD001 AD002 AD003AD004 AD005 AD006 AD007 AD008 ZZ X
-1 X0 X
Age 6 X12 X18 X19 X
999 X$0 X X
Admission $5 X Fee $8 X
$10 XErr msg X X X
Category abnrm bound norm norm norm norm norm abnrmCode inspection 5-May 5-May 5-May 5-May 5-May 5-May 5-May 6-MayMachine test 1- J un 1- J un 2- J un 2- J un 2- J un 2- J un 3- J un 3- J un
admission fee: $0: age 6 or under, $5: age 7 - 12, $8: age 13 - 18, $10: age 19 and above
![Page 7: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/7.jpg)
7
From when ?
All the compilation errors are eliminated (from the code
inspection phase).
What must be reported ?
a. when detected
ID number, symptom of the bug, who caught the bug,
when, seriousness, etc.
(2) Statistical bug data collection throughout the life cycle
Developers must report all the bugs (Bug Report)
b. when fixed
Modules in which the bug resides, explanation of the bug,
who fixed, when fixed and what was tested by whom, code
before and after.
![Page 8: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/8.jpg)
8
(3) Bug Data Analysis by Project Manager
- A project manager drows the « test progress plans» before
getting into an actual testing.
numb. of untested cases (target)
cumulative bugs expected
time
numb. oftest cases
Dec/30 Jan/5 Jan/12 Jan/19 Jan/26 Feb/2 Feb/9
![Page 9: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/9.jpg)
9
(3) Bug Data Analysis by Project Manager
The Project Manager plots the test progress on the chart:
- to determine the best strategies to improve the quality
- to predict the release date.
numb. of untested cases (target)
numb. of untested cases (actual)
cumulative bugs found
cumulative bugs expected
time
numb. oftest cases
Dec/30 Jan/5 Jan/12 Jan/19 Jan/26 Feb/2 Feb/9
![Page 10: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/10.jpg)
10
(3) Bug Data Analysis by Project Manager (continued)
remaining test cases
expectedactual
bugs found
bugs expected
best case
remaining test cases
expected actual
bugs found
bugs expected
low quality in previous process
remaining test cases
expectedactual
bugs found
bugs expected
inexperienced testers
remaining test cases
expected
actual
bugs found
bugs expected
low test case quality
![Page 11: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/11.jpg)
11
(3) Bug Data Analysis by Project Manager
numb. of untested cases (target)
numb. of untested cases (actual)
Generation of bugs gets flat
All the test cases are executed.
time
numb. oftest cases
Dec/30 Jan/5 Jan/12 Jan/19 Jan/26 Feb/2 Feb/9
Project Manager thinks it’s time release when:
- All the test cases are executed.
- The growth of bugs gets flat.
- All the bugs are fixed.
- 48-hour continuous operation successfully completed.
![Page 12: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/12.jpg)
12
(4) Independent QA engineers test products
We have QA departments which are cpmpletely independentfrom the development departments.
Q1: How many engineers are assigned to the QA departments ?A1: Approximately 8% of the entire engineers (330 out of 4,000).
Q2: What do they test ?A2: All the developers’ output (requirement specifications,
design documents, programs, test cases, etc).
![Page 13: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/13.jpg)
13
Q4: How do they test ?
A4: From the customer’s vew point. They design, and
write test cases, which are completely independent
from those created by developers.
(4) Independent QA engineers test products (continued)
Q3: How powerful are they ?A3: A product cannot be released without their ‘go-ahead.’
![Page 14: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/14.jpg)
14
The cost of the quality (There is no free lunch !)
Assumption : A 12-month project with 10 developers works on
C-based software with 100,000 LOC.
Months for each phase
- requirement specification : 2 months
- designing : 3 months
- coding : 2 months
- debugging : 3 months
- testing (by QA engineers) : 2 months
Details of the 3-month debugging phase
-100,000 LOC software needs 10,000 test cases (1,000 per developer).
-10 days to design 10,000 test cases (100 a day per developer)
-10 days to check 1,000 test cases in code inspection (10 cs/day-develpr)
-40 days to check 10,000 test cases in machine debugging (25cs/day-dvlp)
![Page 15: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/15.jpg)
15
Q: How did we attain such a high goal (99.98% bug-free) ?
Are we better than others ?
A: Hopefully yes, but I am not sure.
The main reasons are:
- We have not changed the QA methods for more than
30 years.
- Our turnover rate is quite low (an average project
manager, for example, has worked for the company
for more than 15 years).
- Our review process works pretty well.
![Page 16: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/16.jpg)
16
Applicability of Our QA Method to Development Models
[1] Waterfall Model
Our QA approach fits very well, because:
- The development lifecycle spans several months (or years),
- The start and end of each development phase are clear,
- Each development phase goes sequentially.
[2] Object Oriented
We are in the researching stage. We may need some
modification, but perhaps existing method can be used.
[3] Spiral Model
If the development lifecycle is very quick (for example, a
product comes out in every week), our QA approach may
not be suitable.
![Page 17: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/17.jpg)
17
Q: Do you think our QA approach is applicable to the programmers in the US and India ? A: Of course NOT !
![Page 18: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/18.jpg)
18
Conflictions in the Software Development Priority
USA ans India vs. Japan = marketability vs. quality
American/Indian type Japan type
It does not mean a thing if it does not sell well.
Quality is it.
differentreligions
![Page 19: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/19.jpg)
19
magnitude
release timing
impact to market
quality maturity
USA/India Japan
Differences in sales strategies (Japan before 2000)
cost < qulity < release cost < release < qulity
![Page 20: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/20.jpg)
20
Differences in Software Industries in Japan and the US
1. Success as a team is important.
- Honest data can be easily
collected.
- Statistics-based QA is
applicable.
1. Personal achievement is more important than a team success.
- “Making bugs” = “I have failed.”
- The bug data is unintentionally
cooked.
Software Industries in Japan Software Industries in the US
![Page 21: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/21.jpg)
21
Differences in Software Industries in Japan and the US (cont.)
2. Quality is more important than cost and release schedule to avoid huge bug fixing cost.
2. Dominating the market by early release is more important than quality.
Software Industries in Japan Software Industries in the US
![Page 22: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/22.jpg)
22
Differences in Software Industries in Japan and the US (cont.)
3. Looks for “industrialization” in
software.
- Key word = “Play safe”
- Tends to plan riskless products
- Follows the technologies
invented in the US / India.
3. Seeks “novelty” and “technology in
the next generation.”
- Key word = “Challenge”
- Goes for ambitious products
- Delivers epoch-making
technologies.
Software Industries in Japan Software Industries in the US
![Page 23: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/23.jpg)
23
Differences in sales strategies (What Japan faces after 2000: Part 1)
magnitude
release timing
impact to market
quality maturity
USA/India Japancost < qulity < release cost < qulity < release ?
・ The marketing strategy has changed from “heavy/thick/big/long” to “light/thin/small/short.” When you want sell inexpensive goods in a big quantity, sales timing is crusial (e.g., If you cannot release by Christmas season, you will miss a sale chance).・ A strong demand from customers (e.g., Must be released by the beginning of a new fiscal year)
Businessstrategies in
the Era of Web and embedded
system
![Page 24: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/24.jpg)
24
Differences in sales strategies (What Japan faces after 2000: Part 2)
magnitude
release timing
impact to market
quality maturity
USA/India Japancost < qulity < release cost < qulity = release
・ We have to shorten the release timing as well as to retain the high quality.
![Page 25: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/25.jpg)
25
CMM Issue
Qustion: India is the No. 1 country that has most CMM-level-5 companies
in the world. Does that mean they really make high quality software ?
Answer: - CMM checks if a software company furnishes software development
processes to improve quality.
- CMM does not consider if a company makes quality software.
- A little league baseball team will get CMM level 5 if the team
has a pitching coach, a batting coach, health checking stuff.
If, on the other hand, a major league baseball team does not have
any coaches, it will get level 1.
- CMM considers the processes to develop quality software, not
the actual actual quality.
Differences in sales strategies (What Japan faces after 2000: Part 3)
![Page 26: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/26.jpg)
26
- In order to cut the development cost, and to shorten the development
time frame, we have to use off-the-shelf software (like MS Word, Excel)
in stead of develop everything.
→ Bugs in the off-the-shelf software will not be fixed in a timely fashion.
→ You have to assumu such bugs as « Specifications. »
Differences in sales strategies (What Japan faces after 2000: Part 3)
- The technology of software tends to be highly specialized, you have to
completely depend on a particular company that has such special
technology, such as the browser of a mobile phone.
→ The quality of your software depends on the company.
→ You cannot force them your quality processes.
![Page 27: Business Strategy](https://reader034.vdocuments.site/reader034/viewer/2022051322/5463f01faf795967228b4c58/html5/thumbnails/27.jpg)
27
One of the promissing solutions is:
- Open Sourece
- Standardized Souftware