Rating & Aggregation Rule

My this article is intended to Assessor community &

NOT to Process Improvement practitioners.

Rating Rule:

Existing rule: ISO/IEC 15504 Part 2 requires to rate Process Attribute (PA).

In Practice: In most of the  assessments, BPs & sometimes GPs are also rated. These ratings have been used to derive Process Attribute rating.

Solution: ISO/IEC 33020 (which replaces ISO/IEC 15504 Part 2) now defines three types of rating, R1, R2 & R3. The meaning of these ratings is:

R1:
a) Characterize process outcomes, i.e., Characterize BP for each process instance 
b) Characterize process attribute outcomes, i.e., Characterize GP for each process instance
c) Aggregate BP characterization of all process instances to PA1.1 rating
d) Aggregate GP characterization of all process instances to respective PA rating

R2:
a) Characterize process attribute for each process instance 
b) Aggregate process attribute characterization of all instances to PA rating

R3:
Rate process attribute for each process instance without aggregation.

Aggregation method:

Existing rule: ISO/IEC TR 15504 Part-7 defines derive CL of each process & converting to aggregated CL rating.

In Practice: Concept of SCAMPI-A method with expert judgment had been used frequently in ML assessment.

Solution: ISO/IEC 33020 now defines two major concept of aggregating characterization to derive rating.

1. Expert Judgment

2. Mathematical approach: Using Mean or Median

To apply mathematical approach, first convert ordinal scale (N, P, L, F) characterization to corresponding interval scale (0, 1, 2, 3) or (N, P-, P+, L-, L+, F) to (0, 1, 2, 3, 4, 5).

2.1 Mean:

• Calculate mean value of all characterization for particular Process Attribute
• Round-off the value in case of decimal value
• Convert to corresponding Ordinal scale (N, P, L, F)

2.2 Median:

• Calculate median of all characterization for particular Process Attribute
• In case of even count, calculate mean of center two numbers
• Convert to corresponding Ordinal scale (N, P, L, F)  

Note: Mathematical concept shall be applied when sufficient number of data points are available. Class 1 & Class 2 type of assessment are potential candidate to apply this concept.