BOD Quality Assurance (QA) and Quality Control (QC)

Limit of Detection (LOD)

BOD detection limits are theoretically based. A key assumptionon which the concept of detection for BOD is based is that the LEAST amount of depletion allowable is 2 mg/L. The LOD is also based on the highest volume of sample used in a dilution series. What results is the detection capability for a specific dilution series, or sample. This technique doesn’t should include seed correction, because we want to identify depletion due to the sample itself, and not due to the seed material.

The calculation involved is simply:

LOD mg/L =     2 mg/L X           300 mL         
                                      mL of sample in LEAST dilution

EXAMPLE: sample with dilutions of 100, 50, and 25 mLs
=   2 mg/L X           300 mL         
                                 100 mL

= 2 mg/L  X  3
= 6 mg/L LOD for that sample

Blanks

• Oxygen depletion MUST be < 0.2 mg/L

Back to Top

Seed Control

• Need at least 2 dilutions
• Best to do 3 dilutions
• Calculate seed correction factor
• Should deplete between 0.6 to 1.0 mg/L
• Standard Methods changing its position on this. Less emphasis on Seed Control; More on GGA

Back to Top

Glucose molecule.

GGA

• MUST be glucose + glutamic acid (“Alphatrol” not allowed)
• GGA solution MUST be 150 mg/L of each
• MUST bring up to room temperature before use.
• NEVER pipet out of the GGA reagent bottle
• MUST use exactly 6 mLs of GGA solution
• MUST be seeded
• Acceptance criteria MUST be 198 + 30.5 (167.5-228.5 mg/L)
• If you prepare more than one, ALL must meet criteria
• Consider: GGA #1 = 150, GGA #2 = 250, average=200
• THIS would constitute acceptable performance???????
• What about two results: 225 and 230 mg/L
• Analysis required weekly (1 per 20 if > 20 samples/week)

Back to Top

Gluctamic acid molecule - note presence of ammonia group (NH2).

Glucose

C₆H₁₂O₆

Molecular Weight= 180.16

Glutamic acid

C₅H₉NO₄

Molecular Weight= 147.13

Atomic Mass of Nitrogen (N) is 14
thus Nitrogen represents 14/147.13 or 9.5% of the mass of glutamic acid

GGA is a 50:50 mix of glucose and glutamic acid
thus Nitrogen represents about 4.75% of the weight of GGA

CRITICAL POINT: There’s NITROGEN in GGA!!!

That means that if your seed source contains nitrifiers, then your GGA will be biased high as the nitrifiers use oxygen to convert the nitrogen ammonia and then to nitrate

That further means that if your facility recycles final effluent into primary clarifiers--and you use primary as your seed source--, you could be adding nitrifying organisms to the seed. The result could mean high bias in your GGA data.

Back to Top

Precision (Replicates)

Precision is a measure of how closely multiple determinations performed on the same sample will agree with each other. To examine precision under the brightest light, consider the implications of analyzing replicates of a final effluent sample in which one of the determinations measures only 80% of a daily maximum permit limit, and the other exceeds the daily maximum by 20%.

Which result is correct? One would have an overwhelming desire to report the value that is under the daily maximum, but is that a correct, or even defensible approach? The simple answer is, “No.”

Back to Top

Replicates

• MUST use same dilutions as used for sample
• Example: if effluent dilutions are 100, 200, and 300 mL
  then replicate must be 100, 200, and 300 mL
• Required after every 20 samples of the same matrix
• Basic rule: if you report results on DMR, those samples count towards "the 20"
• influent and effluent considered separate matrices
• If you analyze industrial samples, those are a separate matrix

Evaluating Replicates

Precision, in BOD, is based on absolute difference (Range) or Relative percent difference (RPD) between duplicates.

Range
Range is expressed in the same units as values (i.e., mg/L)
Range is the absolute difference of the Larger value – smaller value

The term "Absolute Difference" is a mathematical expression that means the result cannot be less than zero. A negative result becomes positive. Therefore the absolute difference of 1-3 is 2 (the negative 2 becomes a positive). One way to look at this is that the smaller of two values is always subtracted from the larger value.

Example Sample = 22 Replicate =18
Range = 22 - 18 = 4

RPD (Relative Percent Difference)
RPD is expressed as %
RPD = [Range of replicates) ÷ Mean (of replicates)] X 100 Example Sample = 22 Replicate =18
Range = 22- 18
Range = 4 (see above)
Mean= (22 + 18) ÷ 2 = 40 ÷ 2 = 20
RPD = (4 ÷ 20) x 100 = 20%

Back to Top

Range is concentration dependent

Concentration dependency means that as concentration changes, so does the range. If you were told that a lab's range control limit was 25 mg/L, and asked whether that was "good", your response would likely be that you don't have enough information to make an assessment. If you were told that it was associated with effluent samples, you may now say something along the lines of " That's terrible!". On the other hand, if you were told that the range control limit of 25 was for influents, you may reverse your earlier assessment.

That is precisely what "concentration dependency" implies: that you need to know th3e concentrations involved in order to evaluate the precision.

• If the two values in question are 500 and 525, a range of 25 indicates very good precision.
• But…if the two values are 30 and 5, a range of 25 now indicates extremely POOR precision.
• The use of RPD (instead of Range) offers a means of normalizing this variance.
• With values of 500 and 525, the RPD is 4.9%
• With values of 5 and 30, however, the RPD is 143%
• You still need to know that the lower the RPD, the better the precision, but knowing that, the disparity between these sets of replicates becomes clear.

How do you deal with concentration dependency?

• Establish separate control limits based on concentration range
• Example: A typical BOD effluent may range 5-10 mg/L, but following rain events often runs 20-30 mg/L. During rain events, range of replicates may exceed control limits!!!!!
• Establish interim limits to deal with non-routine concentrations.

Back to Top

Which to use to monitor precision? Range or RPD?

As concentration increases, the absolute range usually increases as well with NO CHANGE in RPD.
For a given range, as concentration increases, RPD decreases.
For a given range, as concentration decreases, RPD increases.

When might this be of use?

Back to Top

Control Limits

RPD Control limits

1. Test the data for and eliminate outliers before proceeding.
2. Calculate the mean and standard deviation of the data.
3. Warning limits = Mean + 2 standard deviations
4. Control limits = Mean + 3 standard deviations
   NOTE: RPD is a "1-tailed test" (which means you cant have an RPD less than zero) , so we use only UPPER control limits. The target for spike recovery is 100%, and recoveries can be both less than and greater than 100%. Therefore spike control limits have upper and LOWER control limits, termed a "two-tailed" test.

Range Control Limits

1. Test the data for and eliminate outliers before proceeding.
2. Calculate the mean of the data.
3. Warning limits = 2.51 x Mean of the data
4. Control limits = 3.27 x Mean of the data

Review: Calculating Arithmetic Means
The arithmetic mean is calculated by summing the data involved and divided by the number of data points.

Back to Top

Calculating Replicate Control Limits

Precision is a measure of how closely multiple determinations performed on the same sample will agree with each other. To examine precision under the brightest light, consider the implications of analyzing replicates of a final effluent sample in which one of the determinations measures only 80% of a daily maximum permit limit, and the other exceeds the daily maximum by 20%.

Which result is correct? One would have an overwhelming desire to report the value that is under the daily maximum, but is that a correct, or even defensible approach? The simple answer is, “No.”

Step 1 - Calculate the range (R) of each set of replicate samples

R = | (Sample result) - (Replicate result) |

where: |expression | = the absolute value of "expression"

NOTE: The absolute value simply means that negative values are converted to positive values, thus |2 - 5| = 3 rather than -3. Range can be thought of as the difference between two sample results that is always a positive value.

Step 2 - Calculate the average Range (of at least 20 replicate pairs)

The average range [R] is calculated for n replicate sample sets (to be statistically significant, n must be at least 20):

Back to Top

Step 3 - Calculate control limits

The control and warning limits are calculated:

Back to Top

Identifying Outlier Data

A number of statistical tests to identify outliers are available, including a procedure used by the U.S.G.S., Dixon's test, the t-test, and Grubbs' test. While any of these tests can be employed to identify outlier data, Grubbs' test will be mentioned here due to its relative simplicity.

First, it is critical to understand that the basic premise of any outlier test is simply to assign a statistical likelihood that a given suspect outlier point represents a different population than the other values in a data set. It cannot -and should not-tell you what to do with that point. Because, however, the consequences associated with failing to exclude outliers (control limits become unreasonably broad allowing inaccurate data ) far outweighs the risk of excluding them (control limits become too tight and too many data points become suspect), we recommend that outliers be excluded from further use.

Grubbs' test is based on the calculation of a “Z-score” which is then evaluated against a set of criterion values based on the number of data points in the data set.

Z-score (Z) = | Mean - [Suspect result] |
                       -----------------------------------
                         standard deviation

where: |expression | = absolute value of "expression"

If the resultant Z-score is greater than the corresponding criterion value based on the number of data points, then the suspect value is indeed an outlier, and should be excluded and the control limits re-calculated.

Given the following data set:
1.5, 1.1, 0.0, 1.0, 0.4, 0.8, 1.0, 0.1, 2.7, 0.2, 0.8, 1.1, 0.1, 1.2, 0.5, 0.9, 0.4, 0.8, * 5.8 *, 2.1

First, order the data from smallest to largest:
0, 0.1, 0.1, 0.2, 0.4, 0.4, 0.5, 0.8, 0.8, 0.8, 0.9, 1.0, 1.0, 1.1, 1.1, 1.2, 1.5, 2.1, 2.7, * 5.8 *

Back to Top

You would test ONLY the maximum value (5.8). The mean for the data set (including the suspect point) is 1.13, and the standard deviation is 1.294. This yields a Z-score of 3.631 for the maximum range value (5.8) in the data set.

The Z-criterion value for a set of 20 data points is 2.71. Since the suspect data point's Z-score is greater than the criterion Z-score, the suspect data point (5.8) is indeed an outlier. The data point must be removed and the outlier test repeated using the next highest point (2.7)

Using the ordered data without the "5.8" value:
0, 0.1, 0.1, 0.2, 0.4, 0.4, 0.5, 0.8, 0.8, 0.8, 0.9, 1.0, 1.0, 1.1, 1.1, 1.2, 1.5, 2.1, * 2.7 *

Now we test the new maximum value (2.7). The mean for the reduced data set (including the suspect point) is 0.879, and the standard deviation is 0.689. This yields a Z-score of 2.64.

Subtract the mean from the suspect data point (2.7 - 0.88)= 1.82.
Then divide 1.82 by the standard deviation (0.689) 1.82 ÷ 0.689= 2.64.

The Z-criterion value for a set of 19 data points is 2.68. Since the suspect data point's Z-score is LESS than the criterion Z-score (although just barely!), the suspect data point (2.7) is NOT an outlier.

The control and warning limits are calculated:

The Range Control limits resulting from the inclusion of this data point would be 2.88 (Warning Limit = 2.26) , while if the outlier data point is NOT excluded, the control limit would be 3.69 (Warning Limit = 2.90).

Back to Top

Corrective Action

To put it simply, corrective action is required whenever any control limit is exceeded. The goal of corrective action is to develop a historical record which details the type of problems a laboratory encounters as well as the resolution process itself. To put this into perspective, most operators who have analyzed BOD using a DO probe come to realize that the DO probe membrane degrades over time, and at some certain point, you need to replace it. Most of this information, unfortunately, is kept in the operator's head. A new operator-perhaps an individual without extensive analytical experience-who takes over laboratory responsibilities, may not recognize the “warning signs” that indicate a need to change the membrane. Without a corrective action trail, which essentially can be reduced to documentation in the form of “When you see this, do the following”, analysis will continue, until the fateful day that a GGA sample fails to meet acceptance criteria. Once data has been generated under circumstances in which one or more QC checks exceed their associated acceptance criteria, it is too late, and further effort is mandatory.

Corrective action, then, should be considered to be a sort of preventive medicine against future illnesses. We should learn from our past “difficulties” to guard against going through the same difficulties in the future. As an example, the first time one ever “messes” with a bee's nest, they tend to get stung. The best corrective action for this situation might simply be to stay away from bee's nests. Is that the only corrective action for this situation? Certainly not! The kid that chooses a corrective action of simply throwing the rock (at the bee's nest) from further away may find, however, that his “corrective action” was not adequate to correct the “problem”.

A laboratory needs to document the results obtained from all corrective action efforts so that in the future, the operator/analyst can “home in” on the right solution for a particular problem. A good example of this is to consider what corrective action might be taken in response to failing a matrix spike control limit for recovery. In this example, let's say an operator records a matrix spike recovery of 52% for total phosphorus. Being in a hurry, as many things need to be done around the village, the operator's corrective action is to review his calculations to make sure he didn't make a simple math error. Satisfied that no error was made with calculations, he shrugs his shoulders, chalks it up to fate, and goes about his other duties. This same process goes on for several weeks, and over that time, he gets recoveries between 42% and 58%. He's heard about matrix interferences, and figures that his effluent has suddenly developed a matrix interference. Six months later, however, it's time for an evaluation of his laboratory and the DNR auditor discovers that the real problem is that the standard used to prepare matrix spikes not only should have been prepared fresh 3 months ago, but that an error was made during the preparation. The spike solution is found to be exactly one-half of the concentration the operator believed it to be.

Remember...we need to be sure that the corrective action taken in response to a QC failure is appropriate for the problem at hand, and the laboratory needs to establish a system to check back and ensure that the problem has actually been solved. After all, if one brings their car into the garage to have it looked at because it won't start, they would not be happy to hear that the mechanic's “corrective action” included performing a costly re-alignment!

Last resort - Qualifying your results

Operators will still need to report those analytical results that are associated with any analytical run in which one or more of the quality control samples fails to meet acceptance criteria. These data must be flagged on the DMR reports by putting a check on the Quality Control line in the upper right hand corner of the form as well as by circling or marking the affected data with an asterisk. It is also acceptable to include a narrative that describes which dates and analyses are affected, and specific details regarding the reason for qualification of the data. The operator must also decide whether or not to include the analytical results when calculating weekly or monthly average values. If the decision is made to exclude the values in question from calculating weekly/monthly averages, an explanation for the exclusion(s) must also be provided.

Back to Top

Documentation and Record-Keeping

All records of equipment calibration and maintenance, QC tests, sampling, and sample analysis are retained for at least three (3) years [five (5) years for sludge data] at the treatment facility office in metal file cabinets. Before any result is reported, all raw data and calculations are reviewed for accuracy by someone other than the analyst/lab technician who performed the testing. Depending on the size of the lab/wastewater facility, the individual who performs the review on analytical testing may be another analyst/lab technician, operator or the facility supervisor. In any case, the individual must have sufficient experience to be capable of distinguishing between correct and incorrect data.

It is important that all raw data be kept, no matter how rough in appearance. This information can be very helpful in locating data handling problems. If data contained on any record is transcribed to facilitate summarizing or neatness, the original record must also be kept. Sections NR 149.06 (1) (a) and (b) specifically require that records be maintained such that “...any sample may be traced back to the analyst, date collected, date analyzed, and method used including raw data, intermediate calculations, results, and the final report.” and that “Quality control results shall be traceable to all of the associated sample results.”

A typical wastewater treatment plant operator is charged with a multitude of duties throughout the municipality, and when something has to be overlooked, it tends to be documentation. Unfortunately, documentation is essential to substantiating the quality of your data. A good principle to operate by is that if you did the work - whatever that aspect of the analysis might be-take credit for your efforts, by documenting what you did. The old adage, “If you didn't document it...you didn't do it”, also applies.

Finally, s. NR 149.06 (5) requires that all records “shall be handled in a manner to ensure their permanence and security.” This section of NR 149 also specifies that “Handwritten records shall be recorded in ink.” In keeping with the intent of these requirements, correction fluid of any type, and pens containing “eraseable ink” must not be used. If errors are made when making a notation, a single line should be drawn through the entry and the correction entered directly above. Good laboratory practice also requires that corrections made in this manner should be accompanied by the initials of the individual making the correction and the date the correction was made.

Back to Top