Level 2 Quality Assurance

Body: 

The OSIRIS Level 2 data products are retrieved through the SaskMART iterative numerical method.  Because the result of each iteration feeds into the next, methods of this kind can become unstable under certain conditions.  It is important to detect and screen out these profiles to protect the quality of these products and improve the accuracy of any Level 3 products.
Detection is performed using three major methods, examining Level 1 data for circumstances that are known to cause retrieval instabilities, using statistical methods to detect and remove major outliers, and examining weekly zonal averages and removing scans "by-hand".
The following sections will provide summary explanations for these three methods and describe how they have been implemented.

Level 1 Instability Conditions

Body: 

Current Level 1 Detections

Currently the detection of these issues is performed by a script ran after retrieval of Level 2 products.  It has been written as a MATLAB script that query's the Level 2 SQL database for scans that have not been verified yet, it then uses the MATLAB OsirisLevel1Services API to collect the necessary L1 data.  When an issue is detected a flag is written back to the SQL database, which hides that profile from standard L2 data users.

Thus far two conditions in Level 1 data have been identified to cause potential instabilities in the retrieval process: clouds and radiation bombardment.

Clouds

In general the retrieval algorithm detects the range of altitudes that would hold valid data and this includes detecting where it is likely that cloud cover at low altitudes will interfere with the scan data.  There are cases where clouds are present within this range, which can throw off the retrieval algorithm as it tries to account for the large change in the scattered light profile by adjusting concentrations of aerosols and ozone.

To detect clouds in this case two long wavelength pixels (pixel index of 1176 and 1182) were selected and their radiance was averaged over the profile called

\begin{equation} \bar{I}  \end{equation}.

The value nearest 40km is used as a reference.  A detection vector is constructed using the equation

\begin{equation} v = log \left(\bar{I}/\bar{I_{ref}}\right)./\left( \rho/\rho_{ref} \right) \end{equation} .

where

\begin{equation} \rho \end{equation}.

is the neutral density of the atmosphere.

If anywhere between 15km and 40km this vector has a value that exceeds 0.6, then it is likely that a cloud is present, and the scan is flagged.

The cumulative number of scans rejected due to clouds for the OSIRIS mission is shown in the following figure:

Radiation Hits

Cosmic rays can penetrate the instrument's shielding and hit its CCD array.  This can deposit a lot of energy, which at the detector level is indistinguishable from a really bright signal.   These radiation hits usually only affect  a few number of pixels, producing a small peak in the spectrum that is usually not present at measurements above or below where the collision occured.  The majority of the pixels aren't used for retrieving any Level 2 data, and so they don't need to be checked for radiation hits.

The detection algorithm takes all the L1 data for a scan and at each tangent altitude constructs a normalized spectrum based on the measurements above and below following the equation

 \begin{equation} \frac{log(I_n)}{\left( log(I_{n+1}) + log(I_{n-1}) \right)/2} \end{equation}.

where n refers to the altitude of interest.

Then for each pixel that is used in the retrieval, average and standard deviation of the normalized measuremnt of ten pixels on either side are found.  If there is a spike at the pixel based on the mean and standard deviation, then a flag is set for a radiation hit.

The cumulative number of radiation hits as a function of time for the OSIRIS mission is shown in the following figure:

Future Level 1

In future versions of the retrieval software it would be far better to detect these issues prior to the retrieval.  That way it may be possible to correct for the issue.  For example if a radiation hit is detected at a certain pixel, it may be possible to use a nearby pixel. Or if a cloud is detected, the retrieval limits could be readjusted.
 

Statistical Detection

Body: 

Unstable retrievals may result in extremely large, unphysical values.  To detect the unstable retrievals that cannot be detected by examining Level 1 data for known causes of instabilties we rely on comparing individual values with average values.  Therefore this process requires the calculation of average values then comparing the values.

Binning the Values

Ozone and aerosol values are grouped by week into 10 degree latitude bins at each altitude on the retreival grid.

Iterative Checking

The values in those bins are then used to find average values.  If an outlying value is found in any bin, the entire scan is removed from the bins.  This process of averaging and checking individual values against the average is performed 3 times.

Averages

In each bin the mean and Standard Deviation (SD) are calculated.  The median and Median Absolute Deviation (MAD) are also found. The MAD follows the equation \begin{equation} MAD = median\left( \left|x-median(x)\right| \right) \end{equation}

Comparing for Outliers

The program then checks each value in each bin against and absolute limit, the mean and SD and the median and MAD.  

  • An absolute limit is set at 1 for both ozone and aerosol since they are in units of volume mixing ratios and cannot exceed 1
  • A limit is set at \begin{equation} mean_i \pm a_i SD_i \end{equation}
  • A limit is set at  \begin{equation} median_i \pm b_i MAD_i \end{equation}

For these limits the index "i" represents an altitude and the factors "a" and "b" are set to allow more variability at low altitudes where atmospheric dynamics are likely to create outlying values that are still accurate.

The following figures show the cumulative number of scans removed by statistical detection for the Aerosol and Ozone products:

         

Visual Inspection

Body: 

After screening scans based on issues with Level 1 data and based on Level 2 statistical outliers the scans are screened by visual inspection.  Some outlier scans will remain as the statistical screening is based on averages and measures of deviation from those averages which would be calculated using those very same outliers.

Visual inspection provides the ability to quickly spot outlier scans that would otherwise cannot be detected using using simple algorithms. A simple graphical tool that has been made in MATLAB for this specific purpose displays monthly zonal averages and allows the user to remove outlier scans.  Once a month's data has been viewed using this tool it is marked as approved for use within the SQL database and is available to users.

Known Issues

Time Periods

2005-08-29—2005-09-03, 2005-09-14—2005-09-17

Incorrect Attitude; retrievals excluded.

2005-12-31

Incorrect Attitude due to Leap Second; retrievals excluded.

2008-07-18—2008-07-21

Incorrect Attitude; retrievals excluded.

2008-12-30—2008-12-31

Incorrect Attitude due to Leap Second; retrievals excluded.

2012-3-26—2012-3-31

Incorrect Attitude; retrievals excluded.

2012-5-26—2012-5-31

Incorrect Attitude; retrievals excluded.

2012-7-1—2012-7-5

Incorrect Attitude due to Leap Second; retrievals excluded.

The following figures show the cumulative number of scans removed manually for the Aerosol and Ozone products (excluding the time periods just mentioned):