Submitted by czr137 on
Body: 

Current Level 1 Detections

Currently the detection of these issues is performed by a script ran after retrieval of Level 2 products.  It has been written as a MATLAB script that query's the Level 2 SQL database for scans that have not been verified yet, it then uses the MATLAB OsirisLevel1Services API to collect the necessary L1 data.  When an issue is detected a flag is written back to the SQL database, which hides that profile from standard L2 data users.

Thus far two conditions in Level 1 data have been identified to cause potential instabilities in the retrieval process: clouds and radiation bombardment.

Clouds

In general the retrieval algorithm detects the range of altitudes that would hold valid data and this includes detecting where it is likely that cloud cover at low altitudes will interfere with the scan data.  There are cases where clouds are present within this range, which can throw off the retrieval algorithm as it tries to account for the large change in the scattered light profile by adjusting concentrations of aerosols and ozone.

To detect clouds in this case two long wavelength pixels (pixel index of 1176 and 1182) were selected and their radiance was averaged over the profile called

\begin{equation} \bar{I}  \end{equation}.

The value nearest 40km is used as a reference.  A detection vector is constructed using the equation

\begin{equation} v = log \left(\bar{I}/\bar{I_{ref}}\right)./\left( \rho/\rho_{ref} \right) \end{equation} .

where

\begin{equation} \rho \end{equation}.

is the neutral density of the atmosphere.

If anywhere between 15km and 40km this vector has a value that exceeds 0.6, then it is likely that a cloud is present, and the scan is flagged.

The cumulative number of scans rejected due to clouds for the OSIRIS mission is shown in the following figure:

Radiation Hits

Cosmic rays can penetrate the instrument's shielding and hit its CCD array.  This can deposit a lot of energy, which at the detector level is indistinguishable from a really bright signal.   These radiation hits usually only affect  a few number of pixels, producing a small peak in the spectrum that is usually not present at measurements above or below where the collision occured.  The majority of the pixels aren't used for retrieving any Level 2 data, and so they don't need to be checked for radiation hits.

The detection algorithm takes all the L1 data for a scan and at each tangent altitude constructs a normalized spectrum based on the measurements above and below following the equation

 \begin{equation} \frac{log(I_n)}{\left( log(I_{n+1}) + log(I_{n-1}) \right)/2} \end{equation}.

where n refers to the altitude of interest.

Then for each pixel that is used in the retrieval, average and standard deviation of the normalized measuremnt of ten pixels on either side are found.  If there is a spike at the pixel based on the mean and standard deviation, then a flag is set for a radiation hit.

The cumulative number of radiation hits as a function of time for the OSIRIS mission is shown in the following figure:

Future Level 1

In future versions of the retrieval software it would be far better to detect these issues prior to the retrieval.  That way it may be possible to correct for the issue.  For example if a radiation hit is detected at a certain pixel, it may be possible to use a nearby pixel. Or if a cloud is detected, the retrieval limits could be readjusted.