Step by Step ALE Driver Creation

This guide provides step-by-step instructions for creating an ALE driver for an instrument that is already implemented in ISIS.

Prerequisites

1. Clone and build ISIS locally.
2. Download base and mission kernels related to the driver you plan to implement. One means of downloading this kernels is through ISIS's downloadIsisData script.
3. Fork the ALE repository
4. Clone from your fork (with the recursive flag to clone submodules as well):

   git clone --recurse-submodules -j8 git@github.com:[username]/ale.git
   

5. Add ALE to your ISIS environment
   • In a terminal, navigate to the base directory of your cloned ALE repo
   • Activate your ISIS environment and update with ALE

     conda activate <isis> # <isis> should replaced with the name of your isis environment
     conda env update -n <isis> -f environment.yml
     

   • Set variables and add ISIS binaries to PATH:

     export ISISROOT=[isisRepoLocation]/ISIS3
     export PATH=$PATH:[isisRepoLocation]/ISIS3/bin
     

   • Uninstall any preexisting versions of ALE and link in the development version:

     pip uninstall ale #run 2 or 3x to ensure that all ALE versions are removed
     python setup.py develop
     

Data Setup and Initialization

1. Copy a .cub to a local data area for testing. See Data Sources for a list of data archives.
2. In your local testing area, create 2 copies of the .cub for testing -- one with ALE and one with ISIS

   [cubename].ale.cub
   [cubename].isis.cub
   

3. Initialize the ISIS .cub with spiceinit

   spiceinit from= [cubename].isis.cub
   

Spiceinit driver order

spiceinit uses the first driver that works, and begins by looking for an ALE driver. If no suitable ALE driver is found, then spiceinit will default to the ISIS implementation. This design decision encourages a 'fail fast' approach by first checking that basic elements of the image match those in the driver (such as the instrument id). If you need to initialize a cube with an ISIS driver after creating an ALE driver, it is possible to force a failure in the ALE driver by introducing a syntax error into one of the driver's methods.

Creating the ALE driver

Creating and Naming the Class

1. If it doesn't exist, create a python file in ale/base/drivers for the spacecraft name followed by _drivers (e.g. hayabusa_drivers.py)
2. Create a class
   • See Class Signature for naming conventions
   • Add these common functions (and others as needed):
     • instrument_id
       • map the sensor name to the spacecraft name as shown in the NAIF Toolkit Docs

         @property
         def instrument_id(self):
             id_lookup = {
                 'CaSSIS': 'TGO_CASSIS',
             }
             return id_lookup[super().instrument_id]
         

     • sensor_model_version
       • Should almost always return 1.

         @property
         def sensor_model_version(self):
             return 1
         

Borrow from existing drivers

ALE drivers largely follow the same format. If you feel stuck, borrow the structure of another driver and customize it to fit your instrument!

Creating Methods to Customize the Driver

ALE drivers largely follow the same structure, but individual methods differ among instruments. The mixins with which your class are built (e.g. "LineScanner") provide a best effort to supply default functionality, but it is unlikely that all of your class's methods will use the default values. The process of adding methods is largely a matter of trial and error in 1. Getting spiceinit to fully initialize the image using the ALE driver and 2. getting the ALE driver's output to match ISIS's output.

1. Perform an initial best-attempt at filling in the necessary methods (see the common functions above)
2. Run spiceinit on the ALE cube (spiceinit from= [cubename].ale.cub)
3. Look for errors from the ale driver ([spacecraftname]_driver.py). This should let you see what methods haven't been covered by mixins. You will need to write these methods yourself in the driver.
4. Based on the errors you receive, add necessary methods, test, and repeat until spiceinit succeeds with the ale driver and outputs a cube from source = ale (and not source = isis).
   • To set your ISIS conda environment to use your local ALE instance:

     # Activate your ISIS conda environment
     $ conda activate isis3
     
     # Check which ALE dependency ISIS is using
     $ conda list ale
     
     # Remove only ALE and not its dependencies
     $ conda remove --force ale
     
     # Install dev build to ISIS's conda dir
     $ cd <path-to-ale-repo>/ale/build
     $ rm -rf CMakeCache.txt
     $ cmake .. -DCMAKE_INSTALL_PREFIX=$CONDA_PREFIX/
     $ make
     $ make install
     $ cd ..
     $ python setup.py develop
     
     # Rebuild ISIS to point to dev build
     $ cd ../ISIS3/build
     $ rm -rf CMakeCache.txt
     $ cmake -DJP2KFLAG=ON -Gninja <path-to-isis-repo>/ISIS3/isis
     $ ninja -j24
     
     # Verify ALE dependency is dev build
     $ conda list ale
     

5. Compare ALE .cub to ISIS .cub
   • Generate Labels for the ISIS and ALE Cubes

     catlab from= [cubename].isis.cub to= [cubename]_isis.lbl
     catlab from= [cubename].ale.cub to= [cubename]_ale.lbl
     

   • Compare ALE Label to the ISIS Label to check the accuracy (using the ISIS label as truth data).

     vimdiff [cubename].isis.lbl [cubename].ale.lbl
     

6. Adjust the functions as necessary until the ALE init'd .cub matches the ISIS init'd .cub

Preparing Test Data

1. Create a data directory within ale/tests/pytests/data/ based on your .cub name
2. Slice the Kernels with ALE's kernel slicing notebook (ale/notebooks/KernelSlice.py)
   • This will require some customization based on your instrument, specifically to PVL structure
   • Requires the separate installation of ckslicer (select your operating system -> CKSlicer).
   • If using the KernelSlice notebook is unsuccessful, you may have to manually slice the kernels with NAIF utilities (ckslicer, toxfr, spkmerge) and the appropriate arguments.
3. Copy transfer kernels into the test data directory that you created above
   • See Test Data for what data you will need to copy.
4. Generate an isd using ALE's isd_generate utility and copy it into ale/tests/pytests/data/isds/
5. Copy Test Data
   • In tests/pytests/data, make a directory named after your test cube.
   • Copy an ISD into tests/pytests/data/isds
6. Test with PyTest
   • In ale/pytests/ make a python file called test_[spacecraftname]_drivers.py
   • To start out, you may want to look at the structure of another test.
   • Make a kernels test function and a load test function.
   • Make a class with a test for each method you added to your driver. They should assert the value that was set in that function.
     • if your driver function calls other functions (like scs2e or gdpool), the value should be patched.
   • Run your test, fix mismatched asserts, and fix failing tests.

     pytest tests/pytests/test_[spacecraft]_drivers.py
     

---

References and Supplementary Instructions

Data Sources

Kernels

• A combination of Documentation and Metadata
• isisDataArea/isis_data/[spacecraftname]/kernels/ik
• iak (instead of ik) for addendum kernels

PDS Image Atlas

• https://pds-imaging.jpl.nasa.gov/search/

Lunar Orbital Data Explorer

• https://ode.rsl.wustl.edu/moon/productsearch

ISIS test Cubes

• Cubes contain metadata and an image from a spacecraft.
• isisDataArea/isis_testData/isis/src/[spacecraftname]/unitTestData

Generating Cubes

• If the cube you want is absent, generate it with an ISIS program, documented here.
  (look for an [instrument]2isis program under the appropriate mission.)
• Test Data can often be found here:

  /isis_testData/isis/src/[spacecraft]/apps/[sensor]2isis/tsts/[varies]/input`
  

Camera CPPs implemented in ISIS

• myRepoLocation/ISIS3/isis/src/[spacecraftname]/objs/[sensorname]/[sensorname].cpp

Labels and Fields

For more info on what a specific label or field might mean, check these sources: - ISIS Label Dictionary - NAIF Required Readings for SPICE (You may have to do a bit of digging here.)

Internet

• NAIF PDS Toolkit Docs
• Web search

Class Signature

The class signature for your driver should follow this format. See each letter below for guidance on how to name the class/what mixins to use.

# e.g, class HayabusaAmicaIsisLabelNaifSpiceDriver(Framer, IsisLabel, NaifSpice, RadialDistortion, Driver):
class [A][B][C][D]Driver ([E], [C], [D], [F], Driver):    

A - Spacecraft Name

The name of the spacecraft and/or mission (e.g. Messenger, Hayabusa, Galileo, Odyssey).

B - Instrument Name

The name of the specific sensor on the spacecraft (e.g. AmicaCamera, NarrowAngleCamera, ThemisIr). Often shortened to an acronym or abbreviation (TO DO: When is it shortened and how do I find the short name?).

C - Label Type: IsisLabel or PDS3Label

• Most common: IsisLabel
• PDS3Label is rare
• If dealing with an ISIS converted cube .lbl -> ISIS Label.
• If you haven't done a x_to_isis conversion -> PDS3

D - Auxiliary Data Type: IsisSpice or NaifSpice

• Most common: NaifSpice
• IsisSpice is rare
• If the spice data is already attached, IsisSpice can use the data already attached

E - Camera Type: Framer, LineScan, or (rare) PushFrame

Can be found in ISIS Implementation.

Possibly from the instrument Kernel: see if the CENTER is in the middle of the square (framer) or the first line (line scan). Or the kernel documentation may say directly.

F - Distortion Type

• RadialDistortion and then NoDistortion are the most common
• Usually can find in the IK or IAK in isis_data
• Look in Kernel or IAK for ODK, ODTK, Distortion, Optical
• TRANSX and TRANSY are distortion coefficients

G - Last Mixin is always Driver

Test Data

1. In the last cell of the KernelSlice.ipynb Jupyter Notebook, is the kernels command. As the last step of the Kernel Slicing, run this command to generate a list of the necessary Kernels.
2. For any Binary Kernels ending in .bsp or .bc, the .xsp and .xc, Transfer Kernels generated in the Kernel Slicing output directory should take their place in the ALE test data for your instrument.
3. For the rest of the Kernels listed (not ending in .bsp or .bc), copy them into the the ALE test data for your instrument.
4. Check if your test data folder is missing kernels (like fk and sclk) and copy those kernels from the isis_data area into your test data folder.
5. Copy/paste the naif0012.tls kernel from another ALE test data area.
6. Finally, the isis label (that we made with catlab, ending in _isis.lbl) is needed as truth data. Copy it into the ALE test data directory

For more info on these kernels and what each is for, see the Intro to Kernels or read more about specific Kernels in the NAIF Toolkit Docs.