PANIC Pipeline Documentation¶
Feb 04, 2020
1. This “Documentation” is still a work in progress; some of the material is not organized, and several aspects of PAPI are not yet covered with sufficient detail.
This manual support HAWAII-2RG and HAWAII-4RG detectors.
Welcome! This is the Documentation for PAPI, the Data Reduction Pipeline of PANIC instrument, and for PANIC Quick-Look Tool (PQL), the GUI of PAPI used during the observation; all processing routines used by PQL are implented in PAPI.
PAPI is the automatic image processing pipeline for data taken with the PAnoramic Near Infrared Camera (PANIC) for the 2.2m and 3.5m Telescopes at Calar Alto Observatory (CAHA). The pipeline is written in Python and developed at the Institute of Astrophysics of Andalusia (CSIC). The automated processing steps include basic calibration (removeing instrumental signature), cosmic-ray removal, treatment for electronic ghosts (cross-talk), sky subtraction, non-linear count-rate correction, robust alignment and registration.
This manual is a complete description of the data reduction recipes implemented by the PAPI pipeline, showing the status of the current pipeline version and describing data reduction process of the PANIC data using PAPI.
Although PAPI was developed for the PANIC camera with H2RG detector, it as updated in 2020 to support the new H4RG detector integrated into PANIC.
Moreover, its development was initiated using data from Omega2000 camera at the 3.5m CAHA telescope and HAWK-I camera at the VLT; thus, in principle it should work with data these instruments, although it is not optimized for them.
In addition to this html version of the manual, there is also a pdf version to download.
Development: José-Miguel Ibáñez-Mengual (IAA-CSIC)
Contribution: PANIC Team
Currently PAPI it is able to reduce data taken with the Observing Tool (OT) defining the required observing blocks (OB), or manually through GEIRS scripts. PAPI was primarily developed and optimized for reducing broad-band imaging data of extragalactic sources (such as imaging data taken for field galaxy surveys and galaxy cluster surveys). Other types imaging data have been reduced with PAPI but results can not be as good as desired. (See Troubleshooting for tips). PAPI is not designed to reduce any kind of field taken with PANIC.
- 1. Installation & Configuration
- 2. PANIC Quick-Look Tool (PQL)
- 2.1. Purpose
- 2.2. FITS files and headers
- 2.3. Starting PQL
- 2.4. Configuration files
- 2.5. PQL’s main window
- 2.5.1. Menu bar
- 2.5.2. Tool bar
- 2.5.3. Main panel
- 2.5.4. Current night checkbox
- 2.5.5. Use GEIRS file
- 2.5.6. Filename filter
- 2.5.7. Data list view
- 2.5.8. List view filter
- 2.5.9. QuickLook mode
- 2.5.10. Last file received
- 2.5.11. Buttons
- 2.5.12. Setup Panel
- 2.5.13. Calibrations panel
- 2.5.14. Log panel
- 2.6. Event log window
- 2.7. Pop-up menu
- 2.7.1. Display image
- 2.7.2. Image info
- 2.7.3. Copy files to clipboard
- 2.7.4. Copy files to text file
- 2.7.5. Show Dither pattern
- 2.7.6. Calibrations
- 2.7.7. Focus evaluation
- 2.7.8. Subtract own-sky
- 2.7.9. Subtract near-sky
- 2.7.10. Quick reduction
- 2.7.11. Astrometric calibration
- 2.7.12. Photometric calibration
- 2.7.13. Statistics
- 2.7.14. FWHM mean estimation
- 2.7.15. Background estimation
- 2.7.16. Math operations
- 2.7.17. FITS operations
- 2.8. How do I …?
- 2.8.1. How to determine the telescope focus ?
- 2.8.2. How to determine the field rotation ?
- 2.8.3. How to inspect the profile of the stars in an image ?
- 2.8.4. How do I quick-reduce an observed sequence ?
- 2.8.5. How do I quick-reduce an observed sequence using dark and flat master calibration files ?
- 2.8.6. How do I make mosaics with PQL?
- 2.8.7. How do I make use of parallelisation ?
- 2.8.8. How do I report a issue ?
- 3. PAPI
- 3.1. Purpose
- 3.2. Quickstart
- 3.2.1. Optional Arguments
- 3.2.2. Input FITS data files
- 3.2.3. Show grouped files in a raw directory
- 3.2.4. Show grouped files per filter and coordinates of a raw directory
- 3.2.5. Reduce (quick) a specificied number of sequences of the group list
- 3.2.6. Reduce all the sequences of a given directory
- 3.2.7. Reduce all the sequences of a given set of directories
- 3.2.8. Calibrations
- 3.2.9. Reduce a single detector
- 3.3. Reduction modes
- 3.4. How NOT to use PAPI
- 3.5. Configuration files
- 3.6. Examples
- 3.7. Main config file
- 3.8. Getting PAPI Data
- 3.9. Caveats
- 4. PAPI Reference
- 4.1. Modules
- 4.2. Utilities
- 4.1. Modules
- 5. Data formats
- 6. Processing description
- 6.1. Outline
- 6.2. Computing the master dark
- 6.3. Computing the master flat-field
- 6.4. Computing the Bad Pixel Mask
- 6.5. Sky model
- 7. References
- 8. PAPI FAQ
- 8.1. Installation
- 8.2. Data reduction
- 8.2.1. What is the best way to reduce PAPI data?
- 8.2.2. How can process a broken, interrupted or non OT sequence ?
- 8.2.3. How does PAPI treat bad pixels ?
- 8.2.4. How can we reach hundredth of magnitude accuracy in photometry ?
- 8.2.5. How good is PAPI astrometry and how are PSF variations corrected ?
- 9. Troubleshooting
- 9.1. Why do I get a “command not found” error when trying to run one of the PAPI modules?
- 9.2. PAPI does not recognize a group of files as a well formed sequence.
- 9.3. PAPI can not run the astrometric calibration
- 9.4. PAPI can not run the photometric calibration
- 9.5. PAPI says some files in the input file list does not look a FITS file.
- 9.6. What is the best way to reduce PAPI data?
- 9.7. How can we reach hundredth of magnitude accuracy in photometry ?
- 9.8. How good is PAPI astrometry and how are PSF variations corrected ?
- 9.9. Is there a way to look at or edit FITS headers?
- 9.10. Does PAPI generate a log file of the processing ?
- 10. Acknowledgments
- 11. License
- 12. Glossary
If your research uses PAPI, we’d appreciate it if you could acknowledge the fact by including the following citation:
“This research made use of PAPI, the pipeline of PANIC instrument. It is funded by the Spanish Ministry of Economy and Competitiveness with funds from the European Union (FEDER) and the Spanish national budget, through the grants ICTS-2006-15, ICTS-2007-10, ICTS-2008-24, ICTS-2009-32 and the project Intramural 200450E458 of the Spanish National Research Council.”
Ibáñez Mengual, J.M., Fernández, M., Rodríguez Gómez, J. R., García Segura, A. J., Storz, C., “The PANIC software system”, Proc. SPIE 7740, 77402E (2010)
Ibáñez Mengual,J.M, García A.J, Storz C., Fried J. W., Fernández M., Rodríguez J. F., “Advanced PANIC quick-look tool using Python”, Proc. SPIE 8451, (2012)