Reconstruct obj/sky-pairs individually and combine them afterwards
esorex kmo_sci_red [OPTIONS] FILE.sof
Ideally at least two data frames have to be provided since we need for each IFU pointing to an object as well a sky frame for the same IFU.
If an OH spectrum is given in the SOF file the lambda axis will be corrected using the OH lines as reference.
Every IFU containing an object will be reconstructed and divided by telluric and illumination correction. By default these intermediate cubes are saved to disk. Frames just containing skies won’t produce an output here, so the number of output frames can be smaller than the number of input frames.
Then the reconstructed objects with the same object name are combined. These outputs are also saved to disk, the number of created files depends on the number of reconstructed objects of different name. If the user just wants to combine a certain object, the parameters --name or --ifus can be used.
For exposures taken with the templates KMOS_spec_obs_mapping8 and KMOS_spec_obs_mapping24 the recipe behaves a bit different: All active IFUs will be combined, regardless of the object names.
--imethod The interpolation method used for reconstruction.
--smethod The interpolation method used for shifting.
--name --ifus Since an object can be present only once per exposure and since it can be located in different IFUs for the existing exposures, there are two modes to identify the objects:
* Combine by object names (default) In this case the object name must be provided via the --name parameter.
The object name will be searched for in all primary headers of all provided frames in the keyword ESO OCS ARMx NAME.
* Combine by index (advanced) In this case the --ifus parameter must be provided. The parameter must have the same number of entries as frames are provided, e.g. "3;1;24" for 3 exposures. The index doesn\'t reference the extension in the frame but the real index of the IFU as defined in the EXTNAME keyword.
(e.g. \'IFU.3.DATA\')
--flux Specify if flux conservation should be applied.
--background Specify if background removal should be applied.
--suppress_extension If set to TRUE, the arbitrary filename extensions are supressed. If multiple products with the same category are produced, they will be numered consecutively starting from 0.
--sky_tweak If set to TRUE sky substraction is not done by subtracting the corresponding detector images but subtracting a modified sky cube from the object cube.
It is not allowed that "--sky_tweak" and "--no_subtract" both are TRUE.
--tbsub If set to TRUE subtract the thermal background from the cube resulting from sky tweaking.
Default value is TRUE.
--obj_sky_table The automatic obj-sky-associations can be modified by indicating a file with the desired associations. Therefore the file written to disk by default (without setting this option) can be edited manually. The formatting must absolutely be retained, just the type codes (\'O\' and\'S\') and the associated frame indices should be altered
Advanced reconstruction parameters ----------------------------------
--neighborhoodRange Defines the range to search for neighbors during reconstruction
--b_samples The number of samples in spectral direction for the reconstructed cube.
Ideally this number should be greater than 2048, the detector size.
--b_start --b_end Used to define manually the start and end wavelength for the reconstructed cube. By default the internally defined values are used.
--fast_mode If set to TRUE, the reconstructed cubes will be collapsed (using median) and only then be shifted and combined.
--pix_scale Change the pixel scale [arcsec]. Default of 0.2" results into cubes of 14x14pix, a scale of 0.1" results into cubes of 28x28pix, etc.
--no_subtract If set to TRUE, the found objects and references won\'t be sky subtracted. Additionally all IFUs will be reconstructed, even the ones containing skies.
This option sets the parameter no_combine to TRUE automatically.
--xcal_interpolation If true interpolate the pixel position in the slitlet (xcal) using the two closest rotator angles in the calibration file. Otherwise take the values of the closest rotator angle
--extrapolate By default no extrapolation is applied. This means that the intermediate reconstructed cubes will shrink at most one pixel, which is ok for templates like KMOS_spec_obs_nodtosky or KMOS_spec_obs_freedither. When the cubes will be arranged as a map, a grid is likely to occur between the IFUs. Therefore extrapolation during the shifting process can be switched on in order to get IFUs of original size. For frames taken with mapping templates, extrapolation is switched on automatically.
--velocity_offset Specifies the velocity offset correction in km/s for lambda scale.
Default is 0.0 km/s, i.e. no velocity correction.
--save_interims If set to TRUE the interim object and sky cubes used for sky tweaking are saved to FITS file in the same format as SCI_RECONSTRUCTED Default is FALSE.
Advanced combining parameters ----------------------------------
--edge_nan Set borders of two sides of the cubes to NaN before combining them. This minimises unwanted border effects when dithering.
--no_combine If set to TRUE, the reconstructed cubes will not be combined.
--method There are following sources to get the shift parameters from:
* \'header\' (default) The shifts are calculated according to the WCS information stored in the header of every IFU. The output frame will get larger, except the object is at the exact same position for all exposures. The size of the exposures can differ, but the orientation must be the same for all exposures.
* \'none\' The cubes are directly recombined, not shifting at all. The output frame will have the same dimensions as the input cubes.
If the size differs a warning will be emitted and the cubes will be aligned to the lower left corner. If the orientation differs a warning will be emitted, but the cubes are combined anyway.
* \'center\' The shifts are calculated using a centering algorithm. The cube will be collapsed and a 2D profile will be fitted to it to identify the centre.
With the parameter --fmethod the function to fit can be provided. The size of the exposures can differ, but the orientation must be the same for all exposures.
* \'user\' Read the shifts from a user specified file. The path of the file must be provided using the --filename parameter. For every exposure (except the first one) two shift values are expected per line, they have to be separated with simple spaces. The values indicate pixel shifts and are referenced to the first frame. The 1st value is the shift in x-direction to the left, the 2nd the shift in y-direction upwards. The size of the exposures can differ, but the orientation must be the same for all exposures.
--fmethod see --method=\'center\' The type of function that should be fitted spatially to the collapsed image.
This fit is used to create a mask to extract the spectrum of the object.
Valid values are \'gauss\' and \'moffat\'.
--filename see --method=\'user\'
--cmethod Following methods of frame combination are available:
* \'ksigma\' (Default)
An iterative sigma clipping. For each position all pixels in the spectrum
are examined. If they deviate significantly, they will be rejected
according to the conditions:
val > mean + stdev * cpos_rej
and
val < mean - stdev * cneg_rej
where --cpos_rej, --cneg_rej and --citer are the corresponding
configuration parameters. In the first iteration median and percentile
level are used.
* \'median\' At each pixel position the median is calculated.
* \'average\' At each pixel position the average is calculated.
* \'sum\' At each pixel position the sum is calculated.
* \'min_max\' The specified number of minimum and maximum pixel values will be rejected.
--cmax and --cmin apply to this method.
--cpos_rej --cneg_rej --citer see --cmethod=\'ksigma\'
--cmax --cmin see --cmethod=\'min_max\'
Input files:
DO KMOS category Type Explanation Required #Frames -------- ----- ----------- -------- ------- SCIENCE RAW The science frames Y >=1 XCAL F2D x calibration frame Y 1 YCAL F2D y calibration frame Y 1 LCAL F2D Wavelength calib. frame Y 1 WAVE_BAND F2L Table with start-/end-wavelengths Y 1 MASTER_FLAT F2D Master flat Y 0,1 ILLUM_CORR F2I Illumination correction N 0,1 TELLURIC F1I normalised telluric spectrum N 0,1 OH_SPEC F1S Vector holding OH lines N 0,1
Output files:
DO KMOS
category Type Explanation
-------- ----- -----------
SCI_COMBINED F3I Combined cubes with noise
SCI_RECONSTRUCTED F3I Reconstructed cube with noise
EXP_MASK F3I Exposure time mask (not for mapping-templates!)
SCI_INTERIM_OBJECT F3I (optional) Intermediate reconstructed object
cubes used for sky tweaking, no noise
(set --sky_tweak and --save_interims)
SCI_INTERIM_SKY F3I (optional) Intermediate reconstructed sky
cubes used for sky tweaking, no noise
(set --sky_tweak and --save_interims)
--imethod <str>
Method to use for interpolation during reconstruction. ["NN" (nearest neighbour), "lwNN" (linear weighted nearest neighbor), "swNN" (square weighted nearest neighbor), "MS" (Modified Shepard\'s method)"CS" (Cubic spline)] (str; default: \'CS\'). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.imethod [default = CS].
--smethod <str>
Method to use for interpolation during shifting. ["NN" (nearest neighbour), "CS" (Cubic spline)] (str; default: \'CS\'). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.smethod [default = CS].
--method <str>
The shifting method: \'none\': no shifting, combined directly, \'header\': shift according to WCS (default), \'center\': centering algorithm, \'user\': read shifts from file (str; default: \'header\'). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.method [default = header].
--fmethod <str>
The fitting method (applies only when method=\'center\'): \'gauss\': fit a gauss function to collapsed image (default), \'moffat\': fit a moffat function to collapsed image (str; default: \'gauss\'). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.fmethod [default = gauss].
--name <str>
Name of the object to combine. (str; default: \'\'). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.name [default = ].
--ifus <str>
The indices of the IFUs to combine. "ifu1;ifu2;..." (str; default: \'\'). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.ifus [default = ].
--pix_scale <float>
Change the pixel scale [arcsec]. Default of 0.2" results into cubes of 14x14pix, a scale of 0.1" results into cubes of 28x28pix, etc. (float; default: 0.2). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.pix_scale [default = 0.2].
--suppress_extension <bool>
Suppress arbitrary filename extension.(TRUE (apply) or FALSE (don\'t apply) (bool; default: False). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.suppress_extension [default = False].
--neighborhoodRange <float>
Defines the range to search for neighbors in pixels (float; default: 1.001). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.neighborhoodRange [default = 1.001].
--filename <str>
The path to the file with the shift vectors.(Applies only to method=\'user\') (str; default: \'\'). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.filename [default = ].
--flux <bool>
TRUE: Apply flux conservation. FALSE: otherwise (bool; default: False). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.flux [default = False].
--background <bool>
TRUE: Apply background removal. FALSE: otherwise (bool; default: False). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.background [default = False].
--fast_mode <bool>
FALSE: cubes are shifted and combined,TRUE: cubes are collapsed and then shifted and combined (bool; default: False). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.fast_mode [default = False].
--extrapolate <bool>
Applies only to \'smethod=CS\' when doing sub-pixel shifts: FALSE: shifted IFU will be filled with NaN\'s at the borders,TRUE: shifted IFU will be extrapolated at the borders (bool; default: False). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.extrapolate [default = False].
--xcal_interpolation <bool>
TRUE: Interpolate xcal between rotator angles. FALSE: otherwise (bool; default: True). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.xcal_interpolation [default = True].
--edge_nan <bool>
Set borders of cubes to NaN before combining them.(TRUE (apply) or FALSE (don\'t apply) (bool; default: False). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.edge_nan [default = False].
--no_combine <bool>
Don\'t combine cubes after reconstruction.(TRUE (apply) or FALSE (don\'t apply) (bool; default: False). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.no_combine [default = False].
--no_subtract <bool>
Don\'t sky subtract object and references.(TRUE (apply) or FALSE (don\'t apply) (bool; default: False). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.no_subtract [default = False].
--sky_tweak <bool>
Use modified sky cube for sky subtraction.(TRUE (apply) or FALSE (don\'t apply) (bool; default: False). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.sky_tweak [default = False].
--tbsub <bool>
Subtract thermal background from input cube.(TRUE (apply) or FALSE (don\'t apply) (bool; default: True). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.tbsub [default = True].
--b_samples <long>
The number of samples in wavelength for the reconstructed cube (long; default: 2048). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.b_samples [default = 2048].
--b_start <float>
The lowest wavelength [um] to use when reconstructing. Derived by default, depending on the band (float; default: -1.0). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.b_start [default = -1.0].
--b_end <float>
The highest wavelength [um] to use when reconstructing. Derived by default, depending on the band (float; default: -1.0). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.b_end [default = -1.0].
--obj_sky_table <str>
The path to the file with the modified obj/sky associations. (str; default: \'\'). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.obj_sky_table [default = ].
--velocity_offset <float>
Specify velocity offset correction in km/s for lambda scale (float; default: 0.0). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.velocity_offset [default = 0.0].
--save_interims <bool>
Save interim object and sky cubes. Can only be used together with --sky_tweak (bool; default: False). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.save_interims [default = False].
--cmethod <str>
Apply "average", "median", "sum", "min_max." or "ksigma". (str; default: \'ksigma\'). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.cmethod [default = ksigma].
--cpos_rej <float>
The positive rejection threshold for kappa-sigma-clipping (sigma). (float; default: 3.0). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.cpos_rej [default = 3.0].
--cneg_rej <float>
The negative rejection threshold for kappa-sigma-clipping (sigma). (float; default: 3.0). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.cneg_rej [default = 3.0].
--citer <long>
The number of iterations for kappa-sigma-clipping. (long; default: 3). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.citer [default = 3].
--cmax <long>
The number of maximum pixel values to clip with min/max-clipping. (long; default: 1). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.cmax [default = 1].
--cmin <long>
The number of minimum pixel values to clip with min/max-clipping. (long; default: 1). The full name of this option for the EsoRex configuration file is kmos.kmo_sci_red.cmin [default = 1].
Note that it is possible to create a configuration file containing these options, along with suitable default values. Please refer to the details provided by the 'esorex --help' command.
The full documentation for the kmos pipeline can be downloaded as a PDF file using the following URL:
ftp://ftp.eso.org/pub/dfs/pipelines/kmos/kmos-pipeline-manual-2.9.pdf
An overview over the existing ESO pipelines can be found on the web page http://www.eso.org/sci/software/pipelines/.
Basic documentation about the EsoRex program can be found at the esorex (1) man page.
It is possible to call the pipelines from python using the python-cpl package. See http://packages.python.org/python-cpl/index.html for further information.
The other recipes of the kmos pipeline are kmo_copy(7), kmo_fits_check(7), kmo_fits_stack(7), kmo_noise_map(7), kmo_rotate(7), kmo_extract_spec(7), kmo_multi_reconstruct(7), kmo_sky_tweak(7), kmo_shift(7), kmos_gen_reflines(7), kmo_stats(7), kmo_arithmetic(7), kmos_dark(7), kmo_fits_strip(7), kmo_combine(7), kmo_std_star(7), kmo_illumination_flat(7), kmo_make_image(7), kmos_flat(7), kmo_fit_profile(7), kmo_sky_mask(7), kmos_illumination(7), kmos_wave_cal(7), kmo_reconstruct(7)
kmo_sci_red 1.3.5
Alex Agudo Berbel <[email protected]>
Please report any problems to [email protected]. Alternatively, you may send a report to the ESO User Support Department <[email protected]>.
This file is part of the CRIRES Instrument Pipeline Copyright (C) 2002,2003 European Southern Observatory
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA