goodman_pipeline.core package¶
Submodules¶
goodman_pipeline.core.check_version module¶
- v1.0.0
First public release.
-
goodman_pipeline.core.check_version.
am_i_updated
(version)¶
-
goodman_pipeline.core.check_version.
get_github_api_url
(token_env_name)¶
-
goodman_pipeline.core.check_version.
get_last
(github_api_token='GITHUB_ACCESS_TOKEN')¶ Returns the version of the last release on GitHub.
- Parameters
(str, optional) (github_api_token) –
token for the API (access) –
- Returns
version (LooseVersion)
- Return type
the last version of the pipeline.
goodman_pipeline.core.core module¶
-
class
goodman_pipeline.core.core.
GenerateDcrParFile
(par_file_name='dcr.par')¶ Bases:
object
Creates dcr.par file based on lookup table
dcr parameters depend heavily on binning, this class generates a file using the default format. The lookup table considers camera and binning.
-
class
goodman_pipeline.core.core.
IdentifySpectroscopicTargets
¶ Bases:
object
-
filter_peaks
(spatial_profile=None, detected_peaks=None, nfind=None, background_threshold=None, file_name=None, plots=False)¶ - Parameters
spatial_profile –
detected_peaks –
nfind –
background_threshold –
file_name –
plots –
Returns:
-
fit_background
(spatial_profile=None, file_name=None, plots=False)¶ - Parameters
spatial_profile –
file_name (String) –
plots –
Returns:
-
fit_model
(spatial_profile=None, selected_peaks=None, order=None, model_name=None, file_name=None, plots=False)¶
-
get_peaks
(spatial_profile=None, order=None, file_name=None, plots=False)¶ - Parameters
spatial_profile – Background subtracted profile
order –
file_name –
plots –
Returns:
-
subtract_background
(spatial_profile=None, background_model=None, file_name=None, plots=False)¶ - Parameters
spatial_profile –
background_model –
file_name –
plots –
Returns:
-
-
class
goodman_pipeline.core.core.
NightDataContainer
(path, instrument, technique)¶ Bases:
object
This class is designed to be the organized data container. It doesn’t store image data but a list of
DataFrame
objects. Also it stores critical variables such as sunrise and sunset times.-
add_bias
(bias_group)¶ Adds a bias group
- Parameters
bias_group (DataFrame) – A
DataFrame
Contains a set of keyword values of grouped image metadata
-
add_comp_group
(comp_group)¶ Adds a comp-only group
All comparison lamps groups are added here. The ones that may have been taken in the afternoon (isolated) or along science target. This will act as a pool of comparison lamp groups for eventual science targets taken without comparison lamps.
- Parameters
comp_group (DataFrame) – A
DataFrame
Contains a set of keyword values of grouped image metadata
-
add_data_group
(data_group)¶ Adds a data group
- Parameters
data_group (DataFrame) – A
DataFrame
Contains a set of keyword values of grouped image metadata
-
add_day_flats
(day_flats)¶ “Adds a daytime flat group
- Parameters
day_flats (DataFrame) – A
DataFrame
Contains a set of keyword values of grouped image metadata
-
add_object_group
(object_group)¶ Adds a object-only group
- Parameters
object_group (DataFrame) – A
DataFrame
Contains a set of keyword values of grouped image metadata
-
add_spec_group
(spec_group)¶ Adds a data group containing object and comp
The comparison lamp groups are also added to a general pool of comparison lamps.
- Parameters
spec_group (DataFrame) – A
DataFrame
Contains a set of keyword values of grouped image metadata
-
data_groups
= None¶ For spectroscopy use
-
is_empty
= None¶ For imaging use
-
set_readout
(gain, rdnoise, roi)¶ Set Gain, Read noise and ROI.
- Parameters
gain (float) – Gain from header
rdnoise (float) – Read noise from header.
roi (str) – ROI from header.
-
set_sun_times
(sun_set, sun_rise)¶ Sets values for sunset and sunrise
- Parameters
sun_set (str) – Sun set time in the format ‘YYYY-MM-DDTHH:MM:SS.SS’
sun_rise (str) – Sun rise time in the format ‘YYYY-MM-DDTHH:MM:SS.SS’
-
set_twilight_times
(evening, morning)¶ Sets values for evening and morning twilight
- Parameters
evening (str) – Evening twilight time in the format ‘YYYY-MM-DDTHH:MM:SS.SS’
morning (str) – Morning twilight time in the format ‘YYYY-MM-DDTHH:MM:SS.SS’
-
spec_groups
= None¶ Time reference points
-
-
exception
goodman_pipeline.core.core.
NoMatchFound
(message='No match found')¶ Bases:
Exception
Exception for when no match is found.
-
exception
goodman_pipeline.core.core.
NoTargetException
¶ Bases:
Exception
Exception to be raised when no target is identified
-
exception
goodman_pipeline.core.core.
NotEnoughLinesDetected
¶ Bases:
Exception
Exception for when there are no lines detected.
-
class
goodman_pipeline.core.core.
ReferenceData
(reference_dir)¶ Bases:
object
Contains spectroscopic reference lines values and filename to templates.
- This class stores:
file names for reference fits spectrum
file names for CSV tables with reference lines and relative intensities
line positions only for the elements used in SOAR comparison lamps
-
check_comp_group
(comp_group)¶ Check if comparison lamp group has matching reference lamps
- Parameters
comp_group (DataFrame) – A
DataFrame
instance that contains meta-data for a group of comparison lamps.
Returns:
-
get_reference_lamp
(header)¶ Finds a suitable template lamp from the catalog
- Parameters
header (Header) – FITS header of image we are looking a reference lamp.
- Returns
full path to best matching reference lamp.
-
lamp_exists
(header)¶ Checks whether a matching lamp exist or not
- Parameters
object_name (str) – Name of the lamp from ‘OBJECT’ keyword.
grating (str) – Grating from ‘GRATING’ keyword.
grt_targ (float) – Grating target from keyword ‘GRT_TARG’.
cam_targ (float) – Camera target from keyword ‘CAM_TARG’.
- Returns
True of False depending if a single matching lamp exist.
- Raises
NotImplementedError if there are more than one lamp found. –
-
class
goodman_pipeline.core.core.
SaturationValues
(ccd=None)¶ Bases:
object
Contains a complete table of readout modes and 50% half well
-
get_saturation_value
(ccd)¶ Defines the saturation_threshold level
- Parameters
ccd (CCDData) – Image to be tested for saturation_threshold
- Returns
The saturation_threshold value or None
-
property
saturation_value
¶ Saturation value in counts
In fact the value it returns is the 50% of full potential well, Some configurations reach digital saturation_threshold before 50% of full potential well, they are specified in the last column:
saturates_before
.- Returns
None if the value has not been defined
-
-
class
goodman_pipeline.core.core.
SpectroscopicMode
¶ Bases:
object
-
get_cam_grt_targ_angle
(grating, mode)¶ Get the camera and grating target values grating and mode
- Parameters
grating (float) – Grating frequency in lines/mm (unitless value)
mode (str) – Name of the grating’s mode for which the camera and grating target values are required.
- Returns
Camera and grating target values. None and None if no such values exists.
-
get_mode
(grating, camera_targ, grating_targ, blocking_filter)¶ Get the camera’s optical configuration mode.
This method is useful for data that does not have the WAVMODE keyword
- Parameters
grating (str) – Grating frequency as string
camera_targ (str) – Camera target angle as in the header.
grating_targ (str) – Grating target angle as in the header.
blocking_filter (str) – Order blocking filter listed on the header.
- Returns
string that defines the wavelength mode used
-
-
goodman_pipeline.core.core.
add_linear_wavelength_solution
(ccd, x_axis, reference_lamp, crpix=1)¶ Add wavelength solution to the new FITS header
Defines FITS header keyword values that will represent the wavelength solution in the header so that the image can be read in any other astronomical tool. (e.g. IRAF)
- Parameters
ccd (CCDData) –
x_axis (ndarray) – Linearized x-axis in angstrom
reference_lamp (str) – Name of lamp used to get wavelength solution.
crpix (int) – reference pixel for defining wavelength solution.
1. For most cases 1 should be fine. (Default) –
- Returns
- ccd (CCDData) A
CCDData
instance with linear wavelength solution on it.
- ccd (CCDData) A
-
goodman_pipeline.core.core.
add_wcs_keys
(ccd)¶ Adds generic keyword for linear wavelength solution to the header
Linear wavelength solutions require a set of standard fits keywords. Later on they will be updated accordingly. The main goal of putting them here is to have consistent and nicely ordered headers.
Notes
This does NOT add a WCS solution, just the keywords.
- Parameters
ccd (CCDData) – class:~astropy.nddata.CCDData` instance with no wcs keywords.
- Returns
- ccd (CCDData) A
CCDData
instance with modified header with added WCS keywords
- ccd (CCDData) A
-
goodman_pipeline.core.core.
astroscrappy_lacosmic
(ccd, red_path=None, save_mask=False)¶
-
goodman_pipeline.core.core.
bin_reference_data
(wavelength, intensity, serial_binning)¶ Bins a 1D array
This method reduces the size of an unbinned array by binning. The function to combine data is numpy.mean.
- Parameters
wavelength (array) – Wavelength axis
intensity (array) – Intensity
serial_binning (int) – Serial Binning is the binning in the
axis. (dispersion) –
- Returns
Binned wavelength and intensity arrays.
-
goodman_pipeline.core.core.
call_cosmic_rejection
(ccd, image_name, out_prefix, red_path, keep_files=False, prefix='c', method='dcr', save=False)¶ Call for the appropriate cosmic ray rejection method
There are four options when dealing with cosmic ray rejection in this pipeline, The default option is called
default
and it will choose the rejection method based on the binning of the image. Note that there are only two real methdos:dcr
andlacosmic
.For
binning 1x1
the choice will bedcr
forbinning 2x2
and3x3
will belacosmic
.The method
dcr
is a program written in C by Wojtek Pych (http://users.camk.edu.pl/pych/DCR/) that works very well for spectroscopy the only negative aspect is that integration with python was difficult and not natively (through subprocess).The method lacosmic is well known but there are different implementations, we started using
cosmicray_lacosmic()
but later we shifted towardsastroscrappy.detect_cosmics
. The LACosmic method was developed by Pieter G. van Dokkum. See <http://www.astro.yale.edu/dokkum/lacosmic/>There is also the option of skipping cosmic ray removal by using
none
.- Parameters
ccd (CCCData) – a
CCDData
instance.image_name (str) – Science image name.
out_prefix (str) – Partial prefix to be added to the image name. Related to previous processes and not cosmic ray rejection.
red_path (str) – Path to reduced data directory.
keep_files (bool) – If True, the original file and the cosmic ray mask will not be deleted. Default is False.
prefix (str) – Cosmic ray rejection related prefix to be added to image name.
method (str) – Method to use for cosmic ray rejection. There are four options: default, dcr, lacosmic and none.
save (bool) – Disables by default saving the images
- Returns
CCDData
instance and out_prefix which is theprefix added to the image name.
- Raises
NotImplementedError if the method argument is not dcr, lacosmic –
nor none. –
-
goodman_pipeline.core.core.
classify_spectroscopic_data
(path, search_pattern)¶ Classify data by grouping them by a set of keywords.
This function uses
ImageFileCollection
. First it creates a collection of information regarding the images located inpath
that match the patternsearch_pattern
. The information obtained are all keywords listed in the listkeywords
. TheImageFileCollection
object is translated intoDataFrame
and then is used much like an SQL database to select and filter values and in that way put them in groups that areDataFrame
instances. The keywords retrieved are: -date
-slit
-date-obs
-obstype
-object
-exptime
-obsra
-obsdec
-grating
-cam_targ
-grt_targ
-filter
-filter2
-gain
-rdnoise
.Then all data is grouped by matching the following keywords:
slit
radeg
decdeg
grating
cam_targ
grt_targ
filter
filter2
gain
rdnoise
And finally, every group is classified as: a comparison lamp-only group, an object-only group or a group of object and comparison lamps. The comparison lamps present in the last group (
COMP
+OBJECT
) are also added in the first one (COMP
-only).- Parameters
path (str) – Path to data location
search_pattern (str) – Prefix to match files.
- Returns
Instance of
goodman_pipeline.core.core.NightDataContainer
-
goodman_pipeline.core.core.
combine_data
(image_list, dest_path, prefix=None, output_name=None, method='median', save=False)¶ Combine a list of
CCDData
instances.- Parameters
image_list (list) – Each element should be an instance of
CCDData
dest_path (str) – Path to where the new image should saved
prefix (str) – Prefix to add to the image file name
output_name (str) – Alternatively a file name can be parsed, this will ignore prefix.
method (str) – Method for doing the combination, this goes straight to the call of ccdproc.combine function.
save (bool) – If True will save the combined images. If False it will ignore prefix or output_name.
- Returns
A combined image as a
CCDData
object.
-
goodman_pipeline.core.core.
convert_time
(in_time)¶ Converts time to seconds since epoch
- Parameters
in_time (str) – time obtained from header’s keyword DATE-OBS
- Returns
time in seconds since epoch
-
goodman_pipeline.core.core.
create_master_bias
(bias_files, raw_data, reduced_data, technique)¶ Create Master Bias
Given a
DataFrame
object that contains a list of compatible bias. This function creates the master flat using ccdproc.combine using median and 3-sigma clipping.- Parameters
bias_files (list) – List of all bias files to be combined. They have
be compatible with each other as no check is done in this method. (to) –
raw_data (str) – Full path to raw data location.
reduced_data (str) – Full path to were reduced data will reside.
technique (str) – Name of observing technique. Imaging or
Spectroscopy. –
- Returns
master_bias_name (str):
- Return type
master_bias (object)
-
goodman_pipeline.core.core.
create_master_flats
(flat_files, raw_data, reduced_data, technique, overscan_region, trim_section, master_bias_name, new_master_flat_name, saturation_threshold, ignore_bias=False)¶ Creates master flats
Using a list of compatible flat images it combines them using median and 1-sigma clipping. Also it apply all previous standard calibrations to each image.
- Parameters
flat_files (list) – List of files previously filtered, there is no
check in this function and is assumed the files are (compatibility) –
combinables. –
raw_data (str) – Full path to raw data.
reduced_data (str) – Full path to reduced data. Where reduced data
be stored. (should) –
technique (str) – Observing technique. Imaging or Spectroscopy.
overscan_region (str) – Defines the area to be used to estimate the
region for overscan correction. Should be in the format. (overscan) –
`[x1 –
x2.,y1:y2]`.
trim_section (str) – Defines the area to be used after trimming
selected parts (unusable) – x2.,y1:y2]`.
master_bias_name (str) – Master bias name, can be full path or not.
it is a relative path, the path will be ignored and will define (If) –
full path as raw_path + basename. (the) –
new_master_flat_name (str) – Name of the file to save new master
Can be absolute path or not. (flat.) –
saturation_threshold (int) – Saturation threshold, defines the percentage of
above saturation level allowed for flat field images. (pixels) –
ignore_bias (bool) – Flag to create master bias without master bias.
- Returns
The master flat
CCDData
instance and the name of under which the master flat was stored. If it can’t build the master flat it will return None, None.
-
goodman_pipeline.core.core.
cross_correlation
(reference, compared, slit_size, serial_binning, mode='full', plot=False)¶ Do cross correlation of two 1D spectra
It convolves the reference lamp depending on the slit size of the new_array that corresponds with a comparison lamp. If the slit is larger than 3 arcseconds the reference lamp is convolved with a ~astropy.convolution.Box1DKernel because spectral lines look more like a block than a line. And if it is smaller or equal to 3 it will use a ~astropy.convolution.Gaussian1DKernel ponderated by the binning. All reference lamp are unbinned, or binning is 1x1.
- Parameters
reference (array) – Reference array.
compared (array) – Array to be matched. A new reference lamp.
slit_size (float) – Slit width in arcseconds
serial_binning (int) – Binning in the spectral axis
mode (str) – Correlation mode for scipy.signal.correlate.
plot (bool) – Switch debugging plots on or off.
- Returns
Shift value in pixels.
- Return type
correlation_value (int)
-
goodman_pipeline.core.core.
dcr_cosmicray_rejection
(data_path, in_file, prefix, keep_cosmic_files=False, save=True)¶ Runs an external code for cosmic ray rejection
DCR was created by Wojtek Pych and the code can be obtained from http://users.camk.edu.pl/pych/DCR/ and is written in C. Contrary to ccdproc’s LACosmic it actually applies the correction, and also doesn’t update the mask attribute since it doesn’t work with
CCDData
instances.The binary takes three positional arguments, they are: 1. input image, 2. output image and 3. cosmic rays image. Also it needs that a dcr.par file is located in the directory. All this is implemented in this function, if delete is True it will remove the original image and the cosmic rays image. The removal of the original image is absolutely safe when used in the context of the goodman pipeline, however if you want to implement it somewhere else, be careful.
Notes
This function operates an external code therefore it doesn’t return anything natively, instead it creates a new image. A workaround has been created that loads the new image and deletes the file.
- Parameters
data_path (str) – Data location
in_file (str) – Name of the file to have its cosmic rays removed
prefix (str) – Prefix to add to the file with the cosmic rays removed
keep_cosmic_files (bool) – True for deleting the input and cosmic ray file.
save (bool) – Toggles the option of saving the image.
-
goodman_pipeline.core.core.
define_trim_section
(sample_image, technique)¶ Get the initial trim section
The initial trim section is usually defined in the header with the keyword
TRIMSEC
but in the case of Goodman HTS this does not work well. In particular for spectroscopy where is more likely to have combined binning and so on.- Parameters
sample_image (str) – Full path to sample image.
technique (str) – The name of the technique, the options are: Imaging or Spectroscopy.
- Returns
x2, y1:y2]``
- Return type
The trim section in the format ``[x1
-
goodman_pipeline.core.core.
evaluate_wavelength_solution
(clipped_differences)¶ Calculates Root Mean Square Error for the wavelength solution.
- Parameters
clipped_differences (ndarray) – Numpy masked array of differences between reference line values in angstrom and the value calculated using the model of the wavelength solution.
- Returns
- Root Mean Square Error, number of points and number of points
rejected in the calculation of the wavelength solution.
-
goodman_pipeline.core.core.
extract_fractional_pixel
(ccd, target_trace, target_fwhm, extraction_width, background_spacing=3)¶ Performs an spectrum extraction using fractional pixels.
- Parameters
ccd (CCDData) –
a 2D spectrum. (contains) –
target_trace (object) – Instance of astropy.modeling.models.Model that defines the trace of the target on the image (ccd).
target_fwhm (float) – FWHM value for the spatial profile fitted to the target.
extraction_width (int) – Width of the extraction area as a function of target_fwhm. For instance if extraction_with is set to 1 the function extract 0.5 to each side from the center of the traced target.
background_spacing (float) – Number of target_stddev to separate the target extraction to the background. This is from the edge of the extraction zone to the edge of the background region.
-
goodman_pipeline.core.core.
extract_optimal
()¶ Placeholder for optimal extraction method.
- Raises
NotImplementedError –
-
goodman_pipeline.core.core.
extraction
(ccd, target_trace, spatial_profile, extraction_name)¶ Calls appropriate spectrum extraction routine
This function calls the appropriate extraction function based on extraction_name
Notes
Optimal extraction is not implemented.
- Parameters
ccd (CCDData) – Instance of
CCDData
containing a 2D spectrumtarget_trace (object) – Instance of astropy.modeling.Model, a low order polynomial that defines the trace of the spectrum in the ccd object.
spatial_profile (Model) – Instance of
Model
, a Gaussian model previously fitted to the spatial profile of the 2D spectrum contained in the ccd object.extraction_name (str) – Extraction type, can be fractional or optimal though the optimal extraction is not implemented yet.
- Returns
Instance of
CCDData
containing a 1D spectrum. The attribute ‘data’ is replaced by the 1D array resulted from the extraction process.- Return type
ccd (CCDData)
- Raises
NotImplementedError – When extraction_name is optimal.
-
goodman_pipeline.core.core.
fix_keywords
(path, pattern='*.fits')¶ Fix FITS uncompliance of some keywords
Uses automatic header fixing by
CCDData
. Note that this only fixes FITS compliance.- Parameters
path (str) – Path to raw data
pattern (str) – Search pattern for listing file in path.
-
goodman_pipeline.core.core.
fractional_sum
(data, index, low_limit, high_limit)¶ Performs a fractional pixels sum
A fractional pixels sum is required several times while extracting a 1D spectrum from a 2D spectrum. The method is actually very simple.
It requires the full data, the column and the range to sum, this range is given as real numbers. First it separates the limits values as an integer and fractional parts. Then it will sum the integer’s interval and subtract the low_limit’s fractional part and sum the high_limit’s fractional part.
The sum is performed in one operation. It does not do background subtraction, for which this very same method is used to get the background sum to be subtracted later.
- Parameters
data (numpy.ndarray) – 2D array that contains the 2D spectrum/image
index (int) – Index of the column to be summed.
low_limit (float) – Lower limit for the range to be summed.
high_limit (float) – Higher limit for the range to be summed.
- Returns
Sum in ADU of all pixels and fractions between low_limit and high_limit.
-
goodman_pipeline.core.core.
get_best_flat
(flat_name, path)¶ Look for matching master flat
Given a basename for master flats defined as a combination of key parameters extracted from the header of the image that we want to flat field, this function will find the name of the files that matches the base name and then will choose the first. Ideally this should go further as to check signal, time gap, etc. After it identifies the file it will load it using
CCDData
and return it along the filename. In the case it fails it will return None instead of master_flat and another None instead of master_flat_name.- Parameters
flat_name (str) – Full path of master flat basename. Ends in ‘*.fits’ for using glob.
path (str) – Location to look for flats.
- Returns
A
CCDData
instance. master_flat_name (str): Full path to the chosen master flat.- Return type
master_flat (object)
-
goodman_pipeline.core.core.
get_central_wavelength
(grating, grt_ang, cam_ang)¶ Calculates the central wavelength for a given spectroscopic mode
The equation used to calculate the central wavelength is the following
\[\lambda_{central} = \frac{1e6}{GRAT} \sin\left(\frac{\alpha \pi}{180}\right) + \sin\left(\frac{\beta \pi}{180}\right)\]- Parameters
grating (str) – Grating frequency as a string. Example ‘400’.
grt_ang (str) – Grating Angle as a string. Example ‘12.0’.
cam_ang (str) – Camera Angle as a string. Example ‘20.0’
- Returns
Central wavelength as a float value.
- Return type
central_wavelength (float)
-
goodman_pipeline.core.core.
get_lines_in_lamp
(ccd, plots=False)¶ Identify peaks in a lamp spectrum
Uses scipy.signal.argrelmax to find peaks in a spectrum i.e emission lines, then it calls the recenter_lines method that will recenter them using a “center of mass”, because, not always the maximum value (peak) is the center of the line.
- Parameters
ccd (CCDData) – Lamp ccdproc.CCDData instance.
plots (bool) – Wether to plot or not.
- Returns
- A common list containing pixel values at
approximate location of lines.
- Return type
lines_candidates (list)
-
goodman_pipeline.core.core.
get_overscan_region
(sample_image, technique)¶ Get the right overscan region for spectroscopy
- It works for the following ROI:
Spectroscopic 1x1 Spectroscopic 2x2 Spectroscopic 3x3
The limits where measured on a Spectroscopic 1x1 image and then divided by the binning size. This was checked that it actually works as expected.
Notes
The regions are 1-based i.e. different to Python convention. For Imaging there is no overscan region.
- Parameters
sample_image (str) – Full path to randomly chosen image.
technique (str) – Observing technique, either Spectroscopy or
Imaging –
- Returns
- overscan_region (str) Region for overscan in the format
- ’[min:max,:]’ where min is the starting point and max is the end
point of the overscan region.
-
goodman_pipeline.core.core.
get_slit_trim_section
(master_flat)¶ Find the slit edges to trim all data
Using a master flat, ideally with good signal to noise ratio, this function will identify the edges of the slit projected into the detector. Having this done will allow to reduce the overall processing time and also reduce the introduction of artifacts due to non-illuminated regions in the detectors, such as NaNs -INF +INF, etc.
- Parameters
master_flat (CCDData) – A
CCDData
instance.- Returns
- Trim section in spatial direction in the format
[:,slit_lower_limit:slit_higher_limit]
- Return type
slit_trim_section (str)
-
goodman_pipeline.core.core.
get_spectral_characteristics
(ccd, pixel_size, instrument_focal_length)¶ Calculates some Goodman’s specific spectroscopic values.
From the header value for Grating, Grating Angle and Camera Angle it is possible to estimate what are the wavelength values at the edges as well as in the center. It was necessary to add offsets though, since the formulas provided are slightly off. The values are only an estimate.
- Parameters
ccd (CCDData) – Lamp ccdproc.CCDData instance
pixel_size (float) – Pixel size in microns
instrument_focal_length (float) – Instrument focal length
- Returns
- Contains the following parameters:
center: Center Wavelength blue: Blue limit in Angstrom red: Red limit in Angstrom alpha: Angle beta: Angle pix1: Pixel One pix2: Pixel Two
- Return type
spectral_characteristics (dict)
-
goodman_pipeline.core.core.
get_twilight_time
(date_obs)¶ Get end/start time of evening/morning twilight
Notes
Taken from David Sanmartim’s development
- Parameters
date_obs (list) – List of all the dates from data.
- Returns
- Evening twilight time in the format
’YYYY-MM-DDTHH:MM:SS.SS’
- twilight_morning (str): Morning twilight time in the format
’YYYY-MM-DDTHH:MM:SS.SS’
- sun_set_time (str): Sun set time in the format
’YYYY-MM-DDTHH:MM:SS.SS’
- sun_rise_time (str): Sun rise time in the format
’YYYY-MM-DDTHH:MM:SS.SS’
- Return type
twilight_evening (str)
-
goodman_pipeline.core.core.
identify_targets
(ccd, fit_model, background_threshold, nfind=3, plots=False)¶ Identify Spectroscopic Targets
Wrapper to the class IdentifySpectroscopicTargets.
- Parameters
ccd (CCDData) – Image containing spectra
fit_model (str) – Name of the model to be fitted moffat or gaussian.
background_threshold (int) – Number of background levels for target
discrimination. –
nfind (int) – Maximum number of targets passing the background threshold
be returned, they are order from most intense peak to least intense. (to) –
plots (bool) – Flat for plotting results.
- Returns
List of models successfully fitted.
- Return type
identified_targets (list)
-
goodman_pipeline.core.core.
image_overscan
(ccd, overscan_region, add_keyword=False)¶ Apply overscan correction to data
Uses ccdproc.subtract_overscan to perform the task.
Notes
The overscan_region argument uses FITS convention, just like IRAF, therefore is 1 based. i.e. it starts in 1 not 0.
- Parameters
ccd (CCDData) – overscan corrected.
overscan_region (str) – The overscan region in the format [x1:x2,y1:y2] where x is the spectral axis and y is the spatial axis.
add_keyword (bool) – Tells ccdproc whether to add a keyword or not. Default False.
- Returns
- ccd (CCDData) Overscan corrected
CCDData
instance
- ccd (CCDData) Overscan corrected
-
goodman_pipeline.core.core.
image_trim
(ccd, trim_section, trim_type='trimsec', add_keyword=False)¶ Trim image to a given section
Notes
The overscan_region argument uses FITS convention, just like IRAF, therefore is 1 based. i.e. it starts in 1 not 0.
- Parameters
ccd (CCDData) –
trim_section (str) – The trimming section in the format [x1:x2,y1:y2] where x is the spectral axis and y is the spatial axis.
trim_type (str) – trimsec or slit trim.
add_keyword (bool) – Tells ccdproc whether to add a keyword or not. Default False.
- Returns
ccd (CCDData) Trimmed
CCDData
instance
-
goodman_pipeline.core.core.
interpolate
(spectrum, interpolation_size)¶ Creates an interpolated version of the input spectrum
This method creates an interpolated version of the input array, it is used mainly for a spectrum but it can also be used with any unidimensional array. The reason for doing interpolation is that it allows to find the lines and its respective center more precisely.
- Parameters
spectrum (array) – an uncalibrated spectrum or any unidimensional array.
interpolation_size (int) – Number of points to interpolate. (points added between two existing ones)
- Returns
- Two dimensional array containing x-axis and interpolated array.
The x-axis preserves original pixel values.
-
goodman_pipeline.core.core.
is_file_saturated
(ccd, threshold)¶ Detects a saturated image
It counts the number of pixels above the saturation_threshold level, then finds which percentage they represents and if it is above the threshold it will return True. The percentage threshold can be set using the command line argument
--saturation_threshold
.- Parameters
ccd (CCDData) – Image to be tested for saturation_threshold
threshold (float) – Percentage of saturated pixels allowed. Default 1.
- Returns
True for saturated and False for non-saturated
-
goodman_pipeline.core.core.
linearize_spectrum
(data, wavelength_solution, plots=False)¶ Produces a linearized version of the spectrum
Storing wavelength solutions in a FITS header is not simple at all for non-linear solutions therefore is easier for the final user and for the development code to have the spectrum linearized. It first finds a spline representation of the data, then creates a linear wavelength axis (angstrom) and finally it resamples the data from the spline representation to the linear wavelength axis.
It also applies a median filter of kernel size three to smooth the linearized spectrum. Sometimes the splines produce funny things when the original data is too steep.
- Parameters
data (Array) – The non-linear spectrum
wavelength_solution (object) – Mathematical model representing the
solution. (wavelength) –
plots (bool) – Whether to show the plots or not.
- Returns
Contains two elements: Linear wavelength axis and the smoothed linearized data itself.
- Return type
linear_data (list)
-
goodman_pipeline.core.core.
name_master_flats
(header, technique, reduced_data, sun_set, sun_rise, evening_twilight, morning_twilight, target_name='', get=False)¶ Defines the name of a master flat or what master flat is compatible with a given data
Given the header of a flat image this method will look for certain keywords that are unique to a given instrument configuration therefore they are used to discriminate compatibility.
It can be used to define a master flat’s name when creating it or find a base name to match existing master flat files thus finding a compatible one for a given non-flat image.
- Parameters
header (object) – Fits header. Instance of
:param
Header
: :param technique: Observing technique, either Spectroscopy or :type technique: str :param Imaging.: :param reduced_data: Full path to reduced data directory :type reduced_data: str :param sun_set: Sunset time formatted as “%Y-%m-%dT%H:%M:%S.%f” :type sun_set: str :param sun_rise: Sunrise time formatted as “%Y-%m-%dT%H:%M:%S.%f” :type sun_rise: str :param evening_twilight: End of evening twilight formatted as :type evening_twilight: str :param “%Y-%m-%dT%H: %M:%S.%f” :param morning_twilight: Start of morning twilight in the format :type morning_twilight: str :param “%Y-%m-%dT%H: %M:%S.%f” :param target_name: Optional science target name to be added to themaster flat name.
- Parameters
get (bool) – This option is used when trying to find a suitable master flat for a given data.
- Returns
A master flat name, or basename to find a match among existing files.
-
goodman_pipeline.core.core.
normalize_master_flat
(master, name, method='simple', order=15)¶ Master flat normalization method
- This function normalize a master flat in three possible ways:
mean: simply divide the data by its mean
simple: Calculates the median along the spatial axis in order to obtain the dispersion profile. Then fits a
Chebyshev1D
model and apply this to all the data.full: This is for experimental purposes only because it takes a lot of time to process. It will fit a model to each line along the dispersion axis and then divide it by the fitted model. I do not recommend this method unless you have a good reason as well as a very powerful computer.
- Parameters
master (CCDData) – Master flat. Has to be a
CCDData
instance.name (str) – Full path of master flat prior to normalization.
method (str) – Normalization method, ‘mean’, ‘simple’ or ‘full’.
order (int) – Order of the polynomial to be fitted.
- Returns
- The normalized master flat.
CCDData
instance.
- Return type
master (CCDData)
-
goodman_pipeline.core.core.
ra_dec_to_deg
(right_ascension, declination)¶ Converts right ascension and declination to degrees
- Parameters
right_ascension (str) – Right ascension in the format hh:mm:ss.sss
declination (str) – Declination in the format dd:mm:ss.sss
- Returns
Right ascension in degrees declination_deg (float): Declination in degrees
- Return type
right_ascension_deg (float)
-
goodman_pipeline.core.core.
read_fits
(full_path, technique='Unknown')¶ Read fits files while adding important information to the header
It is necessary to record certain data to the image header so that’s the reason for this wrapper of
read()
to exist. It will add the following keywords. In most cases, if the keyword already exist it will skip it except for GSP_FNAM, GSP_PATH and BUNIT. GSP_VERS: Goodman Spectroscopic Pipeline version number GSP_ONAM: Original File name GSP_PNAM: Parent file name or name of the file from which this one originated after some process or just a copy. GSP_FNAM: Current file name. GSP_PATH: Path to file at the moment of reading. GSP_TECH: Observing technique. Spectroscopy or Imaging. GSP_DATE: Date of first reading. GSP_OVER: Overscan region. GSP_TRIM: Trim section (region). GSP_SLIT: Slit trim section, obtained from the slit illuminated area. GSP_BIAS: Master bias image used. Default none. GSP_FLAT: Master flat image used. Default none. GSP_SCTR: Science target file GSP_NORM: Flat normalization method. GSP_COSM: Cosmic ray rejection method. GSP_EXTR: Extraction window at first column GSP_BKG1: First background extraction zone GSP_BKG2: Second background extraction zone GSP_WRMS: Wavelength solution RMS Error. GSP_WPOI: Number of points used to calculate the wavelength solution Error. GSP_WREJ: Number of points rejected.- Parameters
full_path (str) – Full path to file.
technique (str) – Observing technique. ‘Imaging’ or ‘Spectroscopy’.
- Returns
- Instance of
CCDData
corresponding to the file from full_path.
- Instance of
-
goodman_pipeline.core.core.
recenter_broad_lines
(lamp_data, lines, order)¶ Recenter broad lines
Notes
This method is used to recenter broad lines only, there is a special method for dealing with narrower lines.
- Parameters
lamp_data (ndarray) – numpy.ndarray instance. It contains the lamp data.
lines (list) – A line list in pixel values.
order (float) – A rough estimate of the FWHM of the lines in pixels in the data. It is calculated using the slit size divided by the pixel scale multiplied by the binning.
- Returns
A list containing the recentered line positions.
-
goodman_pipeline.core.core.
recenter_lines
(data, lines, plots=False)¶ Finds the centroid of an emission line
For every line center (pixel value) it will scan left first until the data stops decreasing, it assumes it is an emission line and then will scan right until it stops decreasing too. Defined those limits it will use the line data in between and calculate the centroid.
Notes
This method is used to recenter relatively narrow lines only, there is a special method for dealing with broad lines.
- Parameters
data (ndarray) – numpy.ndarray instance. or the data attribute of a
CCDData
instance.lines (list) – A line list in pixel values.
plots (bool) – If True will plot spectral line as well as the input center and the recentered value.
- Returns
A list containing the recentered line positions.
-
goodman_pipeline.core.core.
record_trace_information
(ccd, trace_info)¶ Adds trace information to fits header
Notes
Example of trace_info. OrderedDict([(‘GSP_TMOD’, [‘Polynomial1D’, ‘Model name used to fit trace’]),
(‘GSP_TORD’, [2, ‘Degree of the model used to fit target trace’]), (‘GSP_TC00’, [80.92244303468138, ‘Parameter c0’]), (‘GSP_TC01’, [0.0018921968204536187, ‘Parameter c1’]), (‘GSP_TC02’, [-7.232545448865748e-07, ‘Parameter c2’]), (‘GSP_TERR’, [0.18741058188097284, ‘RMS error of target trace’])])
- Parameters
ccd (CCDData) – CCDData instance to have trace info recorded into its
header. –
trace_info (OrderedDict) – Ordered Dictionary with a set of fits keywords
to a list of values corresponding to value and comment. (associated) –
- Returns
Same CCDData instance with the header modified.
- Return type
ccd (CCDData)
-
goodman_pipeline.core.core.
save_extracted
(ccd, destination, prefix='e', target_number=1)¶ Save extracted spectrum while adding a prefix.
- Parameters
ccd (CCDData) –
destination (str) – Path where the file will be saved.
prefix (str) – Prefix to be added to images. Default e.
target_number (int) –
- Returns
CCDData
instance of the image just recorded.although is not really necessary.
-
goodman_pipeline.core.core.
search_comp_group
(object_group, comp_groups, reference_data)¶ Search for a suitable comparison lamp group
In case a science target was observed without comparison lamps, usually right before or right after, this function will look for a compatible set obtained at a different time or pointing.
Notes
This methodology is not recommended for radial velocity studies.
- Parameters
object_group (DataFrame) – A
DataFrame
instances containing a group of images for a given scientific target.comp_groups (list) – A list in which every element is a
DataFrame
that contains information regarding groups of comparison lamps.reference_data (ReferenceData) – Instance of goodman.pipeline.core.ReferenceData contains all information related to the reference lamp library.
Returns:
-
goodman_pipeline.core.core.
setup_logging
(debug=False, generic=False)¶ configures logging
Notes
Logging file name is set to default ‘goodman_log.txt’. If –debug is activated then the format of the message is different.
-
goodman_pipeline.core.core.
trace
(ccd, model, trace_model, model_fitter, sampling_step, nfwhm=1, plots=False)¶ Find the trace of a spectrum
This function is called by the trace_targets function, the difference is that it only takes single models only not CompoundModels so this function is called for every single target. CompoundModels are a bit tricky when you need each model separated so all CompoundModels have been removed.
Notes
This method forces the trace to go withing a rectangular region of center model.mean.value and width 2 * nsigmas, this is for allowing the tracing of low SNR targets. The assumption is valid since the spectra are always well aligned to the detectors’s pixel columns. (dispersion axis)
- Parameters
ccd (CCDData) –
model (Model) – An astropy.modeling.Model instance that contains information regarding the target to be traced.
trace_model (object) – An astropy.modeling.Model instance, usually a low order polynomial.
model_fitter (Fitter) – An astropy.modeling.fitting.Fitter instance. Will fit the sampled points to construct the trace model
sampling_step (int) – Step for sampling the spectrum.
nfwhm (int) – Number of fwhm to each side of the mean to be used for searching the trace.
plots (bool) – Toggles debugging plot
- Returns
An astropy.modeling.Model instance, that defines the trace of the spectrum.
-
goodman_pipeline.core.core.
trace_targets
(ccd, target_list, sampling_step=5, pol_deg=2, nfwhm=5, plots=False)¶ Find the trace of the target’s spectrum on the image
This function defines a low order polynomial that trace the location of the spectrum. The attributes pol_deg and sampling_step define the polynomial degree and the spacing in pixels for the samples. For every sample a gaussian model is fitted and the center (mean) is recorded and since spectrum traces vary smoothly this value is used as a new center for the base model used to fit the spectrum profile.
Notes
This doesn’t work for extended sources. Also this calls for the function trace for doing the actual trace, the difference is that this method is at a higher level.
- Parameters
ccd (CCDData) –
target_list (list) – List of single target profiles.
sampling_step (int) – Frequency of sampling in pixels
pol_deg (int) – Polynomial degree for fitting the trace
plots (bool) – If True will show plots (debugging)
nfwhm (int) – Number of fwhm from spatial profile center to search for
target. default 10. (a) –
- Returns
- List that contains traces that are
astropy.modeling.Model instance
- Return type
all_traces (list)
-
goodman_pipeline.core.core.
validate_ccd_region
(ccd_region, regexp='^\\[\\d*:\\d*, \\d*:\\d*\\]$')¶
-
goodman_pipeline.core.core.
write_fits
(ccd, full_path, combined=False, parent_file=None, overwrite=True)¶ Write fits while adding information to the header.
This is a wrapper for allowing to save files while being able to add information into the header. Mostly for historical reasons.
- Parameters
ccd (CCDData) – to fits.
full_path (str) – Full path of file.
combined (bool) – True if ccd is the result of combining images.
parent_file (str) – Name of the file from which ccd originated. If combined is True this will be set to combined.
overwrite (bool) – Overwrite files, default True.
- Returns
CCDData
instance.