goodman_pipeline.core package#

Submodules#

goodman_pipeline.core.check_version module#

v1.0.0

  • First public release.

goodman_pipeline.core.check_version.am_i_updated(version: str) → bool#
goodman_pipeline.core.check_version.get_last(github_api_token: str = 'GITHUB_ACCESS_TOKEN') → Version#

Returns the version of the last release on GitHub.

Parameters:

  • (str (github_api_token) – holding the github access token for the API

  • optional) (Name of the environment variable) – holding the github access token for the API

Returns:

version (object)

Return type:

A pkg_resources.extern.packaging.version.Version the last version of the pipeline.

goodman_pipeline.core.core module#

class goodman_pipeline.core.core.GenerateDcrParFile(par_file_name='dcr.par')#

Bases: object

Creates dcr.par file based on lookup table

dcr parameters depend heavily on binning, this class generates a file using the default format. The lookup table considers camera and binning.

class goodman_pipeline.core.core.IdentifySpectroscopicTargets#

Bases: object

filter_peaks(spatial_profile=None, detected_peaks=None, nfind=None, background_threshold=None, file_name=None, plots=False)#
Parameters:

  • spatial_profile

  • detected_peaks

  • nfind

  • background_threshold

  • file_name

  • plots

Returns:

fit_background(spatial_profile=None, file_name=None, plots=False)#
Parameters:

  • spatial_profile

  • file_name (String)

  • plots

Returns:

fit_model(spatial_profile=None, selected_peaks=None, order=None, model_name=None, file_name=None, plots=False)#
get_peaks(spatial_profile=None, order=None, file_name=None, plots=False)#
Parameters:

  • spatial_profile – Background subtracted profile

  • order

  • file_name

  • plots

Returns:

subtract_background(spatial_profile=None, background_model=None, file_name=None, plots=False)#
Parameters:

  • spatial_profile

  • background_model

  • file_name

  • plots

Returns:

class goodman_pipeline.core.core.NightDataContainer(path, instrument, technique)#

Bases: object

This class is designed to be the organized data container. It doesn’t store image data but a list of DataFrame objects. Also it stores critical variables such as sunrise and sunset times.

add_bias(bias_group)#

Adds a bias group

Parameters:

bias_group (DataFrame) – A DataFrame Contains a set of keyword values of grouped image metadata

add_comp_group(comp_group)#

Adds a comp-only group

All comparison lamps groups are added here. The ones that may have been taken in the afternoon (isolated) or along science target. This will act as a pool of comparison lamp groups for eventual science targets taken without comparison lamps.

Parameters:

comp_group (DataFrame) – A DataFrame Contains a set of keyword values of grouped image metadata

add_data_group(data_group)#

Adds a data group

Parameters:

data_group (DataFrame) – A DataFrame Contains a set of keyword values of grouped image metadata

add_day_flats(day_flats)#

“Adds a daytime flat group

Parameters:

day_flats (DataFrame) – A DataFrame Contains a set of keyword values of grouped image metadata

add_object_group(object_group)#

Adds a object-only group

Parameters:

object_group (DataFrame) – A DataFrame Contains a set of keyword values of grouped image metadata

add_spec_group(spec_group)#

Adds a data group containing object and comp

The comparison lamp groups are also added to a general pool of comparison lamps.

Parameters:

spec_group (DataFrame) – A DataFrame Contains a set of keyword values of grouped image metadata

data_groups#

For spectroscopy use

is_empty#

For imaging use

set_readout(gain, rdnoise, roi)#

Set Gain, Read noise and ROI.

Parameters:

  • gain (float) – Gain from header

  • rdnoise (float) – Read noise from header.

  • roi (str) – ROI from header.

set_sun_times(sun_set, sun_rise)#

Sets values for sunset and sunrise

Parameters:

  • sun_set (str) – Sun set time in the format ‘YYYY-MM-DDTHH:MM:SS.SS’

  • sun_rise (str) – Sun rise time in the format ‘YYYY-MM-DDTHH:MM:SS.SS’

set_twilight_times(evening, morning)#

Sets values for evening and morning twilight

Parameters:

  • evening (str) – Evening twilight time in the format ‘YYYY-MM-DDTHH:MM:SS.SS’

  • morning (str) – Morning twilight time in the format ‘YYYY-MM-DDTHH:MM:SS.SS’

spec_groups#

Time reference points

exception goodman_pipeline.core.core.NoMatchFound(message='No match found')#

Bases: Exception

Exception for when no match is found.

exception goodman_pipeline.core.core.NoTargetException#

Bases: Exception

Exception to be raised when no target is identified

exception goodman_pipeline.core.core.NotEnoughLinesDetected#

Bases: Exception

Exception for when there are no lines detected.

class goodman_pipeline.core.core.ReferenceData(reference_dir)#

Bases: object

Contains spectroscopic reference lines values and filename to templates.

This class stores:

  • file names for reference fits spectrum

  • file names for CSV tables with reference lines and relative intensities

  • line positions only for the elements used in SOAR comparison lamps

check_comp_group(comp_group)#

Check if comparison lamp group has matching reference lamps

Parameters:

comp_group (DataFrame) – A DataFrame instance that contains meta-data for a group of comparison lamps.

Returns:

get_reference_lamp(header)#

Finds a suitable template lamp from the catalog

Parameters:

header (Header) – FITS header of image we are looking a reference lamp.

Returns:

full path to best matching reference lamp.

lamp_exists(header)#

Checks whether a matching lamp exist or not

Parameters:

  • object_name (str) – Name of the lamp from ‘OBJECT’ keyword.

  • grating (str) – Grating from ‘GRATING’ keyword.

  • grt_targ (float) – Grating target from keyword ‘GRT_TARG’.

  • cam_targ (float) – Camera target from keyword ‘CAM_TARG’.

Returns:

True of False depending if a single matching lamp exist.

Raises:

NotImplementedError if there are more than one lamp found. –

class goodman_pipeline.core.core.SaturationValues(ccd=None)#

Bases: object

Contains a complete table of readout modes and 50% half well

get_saturation_value(ccd)#

Defines the saturation_threshold level

Parameters:

ccd (CCDData) – Image to be tested for saturation_threshold

Returns:

The saturation_threshold value or None

property saturation_value#

Saturation value in counts

In fact the value it returns is the 50% of full potential well, Some configurations reach digital saturation_threshold before 50% of full potential well, they are specified in the last column: saturates_before.

Returns:

None if the value has not been defined

class goodman_pipeline.core.core.SpectroscopicMode#

Bases: object

get_cam_grt_targ_angle(grating, mode)#

Get the camera and grating target values grating and mode

Parameters:

  • grating (float) – Grating frequency in lines/mm (unitless value)

  • mode (str) – Name of the grating’s mode for which the camera and grating target values are required.

Returns:

Camera and grating target values. None and None if no such values exists.

get_mode(grating, camera_targ, grating_targ, blocking_filter)#

Get the camera’s optical configuration mode.

This method is useful for data that does not have the WAVMODE keyword

Parameters:

  • grating (str) – Grating frequency as string

  • camera_targ (str) – Camera target angle as in the header.

  • grating_targ (str) – Grating target angle as in the header.

  • blocking_filter (str) – Order blocking filter listed on the header.

Returns:

string that defines the wavelength mode used

goodman_pipeline.core.core.add_linear_wavelength_solution(ccd, x_axis, reference_lamp, crpix=1)#

Add wavelength solution to the new FITS header

Defines FITS header keyword values that will represent the wavelength solution in the header so that the image can be read in any other astronomical tool. (e.g. IRAF)

Parameters:

  • ccd (CCDData)

  • x_axis (ndarray) – Linearized x-axis in angstrom

  • reference_lamp (str) – Name of lamp used to get wavelength solution.

  • crpix (int) – reference pixel for defining wavelength solution.

  • fine. (Default 1. For most cases 1 should be)

Returns:

ccd (CCDData) A CCDData instance with

linear wavelength solution on it.

goodman_pipeline.core.core.add_wcs_keys(ccd)#

Adds generic keyword for linear wavelength solution to the header

Linear wavelength solutions require a set of standard fits keywords. Later on they will be updated accordingly. The main goal of putting them here is to have consistent and nicely ordered headers.

Notes

This does NOT add a WCS solution, just the keywords.

Parameters:

ccd (CCDData) – class:~astropy.nddata.CCDData` instance with no wcs keywords.

Returns:

ccd (CCDData) A CCDData instance with modified

header with added WCS keywords

goodman_pipeline.core.core.astroscrappy_lacosmic(ccd, red_path=None, save_mask=False)#
goodman_pipeline.core.core.bias_subtract(ccd, master_bias, master_bias_name)#

Subtract bias from file.

Wrapper for subtract_bias(). The main goal is to have a consistent API for apps using the Goodman Pipeline as a library.

Parameters:

  • ccd (CCDData) – A file to be bias-subtracted

  • master_bias (CCDData)

  • master_bias_name (str) – Full path to master bias file, this is added to

  • GSP_BIAS. (the bias-subtracted ccd under)

Returns:

A bias-subtracted file.

goodman_pipeline.core.core.bin_reference_data(wavelength, intensity, serial_binning)#

Bins a 1D array

This method reduces the size of an unbinned array by binning. The function to combine data is numpy.mean.

Parameters:

  • wavelength (array) – Wavelength axis

  • intensity (array) – Intensity

  • serial_binning (int) – Serial Binning is the binning in the

  • axis. (dispersion)

Returns:

Binned wavelength and intensity arrays.

goodman_pipeline.core.core.call_cosmic_rejection(ccd, image_name, out_prefix, red_path, keep_files=False, prefix='c', method='dcr', save=False)#

Call for the appropriate cosmic ray rejection method

There are four options when dealing with cosmic ray rejection in this pipeline, The default option is called default and it will choose the rejection method based on the binning of the image. Note that there are only two real methods: dcr and lacosmic.

For binning 1x1 the choice will be dcr for binning 2x2 and 3x3 will be lacosmic.

The method dcr is a program written in C by Wojtek Pych (http://users.camk.edu.pl/pych/DCR/) that works very well for spectroscopy the only negative aspect is that integration with python was difficult and not natively (through subprocess).

The method lacosmic is well known but there are different implementations, we started using cosmicray_lacosmic() but later we shifted towards astroscrappy.detect_cosmics. The LACosmic method was developed by Pieter G. van Dokkum. See <http://www.astro.yale.edu/dokkum/lacosmic/>

There is also the option of skipping cosmic ray removal by using none.

Parameters:

  • ccd (CCCData) – a CCDData instance.

  • image_name (str) – Science image name.

  • out_prefix (str) – Partial prefix to be added to the image name. Related to previous processes and not cosmic ray rejection.

  • red_path (str) – Path to reduced data directory.

  • keep_files (bool) – If True, the original file and the cosmic ray mask will not be deleted. Default is False.

  • prefix (str) – Cosmic ray rejection related prefix to be added to image name.

  • method (str) – Method to use for cosmic ray rejection. There are four options: default, dcr, lacosmic and none.

  • save (bool) – Disables by default saving the images

Returns:

CCDData instance and out_prefix which is the

prefix added to the image name.

Raises:

  • NotImplementedError if the method argument is not dcr, lacosmic –

  • nor none. –

goodman_pipeline.core.core.classify_spectroscopic_data(path, search_pattern)#

Classify data by grouping them by a set of keywords.

This function uses ImageFileCollection. First it creates a collection of information regarding the images located in path that match the pattern search_pattern. The information obtained are all keywords listed in the list keywords. The ImageFileCollection object is translated into DataFrame and then is used much like an SQL database to select and filter values and in that way put them in groups that are DataFrame instances. The keywords retrieved are: - date - slit - date-obs - obstype - object - exptime - obsra - obsdec - grating - cam_targ - grt_targ - filter - filter2 - gain - rdnoise.

Then all data is grouped by matching the following keywords:

  • slit

  • radeg

  • decdeg

  • grating

  • cam_targ

  • grt_targ

  • filter

  • filter2

  • gain

  • rdnoise

And finally, every group is classified as: a comparison lamp-only group, an object-only group or a group of object and comparison lamps. The comparison lamps present in the last group (COMP + OBJECT) are also added in the first one (COMP-only).

Parameters:

  • path (str) – Path to data location

  • search_pattern (str) – Prefix to match files.

Returns:

Instance of goodman_pipeline.core.core.NightDataContainer

goodman_pipeline.core.core.combine_data(image_list, dest_path, prefix=None, output_name=None, method='median', save=False)#

Combine a list of CCDData instances.

Parameters:

  • image_list (list) – Each element should be an instance of CCDData

  • dest_path (str) – Path to where the new image should saved

  • prefix (str) – Prefix to add to the image file name

  • output_name (str) – Alternatively a file name can be parsed, this will ignore prefix.

  • method (str) – Method for doing the combination, this goes straight to the call of ccdproc.combine function.

  • save (bool) – If True will save the combined images. If False it will ignore prefix or output_name.

Returns:

A combined image as a CCDData object.

goodman_pipeline.core.core.convert_time(in_time)#

Converts time to seconds since epoch

Parameters:

in_time (str) – time obtained from header’s keyword DATE-OBS

Returns:

time in seconds since epoch

goodman_pipeline.core.core.create_master_bias(bias_files, raw_data, reduced_data, technique)#

Create Master Bias

Given a DataFrame object that contains a list of compatible bias. This function creates the master bias using ccdproc.combine using median and 3-sigma clipping.

Parameters:

  • bias_files (list) – List of all bias files to be combined. They have

  • method. (to be compatible with each other as no check is done in this)

  • raw_data (str) – Full path to raw data location.

  • reduced_data (str) – Full path to were reduced data will reside.

  • technique (str) – Name of observing technique. Imaging or

  • Spectroscopy.

Returns:

master_bias_name (str):

Return type:

master_bias (object)

goodman_pipeline.core.core.create_master_flats(flat_files, raw_data, reduced_data, technique, overscan_region, trim_section, master_bias_name, new_master_flat_name, saturation_threshold, ignore_bias=False)#

Creates master flats

Using a list of compatible flat images it combines them using median and 1-sigma clipping. Also it apply all previous standard calibrations to each image.

Parameters:

  • flat_files (list) – List of files previously filtered, there is no

  • are (compatibility check in this function and is assumed the files)

  • combinables.

  • raw_data (str) – Full path to raw data.

  • reduced_data (str) – Full path to reduced data. Where reduced data

  • stored. (should be)

  • technique (str) – Observing technique. Imaging or Spectroscopy.

  • overscan_region (str) – Defines the area to be used to estimate the

  • format. (overscan region for overscan correction. Should be in the)

  • `[x1 –

    x2.,y1:y2]`.

  • trim_section (str) – Defines the area to be used after trimming

  • parts (unusable selected) – x2.,y1:y2]`.

  • master_bias_name (str) – Master bias name, can be full path or not.

  • path (If it is a relative)

  • define (the path will be ignored and will)

  • basename. (the full path as raw_path +)

  • new_master_flat_name (str) – Name of the file to save new master

  • not. (flat. Can be absolute path or)

  • saturation_threshold (int) – Saturation threshold, defines the percentage of

  • images. (pixels above saturation level allowed for flat field)

  • ignore_bias (bool) – Flag to create master bias without master bias.

Returns:

The master flat CCDData instance and the name of under which the master flat was stored. If it can’t build the master flat it will return None, None.

goodman_pipeline.core.core.cross_correlation(reference, compared, slit_size, serial_binning, mode='full', plot=False)#

Do cross correlation of two 1D spectra

It convolves the reference lamp depending on the slit size of the new_array that corresponds with a comparison lamp. If the slit is larger than 3 arcseconds the reference lamp is convolved with a ~astropy.convolution.Box1DKernel because spectral lines look more like a block than a line. And if it is smaller or equal to 3 it will use a ~astropy.convolution.Gaussian1DKernel ponderated by the binning. All reference lamp are unbinned, or binning is 1x1.

Parameters:

  • reference (array) – Reference array.

  • compared (array) – Array to be matched. A new reference lamp.

  • slit_size (float) – Slit width in arcseconds

  • serial_binning (int) – Binning in the spectral axis

  • mode (str) – Correlation mode for signal.correlate.

  • plot (bool) – Switch debugging plots on or off.

Returns:

Shift value in pixels.

Return type:

correlation_value (int)

goodman_pipeline.core.core.dcr_cosmicray_rejection(data_path, in_file, prefix, keep_cosmic_files=False, save=True)#

Runs an external code for cosmic ray rejection

DCR was created by Wojtek Pych and the code can be obtained from http://users.camk.edu.pl/pych/DCR/ and is written in C. Contrary to ccdproc’s LACosmic it actually applies the correction, and also doesn’t update the mask attribute since it doesn’t work with CCDData instances.

The binary takes three positional arguments, they are: 1. input image, 2. output image and 3. cosmic rays image. Also it needs that a dcr.par file is located in the directory. All this is implemented in this function, if delete is True it will remove the original image and the cosmic rays image. The removal of the original image is absolutely safe when used in the context of the goodman pipeline, however if you want to implement it somewhere else, be careful.

Notes

This function operates an external code therefore it doesn’t return anything natively, instead it creates a new image. A workaround has been created that loads the new image and deletes the file.

Parameters:

  • data_path (str) – Data location

  • in_file (str) – Name of the file to have its cosmic rays removed

  • prefix (str) – Prefix to add to the file with the cosmic rays removed

  • keep_cosmic_files (bool) – True for deleting the input and cosmic ray file.

  • save (bool) – Toggles the option of saving the image.

goodman_pipeline.core.core.define_trim_section(sample_image, technique)#

Get the initial trim section

The initial trim section is usually defined in the header with the keyword TRIMSEC but in the case of Goodman HTS this does not work well. In particular for spectroscopy where is more likely to have combined binning and so on.

Parameters:

  • sample_image (str) – Full path to sample image.

  • technique (str) – The name of the technique, the options are: Imaging or Spectroscopy.

Returns:

The trim section in the format [x1:x2, y1:y2]

goodman_pipeline.core.core.evaluate_wavelength_solution(clipped_differences)#

Calculates Root Mean Square Error for the wavelength solution.

Parameters:

clipped_differences (ndarray) – Numpy masked array of differences between reference line values in angstrom and the value calculated using the model of the wavelength solution.

Returns:

Root Mean Square Error, number of points and number of points

rejected in the calculation of the wavelength solution.

goodman_pipeline.core.core.extract_fractional_pixel(ccd, target_trace, target_fwhm, extraction_width, background_spacing=3)#

Performs a spectrum extraction using fractional pixels.

Parameters:

  • ccd (CCDData)

  • spectrum. (contains a 2D)

  • target_trace (object) – Instance of astropy.modeling.models.Model that defines the trace of the target on the image (ccd).

  • target_fwhm (float) – FWHM value for the spatial profile fitted to the target.

  • extraction_width (int) – Width of the extraction area as a function of target_fwhm. For instance if extraction_with is set to 1 the function extract 0.5 to each side from the center of the traced target.

  • background_spacing (float) – Number of target_stddev to separate the target extraction to the background. This is from the edge of the extraction zone to the edge of the background region.

goodman_pipeline.core.core.extract_optimal()#

Placeholder for optimal extraction method.

Raises:

NotImplementedError –

goodman_pipeline.core.core.extraction(ccd, target_trace, spatial_profile, extraction_name)#

Calls appropriate spectrum extraction routine

This function calls the appropriate extraction function based on extraction_name

Notes

Optimal extraction is not implemented.

Parameters:

  • ccd (CCDData) – Instance of CCDData containing a 2D spectrum

  • target_trace (object) – Instance of astropy.modeling.Model, a low order polynomial that defines the trace of the spectrum in the ccd object.

  • spatial_profile (Model) – Instance of Model, a Gaussian model previously fitted to the spatial profile of the 2D spectrum contained in the ccd object.

  • extraction_name (str) – Extraction type, can be fractional or optimal though the optimal extraction is not implemented yet.

Returns:

Instance of CCDData containing a 1D spectrum. The attribute ‘data’ is replaced by the 1D array resulted from the extraction process.

Return type:

ccd (CCDData)

Raises:

NotImplementedError – When extraction_name is optimal.

goodman_pipeline.core.core.fix_keywords(path, pattern='*.fits')#

Fix FITS uncompliance of some keywords

Uses automatic header fixing by CCDData. Note that this only fixes FITS compliance.

Parameters:

  • path (str) – Path to raw data

  • pattern (str) – Search pattern for listing file in path.

goodman_pipeline.core.core.fractional_sum(data, index, low_limit, high_limit)#

Performs a fractional pixels sum

A fractional pixels sum is required several times while extracting a 1D spectrum from a 2D spectrum. The method is actually very simple.

It requires the full data, the column and the range to sum, this range is given as real numbers. First it separates the limits values as an integer and fractional parts. Then it will sum the integer’s interval and subtract the low_limit’s fractional part and sum the high_limit’s fractional part.

The sum is performed in one operation. It does not do background subtraction, for which this very same method is used to get the background sum to be subtracted later.

Parameters:

  • data (numpy.ndarray) – 2D array that contains the 2D spectrum/image

  • index (int) – Index of the column to be summed.

  • low_limit (float) – Lower limit for the range to be summed.

  • high_limit (float) – Higher limit for the range to be summed.

Returns:

Sum in ADU of all pixels and fractions between low_limit and high_limit.

goodman_pipeline.core.core.get_best_flat(flat_name, path)#

Look for matching master flat

Given a basename for master flats defined as a combination of key parameters extracted from the header of the image that we want to flat field, this function will find the name of the files that matches the base name and then will choose the first. Ideally this should go further as to check signal, time gap, etc. After it identifies the file it will load it using CCDData and return it along the filename. In the case it fails it will return None instead of master_flat and another None instead of master_flat_name.

Parameters:

  • flat_name (str) – Full path of master flat basename. Ends in ‘*.fits’ for using glob.

  • path (str) – Location to look for flats.

Returns:

A CCDData instance. master_flat_name (str): Full path to the chosen master flat.

Return type:

master_flat (object)

goodman_pipeline.core.core.get_central_wavelength(grating, grt_ang, cam_ang)#

Calculates the central wavelength for a given spectroscopic mode

The equation used to calculate the central wavelength is the following

\[\lambda_{central} = \frac{1e6}{GRAT} \sin\left(\frac{\alpha \pi}{180}\right) + \sin\left(\frac{\beta \pi}{180}\right)\]
Parameters:

  • grating (str) – Grating frequency as a string. Example ‘400’.

  • grt_ang (str) – Grating Angle as a string. Example ‘12.0’.

  • cam_ang (str) – Camera Angle as a string. Example ‘20.0’

Returns:

Central wavelength as a float value.

Return type:

central_wavelength (float)

goodman_pipeline.core.core.get_lines_in_lamp(ccd, peak_percent_for_threshold=3, plots=False)#

Identify peaks in a lamp spectrum

Uses signal.argrelmax to find peaks in a spectrum i.e emission lines, then it calls the recenter_lines method that will recenter them using a “center of mass”, because, not always the maximum value (peak) is the center of the line.

The detection threshold is calculated as follows:

threshold = minimum + (peak_percent_for_threshold / 100.) * maximum

Parameters:

  • ccd (CCDData) – Lamp ccdproc.CCDData instance.

  • peak_percent_for_threshold (float) – Percent of peak intensity above the minimum to use as detection threshold.

  • plots (bool) – Wether to plot or not.

Returns:

A common list containing pixel values at

approximate location of lines.

Return type:

lines_candidates (list)

goodman_pipeline.core.core.get_overscan_region(sample_image, technique)#

Get the right overscan region for spectroscopy

It works for the following ROI:

Spectroscopic 1x1 Spectroscopic 2x2 Spectroscopic 3x3

The limits where measured on a Spectroscopic 1x1 image and then divided by the binning size. This was checked that it actually works as expected.

Notes

The regions are 1-based i.e. different to Python convention. For Imaging there is no overscan region.

Parameters:

  • sample_image (str) – Full path to randomly chosen image.

  • technique (str) – Observing technique, either Spectroscopy or

  • Imaging

Returns:

overscan_region (str) Region for overscan in the format
’[min:max,:]’ where min is the starting point and max is the end

point of the overscan region.

goodman_pipeline.core.core.get_slit_trim_section(master_flat, debug_plots=False)#

Find the slit edges to trim all data

Using a master flat, ideally with good signal to noise ratio, this function will identify the edges of the slit projected into the detector. Having this done will allow to reduce the overall processing time and also reduce the introduction of artifacts due to non-illuminated regions in the detectors, such as NaNs -INF +INF, etc.

Parameters:

  • master_flat (CCDData) – A CCDData instance.

  • debug_plots (bool) – Flag to show debugging plots

Returns:

Trim section in spatial direction in the format

[:,slit_lower_limit:slit_higher_limit]

Return type:

slit_trim_section (str)

goodman_pipeline.core.core.get_spectral_characteristics(ccd, pixel_size, instrument_focal_length)#

Calculates some Goodman’s specific spectroscopic values.

From the header value for Grating, Grating Angle and Camera Angle it is possible to estimate what are the wavelength values at the edges as well as in the center. It was necessary to add offsets though, since the formulas provided are slightly off. The values are only an estimate.

Parameters:

  • ccd (CCDData) – Lamp ccdproc.CCDData instance

  • pixel_size (float) – Pixel size in microns

  • instrument_focal_length (float) – Instrument focal length

Returns:

Contains the following parameters:

center: Center Wavelength blue: Blue limit in Angstrom red: Red limit in Angstrom alpha: Angle beta: Angle pix1: Pixel One pix2: Pixel Two

Return type:

spectral_characteristics (dict)

goodman_pipeline.core.core.get_twilight_time(date_obs)#

Get end/start time of evening/morning twilight

Notes

Taken from David Sanmartim’s development

Parameters:

date_obs (list) – List of all the dates from data.

Returns:

Evening twilight time in the format

’YYYY-MM-DDTHH:MM:SS.SS’

twilight_morning (str): Morning twilight time in the format

’YYYY-MM-DDTHH:MM:SS.SS’

sun_set_time (str): Sun set time in the format

’YYYY-MM-DDTHH:MM:SS.SS’

sun_rise_time (str): Sun rise time in the format

’YYYY-MM-DDTHH:MM:SS.SS’

Return type:

twilight_evening (str)

goodman_pipeline.core.core.identify_targets(ccd, fit_model, background_threshold, nfind=3, profile_min_width=None, profile_max_width=None, plots=False)#

Identify Spectroscopic Targets

Wrapper to the class IdentifySpectroscopicTargets.

Parameters:

  • ccd (CCDData) – Image containing spectra

  • fit_model (str) – Name of the model to be fitted moffat or gaussian.

  • background_threshold (int) – Number of background levels for target

  • discrimination.

  • nfind (int) – Maximum number of targets passing the background threshold

  • returned (to be)

  • intense. (they are order from most intense peak to least)

  • profile_min_width (float) – Minimum FWHM (moffat) or STDDEV (gaussian) for spatial profile model.

  • profile_max_width (float) – Maximum FWHM (moffat) or STDDEV (gaussian) for spatial profile model.

  • plots (bool) – Flat for plotting results.

Returns:

List of models successfully fitted.

Return type:

identified_targets (list)

goodman_pipeline.core.core.identify_technique(target, obstype, slit, grating, wavmode, roi)#

Identify whether is Imaging or Spectroscopic data

Parameters:

  • target (str) – Target name as in the keyword OBJECT this is useful in

  • mode (Automated aquisition)

  • AEON. (such as)

  • obstype (str) – Observation type as in OBSTYPE

  • slit (str) – Value of SLIT keyword.

  • grating (str) – Value of GRATING keyword.

  • wavmode (str) – Value of WAVMODE keyword.

  • roi (str) – Value of ROI keyword.

Returns:

Observing technique as a string. Either Imaging or Spectroscopy.

goodman_pipeline.core.core.image_overscan(ccd, overscan_region, add_keyword=False)#

Apply overscan correction to data

Uses ccdproc.subtract_overscan to perform the task.

Notes

The overscan_region argument uses FITS convention, just like IRAF, therefore is 1 based. i.e. it starts in 1 not 0.

Parameters:

  • ccd (CCDData) – overscan corrected.

  • overscan_region (str) – The overscan region in the format [x1:x2,y1:y2] where x is the spectral axis and y is the spatial axis.

  • add_keyword (bool) – Tells ccdproc whether to add a keyword or not. Default False.

Returns:

ccd (CCDData) Overscan corrected CCDData

instance

goodman_pipeline.core.core.image_trim(ccd, trim_section, trim_type='trimsec', add_keyword=False)#

Trim image to a given section

Notes

The overscan_region argument uses FITS convention, just like IRAF, therefore is 1 based. i.e. it starts in 1 not 0.

Parameters:

  • ccd (CCDData)

  • trim_section (str) – The trimming section in the format [x1:x2,y1:y2] where x is the spectral axis and y is the spatial axis.

  • trim_type (str) – trimsec or slit trim.

  • add_keyword (bool) – Tells ccdproc whether to add a keyword or not. Default False.

Returns:

ccd (CCDData) Trimmed CCDData instance

goodman_pipeline.core.core.interpolate_spectrum(spectrum, interpolation_size)#

Creates an interpolated version of the input spectrum

This method creates an interpolated version of the input array, it is used mainly for a spectrum but it can also be used with any unidimensional array. The reason for doing interpolation is that it allows to find the lines and its respective center more precisely.

Parameters:

  • spectrum (array) – an uncalibrated spectrum or any unidimensional array.

  • interpolation_size (int) – Number of points to interpolate. (points added between two existing ones)

Returns:

Two dimensional array containing x-axis and interpolated array.

The x-axis preserves original pixel values.

goodman_pipeline.core.core.is_file_saturated(ccd, threshold)#

Detects a saturated image

It counts the number of pixels above the saturation_threshold level, then finds which percentage they represents and if it is above the threshold it will return True. The percentage threshold can be set using the command line argument --saturation_threshold.

Parameters:

  • ccd (CCDData) – Image to be tested for saturation_threshold

  • threshold (float) – Percentage of saturated pixels allowed. Default 1.

Returns:

True for saturated and False for non-saturated

goodman_pipeline.core.core.linearize_spectrum(data, wavelength_solution, plots=False)#

Produces a linearized version of the spectrum

Storing wavelength solutions in a FITS header is not simple at all for non-linear solutions therefore is easier for the final user and for the development code to have the spectrum linearized. It first finds a spline representation of the data, then creates a linear wavelength axis (angstrom) and finally it resamples the data from the spline representation to the linear wavelength axis.

It also applies a median filter of kernel size three to smooth the linearized spectrum. Sometimes the splines produce funny things when the original data is too steep.

Parameters:

  • data (Array) – The non-linear spectrum

  • wavelength_solution (object) – Mathematical model representing the

  • solution. (wavelength)

  • plots (bool) – Whether to show the plots or not.

Returns:

Contains two elements: Linear wavelength axis and the smoothed linearized data itself.

Return type:

linear_data (list)

goodman_pipeline.core.core.name_master_flats(header, technique, reduced_data, sun_set, sun_rise, evening_twilight, morning_twilight, target_name='', get=False)#

Defines the name of a master flat or what master flat is compatible with a given data

Given the header of a flat image this method will look for certain keywords that are unique to a given instrument configuration therefore they are used to discriminate compatibility.

It can be used to define a master flat’s name when creating it or find a base name to match existing master flat files thus finding a compatible one for a given non-flat image.

Parameters:

header (object) – Fits header. Instance of

:param Header: :param technique: Observing technique, either Spectroscopy or :type technique: str :param Imaging.: :param reduced_data: Full path to reduced data directory :type reduced_data: str :param sun_set: Sunset time formatted as “%Y-%m-%dT%H:%M:%S.%f” :type sun_set: str :param sun_rise: Sunrise time formatted as “%Y-%m-%dT%H:%M:%S.%f” :type sun_rise: str :param evening_twilight: End of evening twilight formatted as :type evening_twilight: str :param “%Y-%m-%dT%H: %M:%S.%f” :param morning_twilight: Start of morning twilight in the format :type morning_twilight: str :param “%Y-%m-%dT%H: %M:%S.%f” :param target_name: Optional science target name to be added to the

master flat name.

Parameters:

get (bool) – This option is used when trying to find a suitable master flat for a given data.

Returns:

A master flat name, or basename to find a match among existing files.

goodman_pipeline.core.core.normalize_master_flat(master, name, method='simple', order=15)#

Master flat normalization method

This function normalize a master flat in three possible ways:

mean: simply divide the data by its mean

simple: Calculates the median along the spatial axis in order to obtain the dispersion profile. Then fits a Chebyshev1D model and apply this to all the data.

full: This is for experimental purposes only because it takes a lot of time to process. It will fit a model to each line along the dispersion axis and then divide it by the fitted model. I do not recommend this method unless you have a good reason as well as a very powerful computer.

Parameters:

  • master (CCDData) – Master flat. Has to be a CCDData instance.

  • name (str) – Full path of master flat prior to normalization.

  • method (str) – Normalization method, ‘mean’, ‘simple’ or ‘full’.

  • order (int) – Order of the polynomial to be fitted.

Returns:

The normalized master flat.

CCDData instance.

Return type:

master (CCDData)

goodman_pipeline.core.core.ra_dec_to_deg(right_ascension, declination)#

Converts right ascension and declination to degrees

Parameters:

  • right_ascension (str) – Right ascension in the format hh:mm:ss.sss

  • declination (str) – Declination in the format dd:mm:ss.sss

Returns:

Right ascension in degrees declination_deg (float): Declination in degrees

Return type:

right_ascension_deg (float)

goodman_pipeline.core.core.read_fits(full_path, technique='Unknown')#

Read fits files while adding important information to the header

It is necessary to record certain data to the image header so that’s the reason for this wrapper of read() to exist. It will add the following keywords. In most cases, if the keyword already exist it will skip it except for GSP_FNAM, GSP_PATH and BUNIT. GSP_VERS: Goodman Spectroscopic Pipeline version number GSP_ONAM: Original File name GSP_PNAM: Parent file name or name of the file from which this one originated after some process or just a copy. GSP_FNAM: Current file name. GSP_PATH: Path to file at the moment of reading. GSP_TECH: Observing technique. Spectroscopy or Imaging. GSP_DATE: Date of first reading. GSP_OVER: Overscan region. GSP_TRIM: Trim section (region). GSP_SLIT: Slit trim section, obtained from the slit illuminated area. GSP_BIAS: Master bias image used. Default none. GSP_FLAT: Master flat image used. Default none. GSP_SCTR: Science target file GSP_NORM: Flat normalization method. GSP_COSM: Cosmic ray rejection method. GSP_EXTR: Extraction window at first column GSP_BKG1: First background extraction zone GSP_BKG2: Second background extraction zone GSP_WRMS: Wavelength solution RMS Error. GSP_WPOI: Number of points used to calculate the wavelength solution Error. GSP_WREJ: Number of points rejected.

Parameters:

  • full_path (str) – Full path to file.

  • technique (str) – Observing technique. ‘Imaging’ or ‘Spectroscopy’.

Returns:

Instance of CCDData corresponding to the file

from full_path.

goodman_pipeline.core.core.recenter_broad_lines(lamp_data, lines, order)#

Recenter broad lines

Notes

This method is used to recenter broad lines only, there is a special method for dealing with narrower lines.

Parameters:

  • lamp_data (ndarray) – numpy.ndarray instance. It contains the lamp data.

  • lines (list) – A line list in pixel values.

  • order (float) – A rough estimate of the FWHM of the lines in pixels in the data. It is calculated using the slit size divided by the pixel scale multiplied by the binning.

Returns:

A list containing the recentered line positions.

goodman_pipeline.core.core.recenter_lines(data, lines, plots=False)#

Finds the centroid of an emission line

For every line center (pixel value) it will scan left first until the data stops decreasing, it assumes it is an emission line and then will scan right until it stops decreasing too. Defined those limits it will use the line data in between and calculate the centroid.

Notes

This method is used to recenter relatively narrow lines only, there is a special method for dealing with broad lines.

Parameters:

  • data (ndarray) – numpy.ndarray instance. or the data attribute of a CCDData instance.

  • lines (list) – A line list in pixel values.

  • plots (bool) – If True will plot spectral line as well as the input center and the recentered value.

Returns:

A list containing the recentered line positions.

goodman_pipeline.core.core.record_trace_information(ccd, trace_info)#

Adds trace information to fits header

Notes

Example of trace_info. OrderedDict([(‘GSP_TMOD’, [‘Polynomial1D’, ‘Model name used to fit trace’]),

(‘GSP_TORD’, [2, ‘Degree of the model used to fit target trace’]), (‘GSP_TC00’, [80.92244303468138, ‘Parameter c0’]), (‘GSP_TC01’, [0.0018921968204536187, ‘Parameter c1’]), (‘GSP_TC02’, [-7.232545448865748e-07, ‘Parameter c2’]), (‘GSP_TERR’, [0.18741058188097284, ‘RMS error of target trace’])])

Parameters:

  • ccd (CCDData) – ccdproc.CCDData instance to have trace info recorded into its

  • header.

  • trace_info (OrderedDict) – Ordered Dictionary with a set of fits keywords

  • comment. (associated to a list of values corresponding to value and)

Returns:

Same ccdproc.CCDData instance with the header modified.

Return type:

ccd (CCDData)

goodman_pipeline.core.core.save_extracted(ccd, destination, prefix='e', target_number=1)#

Save extracted spectrum while adding a prefix.

Parameters:

  • ccd (CCDData)

  • destination (str) – Path where the file will be saved.

  • prefix (str) – Prefix to be added to images. Default e.

  • target_number (int) – Secuential number of spectroscopic target.

Returns:

CCDData instance of the image just recorded.

although is not really necessary.

goodman_pipeline.core.core.search_comp_group(object_group, comp_groups, reference_data)#

Search for a suitable comparison lamp group

In case a science target was observed without comparison lamps, usually right before or right after, this function will look for a compatible set obtained at a different time or pointing.

Notes

This methodology is not recommended for radial velocity studies.

Parameters:

  • object_group (DataFrame) – A DataFrame instances containing a group of images for a given scientific target.

  • comp_groups (list) – A list in which every element is a DataFrame that contains information regarding groups of comparison lamps.

  • reference_data (ReferenceData) – Instance of goodman.pipeline.core.ReferenceData contains all information related to the reference lamp library.

Returns:

goodman_pipeline.core.core.setup_logging(debug=False, generic=False)#

configures logging

Notes

Logging file name is set to default ‘goodman_log.txt’. If –debug is activated then the format of the message is different.

goodman_pipeline.core.core.trace(ccd, model, trace_model, model_fitter, sampling_step, nfwhm=1, plots=False)#

Find the trace of a spectrum

This function is called by the trace_targets function, the difference is that it only takes single models only not CompoundModels so this function is called for every single target. CompoundModels are a bit tricky when you need each model separated so all CompoundModels have been removed.

Notes

This method forces the trace to go withing a rectangular region of center model.mean.value and width 2 * nsigmas, this is for allowing the tracing of low SNR targets. The assumption is valid since the spectra are always well aligned to the detectors’s pixel columns. (dispersion axis)

Parameters:

  • ccd (CCDData)

  • model (Model) – An astropy.modeling.Model instance that contains information regarding the target to be traced.

  • trace_model (object) – An astropy.modeling.Model instance, usually a low order polynomial.

  • model_fitter (Fitter) – An astropy.modeling.fitting.Fitter instance. Will fit the sampled points to construct the trace model

  • sampling_step (int) – Step for sampling the spectrum.

  • nfwhm (int) – Number of fwhm to each side of the mean to be used for searching the trace.

  • plots (bool) – Toggles debugging plot

Returns:

An astropy.modeling.Model instance, that defines the trace of the spectrum.

goodman_pipeline.core.core.trace_targets(ccd, target_list, sampling_step=5, pol_deg=2, nfwhm=5, plots=False)#

Find the trace of the target’s spectrum on the image

This function defines a low order polynomial that trace the location of the spectrum. The attributes pol_deg and sampling_step define the polynomial degree and the spacing in pixels for the samples. For every sample a gaussian model is fitted and the center (mean) is recorded and since spectrum traces vary smoothly this value is used as a new center for the base model used to fit the spectrum profile.

Notes

This doesn’t work for extended sources. Also this calls for the function trace for doing the actual trace, the difference is that this method is at a higher level.

Parameters:

  • ccd (CCDData)

  • target_list (list) – List of single target profiles.

  • sampling_step (int) – Frequency of sampling in pixels

  • pol_deg (int) – Polynomial degree for fitting the trace

  • plots (bool) – If True will show plots (debugging)

  • nfwhm (int) – Number of fwhm from spatial profile center to search for

  • 10. (a target. default)

Returns:

List that contains traces that are

astropy.modeling.Model instance

Return type:

all_traces (list)

goodman_pipeline.core.core.validate_ccd_region(ccd_region, regexp='^\\[\\d*:\\d*,\\d*:\\d*\\]$')#
goodman_pipeline.core.core.write_fits(ccd, full_path, combined=False, parent_file=None, overwrite=True)#

Write fits while adding information to the header.

This is a wrapper for allowing to save files while being able to add information into the header. Mostly for historical reasons.

Parameters:

  • ccd (CCDData) – to fits.

  • full_path (str) – Full path of file.

  • combined (bool) – True if ccd is the result of combining images.

  • parent_file (str) – Name of the file from which ccd originated. If combined is True this will be set to combined.

  • overwrite (bool) – Overwrite files, default True.

Returns:

CCDData instance.

Module contents#

On this page

This Page