CAPTURE: A continuum imaging pipeline for the uGMRT

Radio interferometric data require post-processing that involves calibration and inverse Fourier transform to obtain an image of the sky. The Giant Metrewave Radio Telescope (GMRT), located near Pune, India, is a radio interferometer that consists of 30 antennas of 45 m diameter each, with the longest baseline $\sim 25$ km, that recently underwent an upgrade of the receivers. Before the upgrade it operated in 5 frequency bands in the range of 120 - 1450 MHz with instantaneous bandwidths of 33.3 MHz. The upgrade included broadbanding of receivers and backend which now allows recording data with instantaneous bandwidths of up to 400 MHz (Gupta et al., 2017). We will refer to the older GMRT with narrow band receivers as the “legacy GMRT” and the one after the upgrade as the upgraded GMRT (uGMRT). The uGMRT is currently operational in four frequency bands, namely, 120 - 250 MHz (band-2), 250 - 500 MHz (band-3), 550 - 850 MHz (band-4) and 1000 - 1460 MHz (band-5).

The standard data analysis software for the legacy GMRT was the NRAO Astronomical Image Processing System (AIPS¹¹1http://www.aips.nrao.edu/index.shtml#WHATISIT). An AIPS based continuum imaging pipeline, Source Peeling and Atmospheric Modelling (SPAM) (Intema, 2014) has been available mainly for the lower frequency bands ($<\mathrm{GHz}$) of the GMRT (Intema et al., 2017).

Analysis of the uGMRT wideband data has been carried out using semi-automated methods so far (e. g. Kale et al., 2018; Rajpurohit et al., 2019) and has been challenging. The fractional bandwidths of uGMRT, $\Delta\nu/\nu\sim 0.3-0.7$ (band-5 to band-2) require imaging algorithms that simultaneously account for the wideband and widefield effects which have not been implemented in AIPS. While the typical datasets for a few hours observing duration under default settings (16s sampling time and 256 frequency channels) were of a few GB in sizes for the legacy GMRT, these are typically of several tens of GB for the uGMRT. The wide banding also meant increased radio frequency interference (RFI) particularly at low radio frequencies, which along with the large data size has made the manual flagging and processing of the data highly challenging and impractical. Currently there is no publicly available pipeline for the analysis of uGMRT wideband data.

The Common Astronomy Software Applications (CASA) package (McMullin et al., 2007) has algorithms for widefield and wideband imaging of the interferometric data such as the w-projection (Cornwell et al., 2005) and multi-scale multi-frequency synthesis (Rau and Cornwell, 2011) implemented. In this paper we present, CAsa Pipeline-cum-Toolkit for Upgraded Giant Metrewave Radio Telescope data REduction (CAPTURE), which is a python program that uses CASA to produce continuum images from radio interferometric data. The data reduction is carried out in the standard steps of flagging (removal of bad data), calibration, imaging and self-calibration. It is possible to start from the rawdata obtained from the observatory in the native lta format and obtain a self-calibrated continuum image using CAPTURE in a fully automated manner. CAPTURE can also be used like a tool-kit to carry out a step by step analysis. In addition it can be adapted for special imaging requirements.

The paper is organised as follows. The structure of CAPTURE is described in Sec.  2. The usage of it is presented in Sec.  3 and the examples in Sec.  4. The method for primary beam correction is described in Sec. 5. Special cases for running CAPTURE are described in Sec. 6 and the current limitations are summarised in Sec. 7. Conclusions are presented in Sec. 8.

The inputs for CAPTURE are the visibilities in either of lta, FITS or Measurement Set (MS) format and the outputs are calibration tables, calibrated target visibilities and self-calibrated images. The operations carried out by CAPTURE are schematically shown in Fig. 1. Once the MS file of the visibilities is available, the analysis steps in CAPTURE begin. The first part is to locate and flag non-working antennas based on the calibrator scans. If a calibrator scan is found to be bad then the adjoining source scans are also flagged. After the initial flagging, the data are calibrated and a round of flagging is done on the calibrated data. With the improved flagging the calibration is carried out again after clearing the earlier calibration. These calibrated data of the target are split into a separate file and inspected for further bad data. After this round of flagging, the data are averaged in frequency according to users’ preference and again inspected for bad data. In the use of automated flagging modes tfcrop and rflag in the task flagdata for the target source, baselines with the central square antennas (C00, …, C14) are flagged with more conservative thresholds than the arm antennas in order to prevent the loss of extended struture due to over-flagging. These data are then taken up for imaging and self-calibration. During self-calibration, flagging is carried out on the residual column data to remove the low lying RFI.

CAPTURE can carry out an end to end processing of raw data to self-calibrated images once the inputs are set by the user. If there is uncertainty regarding the inputs for imaging or if the user feels the need to carry out manual flagging at intermediate stages after inspection of the data, the pipeline has the flexibility to run in stages on the data.

The github repository for CAPTURE contains the files listed in Table 1.

CAPTURE is to be run with the command “casa -c capture.py” on the terminal in the same folder where the data are stored. CAPTURE will create all the new files in the same folder. Two log files will be created for each instance of running CAPTURE with the names “casa-capture$\_$HH$\_$MM$\_$SS$\_$DD$\_$MM$\_$YYYY.log” and “capture$\_$HH$\_$MM$\_$SS$\_$DD$\_$MM$\_$YYYY.log”. The one with prefix “casa” contains the messages from the CASA logger and the other log file contains the reports from the pipeline processing. The log file will also contain a summary of flagging on the visibilities.

We have used CAPTURE for making images using the legacy GMRT data and the uGMRT data at bands 3, 4 and 5. Here we show two examples of the use of CAPTURE pipeline: a uGMRT band-3 dataset and a 608 MHz legacy GMRT dataset. The details of the observations used are given in Table 4.

For the uGMRT dataset, CAPTURE was run in two stages as the case corresponded to that of a file containing multiple targets described in Sec. 3.2. The first stage was to produce calibrated target data for the two targets in the file. The second stage was to run the processing on the single source file. For imaging we used a pixel size of 1 arcsecond and an imagesize of 10000 pixels. A total of 8 self-calibration loops with first 4 phase-only followed by 4 amplitude and phase calibration. For the legacy GMRT data a single pipeline run was used as described in Sec. 3.1. We used a pixel size of 1 arcsecond and an image size of 9000. The same self-calibration loops as for the uGMRT data were used. The central regions of the images produced using CAPTURE are presented in Fig. 2 and Fig. 3.

Additional examples of the images produced using CAPTURE are shown in Fig. 4 (band-5) and Fig. 5 (band-4).

In order to use the images for scientific analysis, they need to be corrected for the primary beam gain. CAPTURE uses the task tclean for imaging without the correction for the primary beam incorporated (the A-projection algorithm, Bhatnagar et al. (2013)) as the primary beam models for the GMRT as required by these algorithms are not yet available.

Due to the wide bandwidth of the uGMRT, the primary beam changes across the frequency band significantly and needs to be accounted for in the correction. The CASA task “widebandpbcor” is the wideband primary beam correction task for the VLA. The CASA task “wbpbgmrt”³³3https://github.com/ruta-k/uGMRTprimarybeam is a modification of “widebandpbcor” to apply the corrections for the GMRT primary beam shapes. Currently wide-band primary beam model in the form of a polynomial approximation is available for the uGMRT⁴⁴4uGMRT primary beam polynomial approximation, Chengalur, J and Lal, D. V. private communication.. This task requires the measurement set used for imaging, the image and a range of frequencies (channel numbers) within the spectral window provided by the user.

For the legacy GMRT, the primary beam correction can be carried out in two steps. First step is to produce the image of the GMRT primary beam based on the polynomial approximation of the beam with the peak at the phase center of the target field. For this step the task “gmrtpb” has been developed⁵⁵5https://github.com/ruta-k/GMRTpbcor. The image of the primary beam created by this task then should be used in the task “impbcor” for the correction to be applied to the image.

There are portions of CAPTURE where it can be easily modified to accommodate a few special cases of the data.

The source names in any given file are compared with the standard names to classify them into calibrators and targets. We use the list of names of secondary calibrators in J2000 format from the VLA calibrator list (https://science.nrao.edu/facilities/vla/observing/callist). If the file contains calibrator names that are diffrent than these, then an easy work around is to find the names of the secondary calibrators and append it to the text file “vla-cals.list”.

In the imaging and self-calibration part of the pipeline, the number of iterations (niter parameter in the task tclean) at the start is “niter_start” and the maximum number of allowed iterations is hard coded to 200000. The user may change the parameter “niter_start” in config_capture.ini to a lower value such as 200 or 500 if the field is crowded with bright sources. The choice of a lower value will use less number of clean components in the model and results in a less chance of picking up artefacts in the model during the initial self-calibration iterations.

The choice of auto-masking in tclean controls the masking and if the user has expertise in tuning the sub-parameters in tclean, it can be done in the corresponding function in the file “ugfunctions.py”. Auto-masking and myniterend prevent the tclean from diverging and in turn the parameter “mJythreshold” is rendered less critical. A value set equal to that of the expected final rms noise has worked well in our test cases. A high value of “mJythreshold” will result in premature stopping of tclean before cleaning the faint sources and therefore is not recommended.

Our approach in this pipeline has been to have no external dependency of software except CASA to make the pipeline usable on any machine where the user can run CASA. Thus use of external flaggers such as the AOFlagger (Offringa, 2010) are not implemented in the pipeline. However given the tool-kit like design of the pipeline, the user can use external flagging at intermediate steps and use the pipeline for further analysis.

It is recommended to have disk space equivalent to at least six times that of the raw data size for running CAPTURE from end-to-end. The original lta, FITS and MS files are retained and their deletion is left to the user’s discretion. The images with all the extensions as produced by tclean are also retained. The intermediate visibilities from self-calibration, except that from the final iteration, are deleted.

At the GMRT, data can be recorded in the Indian polar (total intensity) or the Full Polar modes. In the default Indian Polar mode, only the RR and LL correlations are present. CAPTURE has been written for the Indian Polar mode data, however it has been run on data where RR, LL, RL and LR correlations were present. CAPTURE does not perform polarization calibration and thus the cross polarization data will essentially remain unused though the stokes I images will be produced. A version of CAPTURE incorporating polarization calibration is being developed.

For the uGMRT data presented here it took 4.3 days to complete 8 self-calibration iterations on a machine with 32 GB RAM and 8 cores. Since the user can tune the number of self-calibration loops and the wprojection planes, the run time can change considerably based on these choices. CAPTURE run time is limited by the speeds of CASA tasks and python and thus we only provide numbers for the speed as seen in the CASA logger and rigorous speed tests are beyond the scope of this work. The step that takes most of the running time in CAPTURE is the task tclean. Users can modify the choice of “$nwprojpl=-1$” to 128 and “imsize_pix” to a lower value at the cost of compromising on the w-term correction and the deconvolution of sources outside the area covered in the given image size to make the imaging run faster. The pipeline is currently being tested with the use of “WSclean” (Offringa et al., 2014) to improve upon the speed of the imaging step.

The CAPTURE pipeline presented here does not take into account the direction dependent effects such as due to the ionosphere and the primary beam.

We have presented “CAPTURE” - CAsa Pipeline-cum-Toolkit for uGMRT Data REduction, a python and CASA based pipeline for the analysis of data obtained from the uGMRT at bands 3, 4 and 5 and the legacy GMRT data. CAPTURE has no external dependency except that of CASA and the lta to FITS conversion pre-compiled codes for the uGMRT data. It can be used to produce self-calibrated images from the data in a fully automatic mode and can also be run in steps to allow users to inspect the data at various stages of the analysis. We have shown examples of running CAPTURE on uGMRT band-3 data and obtained an rms noise of $43\,\mu$ Jy beam^-1. CAPTURE can also work with legacy GMRT data and in the example shown here with 608 MHz data we obtained an image with rms noise of $28\,\mu$Jy beam^-1. Additional two examples of images using CAPTURE at band 4 with an rms of $6.7\mu$Jy beam^-1 and at band-5 with an rms $17\mu$Jy beam^-1 are also shown. In all these cases the rms is within the ballpark value expected given the limitations of the GMRT. The primary beam correction for the images is not part of CAPTURE and the tasks for carrying it out are made separately available on github. CAPTURE is publicly available (https://github.com/ruta-k/uGMRT-pipeline, release v1.0.0) and is being developed further to improve on the performance and additional features such as polarization calibration and other imaging options.