Open in Colab: https://colab.research.google.com/github/casangi/casadocs/blob/fe2b08a/docs/notebooks/visibilities_import_export.ipynb
Visibilities Import Export
To use CASA to process your data, you first will need to get it into a form that is understood by the package. These are “MeasurementSets” for synthesis and single dish data, which is the purpose of this chapter. Importing images, or “image tables” as understood by CASA, is explained here.
There are a number of tasks used to fill telescope-specific data and to import/export standard formats. These are:
importasdm — import of ALMA data in ASDM format
importuvfits — import visibility data in UVFITS format
importfitsidi — import visibility data in the FITS-IDI format
importvla — import data from VLA that is in export format
importmiriad — import data from MIRIAD visibilities
importatca — import ATCA data that is in the RPFITS (archive) format
importgmrt — import GMRT data
importasap — convert ASAP (ATNF Spectral Analysis Package) into a CASA visibility data format
importnro — import NRO 45m data
exportasdm — convert a CASA MS into an ASDM
exportuvfits — export a CASA MS in UVFITS format
UV Data Import
Converting Telescope UV Data to a MeasurementSet
There are a number of tasks available to bring data in various forms into CASA as a MeasurementSet:
ALMA and VLA Science Data Model format via importasdm and importevla
historic VLA Archive format data via importvla
ATCA Data via importatca
MIRIAD Data from various telescopes via importmiriad
GMRT Data via importgmrt
UVFITS format can be imported into and exported from CASA (importuvfits, importfitsidi, and exportuvfits)
ALMA and VLA Filling of Science Data Model (ASDM) data
The ALMA and JVLA projects have agreed upon a common archival science data model (ASDM; sometimes also called SDM) format, and have jointly developed the software to fill this data into CASA. In the ASDM format, the bulk of the data is contained in large binary data format (BDF) tables, with the meta-data and ancillary information in XML tables. This is structured as a directory, like the MS, and was designed to be similar to the MS to facilitate conversion.
The content of an ASDM can be listed with the task asdmsummary:
#asdmsummary :: Summarized description of an ASDM dataset.
asdm = '' #Name of input ASDM directory
with an output that contains the list and positions of the antennas, followed by the metadata of each scan like observation time, source name, frequency and polarization setup:
Input ASDM dataset : TDEM0008.sb3373760.eb3580330.55661.22790537037
========================================================================================
ASDM dataset :TDEM0008.sb3373760.eb3580330.55661.22790537037
========================================================================================
Exec Block : ExecBlock_0
Telescope : JVLA
Configuration name : B
Observer name : Dr. Juergen Ott
The exec block started on 2011-04-10T05:28:13.200000000 and ended on 2011-04-10T10:27:12.300000256
27 antennas have been used in this exec block.
Id Name Make Station Diameter X Y Z
Antenna_0 ea01 UNDEFINED W36 25 -1606841.96 -5042279.689 3551913.017
Antenna_1 ea02 UNDEFINED E20 25 -1599340.8 -5043150.965 3554065.219
Antenna_2 ea03 UNDEFINED E36 25 -1596127.728 -5045193.751 3552652.421
Antenna_3 ea04 UNDEFINED W28 25 -1604865.649 -5042190.04 3552962.365
Antenna_4 ea05 UNDEFINED W08 25 -1601614.091 -5042001.653 3554652.509
Antenna_5 ea06 UNDEFINED N24 25 -1600930.06 -5040316.397 3557330.397
Antenna_6 ea07 UNDEFINED E32 25 -1597053.116 -5044604.687 3553058.987
Antenna_7 ea08 UNDEFINED N28 25 -1600863.684 -5039885.318 3557965.319
Antenna_8 ea09 UNDEFINED E24 25 -1598663.09 -5043581.392 3553767.029
Antenna_9 ea10 UNDEFINED N32 25 -1600781.039 -5039347.456 3558761.542
Antenna_10 ea11 UNDEFINED E04 25 -1601068.79 -5042051.91 3554824.835
Antenna_11 ea12 UNDEFINED E08 25 -1600801.926 -5042219.366 3554706.448
Antenna_12 ea14 UNDEFINED W12 25 -1602044.903 -5042025.824 3554427.832
Antenna_13 ea15 UNDEFINED W24 25 -1604008.742 -5042135.828 3553403.707
Antenna_14 ea16 UNDEFINED N12 25 -1601110.052 -5041488.079 3555597.439
Antenna_15 ea17 UNDEFINED W32 25 -1605808.656 -5042230.082 3552459.202
Antenna_16 ea18 UNDEFINED N16 25 -1601061.961 -5041175.88 3556058.022
Antenna_17 ea19 UNDEFINED W04 25 -1601315.893 -5041985.32 3554808.305
Antenna_18 ea20 UNDEFINED N36 25 -1600690.606 -5038758.734 3559632.061
Antenna_19 ea21 UNDEFINED E12 25 -1600416.51 -5042462.45 3554536.041
Antenna_20 ea22 UNDEFINED N04 25 -1601173.979 -5041902.658 3554987.518
Antenna_21 ea23 UNDEFINED E16 25 -1599926.104 -5042772.967 3554319.789
Antenna_22 ea24 UNDEFINED W16 25 -1602592.854 -5042054.997 3554140.7
Antenna_23 ea25 UNDEFINED N20 25 -1601004.709 -5040802.809 3556610.133
Antenna_24 ea26 UNDEFINED W20 25 -1603249.685 -5042091.404 3553797.803
Antenna_25 ea27 UNDEFINED E28 25 -1597899.903 -5044068.676 3553432.445
Antenna_26 ea28 UNDEFINED N08 25 -1601147.94 -5041733.837 3555235.956
Number of scans in this exec Block : 234
scan #1 from 2011-04-10T05:28:13.200000000 to 2011-04-10T05:33:35.500000256
Intents : OBSERVE_TARGET
Sources : 1331+305=3C286
Subscan #1 from 2011-04-10T05:28:13.200000000 to 2011-04-10T05:33:35.500000256
Intent : UNSPECIFIED
Number of integrations : 322
Binary data in uid:///evla/bdf/1302413292901
Number of integrations : 322
Time sampling : INTEGRATION
Correlation Mode : CROSS_AND_AUTO
Spectral resolution type : FULL_RESOLUTION
Atmospheric phase correction : AP_UNCORRECTED
SpectralWindow_0 : numChan = 256, frame = TOPO,
firstChan = 8484000000, chandWidth = 125000 x Polarization_0 : corr = RR,LL
scan #2 from 2011-04-10T05:33:35.500000256 to 2011-04-10T05:35:35.200000000
Intents : OBSERVE_TARGET
Sources : 1331+305=3C286
Subscan #1 from 2011-04-10T05:33:35.500000256 to 2011-04-10T05:35:35.200000000
Intent : UNSPECIFIED
Number of integrations : 119
Binary data in uid:///evla/bdf/1302413293280
Number of integrations : 119
Time sampling : INTEGRATION
Correlation Mode : CROSS_AND_AUTO
Spectral resolution type : FULL_RESOLUTION
Atmospheric phase correction : AP_UNCORRECTED
SpectralWindow_0 : numChan = 256, frame = TOPO,
firstChan = 8484000000, chandWidth = 125000 x Polarization_0 : corr = RR,LL
scan #3 from 2011-04-10T05:35:35.200000000 to 2011-04-10T05:36:34.999999488
Intents : OBSERVE_TARGET
Sources : 1331+305=3C286
Subscan #1 from 2011-04-10T05:35:35.200000000 to 2011-04-10T05:36:34.999999488
...
The importasdm task will fill SDM format data from ALMA and the JVLA into a CASA visibility data set (MS). All SDM formats from version 2 to the current version can be filled by importasdm.
The default inputs of importasdm are:
# importasdm -- Convert an ALMA Science Data Model observation into a
# CASA visibility file (MS)
asdm = '' # Name of input asdm directory (on
# disk)
vis = '' # Root name of the MS to be created.
# Note the .ms is NOT added
createmms = False # Create a multi-MS output
corr_mode = 'all' # specifies the correlation mode to be
# considered on input. A quoted string
# containing a sequence of ao, co,
# ac,or all separated by whitespaces
# is expected
srt = 'all' # specifies the spectral resolution
# type to be considered on input. A
# quoted string containing a sequence
# of fr, ca, bw, or all separated by
# whitespaces is expected
time_sampling = 'all' # specifies the time sampling
# (INTEGRATION and/or SUBINTEGRATION)
# to be considered on input. A quoted
# string containing a sequence of i,
# si, or all separated by whitespaces
# is expected
ocorr_mode = 'ca' # output data for correlation mode
# AUTO_ONLY (ao) or CROSS_ONLY (co) or
# CROSS_AND_AUTO (ca)
compression = False # Flag for turning on data compression
lazy = False # Make the MS DATA column read the ASDM
# Binary data directly (faster import,
# smaller MS)
asis = '' # Creates verbatim copies of the
# ASDMtables in the ouput MeasurementSet.
# Value given must be a string
# of table names separated by spaces;
# A * wildcard is allowed.
wvr_corrected_data = 'no' # Specifies which values are considerd
# in the SDM binary data to fill the
# DATA column in the MAIN table of the
# MS; yes for corrected, no for
# uncorrected, both for corrected and
# uncorrected (resulting in two MSs)
scans = '' # Processes only the specified scans.
# A scan specification consists of an
# exec block index followed by the :
# character, followed by a comma
# separated list of scan indexes or
# scan index ranges.
# (e.g. 0:1;1:2~6,8;2:,3:24~30),
ignore_time = False # All the rows of the tables Feed,
# History, Pointing, Source, SysCal,
# CalDevice, SysPower, and Weather are
# processed independently of the time
# range of the selected exec
# block / scan.
process_syspower = True # Process the SysPower table?
process_caldevice = True # Process the CalDevice table?
process_pointing = True # Process the Pointing table?
process_flags = True # Create online flags in the FLAG_CMD
# sub-table?
tbuff = 0.0 # Time padding buffer (seconds)
applyflags = False # Apply the flags to the MS.
savecmds = False # Save flag commands to an ASCII file
outfile = '' # Name of ASCII file to save flag
# commands
flagbackup = True # Back up flag column before applying
# flags.
verbose = False # Output lots of information while the
# filler is working
overwrite = False # Over write an existing MS(s)
bdfflags = False # Set the MS FLAG column according to
# the ASDM _binary_ flags
with_pointing_correction = False # Add (ASDM::Pointing::encoder -
# ASDM::Pointing::pointingDirection)
# to the value to be written in
# MS::Pointing::direction
convert_ephem2geo = True # if True, convert any attached
# ephemerides to the GEO reference
# frame (time-spacing not changed)
polyephem_tabtimestep = 0.0 # Timestep (days) for the tabulation
# of polynomial ephemerides.
# A value <= 0 disables tabulation.
If scans is set, then importasdm processes only the scans specified in the option’s value. This value is a semicolon separated list of scan specifications. A scan specification consists of an exec block index followed by the character ‘:’ followed by a comma separated list of scan indexes or scan index ranges. A scan index is relative to the exec block it belongs to. Scan indexes are 1-based while exec blocks are 0-based. The expressions
"0:1"
"2:2~6"
"0:1,1:2~6,8;2:,3:24~30"
"1,2"
"3:"
are all valid values for the selection. The “3:” selector will be interpreted as ‘all the scans of the exec block 3’. A scan index or a scan index range not preceded by an exec block index will be interpreted as ‘all the scans with such indexes in all the exec blocks’. By default all the scans are considered.
When process_flags=True the task will create online flags based on the Flag.xml, Antenna.xml and SpectralWindow.xml files and copy them to the FLAG_CMD sub-table of the MS. The flags will NOT be applied unless the parameter applyflags is set to True. Optionally, the flags can also be saved to an external ASCII file if savecmds is set to True. The flags can later be applied to the MS using task flagdata in list mode.
When bdfflags=True the task will apply online flags contained in the ASDM BDF data by calling the executable bdflags2MS which the user can also do from the OS prompt. This is recommended for ALMA data.
The option createmms prepares the output file for parallel processing and creates a multi-MS.
Specifics on importing Janksy VLA data with importasdm
As of CASA 5.4, the task importevla is no longer available to import JVLA data, but a lot of its functionality is replaced by importasdm. However, several additional steps are required to duplicate the behaviour of importevla when using importasdm, involving a difference in default parameters and the fact that some of the on-the-go flagging cannot be performed by importasdm.
To mimic the behaviour of importevla, change the following parameters in importasdm from their default settings:
ocorr_mode = ‘co’ to import cross-correlations only (discarding auto-correlations)* *
with_pointing_correction = True to add pointing corrections* *
process_flags = True (default) to read in the online flags, then applyflags = True to apply the online flags and/or savecmd = True to save flag commands to an ascii table.
For ephemeris objects: convert_ephem2geo = False
While online flags can thus be created by leaving the parameter process_flags = True by default, additional flagging steps need to be performed after importasdm to flag zero values and shadowing of antennas:
Shadow flags: use task flagdata, with mode = ‘shadow’ (and optionally reason = ‘shadow’). The parameters tolerance and addantenna can be specified in flagdata in the same way they were used in importevla. * *
Zero clipping flags: use task flagdata, with mode = ‘clip’, correlation = ‘ABS_ALL’, and clipzeros = True (and optionally reason = ‘clip’). Note that the non-default case in importevla where flagpol = False can be replicated by setting correlation=”ABS_RR, ABS_LL”.
Like importasdm, the task flagdata can also save the flagging commands to an ascii table by setting savepars = True. To NOT apply the flags (applyflags=False in importevla) add action=’calculate’ to flagdata. You may also chose to add a reason using the cmdreason argument, e.g. cmdreason=”CLIP_ZERO_ALL”.
WARNING: The task flagdata can only write out the flag commands for that invocation of flagdata. The default overwrite=True must be used to overwrite an existing file. In order to save the commands from all 3 possible flagging steps (importasdm, zero, and shadow) each step must be saved to a separate file, which must then be concatenated into a single file to be used to flag the data.
Import of ASDM data with option lazy=True
With lazy=False, importasdm will fill the visibilities into a newly created DATA column of the MS converting them from their binary format in the ASDM to the CASA Table format.
If, however, lazy is set to True, the task will create the DATA column with an ALMA data-specific storage manager, the asdmstman, which enables CASA to directly read the binary data from the ASDM with on-the-fly conversion. No redundant copy of the raw data is created.
This procedure has the advantage that it saves more than 60% disk space and at least in some cases makes the access to the DATA column ≥ 10% faster because the data I/O volume is decreased. For the same reason, it also accelerates the import itself by ca. a factor 2. The acceleration is particularly large in the applycal task and here particularly on standard SATA disks.
E.g., if your ASDM has a size of 36 GB, the import with default parameters will turn this into an MS of 73 GB size (total disk space consumption = 36 GB + 73 GB = 109 GB). With lazy=True, the imported MS has a size of only 2 GB (total disk space consumption = 36 GB + 2 GB = 38 GB). I.e. your total disk space savings are ca. 65%. Even when you compare to the case where you delete the ASDM after normal import, the solution with lazy import and keeping the ASDM will save you ca. 48% disk space (in the example above 38 GB compared to 73 GB).
The only caveats are the following:
You must not delete your ASDM. You can, however, move it but you have to update the reference stored in the MS. Symbolic links will work. See below on how to use the tool method ms.asdmref() to manipulate the ASDM reference.
The lazily imported DATA column is read-only. But in any normal data reduction, the DATA column (as opposed to CORRECTED_DATA) is treated as read-only anyway.
The lazily imported MS is numerically identical with the traditionally imported MS and so are all results derived from the MSs.
An important additional tool to manipulate lazily imported MSs is the new method ms.asdmref() in the ms tool. If the MS is imported from an ASDM with option lazy=True, the DATA column of the MS is virtual and directly reads the visibilities from the ASDM. A reference to the original ASDM is stored with the MS. If the ASDM needs to be moved to a different path, the reference to it in the MS needs to be updated. This can be achieved with ms.asdmref().
The method takes one argument: abspath. When called with abspath equal to an empty string (default), the method just reports the currently set ASDM path or an empty string if the ASDM path was not set, i.e. the MS was not lazily imported.
If you want to move the referenced ASDM to a different path, you can set the new absolute path by providing it as the value of abspath to the method.
ms.open('uid___A12345_X678_X910.ms',False)
ms.asdmref('/home/alma/myanalysis/uid___A12345_X678_X910')
ms.close()
will set the new location of the referenced ASDM to /home/alma/myanalysis/uid___A12345_X678_X910. Contrary to what one would expect from the parameter name, you can also provide a relative path as abspath. This path will be interpreted as relative to the location of the MS.
Info: the lazily imported MS itself can be moved without any restrictions independently from the referenced ASDM as long as the path to the ASDM remains accessible, even across file systems.
VLA: Filling data from archive format (importvla)
VLA data in archive format (i.e., as downloaded from the historic VLA data archive) are read into CASA from disk using the importvla task. The inputs are:
#importvla :: import VLA archive file(s) to a MeasurementSet:
archivefiles = '' #Name of input VLA archive file(s)
vis = '' #Name of output visibility file
bandname = '' #VLA frequency band name:''=>obtain all bands in archive files
frequencytol = 150000.0 #Frequency shift to define a unique spectral window (Hz)
project = '' #Project name: '' => all projects in file
starttime = '' #start time to search for data
stoptime = '' #end time to search for data
applytsys = True #apply nominal sensitivity scaling to data & weights
autocorr = False #import autocorrelations to ms, if set to True
antnamescheme = 'new' #'old' or 'new'; 'VA04' or '4' for ant 4
keepblanks = False #Fill scans with empty source names (e.g. tipping scans)?
evlabands = False #Use updated eVLA frequencies and bandwidths
The main parameters are archivefiles to specify the input VLA Archive format file names, and vis to specify the output MS name.
Info: The scaling of VLA data both before and after the June 2007 Modcomp-turnoff is fully supported, based on the value of applytsys.
Note that archivefiles takes a string or list of strings, as there are often multiple files for a project in the archive.
For example:
archivefiles = ['AP314_A950519.xp1','AP314_A950519.xp2']
vis = 'NGC7538.ms'
The importvla task allows selection on the frequency band. Suppose that you have 1.3 cm line observations in K-band and you have copied the archive data files AP314_A95019.xp* to your working directory and started casa. Then,
default('importvla')
archivefiles = ['AP314_A950519.xp1','AP314_A950519.xp2','AP314_A950519.xp3']
vis = 'ngc7538.ms'
bandname = 'K'
frequencytol = 10e6
importvla()
If the data is located in a different directory on disk, then use the full path name to specify each archive file, e.g.:
archivefiles=['/home/rohir2/jmcmulli/ALMATST1/Data/N7538/AP314_A950519.xp1',
'/home/rohir2/jmcmulli/ALMATST1/Data/N7538/AP314_A950519.xp2',
'/home/rohir2/jmcmulli/ALMATST1/Data/N7538/AP314_A950519.xp3']
Info: importvla will import the on-line flags (from the VLA system) along with the data. Shadowed antennas will also be flagged. The flags will be put in the MAIN table and thus available to subsequent tasks and tools. If you wish to revert to unflagged data, use flagmanager to save the flags (if you wish), and then use flagdata with mode=’manual’ and unflag=True to toggle off the flags.
The other parameters are:
Parameter applytsys
The applytsys parameter controls whether the nominal sensitivity scaling (based on the measured TSYS, with the weights scaled accordingly using the integration time) is applied to the visibility amplitudes or not. If True, then it will be scaled so as to be the same as AIPS FILLM (i.e. approximately in deciJanskys). Note that post-Modcomp data is in raw correlation coefficient and will be scaled using the TSYS values, while Modcomp-era data had this applied online. In all cases importvla will do the correct thing to data and weights based on an internal flag in the VLA Archive file, either scaling it or unscaling based on your choice for applytsys.
If applytsys=True and you see strange behavior in data amplitudes, it may be due to erroneous TSYS values from the online system. You might want to then fill with applytsys=False and look at the correlation coefficients to see if the behavior is as expected.
Parameter bandname
The bandname indicates the VLA Frequency band(s) to load, using the traditional bandname codes. These are:
‘4’ = 48-96 MHz
‘P’ = 298-345 MHz
‘L’ = 1.15-1.75 GHz
‘C’ = 4.2-5.1 GHz
‘X’ = 6.8-9.6 GHz
‘U’ = 13.5-16.3 GHz
‘K’ = 20.8-25.8 GHz
‘Q’ = 38-51 GHz
‘’ = all bands (default)
Note that as the transition from the VLA to JVLA progressed, the actual frequency ranges covered by the bands expanded, and additional bands were added (namely ‘S’ from 2-4 GHz and ‘A’ from 26.4-40 GHz).
Parameter frequencytol
The frequencytol parameter specifies the frequency separation tolerated when assigning data to spectral windows. The default is frequencytol=150000 (Hz). For Doppler tracked data, where the sky frequency changes with time, a frequencytol < 10000 Hz may produce too many unnecessary spectral windows.
Parameter project
You can specify a specific project name to import from archive files. The default ‘’ will import data from all projects in file(s) archivefiles.
For example for VLA Project AL519:
project = 'AL519' #this will work project = 'al519' #this will also work
while project=’AL0519’ will NOT work (even though that is what queries to the VLA Archive will print it as - sorry!).
Parameters starttime and stoptime
You can specify start and stop times for the data, e.g.:
starttime = '1970/1/31/00:00:00' stoptime = '2199/1/31/23:59:59'
Note that the blank defaults will load all data fitting other criteria.
Parameter autocorr
Note that autocorrelations are filled into the data set if autocorr=True. Generally for the VLA, autocorrelation data is not useful, and furthermore the imaging routine will try to image the autocorrelation data (it assumes it is single dish data) which will swamp any real signal. Thus, if you do fill the autocorrelations, you will have to flag them before imaging.
Parameter antnamescheme
The antnamescheme parameter controls whether importvla will try to use a naming scheme where JVLA antennas are prefixed with EA (e.g. ‘EA16’) and old VLA antennas have names prefixed with VA (e.g. ‘VA11’). Our method to detect whether an antenna is JVLA is not yet perfected, and thus unless you require this feature, simply use antnamescheme=’old’.
Parameter evlabands
The evlabands=True option is provided to allow users to access JVLA frequencies outside the standard VLA tunings (e.g. the extended C-band above 6 GHz).
ALERT: use of this option for standard VLA data will cause unexpected associations, such as X-band data below 8 GHz being extracted to C-band (as the JVLA C-band is 4–8 GHz). Use with care.
Import ATCA and CARMA data
There are several ways to import data from ATCA and CARMA into CASA. The data from these arrays has historically been processed in MIRIAD. For simple cases (single source and frequency) exporting from MIRIAD to UVFITS format and importing using importuvfits often works ok, although some fixes to the resulting MeasurementSet may be needed.
The importmiriad task reads MIRIAD visibility data and can handle multiple frequencies and sources in the input. Since it does not apply any calibration, make sure to apply it beforehand in MIRIAD.
The importatca task reads the ATCA archive format (RPFITS) directly, avoiding the need to go through MIRIAD to load the data. It can handle ATCA data from both the old and new (CABB) correlator.
Import MIRIAD visibilities (importmiriad)
The task importmiriad allows one to import visibilities in the MIRIAD data format to be converted to a MS. The task has mainly been tested on data from the ATCA and CARMA telescopes and the inputs are:
#importmiriad :: Convert a Miriad visibility file into a CASA MeasurementSet
mirfile = '' #Name of input Miriad visibility file
vis = '' #Name of output MeasurementSet
tsys = False #Use the Tsys to set the visibility weights
spw = 'all' #Select spectral windows
vel = '' #Select velocity reference (TOPO,LSRK,LSRD)
linecal = False #(CARMA) Apply line calibration
wide = 'all' #(CARMA) Select wide window averages
debug = 0 #Display increasingly verbose debug messages
The mirfile parameter specifies a single MIRIAD visibility file which should have any calibration done in MIRIAD already applied to it.
Set the tsys parameter to True to change the visibility weights from the MIRIAD default (usually the integration time) to the inverse of the noise variance using the recorded system temperature.
The spw parameter can be used to select all or some of the simultaneous spectral windows from the input file. Use the default of ‘all’ for all the data or use e.g., spw=’0,2’ to select the first and third window.
The vel parameter can be used to set the output velocity frame reference. For ATCA this defaults to ‘TOPO’ and for CARMA it defaults to ‘LSRK’. Only change this if your data comes out with the incorrect velocity.
The linecal parameter is only useful for CARMA data and can apply the line calibration if it is stored with the MIRIAD data.
The wide parameter is only useful for CARMA data and can select which of the wide-band channels should be loaded.
Import ATCA RPFITS data (importatca)
The data from the ATCA is available from the archive in files in the RPFITS format. These files can be imported into CASA with the importatca task.
#importatca :: Import ATCA RPFITS file(s) to a MeasurementSet
files =['*.C1234'] #Name of input ATCA RPFits file(s)
vis = 'c1234.ms' #Name of output visibility file
#(MeasurementSet)
options = '' #Processing options: birdie, reweight,
#noxycorr, fastmosaic, hires, noac
#(comma separated list)
spw = [-1] #Specify the spectral windows to use,
#default=all
nscans = [0, 0] #Number of scans to skip followed by
#number of scans to read
lowfreq = '0.1GHz' #Lowest reference frequency to select
highfreq = '999GHz' #Highest reference frequency to select
fields = [''] #List of field names to select
edge = 8 #Percentage of edge channels to flag.
#For combined zooms, this specifies
#the percentage for a single zoom
#window
The files parameter can take a string or a list of strings as input and also allows the use of wildcards as shown in the example above.
For older ATCA continuum data (before the CABB correlator, April 2009) use options=’birdie,reweight’ to suppress internally generated RFI.
The options parameter:
birdie - (pre-CABB data only) Discard edge channels and channels affected by internal RFI.
reweight - (pre-CABB data only) Suppress ringing of RFI spikes by reweighting of the lag spectrum
noxycorr - do not apply the xy phase correction as derived from the switched noise calibration, by default this is applied during loading of the data.
fastmosaic - use this option if you are loading mosaic data with many pointings and only one or two integrations per pointing. This option changes the tiling of the data to avoid excessive I/O.
hires - use this option if you have data in time binning mode (as used for pulsars) but you want to make it look like data with very short integration time (no bins).
noac - discard the auto-correlation data
The spw parameter takes a list of integers and can be used to select one or more of the simultaneous frequencies. With CABB there can be up to 34 spectra. The order of the frequency bands in the RPFITS file is: the two continuum bands (0 and 1), followed by the zoom bands for the first frequency and then the zoom bands for the second frequency. Note that this spw parameter does not take a string with wildcards. Use spw=-1 to get all the data.
The nscans parameter can be used to select part of a file, e.g., to retrieve a few test scans for a quick look.
The lowfreq and highfreq parameters select data based on the reference frequency.
The fields parameter selects data based on the field/source name.
The edge parameter specifies how many edge channels to discard as a percentage of the number of channels in each band. E.g., the default value of 8 will discard 8 channels from the top and bottom of a 2048 channel spectrum.
UVFITS Import
The UVFITS format is not exactly a standard, but is a popular archive and transport format nonetheless. CASA supports UVFITS files written by the AIPS FITTP task, and others.
UVFITS is supported for both import and export.
Import using importuvfits
To import UVFITS format data into CASA, use the importuvfits task:
#In CASA: inp(importuvfits)
fitsfile = '' #Name of input UVFITS file
vis = '' #Name of output visibility file (MS)
antnamescheme = 'old' #For VLA only; 'new' or 'old'; 'VA04' or '04' for VLA ant 4
This is straightforward, since all it does is read in a UVFITS file and convert it as best it can into a MS.
For example:
importuvfits(fitsfile='NGC5921.fits',vis='ngc5921.ms')
Here is a hint for handling CARMA data loaded into CASA using importuvfits:
tb.open("c0104I/ANTENNA",nomodify=False)
namelist=tb.getcol("NAME").tolist()
for i in range(len(namelist)):
name = 'CA'+namelist[i]
print ' Changing '+namelist[i]+' to '+name
namelist[i]=name
tb.putcol("NAME",namelist)
tb.close()
Import using importfitsidi
Some uvfits data is written in the FITS-IDI standard. Those files can be imported into CASA with the importfitsidi task:
#importfitsidi :: Convert a FITS-IDI file to a CASA visibility data set
fitsidifile = [''] #Name(s) of input FITS-IDI file(s)
vis = '' #Name of output visibility file (MS)
constobsid = False #If True, give constant obs ID==0 to
#the data from all input fitsidi
#files (False = separate obs id for
#each file)
scanreindexgap_s = 0.0 #min time gap (seconds) between
#integrations to start a new scan
The constobs parameter can be used to give all visibilities the same observation id of 0. scanreindexgap_s controls the gap that defines different scans.
Example:
importfitsidi(fitsidifile='NGC1300.fits',vis='NGC1300.ms')
MeasurementSet Export
Convert a MeasurementSet to UVFITS
Export using exportuvfits
The exportuvfits task will take a MS and write it out in UVFITS format. The defaults are:
#exportuvfits :: Convert a CASA visibility data set to a UVFITS file:
vis = '' #Name of input visibility file
fitsfile = '' #Name of output UV FITS file
datacolumn = 'corrected' #Visibility file data column
field = '' #Select field using field id(s) or field name(s)
spw = '' #Select spectral window/channels
antenna = '' #Select data based on antenna/baseline
timerange = '' #Select data based on time range
avgchan = 1 #Channel averaging width (value > 1 indicates averaging)
writesyscal = False #Write GC and TY tables, (Not yet available)
multisource = True #Write in multi-source format
combinespw = True #Export the spectral windows as IFs
padwithflags = True #Fill in missing data with flags to fit IFs
writestation = True #Write station name instead of antenna name
overwrite = False #Overwrite output file if it exists?
For example:
exportuvfits(vis='ngc5921.split.ms',
fitsfile='NGC5921.split.fits',
multisource=False)
The MS selection parameters field, spw, antenna, and timerange follow the standard selection syntax.
The datacolumn parameter chooses which data-containing column of the MS is to be written out to the UV FITS file. Choices are: ‘data’, ‘corrected’, and ‘model’.
There are a number of special parameters that control what is written out. These are mostly here for compatibility with AIPS.
The writesyscal parameter toggles whether GC and TY extension tables are written. These are important for VLBA data, and for JVLA data.
ALERT: The writesyscal option is not yet available.
The multisource parameter determines whether the UV FITS file is a multi-source file or a single-source file, if you have a single-source MS or choose only a single source. Note: the difference between a single-source and multi-source UVFITS file here is whether it has a source (SU) table and the source ID in the random parameters. Some programs (e.g. difmap) only accept single-source files. If you select more than one source in fields, then the multisource parameter will be overridden to be True regardless.
The combinespw parameter allows, if some conditions are met, exporting of all spectral windows (SpW) as a set of “IF”s in a single “FREQID” setup instead of giving each SpW its own FREQID in the FITS file. In this context an IF (Intermediate Frequency) is a specialization of an SpW, where each IF in a UV FITS file must have the same number of channels and polarizations, each channel must have the same width, and each IF must be present (even if flagged) throughout the entire observation. If these conditions are not met the data must be exported using multiple FREQIDs, the UV FITS equivalent of a general SpW. This matters since many (sub)programs will work with multiple IFs, but not multiple FREQIDs. For example, a UV FITS file with multiple FREQIDs can be read by AIPS, but you may find that you have to separate the FREQIDs with SPLIT before you can do very much with them. Therefore combinespw should be True if possible. Typically MSes where each band was observed simultaneously can be exported with combinespw=True. MSes where the tuning changed with time, e.g. 10 minutes at 4.8 GHz followed by 15 minutes at 8.4 GHz, should be exported to multiple UV FITS files using spw to select one tuning (set of simultaneous SpWs) per file.
The writestation parameter toggles the writing of the station name instead of antenna name.