#
# stub class definition file for docstring parsing
#
[docs]class table:
r"""
Access tables from casapy
table is the tool that contains all the functions relevant for table handling.
"""
[docs] def table(self, tablename='', lockoptions='', nomodify=True):
r"""
Use this constructor to construct a table tool inside casapy from the name of a disk file containing a Table. A new table may also be created from a table descriptor (see tablecreatedesc). When creating a new table, detailed data manager information can be given using the ``dminfo`` argument. This is a record as returned by the getdminfo function.
Most of the arguments are rarely used: most of the time, you’ll just need to use the tablename, and perhaps nomodify.
| A table can be shared by multiple processes by using the appropriate locking options. The possible options are: auto: let the system take care of locking. At regular time intervals these autolocks are released to give other processes the opportunity to access the table. The aipsrc variable
| texttttable.relinquish.reqautolocks.interval defines the number of seconds between releasing autolocks on tables needed in another process.
| texttttable.relinquish.allautolocks.interval defines the number of seconds between releasing all autolocks. autonoread: as auto, but no read locking is needed. This must be used with care, because it means that reading can be done while the table tool is not synchronized with the table file (as is normally done when a lock is acquired). The function ``resync`` can be used to explicitly synchronize the table tool user: the user takes care by explicit calls to lock and unlock usernoread: as user and the no readlocking behaviour of autonoread. permanent: use a permanent lock; the constructor fails when the table is already in use in another process permanentwait: as above, but wait until the other process releases its lock default: this is the default option. If the given table is already open, the locking option in use is not changed. Otherwise it reverts to auto. auto locking is used, it is possible to give a record containing the fields option, interval, and/or maxwait. In this way advanced users have full control over the locking options. In practice this is hardly ever needed.
When creating a new table, the endian format in which the data should be stored, can be specified. The possible values are: big: big endian format (as used on e.g. SUN) little: little endian format (as used on e.g. PC) local: use the endian format of the machine being used aipsrc: use the endian format specified in aipsrc variable table.endianformat (which defaults to big). default is aipsrc. that usually it is best to store data in local endian format, because that requires the least amount of byte swapping. However, if the table must be accessible with AIPS++ version 1.7 or before, big endian should be used.
When creating a new table, the table will normally reside on disk. It is, however, possible to specify that the table should be held in memory. In such a case the table is process specific, thus cannot be seen by other processes. Note that a memory table uses the MemoryStMan storage manager for all its stored columns, but it is still possible to use virtual columns as well.
.. rubric:: Parameters
- ``tablename (string='')``
- ``lockoptions (record='')`` - locking dictionary to be used : dict keys are 'option', 'interval', 'maxwait'
- ``nomodify (bool=True)``
.. rubric:: Returns
``table``
.. rubric:: Examples
::
table1:=table("3C273XC1.MS");
table1.browse();
"""
pass
[docs] def fromfits(self, tablename='', fitsfile='', whichhdu=1, storage='standard', convention='none', nomodify=True, ack=True):
r"""
| Create a table from binary FITS format. This generates a CASA table from the binary FITS table in the given HDU (header unit) of the FITS file. Note that other FITS formats (*e.g.* Image FITS and UVFITS) are read by other means. is possible to specify the storage manager to use for the table:
| textttstandard is the default storage manager.
| textttincremental is efficient for slowly varying data.
| textttmemort is for in memory use for e.g to grab given columns via getcol.
.. rubric:: Parameters
- ``tablename (string='')`` - Name of table to be created
- ``fitsfile (string='')`` - Name of FITS file to be read
- ``whichhdu (int=1)`` - Which HDU to read (0-relative to primary HDU i.e. 1 is the smallest valid value)
- ``storage (string='standard')`` - Storage manager to use (standard or incremental or memory)
- ``convention (string='none')`` - Convention to use (sdfits or none)
- ``nomodify (bool=True)`` - Open Read-only?
- ``ack (bool=True)`` - Acknowledge creations, etc
.. rubric:: Returns
``table``
"""
pass
[docs] def fromascii(self, tablename='', asciifile='', headerfile='', autoheader=False, autoshape=[-1], sep='', commentmarker='', firstline=0, lastline=-1, nomodify=True, columnnames='', datatypes=''):
r"""
Create a table from an ASCII file. Columnar data as well as table and column keywords may be specified. the table is created from the ASCII data, it is opened in the specified mode by the table tool.
The table columns are filled from a file containing the data values separated by a separator (one line per table row). The default separator is a blank. Blanks after the separator are ignored. a non-blank separator is used, values can be empty. Such values default to 0, empty string, or F depending on the data type. E.g. 1,,2, has 4 values of which the 2nd and 4th are empty and default to 0. Similarly if fewer values are given than needed, the missing values get the default value.
Either the data format can be explicitly specified or it can be found automatically. The former gives more control in ambiguous situations. Both scalar and array columns can be generated from the ASCII input. The format string determines the type and optional shape.
In automatic mode (``autoheader=True``) the first line of the ASCII data is analyzed to deduce the data types. Only the types I, D, and A can be recognized. A number without decimal point or exponent is I (integer), otherwise it is D (double). Any other string is A (string). Note that a number may contain a leading sign (+ or -). The ``autoshape`` argument can be used to specify if the input should be stored as multiple scalars (the default) or as a single array. In the latter case one axis in the shape can be defined as variable length by giving it the value 0. It means that the actual array shape in a row is determined by the number of values in the corresponding input line. Columns get the names ``Column1``, ``Column2``, etc.. example:
#. ``autoshape=[]`` (which is the default) means that all values are to be stored as scalar columns.
#. ``autoshape=0`` means that all values in a row are to be stored as a variable length vector.
#. ``autoshape=10`` defines a fixed length vector. If an input line contains less than 10 values, the vector is filled with default values. If more than 10 values, the latter values are ignored.
#. ``autoshape=[5,0]`` defines a 2-dim array of which the 2nd axis is variable. Note that if an input line does not contain a multiple of 5 values, the array is filled with default values.
If the format of the table is explicitly specified, it has to be done either in the first two lines of the data file (named by the argument filename), or in a separate header file (named by the argument headerfile). In both forms, table keywords may also be specified before the column definitions. The column names and types can be described by two lines:
#. The first line contains the names of the columns. These names may be enclosed in quotes (either single or double).
#. The second line contains the data type and optionally the shape of each column. Valid types are:
- S for Short data
- I for Integer data
- R for Real data
- D for Double Precision data
- X for Complex data (Real followed by Imaginary)
- Z for Complex data (Amplitude then Phase)
- DX for Double Precision Complex data (Real followed by Imaginary)
- DZ for Double Precision Complex data (Amplitude then Phase)
- A for ASCII data (a value must be enclosed in single or double quotes if it contains whitespace)
- B for Boolean data (False are empty string, 0, or any string starting with F, f, N, or n).
If a column is an array, the shape has to be given after the data type without any whitespace. E.g. ``I10`` defines an integer vector of length 10. ``A2,5`` defines a 2-dim string array with shape [2,5]. Note that ``I`` is not the same as ``I1`` as the first one defines a scalar and the other one a vector with length 1. The last column can have one variable length axis denoted by the value 0. It "consumes" the remainder of the input line.
If the argument headerfile is set then the header information is read from that file instead of the first lines of the data file.
To give a simple example of the form where the header information is located at the top of the data file:
::
COLI COLF COLD COLX COLZ COLS
I R D X Z A
1 1.1 1.11 1.12 1.13 1.14 1.15 Str1
10 11 12 13 14 15 16 ""
Note that a complex number consists of 2 numbers. note that an empty string can be given.
Let us now give an example of a separate header file that one might use to get interferometer data into :
::
U V W TIME ANT1 ANT2 DATA
R R R D I I X1,0
The data file would then look like:
::
124.011 54560.0 3477.1 43456789.0990 1 2 4.327 -0.1132
34561.0 45629.3 3900.5 43456789.0990 1 3 5.398 0.4521
Note that the DATA column is defined as a 2-dim array of 1 correlation and a variable number of channels, so the actual number of channels is determined by the input. In this example both rows will have 1 channel (note that a complex value contains 2 values).
Tables may have keywords in addition to the columns. The keywords are useful for holding information that is global to the entire table (such as author, revision, history, *etc,*). keywords in the header definitions must preceed the column descriptions. They must be enclosed between a line that starts with ".key..." and a line that starts with ".endkey..." (where ... can be anything). Between these two lines each line should contain the following as listed below. A table keywordset and column keywordsets can be specified. The latter can be specified by specifying the column name after the .keywords string.
- The keyword name, e.g., ANYKEY
- The datatype and optional shape of the keyword (cf. list of valid types above)
- The value or values for the keyword (the keyword may contain a scalar or an array of values). e.g., 3.14159 21.78945
Thus to continue the example above, one might wish to add keywords as follows:
::
.keywords
DATE A "97/1/16"
REVISION D 2.01
AUTHOR A "Tim Cornwell"
INSTRUMENT A "VLA"
.endkeywords
.keywords TIME
UNIT A "s"
.endkeywords
U V W TIME ANT1 ANT2 DATA
R R R D I I X1,0
Similarly to the column format string, the keyword formats can also contain shape information. The only difference is that if no shape is given, a keyword can have multiple values (making it a vector).
It is possible to ignore comment lines in the header and data file by giving the ``commentmarker``. It indicates that lines starting with the given marker are ignored. Note that the marker can be a regular expression (e.g. texttt’ \*//’ tells that lines starting with // and optionally preceeded by blanks have to be ignored).
With the arguments ``firstline`` and ``lastline`` one can specify which lines have to be taken from the input file. A negative value means 1 for ``firstline`` or end-of-file for ``lastline``. Note that if the headers and data are combined in one file, these line arguments apply to the whole file. If headers and data are in separate files, these line arguments apply to the data file only.
Also note that ignored comment lines are counted, thus are used to determine which lines are in the line range.
The number of rows is determined by the number of lines read from the data file.
.. rubric:: Parameters
- ``tablename (string='')`` - Name of table to be created
- ``asciifile (string='')`` - Name of ASCII file to be read
- ``headerfile (string='')`` - Name of an optional file defining the format
- ``autoheader (bool=False)`` - Determine header information automatically
- ``autoshape (intVec=[-1])`` - Shape to be used if autoheader=True
- ``sep (string='')`` - Value separator
- ``commentmarker (string='')`` - Regex indicating comment line
- ``firstline (int=0)`` - First line to use
- ``lastline (int=-1)`` - Last line to use
- ``nomodify (bool=True)`` - Open Read-only?
- ``columnnames (stringVec='')`` - Column Names
- ``datatypes (stringVec='')`` - Data types
.. rubric:: Returns
``bool``
"""
pass
[docs] def open(self, tablename='', lockoptions='', nomodify=True):
r"""
Opens a disk file containing an existing Table.
Most of the time you just need to specify the tablename and perhaps nomodify.
A table can be shared by multiple processes by using the appropriate locking options. The possible options are: auto: let the system take care of locking. At regular time intervals these autolocks are released to give other processes the opportunity to access the table. autonoread: as auto, but no read locking is needed. This must be used with care, because it means that reading can be done while the table tool is not synchronized with the table file (as is normally done when a lock is acquired). The function ``resync`` can be used to explicitly synchronize the table tool user: the user takes care by explicit calls to lock and unlock usernoread: as user and the no readlocking behaviour of autonoread. permanent: use a permanent lock; the constructor fails when the table is already in use in another process permanentwait: as above, but wait until the other process releases its lock default: this is the default option. If the given table is already open, the locking option in use is not changed. Otherwise it reverts to auto. auto locking is used, it is possible to give a record containing the fields option, interval, and/or maxwait. In this way advanced users have full control over the locking options. In practice this is hardly ever needed.
.. rubric:: Parameters
- ``tablename (string='')``
- ``lockoptions (record='')`` - locking dictionary to be used : dict keys are 'option', 'interval', 'maxwait'
- ``nomodify (bool=True)``
.. rubric:: Returns
``bool``
.. rubric:: Examples
::
# First let's make a table for testing
def maketesttable():
# Get path to CASA home directory by stipping name from '$CASAPATH'
pathname=os.environ.get("CASAPATH").split()[0]
# This is where the 3C273XC1.fits data should be
fitsdata=pathname+"/data/demo/3C273XC1.fits"
# Remove old table if present
!rm -rf 3C273XC1.MS
ms.fromfits("3C273XC1.MS",fitsdata)
ms.close()
maketesttable()
tb.open("3C273XC1.MS")
tb.browse()
tb.close()
The first line opens an existing table 3C273XC1.MS, the second browses
it using the browse function.
tb.open("3C273XC1.MS", nomodify=False, lockoptions={'option':'user'})
tb.lock();
tb.addrows();
tb.unlock();
In this example explicit user locking is used. The function lock
is needed to acquire a (write) lock before the addrows is done.
Thereafter the lock is released to give other processes the chance
to operate on the table.
\\Note that releasing a lock implies flushing the table, so doing
that very often can be quite expensive.
"""
pass
[docs] def create(self, tablename='', tabledesc='', lockoptions='default', endianformat='', memtype='', nrow=0, dminfo=''):
r"""
Create a new Table.
Most of the time you just need to specify the table’s name and a description of its format.
A table can be shared by multiple processes by using the appropriate locking options. The possible options are: auto: let the system take care of locking. At regular time intervals these autolocks are released to give other processes the opportunity to access the table. autonoread: as auto, but no read locking is needed. This must be used with care, because it means that reading can be done while the table tool is not synchronized with the table file (as is normally done when a lock is acquired). The function ``resync`` can be used to explicitly synchronize the table tool user: the user takes care by explicit calls to lock and unlock usernoread: as user and the no readlocking behaviour of autonoread. permanent: use a permanent lock; the constructor fails when the table is already in use in another process permanentwait: as above, but wait until the other process releases its lock default: this is the default option. If the given table is already open, the locking option in use is not changed. Otherwise it reverts to auto. auto locking is used, it is possible to give a record containing the fields option, interval, and/or maxwait. In this way advanced users have full control over the locking options. In practice this is hardly ever needed.
.. rubric:: Parameters
- ``tablename (string='')``
- ``tabledesc (record='')`` - description of the table's format
- ``lockoptions (record='default')`` - locking to be used
- ``endianformat (string='')``
- ``memtype (string='')``
- ``nrow (int=0)``
- ``dminfo (record='')`` - Data Manager information
.. rubric:: Returns
``bool``
.. rubric:: Examples
::
# First let's get sample descriptions of a table and its data managers.
import os, shutil
def get_tabledesc_and_dminfo(tabname="3C273XC1.MS"):
made_copy = False
# Fetch new table if tabname not present
if not os.path.isdir(tabname):
# Get path to CASA root directory by stripping name from '$CASAPATH'
pathname = os.environ.get("CASAPATH").split()[0]
# There should be some data here
fitsdata = pathname + "/data/demo/3C273XC1.fits"
tabname = "3C273XC1.MS"
ms.fromfits(tabname, fitsdata)
ms.close()
made_copy = True
tb.open(tabname)
tabdesc = tb.getdesc()
dminfo = tb.getdminfo()
print tabname, "has", tb.nrows(), "rows."
tb.close()
# Clean up
if made_copy:
shutil.rmtree(tabname)
return tabdesc, dminfo
tabdesc, dmi = get_tabledesc_and_dminfo()
tabdesc # prints tabdesc
dmi # prints dmi
# You could alter tabdesc and/or dmi at this point.
# Unnecessary, but just to show there is nothing up my sleeve...
tb.close()
tb.create("myempty.ms", tabdesc, dminfo=dmi)
tb.nrows() # 0L
tb.addrows(5) # Add the rows _before_ filling the columns.
tb.putcol('ARRAY_ID', numpy.array([0 for i in range(5)]))
tb.putcol('ANTENNA1', numpy.array(range(5)))
tb.putcol('ANTENNA2', numpy.array(range(1,6)))
tb.browse() # Still mostly, but not completely, empty.
tb.close()
This creates a CASA table using a description of a table and its data managers from an existing MS.
"""
pass
[docs] def flush(self):
r"""
Until a flush is performed, the results of all operations are not reflected in any change to the disk file. Hence you *must* do a flush to write the changes to disk.
"""
pass
[docs] def fromASDM(self, tablename='', xmlfile=''):
r"""
.keywords DATE A "07/7/23" REVISION D 0 AUTHOR A "Paulo C. Cortes" INSTRUMENT A "ALMA" .endkeywords
The main function for this task is to create a CASA::Table from a XML ASDM Table. The classes asdmCasaXMLUtil and asdmCasaSaxHandler are the main objects which implement the task. The asdmCasaSaxHandler encapsulate all the operations returning a reference to a CASA::Table. The class uses xerces-c to parse the XML table and creates the CASA::Table. The implementation assumes the integrity of the XML data, it not attempting to check whether the XML data meets a column format or not. In detail, an ArrayString column should agree with the following format: nd nx ... data, where nd is the number of dimensions, nx is the size of the first dimension (implemented upto a cube, i.e. nx,ny,nz), and data is the array itself which should have the appropiate number of elements. For example, a VectorString column could be: 1 2 "I" "Q" or dimension 1, size 2, and two string elements. Due to the lack of data type spefication in the XML tables, the column names are hardcoded into the asdmCasaSaxHandler based on the ASDM specification (see http://aramis.obspm.fr/ alma/ASDM/ASDMEntities/index.html). While missing data from a table column will be accepted by the task, any new column beyond the specification has to be added into the class, also, any change in data types form the specificatin will produce a crash, CASA is picky with data types integrity. So far, the list of tables included in the class is:
AlmaCorrelatorMode.xml, Antenna.xml ConfigDescription.xml, DataDescription.xml, ExecBlock.xml, Feed.xml, Field.xml, Main.xml, Polarization.xml, Processor.xml, Receiver.xml, SBSummary.xml, Scan.xml, Source.xml, SpectralWindow.xml, State.xml, Station.xml, Subscan.xml, SwitchCycle.xml, CalCurve.xml, CalData.xml, CalPhase.xml
more tables will follow. The usage of fromASDM is simple, it gets two string, tablename and xmlfile, where tablename is the CASA::Table to be written and xmlfile represents the ASDM XML table. To call it do: tb.fromasdm(tablename,xmlfile)
.. rubric:: Parameters
- ``tablename (string='')`` - Name of table to be created
- ``xmlfile (string='')`` - Name of the XML file to be read
.. rubric:: Returns
``bool``
"""
pass
[docs] def resync(self):
r"""
Acquiring a read or write lock automatically synchronizes the internals of the table tool with the actual contents of the table files. In this way different processes accessing the same table always use the same table data. , a table can be used without read locking. In that case the table tool internals are not synchronized automatically. The resync function offers a way to do explicit synchronization. It is only useful if the table is opened with locking mode ``autonoread`` or ``usernoread``.
"""
pass
[docs] def close(self):
r"""
First a flush is done, then the table is closed inside casapy and is no longer available for use.
"""
pass
[docs] def copy(self, newtablename='', deep=False, valuecopy=False, dminfo='', endian='aipsrc', memorytable=False, returnobject=False, norows=False):
r"""
Copy the table. All subtables are also copied. References to another table are preserved.
The argument ``deep`` determines how a reference table (i.e. the result of a query) is copied. By default a file copy is made, thus the resulting table still contains references and no actual data. If, however, ``deep=True`` is given, a deep copy is made which means that the actual data are copied. Also all subtables are copied. a plain table is copied by copying the files. However, if ``deep=True`` and ``valuecopy=True`` are given, a plain table is copied by copying all its values and subtables. This is useful to reorganize the tables, i.e. to regain file space that is wasted by frequent updates to a table. argument ``dminfo`` can be used to specify explicit data manager info for the columns in the new plain table. It can be used to change, for example, a storage manager from IncrStMan to StandardStMan. The ``dminfo`` is a record as returned by the getdminfo If ``dminfo`` is a non-empty record, it forces ``valuecopy=True``.
The standard operation is make the copy to a plain table. It is, however, possible to copy to a memory table by giving ``memorytable=True``.
The endian format for the newly created table can be specified. This is only meaningful if a deep copy is made to a plain table. The possible values are: big: big endian format (as used on e.g. SUN) little: little endian format (as used on e.g. PC) local: use the endian format of the machine being used aipsrc: use the endian format specified in aipsrc variable table.endianformat (which defaults to big). default is aipsrc.
Normally the ``copy`` function only copies the table and does not create a new table tool object. The user can do that by opening the newly created table in the standard way. However, it is possible to get an object back by using ``returnobject=True``. An object is always returned if the copy is made to a memory table.
.. rubric:: Parameters
- ``newtablename (string='')`` - Name of newtable on disk
- ``deep (bool=False)`` - Make a deep copy of a reference table?
- ``valuecopy (bool=False)`` - Make a deep copy of any table?
- ``dminfo (record='')`` - Data manager info for new table
- ``endian (string='aipsrc')`` - Endian format of new table
- ``memorytable (bool=False)`` - Hold new table in memory?
- ``returnobject (bool=False)`` - Return a tool object for the new table
- ``norows (bool=False)`` - Don't copy any rows (useful for copying only the table structure)
.. rubric:: Returns
``table``
"""
pass
[docs] def copyrows(self, outtable='', startrowin=0, startrowout=-1, nrow=-1):
r"""
Copy rows from this table to another. By default all rows of this table are appended to the output table. It is possible though to control which rows are copied. are added to the output table as needed. Because no rows can be added to a reference table, it is only possible to overwrite existing rows in such tables.
Only the data of columns existing in both tables will be copied. Thus by making a reference table consisting of a few columns, it is possible to copy those columns only.
.. rubric:: Parameters
- ``outtable (string='')`` - table object of output table
- ``startrowin (int=0)`` - First row to take from input table
- ``startrowout (int=-1)`` - First row to write in output table, -1 (=end)
- ``nrow (int=-1)`` - Nr of rows to copy, -1 (=all)
.. rubric:: Returns
``bool``
.. rubric:: Examples
::
This example appends rows to the table itself, thus doubles the number
of rows.
tb.open('3C273XC1.MS',nomodify=False)
tb.copyrows('3C273XC1.MS')
tb.close()
This example copies 10 rows of the selected subset of the MS to the
beginning of the output MS.
!rm -rf in.MS out.MS
ms.fromfits('in.MS','3C273XC1.fits') #Make two MSs
ms.fromfits('out.MS','3C273XC1.fits') #for example
ms.close()
tb.open("in.MS")
t1 = tb.query('ANTENNA1==0')
tb.close()
t1.copyrows("out.MS",nrow=10,startrowout=0)
t1.close()
"""
pass
[docs] def done(self):
r"""
Effectively a synonym for function close.
"""
pass
[docs] def iswritable(self):
r"""
Test if the table is opened for write.
"""
pass
[docs] def isopened(self, tablename=''):
r"""
Test if the table is opened in the process this function is invoked
.. rubric:: Parameters
- ``tablename (string='')`` - table name to see if it is opened
.. rubric:: Returns
``bool``
"""
pass
[docs] def lock(self, write=True, nattempts=0):
r"""
Try to acquire a read or write lock on the table. Nothing will be done if the table is already correctly locked by this process. It is only needed when user locking is used. When the lock is acquired, the internal caches will be synchronized with the (possibly changed) contents of the table. is possible to specify the number of attempts to do (1 per second) in case the table is locked by another process. The default 0 is trying indefinitely.
.. rubric:: Parameters
- ``write (bool=True)`` - Write lock? (F=read lock)
- ``nattempts (int=0)`` - Nr of attempts
.. rubric:: Returns
``bool``
"""
pass
[docs] def unlock(self):
r"""
The table is flushed and the lock on the table is released. This function is only needed when user locking is used. However, it is also possible to use it with auto locking. In that case the lock will automatically be re-acquired before the next table operation.
"""
pass
[docs] def datachanged(self):
r"""
This function tests if data in the table have changed (by another process) since the last call to this function.
"""
pass
[docs] def haslock(self, write=True):
r"""
Has this process a read or write lock on the table?
.. rubric:: Parameters
- ``write (bool=True)`` - Has it a write lock? (F=read lock)
.. rubric:: Returns
``bool``
"""
pass
[docs] def lockoptions(self):
r"""
Get the lock options used for this table. It returns a record with the fields: option, interval and maxwait. The record can be used as the lockoptions argument when opening a table.
"""
pass
[docs] def ismultiused(self, checksubtables=False):
r"""
Is the table still in use in another process? If so, the table cannot be deleted.
.. rubric:: Parameters
- ``checksubtables (bool=False)`` - check if subtables are multiused?)
.. rubric:: Returns
``bool``
"""
pass
[docs] def browse(self):
r"""
To start the browser, the environment variable DISPLAY must be set.
"""
pass
[docs] def name(self):
r"""
Gives the name of the table on disk that the table tool has open.
"""
pass
[docs] def createmultitable(self, outputTableName='', tables='', subdirname=''):
r"""
Create a virtually concatenated table
.. rubric:: Parameters
- ``outputTableName (string='')`` - name of the concatenated table
- ``tables (stringVec='')`` - list of the names of the tables to be concatenated
- ``subdirname (string='')`` - optional name of the subdirectory into which the input tables are moved
.. rubric:: Returns
``bool``
"""
pass
[docs] def toasciifmt(self, asciifile='', headerfile='', columns='', sep=''):
r"""
Write a table into an ASCII format approximately compatible with fromascii except that in order to permit variable shaped arrays (as they often occur in MSs), array values are output enclosed in square brackets. The separator between values can be specified and defaults to a blank. Note that columns containing invalid data or record type data are ignored and a warning is issued.
If the argument headerfile is set then the header information is written to that file instead of the first two lines of the data file.
.. rubric:: Parameters
- ``asciifile (string='')`` - Name of ASCII file to be written
- ``headerfile (string='')`` - Name of an optional file defining the format
- ``columns (stringVec='')`` - Names of columns to be written, default is all
- ``sep (string='')`` - Value separator, default is one blank
.. rubric:: Returns
``bool``
.. rubric:: Examples
::
tb.toasciifmt(asciifile='myfile3.dat', headerfile='myfile3.head', columns=['SOURCE_ID', 'NAME', 'PROPER_MOTION'], sep=', ')
will produce a comma separated ASCII output of the three columns 'SOURCE_ID', 'NAME', and 'PROPER_MOTION' in file 'myfile3.dat'
and a format description in 'myfile3.head'.
tb.toasciifmt(asciifile='myfile.dat')
will produce a space separated ASCII output of all table columns into file 'myfile.dat'
with the first two lines containing a format description.
"""
pass
[docs] def taql(self, taqlcommand='TaQL expression'):
r"""
This method Expose TaQL to the user. Details on TaQL maybe found at https://casacore.github.io/casacore-notes/199.html
.. rubric:: Parameters
- ``taqlcommand (string='TaQL expression')`` - TaQL expression
.. rubric:: Returns
``table``
.. rubric:: Examples
::
For more information on TaQL see https://casacore.github.io/casacore-notes/199.html
"""
pass
[docs] def query(self, query='String', name='', sortlist='', columns='', style=''):
r"""
Make a table from a query applied to the current table. It is possible to specify column(s) and/or expressions to sort on and to specify the columns to be contained in the output table. See the example below. A new "on-the-fly" table tool is returned. The new (reference) table can be given a name and will then be written to disk. Note that the resulting table is just a reference to the original table. One can make a deep copy of the query result using the copy function (see example).
.. rubric:: Parameters
- ``query (string='String')`` - Query string
- ``name (string='')`` - Name of resulting reference table
- ``sortlist (string='')`` - Sort string (one or more expressions separated by commas)
- ``columns (string='')`` - List of column names separated by commas
- ``style (string='')`` - How to handle numeric ranges and order axes
.. rubric:: Returns
``table``
.. rubric:: Examples
::
tb.open("3C273XC1.MS")
subt=tb.query("OBSERVATION_ID==0",
sortlist="ARRAY_ID", columns="TIME, DATA, UVW")
print subt.ncols()
# 23
tb.close()
copyt = subt.copy ("3C273XC1_spw1.MS", True)
subt.close()
copyt.close()
From the original table corresponding to the disk file 3C273XC1.MS, only rows
with OBSERVATION\_ID equal to 0 are selected and sorted by ARRAY\_ID. Only the
columns TIME DATA UVW are written. Thereafter a deep copy of the result is
made. This table query command is equivalent to the Table Query Language
(TaQL) command
SELECT TIME, DATA, UVW
FROM 3C273XC1.MS
WHERE OBSERVATION_ID==0
ORDERBY ARRAY_ID
See https://casacore.github.io/casacore-notes/199.html for an explanation of TaQL.
If "style" is not blank, "using style \<style\> " is prepended to the
query. See
https://casacore.github.io/casacore-notes/199.html#x1-70002.2 for
an explanation and list of choices for style. The default (glish) style is
1-based, inclusive end, and Fortran ordering. You may prefer python (0-based,
exclusive end, and C order) style.
tb.open('any_data')
tsel = tb.selectrows([0])
print tsel.nrows() # returns 1
tsel = tb.query('ROWNUMBER()==0')
print tsel.nrows() # returns 0
tsel = tb.query('ROWNUMBER()==0', style='python')
print tsel.nrows() # returns 1
tb.close()
Note that style had no effect on the "OBSERVATION_ID==0" query above.
"""
pass
[docs] def calc(self, expr='', prefix='using style base0, endincl, fortranorder', showtaql=False):
r"""
Get the result from the calculation of an expression on a table
The expression can be any expression that can be given in the WHERE clause of a SELECT expression (thus including subqueries). The given expression determines if the result is a scalar, a vector, or a record containing arrays. See the examples below.
.. rubric:: Parameters
- ``expr (string='')`` - Expression string
- ``prefix (string='using style base0, endincl, fortranorder')`` - TaQL prefix for style and ordering etc ...check TaQL note 199 for usage
- ``showtaql (bool=False)`` - Show the full taql command used
.. rubric:: Returns
``any``
.. rubric:: Examples
::
tb.calc('[select from ngc5921.ms giving [mean(abs(DATA))]]')
find the mean of the abs of each row of the DATA column of the MeasurementSet ngc5921.ms
returns a (potentially enormous) record where a field contains the
value of the expression for the row with that number. Note that it
returns a record because for each row the expression results in an array.
It should be clear that this example is useless. However, something
like this could be useful for a column with (very) small arrays.
tb.calc('[select from ngc5921.ms.contsub giving [nTrue(FLAG)]]')
returns for each row the number of flags set. The result is a vector,
because for each row the expression results in a scalar.
tb.calc('sum([select from ngc5921.ms.contsub giving [nTrue(FLAG)]])')
returns the total number of flags set in the table (in a single
scalar).
using subrow array
tb.calc('median([select from ngc5921.ms where ANTENNA1==3 && ANTENNA2==5 giving [abs(DATA[0,31])]])')
The above will find the median channel 31 and 0th pol of the requested baseline formed with antennas 3 and 5.
Note that the that the default casa order of arrays is fortran order ...pol axis is before channel axis
tb.calc('median([select from ngc5921.ms where ANTENNA1==3 && ANTENNA2==5 giving [abs(DATA[31, 0])]])', prefix='using style python')
Now the same is as the above but using the python style of axis ordering access
"""
pass
[docs] def selectrows(self, rownrs='', name=''):
r"""
Create a (reference) table containing a given subset of rows. It is, for instance, useful when a selection is done on another table containing the row numbers in the main table. It can be useful to apply the casapy function unique to those row numbers, otherwise the same row might be included multiple times (see example).
It is possible to give a name to the resulting table. If given, the resulting table is made persistent with that table name. Otherwise the table is transient and disappears when closed or when casapy exits.
The rownumbers function returns a vector containing the row number in the main table for each row in the selection table. Thus given a row number vector ``rownrs``, the following is always True.
::
rownrs == tb.selectrows(rownrs).rownumbers()
However, it is not True when selectrows is used on a selection table. because ``rownumbers`` does not return the row number in that selection table but in the main table. means that one has to take great care when using ``selectrows`` on a selection table.
.. rubric:: Parameters
- ``rownrs (intVec='')`` - 0-based Row Numbers
- ``name (string='')`` - Name of resulting table
.. rubric:: Returns
``table``
.. rubric:: Examples
::
# EXAMPLE NOT VERIFIED SINCE query IS BROKEN
# Do the query on the main table.
tb.open('SOMENAME')
scantable = tb.query(command)
# Get the column containing the 0-based row numbers in the BACKEND table.
# Make the row numbers unique. NEED TO REPLACE GLISH unique FUNCTION HERE!
backrows = unique(scantable.getcol('NS_GBT_BACKEND_ID'))
# Form the table subset of the BACKEND table containing those rows.
tb.close()
tb.open('SOMENAME/GBT_BACKEND')
scanback = tb.selectrows(backrows);
# Do something with that table.
print scanback.nrows();
"""
pass
[docs] def info(self):
r"""
The info record contains information on the table.
"""
pass
[docs] def putinfo(self, value=''):
r"""
The info record contains information on the table. It is written by applications, and used to determine what type of information is stored in a table.
.. rubric:: Parameters
- ``value (record='')`` - Info record
.. rubric:: Returns
``bool``
"""
pass
[docs] def addreadmeline(self, value=''):
r"""
A readme line is part of the info record associated with a table. It is to inform the user, and is not used by any application directly.
.. rubric:: Parameters
- ``value (string='')`` - readme line
.. rubric:: Returns
``bool``
"""
pass
[docs] def summary(self, recurse=False):
r"""
A (terse) summary of the table contents is sent to the defaultlogger.
.. rubric:: Parameters
- ``recurse (bool=False)`` - Summarize subtables recursively
.. rubric:: Returns
``bool``
.. rubric:: Examples
::
tb.open("tcal")
tb.summary()
# successful nomodify open of table tcal : 9 columns, 11 rows
# Table summary: tcal
# Shape: 9 columns by 11 rows
# Info: [type=Calibration, subType=T Jones, readme=]
# Table keywords: [Type=T Jones, Interval=30, DeltaT=1]
# Columns: StartTime StopTime Gain SolutionOK Fit FitWeight
# iSolutionOK iFit iFitWeight
"""
pass
[docs] def colnames(self):
r"""
The names of the columns in the table are returned as a vector of Strings.
"""
pass
[docs] def rownumbers(self, tab='', nbytes=0):
r"""
!!!NOTE INPUT PARAMETERS IGNORED!!!
This function can be useful after a selection or a sort. It returns the row numbers of the rows in this table with respect to the given table. If no table is given, the original table is used. example:
::
!!!NOTE INPUT PARAMETERS IGNORED!!!
tb.open('3C273XC1.MS')
t1=tb.selectrows([1,3,5,7,9])
t1.rownumbers()
# [1L, 3L, 5L, 7L, 9L]
t2=t1.selectrows([2,4])
t2.rownumbers(t1)
# [2L, 4L]
t2.rownumbers(tb.name())
# [5L, 9L]
t2.rownumbers()
# [5L, 9L]
The last statements show that the function returns the row numbers referring to the given table. Table t2 contains rows 2 and 4 in table t1, which are rows 5 and 9 in table ’3C273XC1.MS’.
Note that when a table is opened using its name, that table can be a reference table. Thus in the example above the last 2 statements may give different results depending on the fact if 3C273XC1.MS is a reference table or not. function should always be called with a table argument. The ability of omitting the argument is only present for backward compatibility.
The function can be useful to get the correct values from the result of a getcol or getcolslice on the original table.
!!!NOTE INPUT PARAMETERS IGNORED!!!
.. rubric:: Parameters
- ``tab (record='')`` - Table to which the row numbers refer
- ``nbytes (int=0)`` - Maximum cache size in bytes
.. rubric:: Returns
``intVec``
.. rubric:: Examples
::
!!!NOTE INPUT PARAMETERS IGNORED!!!
tb.open("3C273XC1.MS")
tb.nrows()
#7669L
data=tb.getcolslice("DATA", [0,0], [0,0])
data.shape
#(1, 1, 7669)
selt=tb.query("ANTENNA1==1")
selt.nrows()
#544L
print len(selt.rownumbers())
#544L
"""
pass
[docs] def setmaxcachesize(self, columnname='', nbytes=''):
r"""
It can sometimes be useful to limit the size of the cache used by a column stored with the tiled storage manager. This function requires some more knowledge about the table system and is not meant for the casual user.
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
- ``nbytes (int='')`` - Maximum cache size in bytes
.. rubric:: Returns
``bool``
.. rubric:: Examples
::
tb.open("3C273XC1.MS")
tb.nrows()
# 7669L
tb.setmaxcachesize ("DATA", 4*1024*1024);
# True
"""
pass
[docs] def isscalarcol(self, columnname=''):
r"""
A column may contain either scalars or arrays in each cell. This tool function tests if the specified column has scalar contents.
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
.. rubric:: Returns
``bool``
.. rubric:: Examples
::
tb.open("tcal")
tb.isscalarcol("StartTime")
# True
tb.open("tcal")
tb.isscalarcol("Gain")
# False
"""
pass
[docs] def isvarcol(self, columnname=''):
r"""
This functions tells if the column contains variable shaped arrays. If so, the function ``getvarcol`` should be used to get the entire column. Otherwise ``getcol`` can be used.
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
.. rubric:: Returns
``bool``
"""
pass
[docs] def coldatatype(self, columnname=''):
r"""
A column may contain various data types. This tool function returns the type of the column as a string.
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
.. rubric:: Returns
``string``
.. rubric:: Examples
::
tb.open("tcal")
tb.coldatatype("StartTime")
# double
tb.open("tcal")
tb.coldatatype("Gain")
# complex
"""
pass
[docs] def colarraytype(self, columnname=''):
r"""
The possible column array types are defined as:
FixedShape
FixedShape means that the shape of the array must be the same in each cell of the column. If not given, the array shape may vary. Option Direct forces FixedShape.
Direct
Direct means that the data is directly stored in the table. Direct forces option FixedShape. If not given, the array is indirect, which implies that the data will be stored in a separate file.
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
.. rubric:: Returns
``string``
.. rubric:: Examples
::
tb.open("tcal")
tb.colarraytype("Gain")
# Direct,FixedShape
"""
pass
[docs] def ncols(self):
r"""
return number of columns
"""
pass
[docs] def nrows(self):
r"""
Note that rows are numbered starting at 0.
"""
pass
[docs] def addrows(self, nrow=1):
r"""
Rows can be added to the end of a table that was opened nomodify=False. The new rows are empty.
.. rubric:: Parameters
- ``nrow (int=1)`` - Number of rows to add
.. rubric:: Returns
``bool``
"""
pass
[docs] def removerows(self, rownrs=''):
r"""
Remove the row numbers specified in the vector from the table. It fails when the table does not support row removal.
.. rubric:: Parameters
- ``rownrs (intVec='')`` - Row numbers to remove
.. rubric:: Returns
``bool``
"""
pass
[docs] def addcols(self, desc='', dminfo=''):
r"""
Columns can be added to a table that was opened nomodify=False. The new columns will be filled with a default value (0 or blank).
!!!THESE COLUMN DESCRIPTION FUNCTIONS HAVE NOT BEEN IMPLEMENTED!!! each column to be added a column description has to be setup using function tablecreatescalarcoldesc or tablecreatearraycoldesc. When multiple columns are used, they have to be combined in a single record using tablecreatedesc. is possible to specify data manager info in order to define a data manager (storage manager or virtual column engine) for the columns to be added.
.. rubric:: Parameters
- ``desc (record='')`` - Description of one or more columns
- ``dminfo (record='')`` - Optional description data manager to use
.. rubric:: Returns
``bool``
.. rubric:: Examples
::
!!!REQUIRES COLUMN DESCRIPTION FUNCTIONS THAT HAVE NOT BEEN IMPLEMENTED!!!
tb.open("mytable", nomodify=False)
dc3=tablecreatescalarcoldesc('C3', 'a')
dc4=tablecreatescalarcoldesc('C4', as_float(0))
dc5=tablecreatearraycoldesc('C5', as_double(0), 2, [10,20])
tb.addcols(dc3)
# True
tb.addcols(tablecreatedesc(dc4, dc5))
# True
A single column can be added as such, but multiple columns have
to be combined.
"""
pass
[docs] def renamecol(self, oldname='', newname=''):
r"""
A column can be renamed in a table that was opened nomodify=False. , renaming is not possible in a (reference) table resulting from a select or sort operation.
.. rubric:: Parameters
- ``oldname (string='')`` - name of column to be renamed
- ``newname (string='')`` - new name of column
.. rubric:: Returns
``bool``
.. rubric:: Examples
::
tb.open("3C273XC1.MS", nomodify=False)
tb.renamecol ('DATA', 'DATA2')
# T
print tb.colnames()
tb.renamecol ('DATA2', 'DATA')
# T
print tb.colnames()
Column \texttt{DATA} is renamed to \texttt{DATA2} and then back to
\texttt{DATA} again..
"""
pass
[docs] def removecols(self, columnames=''):
r"""
Columns can be removed from a table that was opened nomodify=False. may not always be possible to remove a column, because some data managers do not support column removal. However, if all columns of a data manager are removed, it will always succeed. It results in the removal of the entire data manager (and its possible files). that function getdminfo can be used to find which columns are served by which data manager.
.. rubric:: Parameters
- ``columnames (stringVec='')`` - names of columns to be removed
.. rubric:: Returns
``bool``
.. rubric:: Examples
::
tb.open("mytable", nomodify=False)
tb.removecols ("col1 col2")
# T
print tb.colnames()
Two columns are removed.
"""
pass
[docs] def iscelldefined(self, columnname='', rownr=0):
r"""
A column containing variable shaped arrays can have an empty cell (if no array has been put into it). This function tests if a cell is defined (thus is not empty). Note that a scalar column and a fixed shape array column cannot have empty cells.
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
- ``rownr (int=0)`` - Row number, starting at 0
.. rubric:: Returns
``bool``
"""
pass
[docs] def getcell(self, columnname='', rownr=0):
r"""
A cell is the value at one row in one column. It may be a scalar or an array.
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
- ``rownr (int=0)`` - Row number, starting at 0
.. rubric:: Returns
``any``
"""
pass
[docs] def getcellslice(self, columnname='', rownr='', blc='', trc='', incr=[1]):
r"""
A cell is the value at one row in one column. It must be an array. The slice must be specified as blc, trc with an optional stride. blc and trc -1 can be used to indicate all values for a dimension (-1 in blc is equivalent to 0, so -1 is especially useful for trc).
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
- ``rownr (int='')`` - Row number, starting at 0
- ``blc (intVec='')`` - Bottom left corner (e.g. [0,0,0] is start of 3D array)
- ``trc (intVec='')`` - Top right corner
- ``incr (intVec=[1])`` - Stride (defaults to 1 for all axes)
.. rubric:: Returns
``any``
.. rubric:: Examples
::
tb.open("3C273XC1.MS")
data=tb.getcellslice("DATA", 0, [0,0], [1,0])
print data.shape
# [2 1]
"""
pass
[docs] def getcol(self, columnname='', startrow=0, nrow=-1, rowincr=1):
r"""
The entire column (or part of it) is returned. Warning: it might be big! The functions can only be used if all arrays in the column have the same shape. That is guaranteed for columns containing scalars or fixed shaped arrays. For columns containing variable shaped arrays it only succeeds if all those arrays happen to have the same shape. that function ``getvarcol`` can be used to get a column of arbitrary shaped arrays, which also handles empty cells correctly. Function ``isvarcol`` tells if a column contains variable shaped arrays. shaped
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
- ``startrow (int=0)`` - First row to read (default 0)
- ``nrow (int=-1)`` - Number of rows to read (default -1 means till the end)
- ``rowincr (int=1)`` - Increment in rows to read (default 1)
.. rubric:: Returns
``any``
.. rubric:: Examples
::
tb.open("3C273XC1.MS")
# True
gain=tb.getcol("DATA")
print gain.shape
# (4, 1, 7669)
"""
pass
[docs] def getvarcol(self, columnname='', startrow=0, nrow=-1, rowincr=1):
r"""
Function ``getcol`` can only used if values in the column cells to get have the same shape. Function ``getvarcol`` addresses this limitation by returning the values as a record instead of an array. Each field in the record contains the value for a column cell. If the value is undefined (i.e. the cell does not contain a value), the unset value is put in the record. Each field name is the letter r followed by the row number. The length of the record is the number of rows to get. that the function ``isvarcol`` tells if a column contains variable shaped arrays.
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
- ``startrow (int=0)`` - First row to read (default 0)
- ``nrow (int=-1)`` - Number of rows to read (default -1 means till the end)
- ``rowincr (int=1)`` - Increment in rows to read (default 1)
.. rubric:: Returns
``record``
.. rubric:: Examples
::
tb.open("3C273XC1.MS")
gain=tb.getvarcol("DATA")
print len(gain)
# 7669
"""
pass
[docs] def getcolslice(self, columnname='', blc='', trc='', incr='', startrow=0, nrow=-1, rowincr=1):
r"""
A slice from the entire column (or part of it) is returned. Warning: it might be big! blc and trc -1 can be used to indicate all values for a dimension (-1 in blc is equivalent to 1, so -1 is especially useful for trc). Note that blc and trc should not contain the row number, only the blc and trc of the arrays in the column.
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
- ``blc (intVec='')`` - Bottom left corner (e.g. [0,0,0] is start of 3D array)
- ``trc (intVec='')`` - Top right corner
- ``incr (intVec='')`` - Stride (defaults to 1 for all axes)
- ``startrow (int=0)`` - First row to read (default 0)
- ``nrow (int=-1)`` - Number of rows to read (default -1 means till the end)
- ``rowincr (int=1)`` - Increment in rows to read (default 1)
.. rubric:: Returns
``any``
.. rubric:: Examples
::
tb.open("3C273XC1.MS")
data=tb.getcolslice("DATA", [0,0], [1,0])
data.shape
# (2 1 7669)
"""
pass
[docs] def getcoliter(self, columnname='', startrow=0, nrow=-1, rowincr=1, torecord=False):
r"""
This function returns an iterator that will allow for retrieval of the elements of one or more table columns (or a subset of the columns).
.. rubric:: Parameters
- ``columnname (stringVec='')`` - Name of column
- ``startrow (int=0)`` - First row to read (default 0)
- ``nrow (int=-1)`` - Total number of rows to return (default -1 means till the end)
- ``rowincr (int=1)`` - Increment in rows to read (default 1)
- ``torecord (bool=False)`` - when more than one column is retrieved use a dictionary to return the value (default False)
.. rubric:: Returns
``pyobj``
.. rubric:: Examples
::
This example creates an iterator which returns tuples made up of the
values from the TIME column and the EXPOSURE column. It will return
these tuples one at a time, but it will stop after returning five
tuples. It starts at row zero and skips 999 rows to get to the row
used to return the next tuple.
for i in tb.getcoliter(['TIME','EXPOSURE'],startrow=0,nrow=5,rowincr=1000):
print(i)
This example has the same iteration behavior but instead of returning
tuples it returns records containing two key/value pairs, TIME and
EXPOSURE.
for i in tb.getcoliter(['TIME','EXPOSURE'],startrow=0,nrow=5,rowincr=1000,torecord=True):
print(i)
This example shows how getcoliter can be used to load table columns
into a Pandas dataframe.
pd = pandas.DataFrame.from_records(tb.getcoliter(['TIME','DATA'],torecord=True))
This example shows how to use getcoliter to load table columns into
a Dask dataframe by concatenating Pandas dataframes. This creates
chunking based upon the number of rows returned by getcoliter.
import math
import pandas as pd
from dask import delayed, compute
import dask.dataframe as dd
from functools import reduce
from casatools import table
tbx = table(expanduser('2018.1.00879.S_tclean.ms'))
part_start = range( 0, tbx.nrows( ), math.ceil(tbx.nrows( ) / 500) )
part = reduce( lambda acc, x: acc+[(x, math.ceil(tbx.nrows( ) / 500) if x + math.ceil(tbx.nrows( ) / 500) < tbx.nrows() else tbx.nrows()-x)],
part_start, [] )
lst = [delayed(lambda x: pd.DataFrame(table('2018.1.00879.S_tclean.ms').getcoliter(['TIME','DATA'],*x,torecord=True)))(l) for l in part]
df = dd.concat(compute(lst)[0]
If you looked at the elements part and lst you would find:
In [5]: part[:5]
Out[5]: [(0, 632), (632, 632), (1264, 632), (1896, 632), (2528, 632)]
In [6]: lst[:5]
Out[6]:
[Delayed('lambda-af8d8b10-9526-4988-ac7c-8e8e5cd99934'),
Delayed('lambda-b9035365-a074-42ce-b04e-2881179f573a'),
Delayed('lambda-d44d97d8-b6ce-4d6e-a6e0-4326bdfabf01'),
Delayed('lambda-c84f7ed8-af6c-4efc-a225-437353fc1393'),
Delayed('lambda-8fa0e6cb-01c3-4433-acf0-e3e9b8d26a35')]
The Dask dataframe is df, but it is not completely evaluated:
In [7]: df
Out[7]:
Dask DataFrame Structure:
TIME DATA
npartitions=500
object object
... ...
... ... ...
... ...
... ...
Dask Name: concat, 501 graph layers
Evaluating again resolves the contents of the dataframe:
In [8]: df = df.compute()
In [9]: df
Out[9]:
TIME DATA
0 [5053663310.208] [[[(19.649103+0j)], [(19.62976+0j)], [(19.6710...
1 [5053663310.208] [[[(16.316814+0j)], [(16.301018+0j)], [(16.321...
2 [5053663310.208] [[[(15.963612+0j)], [(15.9756+0j)], [(15.98898...
3 [5053663310.208] [[[(17.416065+0j)], [(17.409616+0j)], [(17.393...
4 [5053663310.208] [[[(20.466146+0j)], [(20.48779+0j)], [(20.5097...
.. ... ...
467 [5053664322.4800005] [[[(-0.8585681-0.7359103j)], [(0.8410164+1.934...
468 [5053664322.4800005] [[[(-29.532145-12.631435j)], [(-27.726702-11.6...
469 [5053664322.4800005] [[[(-1.8737663-0.93042445j)], [(-0.79967415-1....
470 [5053664322.4800005] [[[(-1.6989801-1.1293286j)], [(-2.898021-1.001...
471 [5053664322.4800005] [[[(0.22554477+3.4367344j)], [(-2.8623238+1.58...
[315840 rows x 2 columns]
"""
pass
[docs] def putcell(self, columnname='', rownr='', thevalue=''):
r"""
A cell is the the value at one row in one column. It may be a scalar or an array.
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
- ``rownr (intVec='')`` - Row number(s) (0-relative)
- ``thevalue (variant='')`` - Value
.. rubric:: Returns
``bool``
"""
pass
[docs] def putcellslice(self, columnname='', rownr='', value='', blc='', trc='', incr=[1]):
r"""
A cell is the value at one row in one column. It must be an array. The slice must be specified as blc, trc with an optional stride. blc and trc -1 can be used to indicate all values for a dimension (-1 in blc is equivalent to 0, so -1 is especially useful for trc).
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
- ``rownr (int='')`` - Row number, starting at 0
- ``value (variant='')`` - Value
- ``blc (intVec='')`` - Bottom left corner (e.g. [0,0,0] is start of 3D array)
- ``trc (intVec='')`` - Top right corner
- ``incr (intVec=[1])`` - Stride (defaults to 1 for all axes)
.. rubric:: Returns
``bool``
"""
pass
[docs] def putcol(self, columnname='', value='', startrow=0, nrow=-1, rowincr=1):
r"""
put a specific column
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
- ``value (variant='')`` - Array
- ``startrow (int=0)`` - First row to put (default 0)
- ``nrow (int=-1)`` - Number of rows to put (default -1 means till the end)
- ``rowincr (int=1)`` - Increment in rows to put (default 1)
.. rubric:: Returns
``bool``
.. rubric:: Examples
::
tb.open("3C273XC1.MS",nomodify=False)
data=tb.getcol("DATA")
# [could modify data here]
tb.putcol("DATA", data)
tb.flush()
"""
pass
[docs] def putvarcol(self, columnname='', value='', startrow=0, nrow=-1, rowincr=1):
r"""
``putcol`` can only used if values in the column cells to put have the same shape. ``putvarcol`` addresses this limitation by passing the values as a record instead of an array. Each field in the record contains the value for a column cell. So the length of the record has to match the number of rows to put. If a value is the unset value, no put is done for that row.
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
- ``value (record='')`` - Record with values
- ``startrow (int=0)`` - First row to put (default 0)
- ``nrow (int=-1)`` - Number of rows to put (default -1 means till the end)
- ``rowincr (int=1)`` - Increment in rows to put (default 1)
.. rubric:: Returns
``bool``
.. rubric:: Examples
::
tb.open("3C273XC1.MS",nomodify=False)
gain=tb.getvarcol("DATA", 0, 10)
tb.putvarcol("Gain", gain, 10, 10)
tb.flush()
This example copies the values from row 0-9 to row 10-19.
"""
pass
[docs] def putcolslice(self, columnname='', value='', blc='', trc='', incr=[1], startrow=0, nrow=-1, rowincr=1):
r"""
In blc and trc, -1 can be used to indicate all values for a dimension (-1 in blc is equivalent to 0, so -1 is especially useful for trc). Note that blc and trc should not contain the row number, only the blc and trc of the arrays in the column.
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
- ``value (variant='')`` - Array
- ``blc (intVec='')`` - Bottom left corner (e.g. [0,0,0] is start of 3D array)
- ``trc (intVec='')`` - Top right corner
- ``incr (intVec=[1])`` - Stride (defaults to 1 for all axes)
- ``startrow (int=0)`` - First row to put (default 0)
- ``nrow (int=-1)`` - Number of rows to put (default -1 means till the end)
- ``rowincr (int=1)`` - Increment in rows to put (default 1)
.. rubric:: Returns
``bool``
.. rubric:: Examples
::
tb.open("3C273XC1.MS",nomodify=False)
data_all=tb.getcolslice("DATA", [-1,-1], [-1,=1])
print data_all.shape
# (4, 1, 7669)
data=tb.getcolslice("DATA", [0,0],[3,0])
# can modify data here
tb.putcolslice("DATA", data, [0,0],[3,0])
tb.flush()
"""
pass
[docs] def getcolshapestring(self, columnname='', startrow=0, nrow=-1, rowincr=1):
r"""
The shapes of the arrays in the entire column (or part of it) are returned as strings like [20,3]. When the column contains fixed shaped arrays, a single string is returned. Otherwise a vector of strings is returned.
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
- ``startrow (int=0)`` - First row to read (default 0)
- ``nrow (int=-1)`` - Number of rows to read (default -1 means till the end)
- ``rowincr (int=1)`` - Increment in rows to read (default 1)
.. rubric:: Returns
``stringVec``
.. rubric:: Examples
::
tb.open("3C273XC1.MS")
shapes=tb.getcolshapestring("DATA"))
print len(shapes)
"""
pass
[docs] def getkeyword(self, keyword=''):
r"""
The value of the given table keyword is returned. The value can be of any type, including a record and a table. a keyword is a table, its value is returned as a string containing the table name prefixed by ’Table: ’. is possible that the value of a keyword is a record itself (arbitrarily deeply nested). A field in such a subrecord can be read by separating the name with dots.
.. rubric:: Parameters
- ``keyword (variant='')`` - Name or seqnr of keyword: string or int
.. rubric:: Returns
``any``
.. rubric:: Examples
::
tb.open('3C273XC1.MS')
tb.getkeywords()
tb.getkeyword('MS_VERSION')
# 2.0
tb.close()
tb.open('tcal')
tb.getkeyword('rec.fld') # get field from a record
# 3.14
"""
pass
[docs] def getkeywords(self):
r"""
The values of all table keywords are returned. The values can be of any type, including a record and a table. a keyword is a table, its value is returned as a string containing the table name prefixed by ’Table: ’.
"""
pass
[docs] def getcolkeyword(self, columnname='', keyword=''):
r"""
The value of the given column keyword is returned. The value can be of any type, including a record and a table. a keyword is a table, its value is returned as a string containing the table name prefixed by ’Table: ’. is possible that the value of a keyword is a record itself (arbitrarily deeply nested). A field in such a subrecord can be read by separating the name with dots.
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
- ``keyword (variant='')`` - Name or seqnr of keyword: string or int
.. rubric:: Returns
``any``
.. rubric:: Examples
::
tb.open("3C273XC1.MS")
tb.getcolkeyword("UVW", "QuantumUnits")
#array(['m', 'm', 'm'],
# dtype='|S2')
"""
pass
[docs] def getcolkeywords(self, columnname=''):
r"""
The values of all keywords for the given column are returned. The values can be of any type, including a record and a table. a keyword is a table, its value is returned as a string containing the table name prefixed by ’Table: ’.
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
.. rubric:: Returns
``any``
.. rubric:: Examples
::
tb.open("3C273XC1.MS")
tb.getcolkeywords("UVW")
#{'MEASINFO': {'Ref': 'ITRF', 'type': 'uvw'},
# 'QuantumUnits': array(['m', 'm', 'm'],
# dtype='|S2')}
"""
pass
[docs] def putkeyword(self, keyword='', value='', makesubrecord=False):
r"""
Put a table keyword. The value of the keyword can be a scalar or an array of any type or it can be a record. is possible to define a keyword holding a subtable. In that case a special string containing the name of the subtable will be passed to the table client. is possible that the value of a keyword is a record itself (arbitrarily deeply nested). A field in such a subrecord can be written by separating the name with dots. If a subrecord does not exist, an error is returned unless ``makesubrecord=True`` is given. In such a case intermediate records are created when needed.
.. rubric:: Parameters
- ``keyword (variant='')`` - Name or seqnr of keyword: string or int
- ``value (variant='')`` - Value of keyword
- ``makesubrecord (bool=False)`` - Create intermediate records
.. rubric:: Returns
``bool``
.. rubric:: Examples
::
tb.open("3C273XC1.MS", nomodify=False)
tb.putkeyword("VERSION", "1.66")
# True
# define ANTENNA subtable
tb.putkeyword("ANTENNA", 'Table: 3C273XC1.MS/ANTENNA')
tb.flush()
# True
# write a field in a record and create subrecords when needed
tb.putkeyword("REC.SUB.FLD", "val", True)
# True
# write a keyword with a record value
tb.putkeyword("REC", {'SUB': {'FLD': 'val'}})
# True
Note that the last example does the same as the previous one (assuming
that \texttt{REC} does not exist yet with other fields).
"""
pass
[docs] def putkeywords(self, value=''):
r"""
Put multiple table keywords. All fields in the given record are put as table keywords. The value of each field can be a scalar or an array of any type or it can be a record. is also possible to define a keyword holding a subtable. This can be done by giving the keyword a string value consisting of the subtable name prefixed by ’Table: ’.
.. rubric:: Parameters
- ``value (record='')`` - Record of keyword=value pairs
.. rubric:: Returns
``bool``
.. rubric:: Examples
::
tb.open('3C273XC1.MS', nomodify=False)
kw=tb.getkeywords()
print kw['MS_VERSION']
# 2.0
kw['MS_VERSION']=2.1
tb.putkeywords(kw)
# !!!BROKEN. Keywords containing float are not handled properly!!!
tb.flush()
# True
"""
pass
[docs] def putcolkeyword(self, columnname='', keyword='', value=''):
r"""
Put a keyword in the given column. The value of the keyword can be a scalar or an array of any type or it can be a record. is possible to define a keyword holding a subtable. In that case a special string containing the name of the subtable will be passed to the table client. is possible that the value of a keyword is a record itself (arbitrarily deeply nested). A field in such a subrecord can be written by separating the name with dots. If a subrecord does not exist, an error is returned unless ``makesubrecord=True`` is given. In such a case intermediate records are created when needed.
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
- ``keyword (variant='')`` - Name or seqnr of keyword,string or int
- ``value (variant='')`` - Value of keyword
.. rubric:: Returns
``bool``
.. rubric:: Examples
::
tb.open("3C273XC1.MS", nomodify=False)
ckw=tb.getcolkeyword("UVW","QuantumUnits")
print ckw
# modify ckw as desired
tb.putcolkeyword("UVW","QuantumUnits",ckw)
# True
tb.flush()
# True
"""
pass
[docs] def putcolkeywords(self, columnname='', value=''):
r"""
Put multiple keywords in the given column. All fields in the given record are put as column keywords. The value of each field can be a scalar or an array of any type or it can be a record. is also possible to define a keyword holding a subtable. This can be done by giving the keyword a string value consisting of the subtable name prefixed by ’Table: ’.
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
- ``value (record='')`` - Record of keyword=value pairs
.. rubric:: Returns
``bool``
.. rubric:: Examples
::
tb.open("3C273XC1.MS", nomodify=False)
kws = tb.getcolkeywords("UVW")
kws
#{'MEASINFO': {'Ref': 'ITRF', 'type': 'uvw'},
# 'QuantumUnits': array(['m', 'm', 'm'],
# dtype='|S2')}
kws['MEASINFO']['Ref']='B1950'
tb.putcolkeywords(kws)
# True
"""
pass
[docs] def removekeyword(self, keyword=''):
r"""
remove a specific table keyword
.. rubric:: Parameters
- ``keyword (variant='')`` - Name or seqnr of keyword: string or int
.. rubric:: Returns
``bool``
.. rubric:: Examples
::
tb.open("3C273XC1.MS", nomodify=False)
tb.removekeyword("MS_VERSION")
# True
tb.flush()
# True
"""
pass
[docs] def removecolkeyword(self, columnname='', keyword=''):
r"""
remove a specific keyword for a column
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
- ``keyword (variant='')`` - Name or seqnr of keyword: string or int
.. rubric:: Returns
``bool``
.. rubric:: Examples
::
tb.open("3C273XC1.MS", nomodify=False)
tb.removecolkeyword("UVW", "QuantumUnits")
# True
tb.flush()
# True
"""
pass
[docs] def getdminfo(self):
r"""
This function returns the types and names of the data managers used. For each data manager it also returns the names of the columns served by it. The information is returned as a record containing a subrecord for each data manager. Each subrecord contains the fields TYPE, NAME and COLUMNS.
"""
pass
[docs] def keywordnames(self):
r"""
This function returns a vector of strings containing the names of all table keywords.
"""
pass
[docs] def fieldnames(self, keyword=''):
r"""
This function returns a vector of strings containing the names of all fields in the given table keyword. It is only valid if the keyword value is a record. no keyword name is given, the names of all table keywords are returned.
.. rubric:: Parameters
- ``keyword (string='')`` - keyword name
.. rubric:: Returns
``stringVec``
"""
pass
[docs] def colkeywordnames(self, columnname=''):
r"""
This function returns a vector of strings containing the names of all keywords in the column with the given name..
.. rubric:: Parameters
- ``columnname (string='')`` - column name
.. rubric:: Returns
``stringVec``
.. rubric:: Examples
::
tb.open('3C273XC1.MS')
tb.colkeywordnames("UVW")
"""
pass
[docs] def colfieldnames(self, columnname='', keyword=''):
r"""
This function returns a vector of strings containing the names of all fields in the given keyword in the given column. It is only valid if the keyword value is a record. no keyword name is given, the names of all keywords in the column are returned.
.. rubric:: Parameters
- ``columnname (string='')`` - column name
- ``keyword (string='')`` - keyword name
.. rubric:: Returns
``stringVec``
"""
pass
[docs] def getdesc(self, actual=True):
r"""
The table description is a casapy record that contains a complete description of the layout of the table (except for the number of rows). By default the actual table description is returned (thus telling the actual shapes and data managers used). It is also possible to get the table description used when creating the table.
.. rubric:: Parameters
- ``actual (bool=True)`` - actual table description?
.. rubric:: Returns
``record``
.. rubric:: Examples
::
tb.open("3C273XC1.MS")
tb.getdesc()
"""
pass
[docs] def getcoldesc(self, columnname=''):
r"""
The column description is a casapy record that contains a complete description of the layout of a specified column (except for the number of rows). It can be used to construct a table description.
.. rubric:: Parameters
- ``columnname (string='')`` - Name of column
.. rubric:: Returns
``record``
.. rubric:: Examples
::
tb.open("3C273XC1.MS")
tb.getcoldesc("DATA")
#{'comment': 'The data column',
# 'dataManagerGroup': 'TiledData',
# 'dataManagerType': 'TiledShapeStMan',
# 'maxlen': 0,
# 'ndim': 2,
# 'option': 0,
# 'valueType': 'complex'}
"""
pass
[docs] def ok(self):
r"""
Perform a number of sanity checks and return T if ok. Failure (returning F) is a sign of a bug.
"""
pass
[docs] def clearlocks(self):
r"""
Occasionally a table will be inretrievably locked to another process no matter how much closing is done. So clearLocks will unlock all the files in the table cache that use AutoLocking.
"""
pass
[docs] def listlocks(self):
r"""
Occasionally a table will be inretrievably locked to another process no matter how much closing is done. So listLocks will list the offending tables (and unoffending ones, too), so we can figure out where the problem might be.
"""
pass
[docs] def statistics(self, column='', complex_value=''):
r"""
This function computes descriptive statistics on the table column. It returns the statistical values as a dictionary. The given column name must be a numerical column. If it is a complex valued column, the parameter complex_value defines which derived real value is used for the statistics computation. This method does not honor any flags in the input table.
.. rubric:: Parameters
- ``column (string='')`` - Column name
- ``complex_value (string='')`` - Which derived value to use for complex columns (amp, amplitude, phase, imag, real, imaginary)
.. rubric:: Returns
``record``
.. rubric:: Examples
::
tb.open("ggtau.1mm.amp.gcal")
s = tb.statistics(column="GAIN", complex_value="phase")
"""
pass
[docs] def showcache(self, verbose=True):
r"""
Show the contents of the table cache.
.. rubric:: Parameters
- ``verbose (bool=True)``
.. rubric:: Returns
``stringVec``
.. rubric:: Examples
::
tb.showcache()
"""
pass
[docs] def testincrstman(self, column=''):
r"""
Checks consistency of an Incremental Store Manager bucket layout
In case of corruption it returns False and a SEVERE msg is posted containing information about the location of the corrupted bucket
.. rubric:: Parameters
- ``column (string='')`` - Column name
.. rubric:: Returns
``bool``
.. rubric:: Examples
::
mytb = tbtool()
mytb.open('uid___A002_X841035_X203.ms.split')
mytb.testincrstman('FLAG_ROW')
"""
pass
[docs] def row(self, columnnames='', exclude=False):
r"""
Return a table accessor object which allows for accessing one or more rows from the table. A single row is returned as a dictionary and multiple rows are returned as a list of dictionaries. The contents of the dictionary by default includes all table columns, but it can be limited by using “columnnames“ parameter. The “exclude“ parameter, which is “False“ by default, indicates whether “columnnames“ lists the columns to \*exclude\* (“exclude=True“) or the columns to include (“exclude=False“).
The returned table accessor object can also be used to store changes to a single row using the “put“ method. This accessor object remains connected to the table object that created it. Closing or deleting a table object invalidates any accessor objects that it created.
.. rubric:: Parameters
- ``columnnames (stringVec='')`` - Column names to be included in (or optionally excluded from) the the returned result.
- ``exclude (bool=False)`` - If set to True, then columnnames describes the coluns to excluded rather than included.
.. rubric:: Returns
``tablerow``
.. rubric:: Examples
::
### open table
tb = table( )
tb.open( 'ngc5921_ut.ms', nomodify=False )
### read a single row
print(f'row 3 is {tb.row( )[3]}')
### contents of the DATA column from the first 5 rows
first_five_data = tb.row(columnnames=['DATA'])[:5]
### ENTIRE contents of the table
whole_table = tb.row( )[:]
### store a single row
rows = tb.row( )
row = rows.get(22)
row['DATA'].fill( 1.0/row['DATA'].size )
rows.put( 22, { 'DATA': row['DATA'] } )
"""
pass