{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "O5t-PZotJYp1" }, "source": [ "# Release Information\n", "\n", "These are the release notes for CASA 6.7. Changes compared to the [CASA 6.6 release](https://casadocs.readthedocs.io/en/v6.6.5/notebooks/introduction.html) are listed below.\n", "\n", "\n", "\n", "***\n", "\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "4fHFiOpGc90G" }, "source": [ "## Highlights\n", "\n", "- **MacOS 26:** support for MacOS 26 was added. *(CASA 6.7.5)*\n", "\n", "- **pccor:** new task for pulse delay calibration. *(CASA 6.7.5)*\n", "\n", "- **ft:** extended list of data selection parameters. *(CASA 6.7.5)*\n", "\n", "- **fringefit:** performance has been improved with speedup of FFT stage. *(CASA 6.7.5)*\n", "\n", "- **sdcal:** generates fully flagged calibration solution when OFF_SOURCE data are invalid. *(CASA 6.7.5)*\n", "\n", "- **importasdm:** added support for additional optional ASDM columns. *(CASA 6.7.5)*\n", "\n", "- **iclean:** new interactive clean widget now includes Jupyter notebook integration. *(CASA 6.7.3)*\n", "\n", "- **sdimaging:** removed from the code, please use **tsdimaging** instead. *(CASA 6.7.3)*\n", "\n", "- **getantposalma:** new parameter *firstintegration*, to exclude/include data flagged as “firstintegration”. *(CASA 6.7.3)*\n", "\n", "- **simalma/simobserve:** new parameter *correlator*, to control which correlator efficiency is used. *(CASA 6.7.3)*\n", "\n", "- **listcal:** now supports fringefit cal tables. *(CASA 6.7.3)*\n", "\n", "- **getantposalma:** a retry mechanism was added for when the web query times out. *(CASA 6.7.3)*\n", "\n", "- **iclean:** new interactive clean widget. *(CASA 6.7.2)*\n", "\n", "- **msuvbinflag:** new experimental task to flag outliers in the UV plane. *(CASA 6.7.2)*\n", "\n", "- **appendantab:** new task for appending antab files to an MS. *(CASA 6.7.2)*\n", "\n", "- **hanningsmooth:** new parameter *smooth_spw* for selecting spws to smooth. *(CASA 6.7.2)*\n", "\n", "- **sdbaseline:** added applying of sinusoid fitting parameters using baseline table. *(CASA 6.7.2)*\n", "\n", "- **fringefit:** added functionality to combine channels across spectral windows. *(CASA 6.7.2)*\n", "\n", "- **feather/sdintimaging:** support added for per-plane beams in feather. *(CASA 6.7.2)*\n", "\n", "- **importfitsidi:** now labels source positions in the ICRS reference frame (VLBI). *(CASA 6.7.2)*\n", "\n", "- **atmosphere:** ATM (atmosphere) library updated to ALMA Cycle 12. *(CASA 6.7.2)*\n", "\n", "- **casaconfig:** new options and updates available. *(CASA 6.7.2)*\n", "\n", "- **import/export:** \"Beam Waveguide\" antennas (ALT-AZ+BWG-R / ALT-AZ+BWG-L mount types) now experimentally supported. *(CASA 6.7.2)*\n", "\n", "- **config files:** The configuration files for ngVLA version Rev.F and ALMA Cycle 12 are now included. *(CASA 6.7.2)*\n", "\n", "- **Mac Intel:** CASA dropped support for MacOS Intel machines. *(CASA 6.7.0)*\n", "\n", "- **casaviewer:** the CASA Viewer has been removed from MacOS packages. *(CASA 6.7.0)*\n", "\n", "- **sdimaging:** task sdimaging is deprecated, use **tsdimaging** instead. *(CASA 6.7.0)*\n", "\n", "- **tclean:** added new (experimental) gridder option *'awp2'*, a refactoring of *'awproject'*. *(CASA 6.7.0)*\n", "\n", "- **gencal:** added support for the output of task **getantposalma**. *(CASA 6.7.0)*\n", "\n", "- **fringefit:** updates to *corrcomb* parameter, to support *corrcomb='stokes'* (formerly *'all'*) and *corrcomb='parallel'*. *(CASA 6.7.0)*\n", "\n", "- **phaseshift:** added multi-field specification of the *phasecenter* parameter. *(CASA 6.7.0)*\n", "\n", "- **tsdimaging/sdimaging:** weight value properly derived for Stokes I from weights for linear/circular correlations. *(CASA 6.7.0)*\n", "\n", "- **plotbandpass:** upgraded to support band-to-band calibration, plotting of *hsd_skycal* solutions, more robust identification of unique solution timestamps, and added features similar of the corresponding *analysisUtils* version. *(CASA 6.7.0)*\n", "\n", "- **smoothcal:** improvements to smoothing of fringefit solutions. *(CASA 6.7.0)*\n", "\n", "- **msmd.almaspws:** now identifies ALMA FDM spectral windows in cases of heavy online channel averaging. *(CASA 6.7.0)*\n", "\n", "- **tclean:** initial work to support GPU gridding (not yet available for general use). *(CASA 6.7.0)*\n" ] }, { "cell_type": "markdown", "metadata": { "id": "gxYrgSLDJYp2" }, "source": [ "## Release Notes\n", "\n", "**Installation and operation**\n", "\n", "- **MacOS 26:** Support for macOS26 was added. *(CASA 6.7.5)*\n", "\n", "- The old casa5 subtree has been removed from the code. As part of this, the **casatablebrowser** build was updated to work without the casa5 subtree. *(CASA 6.7.3)*\n", "\n", "- **config files:** The configuration files for ngVLA version Rev.F and ALMA Cycle 12 have been included and are available for simulating ngVLA and ALMA data. *(CASA 6.7.2)*\n", "\n", "- **Mac Intel:** CASA dropped support for MacOS Intel machines. The CASA team no longer has the resources to build and test packages for Mac Intel. Therefore, CASA 6.6.5 was the last version that runs on MacOS with an Intel x86 chip. We point users with Intel machines to either CASA 6.6.5, or CASA 6.6.1, which is the official version used by ALMA and the pipelines. *(CASA 6.7.0)*\n", "\n", "- **CASA Viewer:** The CASA Viewer, through the deprecated viewer task **imview/msview**, is now removed from MacOS packages; new exceptions have been added to **imview**, **msview** and interactive modes of **tclean**, **deconvolve** and **sdintimaging**. *(CASA 6.7.0)*\n", "\n", "- **tclean:** initial work to support GPU gridding has been added to the imaging code in CASA 6.7.0. This feature is still under development and is not yet available for general use. *(CASA 6.7.0)*\n", "\n", "\n", "**Import/export and MS structure**\n", "\n", "- **importasdm:** support was added for additional optional ASDM columns. *(CASA 6.7.5)*\n", "\n", "- **importfitsidi:** now labels source positions in the ICRS reference frame, which for VLBI data resolved issues when subsequently exporting the data using *exportfits*. *(CASA 6.7.2)*\n", "\n", "- **importfitsidi/importuvfits/exportuvfits:** \"Beam Waveguide\" antennas are now *experimentally* supported through new ALT-AZ+BWG-R and ALT-AZ+BWG-L mount types. These mount types will be recognized by the importfitsidi, importuvfits and exportuvfits tasks. The calibration tasks will, in principle, apply the correct parallactic angle correction for these antennas.\n", "**Note:** Correlators may not yet encode the correct mount type for this case in the FITS-IDI or UVFITS files they produce. Carefully check the mount type for your antennas and if necessary, adjust the mount type by editing the ANTENNA table of your MeasurementSet. *(CASA 6.7.2)*\n", "\n", "- **msmd.almaspws:** a small change was made to the code of tool *msmd* to correctly identify ALMA FDM spectral windows in cases of heavy online channel averaging. *(CASA 6.7.0)*\n", "\n", "\n", "**Information**\n", "\n", "- **listcal:** task listcal now supports listing fringefit cal tables, showing phase, delay, rate, and dispersive delay, all in convenient units. *(CASA 6.7.3)*\n", "\n", "- **atmosphere (plotbandpass/plotms/plotweather):** the ATM (atmosphere) library has been updated to the version used in ALMA cycle 12. This brings dry path improvements as well as fixes for opacity jumps, especially the one at around 216 GHz. *(CASA 6.7.2)*\n", "\n", "**Flagging**\n", "\n", "- **msuvbinflag:** new task to identify and flag outliers in the UV plane via the UV grid generated by **msuvbin**. This task is *experimental*. *(CASA 6.7.2)*\n", "\n", "\n", "**Calibration**\n", "\n", "- **fringetfit:** the *FFT* stage performance has been improved so as to become nearly negligible compared to the global least-square stage. As part of fringefit FFT stage performance improvements, logging of solution count progress (every ~10%) has been added, and will be evident in other calibration solving tasks. *(CASA 6.7.5)*\n", "\n", "- **pccor:** a new task pccor was added for pulse delay calibration. This task is the equivalent of *PCCOR* in AIPS. *(CASA 6.7.5)*\n", "\n", "- **getantposalma:** a new parameter *firstintegration* was added as an optional boolean parameter. If *True* (default), this excludes measurements flagged as “firstintegration”. If *False*, includes measurements flagged as “firstintegration”. *(CASA 6.7.3)*\n", "\n", "- **getantposalma:** added a retry mechanism when the web query times out. *(CASA 6.7.3)*\n", "\n", "- **appendantab:** this new task allow users to append information available in *antab* files directly to a MeasurementSet. *(CASA 6.7.2)*\n", "\n", "- **getantposalma:** the default value of *snr* changed in task getantposalma, so that the default value used by the web service is used. *(CASA 6.7.2)*\n", "\n", "- **fringefit:** it is now possible to combine channels across spectral windows using the *combine='spw'* and *'spwmap'* options. The task syntax always supported this, but due to a bug the code itself did not. *(CASA 6.7.2)*\n", "\n", "- **applycal:** a warning was added for the case that the *cal library* is incomplete for a caltable for the data selected in applycal (such data is passed over for that caltable, which may be intended by the user). *(CASA 6.7.2)*\n", "\n", "- **gencal:** support has been added for the output of task **getantposalma**. *(CASA 6.7.0)*\n", "\n", "- **fringefit:** the *corrcomb* parameter has been updated to support the following modes:\n", " - *corrcomb='stokes'* (formerly *'all'*) combines the parallel-hand correlations to form formal Stokes I (thereby cancelling any source polarization contribution), with net weight properly propagated from the correlation weights. *(CASA 6.7.0)*\n", " - *corrcomb='parallel'* combines the parallel-hand correlations in a weighted average to optimize sensitivity on the assumption that there is zero source polarization contribution. Along with use of *corrdepflags=True*, *corrcomb='parallel'* will better support combining correlations on VLBI arrays including a subset of single-polarization antennas (i.e., without flagging those antennas). *(CASA 6.7.0)*\n", "\n", "- **smoothcal:** improvements were made to smoothing of fringefit solutions. *(CASA 6.7.0)*\n", "\n", "\n", "**Manipulation**\n", "\n", "- **hanningsmooth:** added a new parameter *smooth_spw* for selecting spectral windows that should have the hanning smooth transformation applied. In addition, reindexing was disabled. *(CASA 6.7.2)*\n", "\n", "- **phaseshift:** added multi-field specification of the *phasecenter* parameter. Every field of the input MS can be shifted to a field-specific center in one single run of the task. *(CASA 6.7.0)*\n", "\n", "\n", "**Imaging**\n", "\n", "- **ft:** the list of data selection parameters was extended for the task interface in ft. *(CASA 6.7.5)*\n", "\n", "- **iclean:** a new task, iclean, was added for [interactive cleaning](interactive_clean.ipynb). This task opens a stand-alone widget that allows for interactive cleaning. This new task operates separately from the interactive cleaning option in tclean, which relies on the CASA Viewer, and does not work anymore on Mac OS. As of CASA 6.7.3, iclean also includes Jupyter Notebook integration. *(CASA 6.7.2/6.7.3)*\n", "\n", "- **tclean:** added new gridder option *'awp2'*, a refactoring of *'awproject'*. This is an experimental option and has only been validated for limited, narrow-field use cases. Further improvements are anticipated in later releases. *(CASA 6.7.0)*\n", "\n", "\n", "\n", "\n", "\n", "**Visualization**\n", "\n", "- **plotbandpass:** in order to support interferometric datasets using band-to-band calibration, plotbandpass now identifies the scans to plot for each requested spw individually rather than as a group. This improvement also enables complete plotting of *hsd_skycal* solutions in single dish spectral scan datasets via the *overlay='time'* option, assuming that the default value of the *solutionTimeThreshold* parameter is used. *(CASA 6.7.0)*\n", "\n", "- **plotbandpass:** The identification of unique solution timestamps has been made more robust by using the *SCAN_NUMBER* information when available (in all Tsys and bandpass caltables since CASA 3.4). *(CASA 6.7.0)*\n", "\n", "- **plotbandpass:** a number of small changes to the task have been made to bring the plots into better agreement with the *analysisUtils* version of plotbandpass (plotbandpass3.py, v2.20), which has had some new features added to the over recent years:\n", " - The number of channels in the solution is now shown in the x-axis label. *(CASA 6.7.0)*\n", " - The maximum altitude of ozone requested in the atmospheric model has been increased from the CASA default to 60km, and the legend of the magenta transmission curve was realigned. *(CASA 6.7.0)*\n", " - Added support for early ALMA data where atmospheric pressure units were wrong in the weather table. *(CASA 6.7.0)*\n", " - When the actual antenna elevation is not available, 45 degrees is now assumed instead of 90 degrees. *(CASA 6.7.0)*\n", " - The *TASK_PLOTBANDPASS_REVISION_STRING* has been updated to \"2.20.0\" in agreement with the corresponding CVS version of *plotbandpass3.py*, stored in a separate repository. *(CASA 6.7.0)*\n", "\n", "\n", "**Single Dish**\n", "\n", "- **sdcal:** sdcal was updated to generate fully flagged calibration solutions when all the *OFF_SOURCE* data are invalid. With fully flagged solutions, uncalibrated *ON_SOURCE* data are flagged by **applycal** when the caltable is applied with *\"strict\"* flagging mode (*\"calflagstrict\"* or *\"flagonlystrict\"*). *(CASA 6.7.5)*\n", "\n", "- **sdimaging:** the deprecated task sdimaging has been removed from the code. Please use **tsdimaging** instead. *(CASA 6.7.3)*\n", "\n", "- **sdbaseline:** applying sinusoid fitting parameters using *baseline table* is now available. *(CASA 6.7.2)*\n", "\n", "- **feather:** support was added for per-plane beams. These are now also handled correctly in sdintimaging. *(CASA 6.7.2)*\n", "\n", "- **sdimaging:** the sdimaging task is deprecated and will be removed in future releases. A deprecation warning has been added to the code. Please use **tsdimaging** instead. *(CASA 6.7.0)*\n", "\n", "- **sdimaging / tsdimaging:** weight values are now properly derived for Stokes I from the weights for linear/circular correlations. *(CASA 6.7.0)*\n", "\n", "\n", "**Simulations**\n", "\n", "- **simalma/simobserve:** a new *correlator* parameter was added tobetter control which correlator efficiency is used in the simulations. *(CASA 6.7.3)*\n", "\n", "\n", "**Bug fixes**\n", "\n", "- **tclean:** fixed *nmajor* parameter when used with *specmode='mvc'*. *(CASA 6.7.5)*\n", "\n", "- **config:** a bug was fixed where config failed when users have no access to Google. *(CASA 6.7.5)*\n", "\n", "- **concat:** different field names having the same phase center will no longer be confused (as occurred in relatively obscure and limited circumstances). This also affects **importfitsidi** on multiple FITS-IDI files. *(CASA 6.7.5)*\n", "\n", "- **concat:** fixed concatenation of *SYSCAL* tables with different sets of spectral windows. This also fixed Tsys measurements for VLBI data sets with multiple frequency setups in **importfitsidi**. *(CASA 6.7.5)*\n", "\n", "- **getephemtable:** now always writes a *NAME* keyword to the output table. *(CASA 6.7.5)*\n", "\n", "- **casacore:** corrections were made for regridding data for tasks that rely on *fftshift* as an interpolation method. *(CASA 6.7.5)*\n", "\n", "- **ia tool:** a bug was fixed where arrays larger than ~2\\*\\*31 in size resulted in empty arrays when using the *ia* tool. *(CASA 6.7.5)*\n", "\n", "- **casampi:** a casampi compatibility issue with Python 3.13 due to exec() API changes was resolved. *(CASA 6.7.5)*\n", "\n", "- **tclean:** fixed a scaling bug when regridding the start model's cell size. Additionally, added a check to enforce that the start model must be in units of Jy/pixel; prior, the start model was assumed to be Jy/pixel regardless of header values. *(CASA 6.7.5)*\n", "\n", "- **simanalyze:** updated to explicitly assume modelimage units are Jy/pixel, to maintain compatibility with the above mentioned tclean bugfix. This preserves the original behavior of simanalyze. *(CASA 6.7.5)*\n", "\n", "- **listcal:** The listcal task no longer crashes when given a (VLBA) gaincurve caltable generated in this release; it should faithfully list the coefficients stored within. VLBA gaincurve tables generated in prior releases should still work when applied; successful use of **listcal** for VLBA gaincurves will require regenerating the cal table with **gencal**. (NB: listcal does not evaluate the gaincurve, which is out-of-scope for listcal.) *(CASA 6.7.3)*\n", "\n", "- **simalma:** task simalma was updated for compatibility with Numpy 2.4. *(CASA 6.7.3)*\n", "\n", "- **applycal/gaincal/bandpass/polcal:** in calibration application (applycal, gaincal/bandpass/polcal with pre-applied gaincal solutions), use of *interp='nearestPD'* or *'linearPD'* with non-trivial *spwmap* now avoids occasional cycle slips in the transferred phase by accurately sensing cycle changes in the applied (pre-transfer) gaincal phase. Flagged solutions in the pre-transferred phase should now be well-tolerated, in general. Transferred phase is now formed from the ratio of the *unselected* centroid spectral window frequency and the solution frequency, thereby ensuring consistent phase calibration behavior in any context. *(CASA 6.7.3)*\n", "\n", "- **plotbandpass:** a bug was fixed where NaN value was shown for solution times in the warning message. *(CASA 6.7.3)*\n", "\n", "- **plotbandpass:** fixed a bug when overlaying solutions from a caltable having only one antenna. *(CASA 6.7.3)*\n", "\n", "- **plotms:** a bug was fixed in the atmospheric transmission calculation when using the POINTING table. *(CASA 6.7.3)*\n", "\n", "- **phaseshift:** an array shape mismatch was fixed that could sometimes results in an error when some fields were not shifted. *(CASA 6.7.3)*\n", "\n", "- **XML:** a bug was fixed in the code generator, where the parser introduced a wrong value for a boolean if no default value was set in an XML file. *(CASA 6.7.3)*\n", "\n", "- **plotms:** a bug was fixed where *showatm* and *showtsky* options did not overlay atmospheric curve if all requested data are flagged, even when *customflaggedsymbol=True*. *(CASA 6.7.3)*\n", "\n", "- **tclean:** a bug was fixed where tclean failed in serial mode for full Stokes imaging with *'Error in tclean: list assignment index out of range'*. *(CASA 6.7.2)*\n", "\n", "- **tclean:** a bug was fixed that lead to an empty beam after changes to *gridwt* propagation. *(CASA 6.7.2)*\n", "\n", "- **tclean:** a bug was fixed where imaging with a user-defined mask gave errors after changes to *gridwt* propagation. *(CASA 6.7.2)*\n", "\n", "- **tsdimaging:** a bug has been fixed where tsdimaging now sets brightness units correctly based on the data column. *(CASA 6.7.2)*\n", "\n", "- **flagdata:** a bug was fixed whereby task flagdata could hang when using list mode with MMS-based parallelization, only in modular CASA installations (without casalith). *(CASA 6.7.2)*\n", "\n", "- **mstransform:** a bug was fixed whereby the history of the output MS was not being updated when using the parameters *spw* or *chanaverage*. *(CASA 6.7.2)*\n", "\n", "- **gencal:** behavior has been restored that affected generation of an *antpos* caltable that will correct for an online troposphere delay application problem during the period Aug-Nov 2016, even when no ordinary antpos corrections are required. *(CASA 6.7.2)*\n", "\n", "- **gaincal:** *gaintype='T'* now supports \"robust\" solmodes *'L1'*, *'R'*, and *'L1R'* for L1 minimization and/or outlier rejection. *(CASA 6.7.2)*\n", "\n", "- **applycal:** in calibration application by cal library (*docallib=True*, in applycal, etc.), a bug was fixed for the (rare) case of multiple cal library entries with intent-specificity for bandpass calibration tables, when there are redundant intents in the dataset. *(CASA 6.7.2)*\n", "\n", "- **tsdimaging:** now falls back to a beam size derived from the physical property of the antenna when additional processing to incorporate the effect of scanning pattern into beam size failed. *(CASA 6.7.2)*\n", "\n", "- **tsdimaging:** an irrelevant warning regarding image size was eliminated. *(CASA 6.7.2)*\n", "\n", "- **sdbaseline:** a bug was fixed to make *model='apply'* work with per spectrum fitting. *(CASA 6.7.2)*\n", "\n", "- **sdintimaging:** a bug was fixed where sdintimaging did not allow running multiple instances simultaneously due to the naming convention of temporary files. *(CASA 6.7.2)*\n", "\n", "- **sdintimaging:** a bug was fixed for correctly handling per-plane vs. global beams in the *setrestoringbeam* parameter. *(CASA 6.7.2)*\n", "\n", "- **tec_maps:** a minor cleanup of behavior was done. *(CASA 6.7.2)*\n", "\n", "- **defintent:** a bug was fixed where an intent would not be added to the selected rows if the same intent name was already in use on other rows. *(CASA 6.7.2)*\n", "\n", "- **casaconfig:** improved exception handling of network issues. *(CASA 6.7.2)*\n", "\n", "- **exportuvfits:** a bug was fixed where the proper motion was being written in the wrong units. *(CASA 6.7.0)*\n", "\n", "- **tclean:** a bug was fixed that could sometimes occur when using a list of MSs and selecting a field by name. *(CASA 6.7.0)*\n", "\n", "- **tclean:** serial and parallel execution should now produce more consistent PSF sizes for certain weighting options. *(CASA 6.7.0)*\n", "\n", "- **tclean:** the correctness of the return dictionary when *niter=0* and the user supplied a mask was improved. *(CASA 6.7.0)*\n", "\n", "- **setjy:** a bug was fixed that sometimes caused a crash to occur in parallel mode when operating on an MMS with *usescratch=False*. *(CASA 6.7.0)*\n", "\n", "- **plotbandpass:** a bug that caused missing spws in amplitude vs. frequency plots that request *overlay='spw'* was fixed by adding a missing elif block that was present in *plotbandpass3.py*. *(CASA 6.7.0)*\n", "\n", "- **plotbandpass:** an undefined variable bug that could cause a crash in rare cases of phase vs. frequency plots was fixed. *(CASA 6.7.0)*\n", "\n", "- **phaseshift:** a metadata inconsistency was fixed when the input MS was not in J2000. *(CASA 6.7.0)*\n", "\n", "- **casaconfig:** *pyproject.toml* was added to casaconfig. *(CASA 6.7.0)*\n", "\n", "- **casaviewer/casalogger:** build infrastructure changes related to *libfreetype* were implemented to fix a bug that could affect the viewer, would trigger an error message for the **casaviewer** and **casalogger** on certain OSs (e.g., Fedora 40). *(CASA 6.7.0)*\n", "\n", "\n", "**Documentation**\n", "\n", "- **tb.copy:** documentation of tb.copy was improved to be consistent with its implementation. *(CASA 6.7.3)*\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "0iNo4MAGJYp3" }, "source": [ "## Known Issues\n", "\n", "**Summary Most Important Issues**\n", "\n", "- **pkg_resources** has been removed and should not longer be used within config.py\n", "- Default CASA installation on **Google Colab** is currently failing (as of 11/11/2024), and requires setting the version of *protobuf* back to 3.20 in Colab Notebooks.\n", "- The **CASA Viewer** is no longer included in Mac OS packages.\n", "- Task **simobserve** may fail on Mac OS 14.\n", "- Task **widebandpbcor** fails if image is in different directory than MeasurementSet\n", "- Task **simanalyze** with two or more input MSs in *vis* may not work.\n", "- In **tclean**, major cycles can be trigger too early due to *psfsidelobe*.\n", "- In **tclean**, *pbmask* uses the wrong default of 0.2 for *usemask='pb'* and *'auto-multithresh'*.\n", "- The modular version of **importfits** may fail with *CFITSIO* error if NASA's HEASoft is installed.\n", "- Tasks **mstransform** and **split** switch the circular (linear) polarization products when input is given as *correlation='ll,rr'* (*'yy,xx'*).\n", "- **CASA 6 startup** may fail on some **Mac OS** where users have set up a file system that is *case-sensitive*.\n", "- Wideband and widefield imaging in **tclean** are only partially validated - please use at own risk and read [wideband](synthesis_imaging.ipynb#Wide-Band-Imaging) and [widefield](synthesis_imaging.ipynb#Wide-Field-Imaging) documentation.\n", "- When imaging large mosaics with *mosweight* in **tclean**, an error *\\\"too many open files\\\"* may occur that may require to increase the limit for open files.\n", "- **stawt** may fail when the correlator integration time changes within an MS and statwt is run with timebin set to an integer value.\n", "- CASA is not using *LD_LIBRARY_PATH* anymore but *CASALD_LIBRARY_PATH* to avoid confusion. \n", "- **cvel** is calculating the velocity incorrectly for ephemeris objects. We recommend to use **mstransform** or its offspring **cvel2**, although the latter should be used with care as it is not fully commissioned yet.\n", "- **fixvis** uses the small angle approximation and may be incorrect for large phase shifts. Use the new task **phaseshift** instead, or use **tclean** for phase center shifts during imaging when applicable.\n", "- With parallel calibration on MMS files, **fixvis** does not write out the the new MMS specified in *outputvis* correctly, hence fixvis solutions are not applied when writing to a new MMS.\n", "- In **tclean**, using the mosaic gridder with the default *nchan=-1* is in some cases known to produce errors.\n", "- Ionospheric TEC corrections are currently validated in CASA only for VLA data.\n", "- *ephemeris* objects are not correctly supported by *virtual model columns*.\n", "- In **tclean**, the combination of *specmode='cube'* and *gridder=\"awproject\"* has not been commissioned for use and may result in errors.\n", "\n", "\n", "### General\n", "\n", "Several issues have been encountered for CASA 6:\n", "\n", "- inp/go syntax does not reset hidden parameters to default between consecutive calls. The most notable issue occurs with the *selectdata* parameters; once set at *selectdata=True*, any associated subparameters that are changed from default (e.g., *spw*, or *field*) will remain even after re-setting to *selectdata=False*. Generally this should not matter as hidden parameters are not used. However, known exceptions exist in **tclean**, **sdintimaging**, **apparentsens**, **setjy**, **listvis**, **sdcal**, and **sdfit** where the value of hidden selection parameters does in fact matter. We are currently investigating the extent of this problem. As a workaround, users can call *default* to manually reset all their parameters.\n", "\n", "- inp/go does not work for the following tasks: msuvbin, browsetable, imview, msview, deconvolve, testconcat. Please invoke the arguments directly when running these tasks in CASA 6.\n", "\n", "### Installation\n", "\n", "- The previously deprecated **pkg_resources** to generate file paths within **config.py** has been removed. Use of pkg_resources now returns the error: *ModuleNotFoundError: No module named 'pkg_resources'*. Users who still have pkg_resources in their config.py should remove the offending import and code.\n", "\n", "- Installing CASA with default commands on Google Colab is currently failing. Reason is that as of 11/11/2024, Google rolled out an update to Colab with updated default packages, of which the *protobuf* package is incompatible with the current default pip version of CASA. CASA is currently stuck to protobuf 3.20 and older, based on choices made for compatibility with the RHEL8 build systems. A solution in CASA is currently under investigation. The problem can be fixed by pinning the *protobuf* version to 3.20 in Colab Notebooks:\n", "```\n", "os.system('pip install protobuf==3.20.3')\n", "import google.protobuf as gpbuf\n", "print('Protobuf version must be 3.20.X, it is: ',gpbuf.__version__)\n", "```\n", "\n", "- Running **importfits** in modular CASA may fail with a *CFITSIO* error if also the NASA HEASoft software is installed, because of a conflict in CFITSIO setting between CASA and heasoft. As a workaround, comment out all parts of heasoft in your environment to made importfits work in the modular environment.\n", "\n", "- CASA 6 startup may fail on some Mac OS where users have set up a file system that is *case-sensitive* (as shown [here](https://support.apple.com/lt-lt/guide/disk-utility/dsku19ed921c/mac)). As a temporary work-around, please manually update the casapy script in /Applications/CASA.app/Contents/MacOS/casapy, by replacing the string \"Macos\" with \"MacOS\", which occurs in lines 36 and 39 of the casaspy script.\n", "\n", "- For Mac OS, the default behavior when downloading multiple versions of CASA is to call it \"CASA\\ X.app\" (i.e., including a space). However, CASA is unable to find the viewer when a space exists in the application name. The workaround is to rename the application excluding the space.\n", "\n", "- NFS mounted disks\n", " - It is not recommended that you run CASA (e.g. have your data) on disks that are NFS mounted. It can be done, but in some cases the files will be NFS locked and this can crash CASA or its tasks. In this case, you have to restart CASA.\n", " \n", " - If you receive messages like xvfb timeout you may try to clean out your /tmp folder, then restart CASA.\n", "\n", "- Python:\n", " - Environment variables set for personal use may be incompatible with CASA 6, given that the CASA comes with a Python version that may be different from the one installed for regular use. It is still unclear which specific errors can occur, but one workaround solution for these types of errors is to unset *PYTHONSTARTUP* before starting casa. We are looking into possible solutions.\n", "\n", " - Files in the current directory with the same name as ipython files will cause errors like this error that occurs when a *new.py* file exists in the current directory: ```python\n", "\n", "```\n", " AttributeError Traceback (most recent call last)\n", " /lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in enable_matplotlib(self, gui) \n", " 2945 gui, backend = pt.find_gui_and_backend(self.pylab_gui_select) \n", " 2946 \n", " -> 2947 pt.activate_matplotlib(backend) \n", " 2948 pt.configure_inline_support(self, backend) \n", " 2949 \n", "\n", " /lib/python2.7/site-packages/IPython/core/pylabtools.pyc in activate_matplotlib(backend) \n", " 292 matplotlib.rcParams['backend'] = backend \n", " 293 \n", " --> 294 import matplotlib.pyplot \n", " 295 matplotlib.pyplot.switch_backend(backend) \n", " 296 \n", "\n", " /lib/python2.7/site-packages/matplotlib/pyplot.py in ()\n", " 21 from matplotlib.cbook import dedent, silent_list, is_string_like, is_numlike \n", " 22 from matplotlib import docstring \n", " ---> 23 from matplotlib.figure import Figure, figaspect \n", " 24 from matplotlib.backend_bases import FigureCanvasBase \n", " 25 from matplotlib.image import imread as _imread \n", "\n", " /lib/python2.7/site-packages/matplotlib/figure.py in ()\n", " 16 import artist \n", " 17 from artist import Artist, allow_rasterization \n", " ---> 18 from axes import Axes, SubplotBase, subplot_class_factory \n", " 19 from cbook import flatten, allequal, Stack, iterable, is_string_like \n", " 20 import _image \n", "\n", " /lib/python2.7/site-packages/matplotlib/axes.py in ()\n", " 8452 \n", " 8453 #This is provided for backward compatibility \n", " -> 8454 Subplot = subplot_class_factory() \n", " 8455 \n", " 8456 docstring.interpd.update(Axes=martist.kwdoc(Axes)) \n", "```\n", "\n", "\n", "### Scripting\n", "\n", "- Starting CASA 6: For *\"execfile\"* calls within a script which itself is run via *\"execfile\"*, it is necessary to add *globals()* as the second argument to those *\"execfile\"* calls in order for the nested script to know about the global variables of the calling script. For example, within a script *'mainscript.py'*, calls to another script *'myscript.py'* should be written as follows: *execfile('myscript.py', globals())*.\n", "- There are cases where two scripts call on each other (i.e., where script \\\"a\\\" uses script \\\"b\\\" and vice versa). In casa 6, the only way to execute these scripts is by running the first script twice:\n", "\n", "```\n", "execfile(\"a.py\",globals())\n", "execfile(\"b.py\",globals())\n", "execfile(\"a.py\",globals())\n", "```\n", "\n", "\n", "### Import/export\n", "\n", "- Import/export of MSs with Nant>255 via UVFITS is implemented according to [AIPS Memo 117](https://library.nrao.edu/public/memos/aips/memos/AIPSM_117.pdf), but should be considered experimental and datasets transported this way may not behave as expected in other packages (e.g., AIPS).\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "9Z8ksWqNNU80" }, "source": [ "### statwt\n", "- In some circumstances when an MS data selection is specified, *chanbin* is not equal to the default value of *spw*, and the WEIGHT_SPECTRUM or SIGMA_SPECTRUM columns don\\'t exist, the **statwt** task may need to be run twice in order to complete successfully due to a known issue with initializing the WEIGHT_SPECTRUM and/or SIGMA_SPECTRUM columns in the code. In these circumstances, an exception will be raised with instructions to restart the task. If you are using the tool method, first close the ms tool, then reopen it using the same data set, apply the same selection, and then run **ms.statwt()**.\n", "\n", "### mstransform/split\n", "- Tasks **mstransform** and **split** switch the circular or linear polarization products when input is given as *correlation=‘ll,rr’* or *correlation=‘yy,xx’*. In those cases, the original *rr (xx)* correlation turns into *ll (yy)* in the output MS, and *ll (yy)* turns into *rr (xx)*. This is opposite from the correct behavior when *correlation=‘rr,ll’* or *correlation=‘xx,yy’*, in which case the polarization products remain unchanged in the output MS. Other parameters do not show this behavior. As a workaround, specify the correlation products in the order that they appear in the input MS, i.e., *correlation=‘rr,ll’* for VLA and *correlation=‘xx,yy’* for ALMA data.\n", "- SPW combination (*combinespws=True*) requires that all the SPWs selected have the same number of channels.\n", "- Some inconsistencies are present in CASA in the SIGMA/WEIGHT columns (and their _SPECTRUM variants) when splitting on datacolumn='data', such as:\n", " - For an MS with WEIGHT_SPECTRUM but no SIGMA_SPECTRUM (as obtained from initweights), SIGMA_SPECTRUM is created and initialized to SIGMA. While split/mstransform correctly initializes the output WEIGHT to 1/SIGMA^2, it does not initialize the output WEIGHT_SPECTRUM to 1/SIGMA_SPECTRUM^2 (instead it copies the original WEIGHT_SPECTRUM).\n", " - For an MS with both WEIGHT_SPECTRUM and SIGMA_SPECTRUM, the output WEIGHT_SPECTRUM is again a copy of the input instead of being initialized to 1/SIGMA_SPECTRUM^2.\n", "\n", " Future work in CASA is planned to address such inconsistencies.\n", "\n", "### cvel\n", "- **cvel** is calculating the velocity incorrectly for ephemeris objects. We recommend to use **mstransform** or its offspring **cvel2**, although the latter should be used with care as it is not fully commissioned yet.\n", "- **cvel** fails on MMS files used for parallel processing. We recommend to use **mstransform** or its offspring **cvel2**, although the latter should be used with care as it is not fully commissioned yet.\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "zHZ31YzPJYp4" }, "source": [ "\n", "\n", "### bandpass\n", "- Currently, **bandpass** will not find good solutions if any correlation (including cross-correlation) in the data is completely flagged. As an interim solution one may split the unflagged data in a separate file and then perform **bandpass**\n", "\n", "### polcal\n", "- Polarization position angle calibration *poltype='X'* or *'Xf'* will be sensitive to any unmodelled position shift in the specified calibrator linear polarization model relative to the centroid location of total intensity (typically the phase center). Excess phase due to the position shift will introduce a bias in the cross-hand phase calibration (which is the same as position angle calibration in the circular feed basis). For this reason, it is best to use truly point-like (in all polarizations) calibrators, if possible, or accurate resolved models.\n", "\n", "### setjy\n", "\n", "- For **setjy**, setting a virtual model with *usescratch=False* (which is the default) can result in a segfault when subsequently running task **gaincal**. To avoid this, please change to *usescratch=True*.\n", "\n", "- Sometimes **setjy** does not properly overwrite a current model in the header of the ms (virtual scratch column). It is recommended to use **delmod** if a model exists and shall be overwritten.\n", "\n", " - *virtual model columns* in the MS do not correctly support *ephemeris* objects, although they will run without generating errors or warnings. If any of your calibrators exhibit significant celestial motion on the timescale of your observation (e.g. , any solar system object), you must set *'usescratch=True'* in calls to **setjy()**.\n", "\n", "### uvcontsub\n", "- *fitorder* should be kept low (\\<= 1) unless the line band is relatively narrow compared to the fit bands. If the image rms markedly rises in the middle line channels after **uvcontsub**, fito*rder* should probably be lowered.\n", "- *fitorder* \\> 0 does not work with *solint* \\> \\'int\\'\n", "\n", "### cal library\n", "- The CASA cal library (*docallib=True* in **applycal**, **gaincal**, **bandpass**, etc.) may exhibit problems when calibration is unavailable for a subset of MS spectral windows. Use of *spwmap* to (transparently, harmless) supply nominal calibration for these spectral windows may help avoid this problem. For antenna position corrections, try *spwmap*=\\[0\\] to relieve a variety of this problem.\n", "\n", "### VLA Switched Power\n", "- In CASA v4.2.2 and higher, the weight calibration for EVLA switched power/Tsys corrections is still being investigated. Visibility corrections are ok. Since switched power calibration is not used by the EVLA pipeline (except for requantizer gain corrections, for which this problem is irrelevant), and since *calwt=F* remains the general recommendation, users should rely on **statwt** to generate appropriate data weights.\n", "\n", "### fringefit\n", "- For task **fringefit**, calibration tables created with CASA 5.5 and before cannot be applied with CASA 5.6 and later. Attempting to do so will fail with an error about non-confirming array sizes.\n", "\n", "### fixvis\n", "- **fixvis** uses the small angle approximation and may be incorrect for large phase shifts. This may result in sources shifting position if large phase shifts are being applied (shifts up to a few beam sizes have been reported). Please use the new task **phaseshift** instead, or use **tclean** for phase center shifts during imaging when applicable.\n", "- With parallel calibration on multi-MS (MMS) files, **fixvis** does not write out the *outputvis* correctly, hence fixvis solutions are not applied when writing to a new MMS. The recommended work-around solution is to over-write the input MMS by leaving the *outputvis* parameter empty. This will change the input MMS, so if you are concerned about that, we recommend to make a copy before running fixvis in parallel mode. Writing output MS files in serial mode is not affected by this bug.\n", "\n", "### tec_map\n", "- Ionospheric TEC corrections in CASA are currently validated only for the VLA. TEC corrections for other observatories are experimental and should be done at your own discretion.\n", "- Do not use CASA 6.1.0 for tec_map corrections.\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "Ws5ABCMuJYp5" }, "source": [ "### tclean\n", "\n", "- **interactive tclean** is not available on Mac OS due to the removal of the CASA Viewer from Mac versions of CASA 6.6.5. Interactive tclean still worked using earlier CASA versions, except on Mac OS 14. For details and workaround solution, see this dedicated [Viewer End-of-Life page](../notebooks/viewer_end_of_life.ipynb).\n", "\n", "- The calculation of *psfsidelobe*, used to determine cyclethreshold, can often pick a value higher than the first sidelobe level, causing major cycles to be triggered too early. Please use *cyclefactor<1.0* to compensate in well-constrained imaging cases where deeper minor cycles are possible.\n", "\n", "- The parameter *pbmask* has a default value of 0.0 for *usermask='user'*, but 0.2 for *usemask='pb'* and *'auto-multithresh'*, even though the documented value is 0.0 for all options. This bug will be fixed in a future release. As a workaround, please set the value manually.\n", "\n", "- For the experimental *'Adaptive Scale Pixel (asp)'* deconvolution algorithm in **tclean**:\n", " - We are currently investigating some intermittent failures on some flavors of Mac OSX. We have follow-up development and test efforts, and encourage users to provide [feedback](https://casa.nrao.edu/index_contact.shtml) on what they find.\n", " - 3rd party code used for the core optimization (lbfgs) is known to produce different numerical results depending on the compiler/build options. We have tested this on all CASA's supported platforms for the use cases included in our [characterization demo](https://github.com/urvashirau/Simulation-in-CASA/blob/master/Multiscale_Wideband_Imaging/Sim_Asp_Demo.ipynb) and for these tests the results are scientifically equivalent.\n", "\n", "- The *gridder='awproject'* has not been fully commisisoned for use with *specmode='cube'* in **tclean**. The following message appears: *The gridder='awproject' has not been fully tested for 'cube' imaging (parallel=True or False). Formal commissioning of this mode is expected in a subsequent release, where 'awproject' will be aligned with recent framework changes.* Until then, please report errors/crashes if seen.\n", "\n", "- In **tclean**, if *gridder='awproject'* is run with *psterm=True*, the output Primary Beam currently still includes the Prolate Spheroidal function. In order to do a primary beam correction, a separate PB needs to be made with *psterm=False*. See the CASA pages on [AWproject](synthesis_imaging.ipynb#Wide-Field-Imaging) for more information.\n", " \n", "- For widefield imaging in **tclean**, the following features still need to implemented and commissioned (for *usepointing=True*, with full heterogenous pointing support):\n", " - *gridder='mosaic'* : Enable accurate pointing corrections for baselines with antennas pointing in different directions\n", "\n", "- In **tclean**, the gridders *'mosaic'* and *'awproject'* include aperture illumination functions in the gridding convolution functions used for the PSF generation. Although this strictly follows the math, it has several undesirable features especially in the situation where data are not uniform across all axes across which they are being combined (i.e. if the mosaic pattern is not relatively flat, if the center of the image has no mosaic pointing, if different pointings have drastically different uv-coverages or frequency coverages). All such variations cause the PSFs to be position-dependent and could relate to potential instabilities during deconvolution, either requiring many major cycles to converge or diverging. For spectral-cube imaging, the effects are lower because PSFs are normalized to peak 1 no matter what their raw peak values are. For multi-term imaging, the ratios between the spectral PSF central values matter and the effect/error is enhanved. When all these uv-coverage variations are avoided (in careful simulations), both algorithms perform as expected for joint wideband mosaics (both with *conjbeams=True* or *False*). For CASA 6.3, the guidelines are: \n", " - For the standard gridder, full-field, single pointing imaging (spectral cube as well as multi-term) will be accurate as long as the image phase center matches the PB pointing center. \n", " - For multi-term wideband joint mosaics, we recommend the use of *gridder=\\'awproject\\'* with *conjbeams=True* as that is the only combination that has demonstrated accurate wideband pb-correction (at *niter=0*) especially in the presence of position-dependent uv-coverage. All other options will need monitoring and several major cycles to ensure convergence. The image should ideally be centered on a PB center.\n", "- In **tclean**, the mosweight parameter for multi-field imaging has a new default value of *mosweight=True* as of CASA 5.4. The new default setting of *mosweight=True* in tclean optimizes noise characteristics for Briggs/uniform weighting, but one should be aware that it has the following disadvantages:\n", " - it may potentially cause memory issues for large VLA mosaics\n", " - the major and minor axis of the synthesized beam may be up to \\~10% larger than with *mosweight=False*\n", " - Please change to *mosweight=False* to get around these issues.\n", "\n", "- When imaging mosaics with a large number of fields and many MSs in **tclean**, an error can occur that specifies *too many open files*. This can happen for both manual and pipeline imaging when using the *mosweight=True* parameter. The reason is that in CASA 5.5, a trade-off was made to reduce memory demands in tclean when using mosweight, by placing the weights on disk using multiple files. Unfortunately, this memory fix may cause open file problems for data sets consisting of many MSs and fields. The problem has been characterized based on the number of MSs and fields: with respect to earlier CASA releases, the imager code now uses *#MSs x #fields x 2* additional files. In CASA 6.2 and 6.3, the number of open files required for the default *perchanweightdensity=True* case has been dramatically reduced, though *perchanweightdensity=False* imaging runs are more limited. While the CASA team is working on a permanent solution for future CASA versions, the recommended work-around solution is to manually increase the limit for the number of open files, e.g.: *ulimit -Sn 8000* or *ulimit -Sn 16000*. In some cases, increasing the hard-limit on number of open files may be necessary, which requires admin/root permissions. As a rule of thumb, each MS requires ~54 simultaneously open files. With a 16k limit and a maximum of 150 fields per MS, for a use case with *perchanweightdensity=False*, *mosweight=True* tclean will encounter too many open files with ~46 MSs or more, while with *perchanweightdensity=True*, *mosweight=True* tclean will encounter too many open files with ~300 MSs or more.\n", "\n", "- There are small, systematic offsets known to occur when using **tclean**. Our initial tests show that the offset in dec is of the order \\~50 milli-arcsec, while the offset in RA is a function of declination, but also amounting to \\~50mas. This issue is currectly being investigated.\n", "\n", "- Currently the parameter type of *niter* is defined as an integer, therefore the integer value larger than 2147483647 will not be set properly as it causes an overflow.\n", "\n", "- Using *deconvolver='mtmfs'*, *nterms=1* and *specmode=cube* does not yet work in parallel imaging mode. Use *specmode='mfs'* instead.\n", " \n", "- The *awproject* gridder in **tclean** does not support the virtual model scheme.\n", " \n", "- Interactive **tclean** only works when a region or mask is selected in the CASA Viewer. There is a known bug that when a region is first selected, and then de-selected to produce an empty mask (filled with zeros), the CASA Viewer that runs interactive tclean will still allow you to proceed, and tclean will detect an empty mask and stop. Please always mark a region/mask to continue interactive tclean (if the entire image should be cleaned, draw a box around the entire image), and do not forget to double-click inside the green contours to select the region.\n", " \n", "- When using interactive **tclean**, hand-edited cyclethresholds do not change back to the auto-calculated values in the GUI until two major cycles later. However, the logger contains the most accurate information about what was used, and the expected behaviour (of hand-edited cyclethresholds applying to only the current minor cycles) is seen and has been tested. Therefore, iteration control and imaging will proceed as expected. This known issue affects CASA versions 5.6 and 5.7/6.1 \n", " \n", "- For imaging of cubes using the *mosaic* gridder, a seg-fault can be triggered if the default *nchan=-1* is used and channel selection is set with *spw*. This can result in either a seg-fault, or image planes with NaN values. It is not yet clear what is the root cause of this issue. The problem was found in VLA data, but it is not clear whether it can affect data from other telescopes. As a work-around solution, we recommend to set the *nchan* parameter to anything different than -1. \n", " \n", "- In the **makemask** task, region files using the minus sign ( - ) to create cutouts are known not to work.\n", "\n", "- The combined imaging of different large MSs where differences in the topo-centric frequencies are large compared to the channel width can fail in **tclean** as a result of memory issues, in particular when imaging in parallel mode. A potential workaround solution is to use the task **concat** to first combined the MSs before running tclean, although further testing is needed to ensure that this solution is valid for all cases. This issue will be fixed in a next CASA release.\n", "\n", "### widebandpbcor\n", "\n", "Task **widebandpbcor** fails if the image to be corrected is in a different directory than the MeasurementSet. As a new alternative method on wideband primary beam correction, one can also try the new parameter *specmode=`mvc'* in tclean.\n", "\n", "\n", "### imregrid\n", "- Position-velocity (PV) images are not supported by **imregrid**, because their coordinate systems are nonstandard, lacking a direction coordinate and having a linear coordinate.\n", "- When converting from between coordinate system that require rotation (e.g., from celestial to galactic coordinates), CASA is known to introduce deviations in position from other software packages that can be several tenths of an arcsec. This could be because the rotation of the rectangular grid in a non-cartesian coordinate system is imperfect, possibly due to internal inconsistencies in the conversion matrices. The conversion between one frame and another in general becomes less accurate as distance from the output image\\'s reference pixel increases. The imregid task and Measures tool suffer from this Known Issue (see [imregrid task page](../api/casatasks.rst)).\n", "\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "BHkht7adJYp5" }, "source": [ "### viewer\n", "\n", "- The CASA Viewer (tasks **imview**, **msview**) is no longer packaged with Mac OS. For details and workaround solution, see this dedicated [Viewer End-of-Life page](../notebooks/viewer_end_of_life.ipynb).\n", "\n", "- The CASA **viewer** does not yet support the entire region shapes and parameters at this stage.\n", "\n", "- For equatorial cubes, i.e. data cubes that include dec=0 (exact), the **viewer** only gives spectra for sources at dec \\> 0. No spectra are produced for any points with `dec <0`.\n", "\n", "- **Viewer** may not properly open saved region files.\n", "\n", "- With the new *region* panel being used now, It may be advisable to rename the *\\$HOME/.casa/viewer/rc* file that stores previous configurations of the **viewer**. \n", "\n", "- **Viewer** - labels are not shown - this can be caused by a conflict between an installed version of *PGPLOT* and the version of *PGPLOT* that comes with the non-root version of CASA. If you do have *PGPLOT* installed in a standard location (e.g. */usr/lib*), you may try moving it aside and see if it resolves the problems. If you do encounter this problem, please report it to the CASA team.\n", "\n", "- Some *X11* settings can make the viewer unstable. We identified that the line *Load \\\"glx\\\"* in */etc/X11/xorg.conf* is such a setting. If you don\\'t need this line for aother applications, it would be better to have it removed.\n", "\n", "- The viewer can only load MeasurementSets (MS) for which all spectral windows have the same channel width. If this is not the case, an *ArrayColumn* error will appear. To get around this, use *SPLIT*`` to place the spectral windows of interest in a separate MS, or try the *table browser* tool.\n", "\n", "- When exiting CASA after using the viewer, a message similar to the following may appear: *proc vtool_1EziEss1P2tH0PxJbGHzzQ is being killed*. This is a cosmetic issue and can be ignored.\n", " \n", "- For some OSs and window managers, parts of the display may be eclipsing interactive elements. We recommend to change the window manager styles for these cases.\n", " \n", "- When multiple animators are open, it can happen that it is not possible to make them active, when the \\'Images\\' animator is inactive. Active the \\'Images\\' animator first to enable the other animators.\n", " \n", "- MeasurementSet with sizes of tens of Gb may not visualize the full data set properly on all machines, which can give the appearance that part of the data is flagged.\n", " \n", "- The line tool in the MAC viewer plots unreadable hex numbers.\n", "\n", "### plotms\n", "- In plotms, PDF and PS exports have been reported not to work with some older sub-versions of RH7 (e.g., RH 7.6). While the problem has not been completely characterized, upgrading to a newer RH version (e.g., RH 7.9) has shown to solve this issue.\n", "\n", "- In RedHat 7 we found that in some circumstances the vertical tab of the viewer appears on the right hand side instead of the left hand side. This eclipses the scrollbar and makes it difficult to use. To fix, add the following to the top of \\~/.config/Trolltech.conf\n", "\n", " ```\n", " [[Qt]]{.error}\n", " \n", " style=GTK+\n", " ```\n", "\n", "- When plotting pointing axes in **plotms** on RHEL6, the tick-values of minutes and seconds on the axes are not multiples of 5\n", "\n", "- For concatenated data sets, **plotms** can create an output error if certain data columns were present in some of the concat input MSs, but missing in others (making concat inset zero values). A practical workaround is to either handle the MSs separately, or delete those columns using the tb.removecols tool (but in case of the latter one has to take care that the columns are not crucial).\n", "\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "CHoyzdy8JYp6" }, "source": [ "### uvmodelfit\n", "- When running **uvmodelfit**, the output componentlist does not contain the uncertainty in flux that the task calculates (and displays at the end of the fitting process).\n", "\n", "### imstat\n", "- The use of the *\"centerbox*\" parameter when specifying a region in **imstat** has a known issue that under very specific circumstances, less pixels are taken into account for the statistics than what is expected. This only occurs when all of the following are true: (1) values are specified in pixels; (2) the width of the box is an even number of pixels (e.g, 4pix, 16pix, or 100pix); and (3) the box is located away from the image center in Right Ascension (progressively more pixels are dropped when moving away from the image center, but only in RA). The issue is a combination of machine rounding errors (when the boundary of the centerbox is exactly at the center of a pixel), and the fact that centerbox has to converts pixel coordinates to sky coordinates to allow all possible combinations of regions. Note that the \"*box*\" parameter is not affected by this, because it can be more strict in only using flat pixel coordinates. As a simple work-around solution, we recommend to always give the width of the centerbox in \"odd\" number of pixels. Please note that because centerbox places the center of a box in the middle of a pixel and CASA only includes full pixels, the width of a centerbox always has an odd number of pixels anyway. For example, `centerbox=[[1000pix,4000pix],[4pix,4pix]]` for an 8000x8000 pixel image should give npts=25, but due to the above issue will result in `npts <25`. Instead, `centerbox=[[1000pix,4000pix],[5pix,5pix]]` will always give npts=25.\n", "\n", "### tb.statistics\n", "- The table tool\\'s statistics function *tb.statistics* currently ignores the useflags parameter, so statistics are calculated for all values in the specified column, and flagged values cannot be avoided.\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "pvS0_ko5JYp6" }, "source": [ "### simobserve / simanalyze\n", "\n", "- Task **simobserve** failed during CASA testing on Mac OS 14. The cause for this is not yet known and currently under investigation.\n", "\n", "- For **simanalyze**, when added two or more input MSs in the *vis* parameter (when combining an INT+SD or INT+INT data), tools may not be appropriately initialized or accessed in CASA 6. As a workaround, use tclean or sdintimaging for combining the simulated output MSs from **simobserve**.\n", "\n", "- When using **simobserve** to simulate a spectral cube, using the *inwidth* parameter with units of velocity (e.g., *kms/s*) is known to produce wrong results. Use inwidth with frequency units instead (e.g., *inwidth=‘1MHz’*).\n", "\n", "- CASA simulations do not yet fully support all spectral types of components (i.e., ability to include spectral lines or spectral indices)\n", "\n", "- When cleaning with a simulated MS, it should be considered best practice to declare the *phasecenter* parameter using the \\'J2000 xx:xx:xx.xxx +xxx.xx.xx.xxx\\' notation to account for possible rounding errors that can create an offset in the image.\n", " \n", "- corruption of simulated MS by an atmospheric phase screen is only available from the toolkit. **simobserve** and **sm**: Under some circumstances, running **sm.setnoise** and **sm.corrupt**, or **simobserve** with thermal noise, twice using the same project name, the noise can be applied a second time, doubling the noise level. Be sure to use different project names for creating different simulations with noise. See [casaguides.nrao.edu](https://casaguides.nrao.edu/index.php/Main_Page) for the latest simulation information\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "bSfGPjsaJYp7" }, "source": [ "### single dish\n", "\n", "- Difficulty in allocating memory to import/processing of Band 9 (fast-mapped, double-circle) data. Use high-performance machines as workaround. \n", "\n", "### tsdimaging\n", "- **tsdimaging** task may fail when more than several MSes are chosen as inputs (infiles) to create single output image. It is because the file descriptor opened by the task exceeds the limit defined by OSes. You can relax the limit of the number of open file descriptors by the command, e.g., ulimit -n 4096.\n", "\n", "### sdintimaging\n", "- For task **sdintimaging**, *gridder='awproject'* is not yet available. It will be enabled in a subsequent release.\n", "\n", "\n" ] }, { "cell_type": "markdown", "source": [ "## CASA Versioning\n", "\n", "CASA has a self-consistent numbering method for version identification, for versions since CASA 4.3.\n", "\n", "All current and future CASA versions are identified by a number of the format X.Y.Z-V (e.g. 6.7.0-31) which is reported when CASA is launched. The X.Y.Z-V number is also reflected in the filename of the corresponding downloadable tarball.\n", "\n", "- X: This is the main CASA version number. An increase of X is rare, and denotes some major change to the package as a whole.\n", "\n", "- Y: An increase in this number normally denotes the issue of a periodic CASA public release version.\n", "\n", "- Z: This number denotes the specific sub-version of the package. For public release versions, a non-zero value denotes a patch release (which is often required for pipeline releases or bug-fixes).\n", "\n", "- V: This is the pre-release version number.\n", "\n", "In addition, as of CASA 6, pipeline numbers are explicitly mentioned in the package name for those CASA versions that include a pipeline." ], "metadata": { "id": "7u80dpyF_48V" } }, { "cell_type": "markdown", "metadata": { "id": "5AIpo-cqFHcd" }, "source": [ "## Compatibility / Testing\n", "\n", "The CASA software works on mutiple LINUX and Mac computer operating systems, and is available for multiple versions of Python.\n", "\n", "**CASA is only validated against the operational Linux RedHat configuration of NRAO instruments, which is currently Linux RHEL8/Python3.10.**\n", "\n", "However, we expect CASA to work on combinations of OS and Python versions listed in the table below, including:\n", "- Linux RedHat\n", "- Linux Ubuntu\n", "- Mac OS\n", "\n", "The CASA team accepts bug reports in case of problems on platforms listed in the table below.\n", "\n", "Other flavors of Linux may also work, but it is up to the user to ensure the pre-requisites for [installation](introduction.ipynb#id1) are met, including all essential OS libraries.\n", "\n", "### Automated testing table\n", "\n", "Automated tests used for verification during the development cycle and for new release versions. These includes tests for CASA tasks, tools, and auxillary repos (e.g., Viewer, PlotMS, almatasks), as well as stakeholder tests, regression tests, and mpi tests. Entries marked with a \"T\" in the following table indicate versions of these automated tests in CASA 6.7.5:\n", "\n", "| OS | casatools | casatasks | casashell | aux repos | stakeholders | regressions | mpi* |\n", "| --------------------- |:------:|:------:|:-----:|:----------:|:------------:|:-----------:|:-----:|\n", "| RHEL 8 + Py 3.12 | T | T | T | T | T | T | T\n", "| RHEL 8 + Py 3.13 | T | T | | | | | \n", "| | | | | | | |\n", "| RHEL 9 + Py 3.12 | T | T | T | T | | T | T\n", "| RHEL 9 + Py 3.13 | | | | | | | \n", "| | | | | | | |\n", "| Ubuntu 24 + Py 3.12 | T | T | | T | | | \n", "| Ubuntu 24 + Py 3.13 | | | | | | |\n", "| | | | | | | |\n", "| MacOS 15 + Py 3.12 | T | T | | | | |\n", "| MacOS 15 + Py 3.13 | T | T | | | | |\n", "| | | | | | | |\n", "| MacOS 26 + Py 3.12 | T | T | | | | |\n", "| MacOS 26 + Py 3.13 | | | | | | |\n", "\n", "*Note: mpicasa, which is required for both mpi and stakeholder tests, is not supported for macOS.\n", "\n", "Older versions of CASA may not be compatible with the latest Operating Systems (for example, an appropriate usage mode on RedHat8 for CASA versions older than 6.3 has not yet been defined/tested). A listing of previous CASA versions and their supported OSs can be found on the [CASA website](https://casa.nrao.edu/casa_obtaining.shtml).\n", "\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "Tqd0_w8VHEhG" }, "source": [ "## Installation\n", "\n", "A full installation of CASA including custom python environment is available as a Linux (.tar) or Mac (.dmg) file from our [Downloads](http://casa.nrao.edu/casa_obtaining.shtml) page ()\n", "\n", "The CASA 6.x series is also available as modular packages, giving users the flexibility to build CASA tools and tasks in their own Python environment. This includes the casatools, casatasks, and casampi modules, allowing for core data processing capabilities in parallel.\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "5rJ_gGXUr9cW" }, "source": [ "### Prerequisite OS Libraries\n", "\n", "CASA requires certain libraries be installed in the user's operating system. Some may already be present by default. In case they are not, supplementary information regarding prerequisite libraries (for both the monolithic and modular CASA versions) can be found in the CASA Docs [Knowledgebase](memo-series.ipynb#Installation) pages on *Installation* for the following OSs:\n", "\n", "- [Linux (RHEL, Ubuntu)](memo-series.ipynb#Pre-requisites-Libraries)\n", "- [Mac OS](memo-series.ipynb#Additional-Pre-requisites-Mac-OS)\n", "\n", "Although the CASA Team does not provide support for other versions of Linux, we have also gathered Knowledgebase tips from users on unsupported OSs:\n", "\n", "- [Linux (Debian, Fedora)](memo-series.ipynb#User-tips-on-installing-CASA-on-unsupported-OSs)\n", "\n", "A system administrator may be required to install OS libraries.\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "gSIlhESeHf6A" }, "source": [ "### Monolithic Distribution\n", "\n", "**On Linux:**\n", "\n", "1. Download the .tar file and place it in a work directory (e.g. \\~/casa)\n", "2. From a Linux terminal window:\n", " \n", " ```\n", " $: tar -xvf casa-xyz.tar.xz\n", " $: ./casa-xyz/bin/casa\n", " ```\n", "\n", "The one caveat is that CASA on Linux currently will not run if the Security-Enhanced Linux option of the linux operating system is set to enforcing. For the non-root install to work, SElinux must be set to disabled or permissive (in */etc/selinux/config*) or you must run (as root): ```$: setsebool -P allow_execheap=1```. Otherwise, you will encounter errors like:\n", "\n", "*error while loading shared libraries: /opt/casa/casa-20.0.5653-001/lib/liblapack.so.3.1.1: cannot restore segment prot after reloc: Permission denied*\n", "\n", "
\n", "**WARNING:** By default, python 3.6 (and earlier versions of python 3) include the current working directory in the python path at startup. Any script in that directory with the same name as a standard python module or a CASA module will be detected and used by python instead of the code that is delivered with CASA. Protections have been included for files called \"new.py\" and \"pickle.py\", but other scripts may cause problems with the CASA startup. For example, do not include a file named runpy.py in the working directory.\n", "
\n", "\n", "\n", "**On Macintosh:**\n", "\n", "1. Download the .dmg disk image file\n", "2. Double click on the disk image file (if your browser does not automatically open it).\n", "3. Drag the CASA application to the *Applications* folder of your hard disk.\n", "4. Eject the CASA disk image.\n", "5. Double click the CASA application to run it for the first time. If the OS does not allow you to install apps from non-Apple sources, please Change the settings in \\\"System Preferences-\\> Security & Privacy -\\> General\\\" and \\\"Allow applications downloaded from: Mac App store and identified developers\\\".\n", "6. Optional: Create symbolic links to the CASA version and its executables (Administrator privileges are required), which will allow you to run `casa` from any terminal command line. To do so, run\n", " ```\n", " !create-symlinks\n", " ```\n", "\n", "**Configuration + data repository (Linux and Mac OS)**\n", "\n", "Each CASA distribution requires a runtime configuration (**casaconfig**) and minimal repository of binary data for CASA to function properly (**casadata**). The casadata must be explicitly installed and updated by the user, and include Earth Orientation Parameters, reference frames, ephemeris data, antenna configurations, beam models, and calibration corrections. Installation instructions for casaconfig and casadata are given in the [External Data](external-data.ipynb#Updating-the-Data-Tables) section of CASA Docs." ] }, { "cell_type": "markdown", "metadata": { "id": "KmtgVmBXHitv" }, "source": [ "### Modular Packages\n", "\n", "Pip wheels for casatools and casatasks are available as Python 3 modules. This allows simple installation and import into standard Python environments. The casatools wheel is necessarily a binary wheel so there may be some compatibility issues for some time as we work toward making wheels available for important Python configurations.\n", "\n", "Make sure you have set up your machine with the necessary [prerequisite libraries](memo-series.ipynb#Pre-requisites-Libraries) first. Then a la carte installation of desired modules (from a Linux terminal window) as follows:\n", "\n", "```\n", "$: python3.12 -m venv myvenv\n", "$: source myvenv/bin/activate\n", "(myvenv) $: pip3.12 install --upgrade pip wheel\n", "```\n", "\n", "Now pick whichever subset of the available CASA packages you are interested in. Package dependencies are handled automatically by pip, with the exception of **casadata** which must be explicitly installed and updated by the user (see [External Data](external-data.ipynb#Updating-the-Data-Tables)). The following packages are available:\n", "\n", "```\n", "(myvenv) $: pip3.12 install casaconfig==1.5.0\n", "(myvenv) $: pip3.12 install casatools==6.7.5.18\n", "(myvenv) $: pip3.12 install casatasks==6.7.5.18\n", "(myvenv) $: pip3.12 install casaplotms==2.9.1\n", "(myvenv) $: pip3.12 install casaviewer==2.4.4\n", "(myvenv) $: pip3.12 install casashell==6.7.5.18\n", "(myvenv) $: pip3.12 install casaplotserver==2.1.1\n", "(myvenv) $: pip3.12 install casatestutils==6.7.5.18\n", "(myvenv) $: pip3.12 install casatablebrowser==0.0.41\n", "(myvenv) $: pip3.12 install casalogger==1.0.24\n", "(myvenv) $: pip3.12 install casafeather==0.0.28\n", "(myvenv) $: pip3.12 install casampi==0.6.0\n", "```\n", "\n", "Users are advised to use a Python virtual environment (venv) and specific module version numbers as shown above. Giving a blank (or invalid) number to the pip install command is an effective way to list all available version numbers.\n", "\n", "**List the latest release version of a module**\n", "\n", "```\n", "(myvenv) $: pip3.12 show casatasks\n", "```\n", "\n", "**List all available versions of a module** (a hack):\n", "\n", "```\n", "(myvenv) $: pip3.12 install casatasks==99999\n", "ERROR: Could not find a version that satisfies the requirement casatasks==99999 (from versions: [list of available versions])\n", "```\n", "\n", "**Start Python in your venv and sanity check**:\n", "\n", "```\n", "(myvenv) $ python3.12\n", "\n", ">>> import casatasks\n", ">>> help(casatasks)\n", "```\n", "\n", "To exit the python venv, type ```deactivate``` from the terminal. However, the rest of this documentation **assumes the venv is active** (to reactivate, type ```source myvenv/bin/activate```)\n", "\n", "The use of python3.12 venv is a simple built-in method of containerizing the pip install such that multiple versions of CASA 6.x can be kept on a single machine in different environments. In addition, CASA is built and tested using standard libraries which can be replicated with a fresh venv, keeping the libraries needed for CASA isolated from other libraries which may already be installed on your machine.\n", "\n", "With the pip installation, CASA may be used in a standard Pythonic manner. Examples can be found in\n", "[this Jupyter Notebook](https://casadocs.readthedocs.io/en/stable/examples/community/casa6_demo.html).\n", "\n", "
\n", "**WARNING:** The pip-wheel version is intended for manual data processing, and is not yet officially endorsed by ALMA or VLA. Currently, pipelines are included in -and tested only for- all-inclusive monolithic CASA distributions. \n", "
\n", "\n", "**Parallel Processing Setup**\n", "\n", "The casampi package provides the task-level [MPI](https://en.wikipedia.org/wiki/Message_Passing_Interface) parallelization infrastructure of CASA. The casatasks module detects when casampi is available and enables the parallel processing capabilities of CASA. Advanced users may also access the casampi package directly to build new or custom parallelization schemes.\n", "\n", "*Make sure you have installed the prerequisite OS libraries for parallel processing.* In particular, users should have mpicc installed, and then follow these general instructions for installing mpi4py: https://mpi4py.readthedocs.io/en/stable/install.html\n", "\n", "To test for correct modular MPI installation, run the following commands (from Linux terminal):\n", "\n", "```\n", "(myvenv) $ echo \"from casampi.MPIEnvironment import MPIEnvironment; print('working?', MPIEnvironment.is_mpi_enabled)\" > test.py\n", "(myvenv) $ mpirun -q -n 2 python3.10 test.py\n", "```\n", "\n", "observe two instances of \\\"working? True\\\"\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "V7DACxTPmEvb" }, "source": [ "### Developer builds\n", "\n", "The above instructions are appropiate for most CASA users. However, if you want to build CASA yourself you can use the developer build instructions found in the CASA git repository [CASA readme.md](https://open-bitbucket.nrao.edu/projects/CASA/repos/casa6/browse/readme.md)\n" ] }, { "cell_type": "markdown", "metadata": { "id": "JRnTvdc9t7N0" }, "source": [ "### Installation Tests\n", "\n", "CASA provides a test script to verify the installation of CASA in the user's system. The script will download testdata from the CASA git repository on-the-fly and run 14 test cases to verify the execution of some casatasks.\n", "The following are some pre-requisites to run the installation tests in your system.\n", "\n", "\n", "* Working directory needs to have read/write permissions\n", "* System needs to be connected to the internet to download testdata\n", "* System needs to have [git-lfs](https://git-lfs.com/) installed for the data download\n", "* When installing modular CASA, the minimum set of packages to install are casatasks, casatestutils and casadata.\n", "\n", "\n", "Follow the instructions to [install CASA for a monolithic distribution](introduction.ipynb#Monolithic-Distribution) and run the test_casatasks.py script which is installed in the site-packages of your CASA installation. The example below shows running the test script from the command-line using a CASA monolithic installation in Linux.\n", "```\n", "./casa-xyz/bin/casa -c ./casa-xyz/lib/py/lib/python3.10/site-packages/casatasks/tests/test_casatasks.py\n", "```\n", "\n", "For Mac OS, the command-line input is as follows:\n", "```\n", "./casa_xyz.app/Contents/darwin/bin/casa -c ./casa_xyz.app/Contents/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/casatasks/tests/test_casatasks.py\n", "\n", "```\n", "\n", "For testing the installation of modular CASA, follow the [installation steps](introduction.ipynb#Modular-Packages), and then point python to the test script as in:\n", "\n", "```\n", "python3.10 ./venv/lib/python3.10/site-packages/casatasks/tests/test_casatasks.py\n", "```\n", "\n", "\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "xeBrpBe0-8DA" }, "source": [ "## Performance\n", "\n", "CASA is now running performance benchmarks against a subset of the `casatasks` API to track various runtime metrics over development history of the project, starting in CASA 6. See the [Performance Benchmark](https://casangi.github.io/casabench/) webpages for an interactive view of the latest test results.\n", "\n", "### casabench\n", "\n", "The tests are implemented using [airspeed-velocity](https://asv.readthedocs.io/en/stable/) with configuration and results tracked by a separate [repository](https://github.com/casangi/casabench). Automated deployment is coordinated with Bamboo and confined to a single computer node.\n", "\n", "The dedicated testing machine has eight Intel(R) Xeon(R) CPUs (E5-2670 @ 2.60GHz), 264GB of DDR3 SDRAM (36KSF2G72PZ-1G6P1 @ 1600 MT/s), 500GB Crucial(R) internal SSD (MX500), and runs Red Hat Enterprise Linux Workstation release 8.10 with kernel version 4.18.0-513.11.1.el8_9.x86_64.\n", "\n", "Tests currently run for each CASA6 pre-release package once other verification steps are complete. New tests and test cases will be added as development continues in CASA.\n" ] } ], "metadata": { "colab": { "name": "introduction.ipynb", "provenance": [] }, "kernelspec": { "display_name": "Python 3", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 0 }