Log of /branches

Increment minor version to 3

Update data/stout/refs.json

Update the Stout refs and species PDFs in docs

Update PDFs in tsuite further

Tell doxygen to not include full paths on output

Add mole_reactions.cpp missing from previous commit

Create file to hold unused header files
    
list_headers.pl creates a file that shows all files that use each header
file.  It takes some parsing to find if there are files not being used.
    
The new file contains all unused header files.

Update author list

Deleting branch update-stout

Fixed cast to of LONG_MAX to double to silence LLVM v12 warning

Add links to Hazy 1-3 and Quickstart PDFs to docs/

Add Gargi Shaw's Markdown for CO reactions

Update md5datafiles.dat, .gitignore

Update energies as per Camilloni et al. 2021, RNAAS

Fix bug in 'save species bands'
    
User Swayam Panda reported that using the 'save species bands' command
more than once in an input deck leads to a segmentation fault.  I have
been able to reproduce the bug.

What causes the bug is a misconception about C++ vector iterators.
When storing the data entered in the commands, the file that holds the
bands is read in, and its data stored in static vector for the bands file
objects by growing the vector (with push_back()).  An iterator to that
memory location is returned to, and stored by the object that holds all
command info.

This assumes that the iterator remains invariant after growing the vector.
That is not true.  C++ vectors are not like C arrays.  Iterators (save the
last one entered) no longer point to valid memory after the vector grows
by one member.

The least disturbing solution in the current scheme is first to read the
band files, and then to store the info for each command.

Update remaining links in Hazy from trac to gitlab

Fix broken links to wiki

Update .gitignore

Fix bug in maser escape probabilities

Patch ported from git.


Original comments:
==================

Merge branch 'maser' into 'master'

See merge request cloudy/cloudy!9

Gary Ferland, 2021-Feb-15:
--------------------------
Add debug print for special maser escape probability
    
This adds a commented-out debug print statement to the
routine that calls esc_CRDwing_1side.  The report
extends over the range of interesting optical depths.  The
range extends from close to the lower
limit of the maser optical depths we can handle due to
floating point limits to a high enough value well into
the positive tau case.

Note that this printout occurs for whatever damping constant
the routine is called the first time.  The print code
exits after doing one loop.  It does not test a range
of damping constant and would not illustrate the problem
being fixed on this branch..

An assert for positive escape probability has been
added to esc_CRDwing_1side.  That assert would have made
it much easier to trace down the original thrown assert.

Add tfile to list of ignored files. 

Improve the usefulness of error print when maser optical depth
becomes too small by giving the smallest optical depth considered.


Gary Ferland, 2021-Feb-08:
--------------------------
Correct algebra on line 197 to preserve intent
of original source.


Gary Ferland, 2021-Feb-07:
--------------------------
BUGFIX - CRD with damping wings did not do masers
    
A sim that used all levels of the new Fe II and Fe III
datasets threw an assert at
 Failed: escgrd_v > 0.
 It happened in the file ../rt_escprob.cpp at line number 330

It was due to a long-standing bug/oversight.  The complete
redistribution with wings escape probability did not protect
against masers and a negative escape probability resulted.
We now use the appropriate routine when the optical depth is
negative.  The sim follows.

This oversight has been present for a long time.  It is
surprising that it has only now bitten.  This is likely
because we now have atomic models that are much larger
than in the past.

---

````
set save prefix "assert"
TURBULENCE= 0.600000 LOG 3.000000
species "Fe+" dataset="Smyth19"
species "Fe+" levels=all
species "Fe+2" levels=all
stop column density 23
stop temperature exceeds 1e7 K
print lines sort wavelength
print lines column
print lines faint 1
iterate
print last
hden 14
table SED "AGN_Jin12_Eddr_mid.sed"
phi(H) 23
normalize to "Inci" 1215 scale factor = 1215
````

update Ha wavelengths that changed with the NIST
> energy levels upgrade

Merged from mainline r13796 through r14361

tsuite/slow/time_cool_cp.in
tsuite/slow/time_cool_cp_eq.in
  - Updated wavelengths following merge from mainline.

Merged from mainline r14168 through r14357

util/register/viz.py
  - Commented out debugging statements in process_tran_WN_data().

Renamed: data/switch-hydro-Aul -> data/hydro_tpnl_switch.pl

util/idata/nrg.py
  - bugfix: Incorrect use of del to remove item from list.  Now using remove()
    method.  Fixed.

util/register/viz.py
  - In term plots, do not mirror y-axis on the right-hand side of the plot.
  - Refactored preparation of file name suffixes for term- and BR update
    plots into function.  It returns the format to be used, which includes
    leading zeros when needed, e.g., the suffix for file 1 of 20 is '01of20',
    intead of '1of20' used until now.

util/idata/nrg.py
  - Logical error: Method BasicEnergy.set_energy() was setting an attribute
    ('_shifted') of the RegisterEnergy() class.  Removed the assignment, and
    introduced method Energy.set_energy(), which calls the homonymous method
    of BasicEnergy(), as well as RegisterEnergy.set_shifted().  The latter has
    been modified to not require indices to be passed on (used with term
    registration).  Fixed.

util/register/viz.py
  - Adapted comparisons against experimental wavelengths to handle datasets that
    have no WL uncertainties, or have no observed wavelengths at all.

util/register/shifts.py
  - bugfix in scipy.optimize.minimize_scalar:
    In some cases, minimization of the change to branching ratios failed to find
    the global minimum, presumably due to the non-linear form of that function.
    In fact, minimization of the level lifetimes produced a smaller BR change
    than the function minimizing the BR change! 
    Tightened energy bracket to 1/4 of the range to the nearest terms, and
    recovered a global minimum lower than that obtained by minimizing the change
    to the lifetimes.  Fixed.

util/idata/nrg.py
  - bugfix in Energies.is_level_present():
    r14334 introduced an overambituous check against a level's Jtot, which
    prevented J=0 levels from being entered.  Removed the check altogether.
    Fixed.

Added symbolic links:
  - data/hydro_tpn.dat -> hydro_tpn_n100.dat
  - data/hydro_tpnl.dat -> hydro_tpnl_n100.dat

Truncated files to n=100.
  - data/hydro_tpn.dat
  - data/hydro_tpnl.dat

adm-ingest-data.py
  - Added command-line option to exclude certain levels from hybrid model.

util/nist/nrg.py
  - Added comment to form_name(), explaining that unphysical configurations,
    such as the O5+ level 1s.2d.3d 4D at 5197910 on NIST, produce undefined
    names.

util/idata/nrg.py
  - bugfix: In Energies.is_level_present(), a level with no name (see above)
    would cause an abort.  Now guarding against that possibility.  Fixed.

test/idata/test_nrg.py
  - Rearranged code.  Inner functions converted to module functions.

adm-ingest-data.py
  - Improved warning for multipole orders discrepant from that determined by ADM.

Added changelog for r14327.
  - util/idata/tp.py
  - util/idata/wl.py
  - util/nist/extract.py

util/idata/wl.py
  - Uncertainty to calculated wavelength is no longer required.

test/idata/test_wl.py
  - Updated unit tests to new interface.
  - Added tests for data records with no uncertainty in calculated wavelength.

util/nist/extract.py
  - Updated call for WLs() object.
  - A file of wrong wavelengths is created only if any are found.

adm-ingest-data.py
  - Disabled reordering (typically degenerate) terms by default.
    Enabled via command line option, -r.

svn merge -r14320:14321 ^/trunk/data/SED

adm-merge-tp.py
  - Only use experimental transitions if their accuracy is 10% or better.

tsuite/auto/agn_warm_absorber_hiU.in
  - Updated wavelength for Fe26 1^2S-2^2P to current value, following Priyanka's
    changes to data/phfit.dat.  See r14306.

tsuite/auto/fix_input_scripts_wl.pl
  - Ported changes to look for suggested wavelength, instead of assuming it is
    always 3 lines after the emitted warning.  First implemented on the Priyanka
    branch in late Aug 2020.

docs/LineLabels.txt
  - Updated to current wavelengths.

svn merge -r14236:14304 ^/trunk 

update H-like n=2 energies.  Priyanka extended the ionization potential
precision in data/phfit.dat to current NIST values.  A polynomial correction to 1/n^2 is now applied
in source/iso_create.cpp to bring the n=2 energies to their NIST values.  Some Ka wavelengths change
slightly as a result.

svn merge -r14301:14302 ^/trunk 

svn merge -r14297:14298 ^/trunk 

Added option to create scatter plots for transition energy updates.
  - adm-visualize.py
  - util/register/trans.py
  - util/register/viz.py

util/register/report.py
  - Fix title of TeX table for term registration.

adm-fine-tune.py
  - Added menu option 'rp' to inspect the wavelength updates in terms of the
    resolving power required to identify the wavelength shifts.

adm-fine-tune.py
  - Changed the formatting of the energy splitting ratio in the output of menu
    option 'td' (transition details).

util/register/viz.py
  - Modified data structure for data in wavelength scatter and resolving power
    plots to hold dicts, instead of tuples, as the most fundamental data
    structure.

util/register/viz.py
  - Decluttered delchi plots of wavelength offsets between hybrid and
    experimental wavelengths, see r14268: Removed error bars; added dotted
    lines at +/-1, instead, to demark range of consistency within one sigma.

util/nist/extract.py
  - Added wavelength validation.  Now air wavelengths are also downloaded
    and are compared against vacuum wavelengths.  The user is notified of
    any problems, and prompted to correct for air refraction. 
    Any changes made are also reported in a text file.
  - Refactored line acquisition to be used with both vacuum and air wavelengths.
    Most notably, process_line_table() has been modified to work over table
    'sections', defined as a header followed by lines of data.  This is because
    tables reporting air wavelengths contain three such sections, each
    reporting the switch between vacuum and air values.

util/nist/files.py
  - Modified get_html_transition_filename() to include '_air' in the output file
    name, controlled by a keyword argument.  By default, the string is omitted.

util/idata/wl.py
  - In BasicWL(), added methods reset_calc_wl() and reset_obs_wl() to update
    existing wavelengths.

test/idata/test_wl.py
  - Exercised new methods in unit tests.

util/nist/phtml.py
  - In TranParser():
    - Enabled parser to read air wavelengths in addition to vacuum.
    - Modified methods obs_wl(), obs_wl_unc(), calc_wl(), and calc_wl_unc() to
      report the wavelengths read, either vacuum or air.
  - In Header(), refactored column validation in method validate_column().

test/nist/test_phtml.py
  - Exercised new capabilities in unit tests.

util/io/log.py
  - Implemented shutting down output to STDOUT from calls to prt() via a keyword
    argument.
    Also adapted calls to error(), warn(), and note() to print their prefix
    (e.g., ERROR) only if the main message is printed.
    Introduced function prep_prefix_kwargs() to handle printing the prefix.

util/nist/phtml.py
  - Refactored code extracting columns from a table line into function
    parse_table_line().

test/nist/test_phtml.py
  - Exercised function in unit tests.

adm-fine-tune.py
  - Added option to sort branching ratios (menu option 'br') in ascending
    order.  The default was descending.

adm-fine-tune.py
  - Added column of theoretical indices to the output of the wavelengths
    option.

util/register/trans.py
  - bugfix: The same dict was used for storing both the calculated and observed
    wavelenghts, causing the calculated wavelengths to be overwritten.  Now the
    observed data are stored in a separate dict, preventing an overwrite.
    Fixed.

util/register/viz.py
  - The word 'Theoretical' was used instead of 'Hybrid' in the output plot
    showing the wavelength differences from the experimental values in units of
    the (exp) uncertainty.  Fixed.

adm-fine-tune.py
  - Added menu option to compare hybrid wavelengths to experimental values.

util/register/trans.py
  - Now recording the experimental indices with wavelength data.

util/ui/upr.py
  - Silence warning in readline_floats() when an empty line is given.

util/register/viz.py
util/register/trans.py
  - Moved functions to trans.py:
    - experm_level_to_theory()
    - experm_level_to_hybrid()
    - prepare_WL_comparison()
  - Modified 'obs' and 'calc' datasets furnished by prepare_WL_comparison() to
    be lists of dicts, not tuples.

util/register/shifts.py
  - Added comment in module docstring for recent changes (r14265).

util/register/shifts.py
  - In getting the energy shift by minimizing the total correction to the TP:
    - bugfix: The multipole order index, k, was being as the exponent of the
      energy ratio in the TP update, instead of 2k+1.  Fixed.
    - Modified code to compute change in the level (inverse) lifetime for the
      given energy shift, instead of the sum of the log of the TP update factors.
      This is because the latter does not discriminate between strong and weak
      lines, leading to an unbiased average that did not work well in many cases.
      By contrast, the current implementation favors strong transitions over weak
      ones. 
      The changes are in ufac(); the TP update factors may be obtained by use of a
      keyword argument.
    - Parallelized exponentiation using numpy.

  - Introduced option to set the energy shift by minimizing the total change in
    branching ratios for this level.

  - Modified minimize_tp_ufac() to accept a function as its first argument, to use
    with both minimization options, and renamed to do_minimize().

  - Now using inner function to produce uniform output to log for all shifting
    options.

svn merge -r14259:14260 ^/trunk

util/register/report.py
  - Updated TeX output to include column for fractional energy change.

adm/adm-fine-tune.py
  - bugfix in output of 'bs' option: Loop was over level indices, leading
    to the first entry of the sorted levels not being displayed.  Now looping
    over the list elements.  Fixed.

Implemented plots of branching ratio updates per level.
  - adm/adm-visualize.py
  - adm/util/register/viz.py

adm/util/register/trans.py
  - Newly added module.
    Contains a refactored function for the weight of a BR update.

adm/adm-fine-tune.py
  - Use refactored function with 'bs' option.

adm/util/register/init.py
  - Added new module to the list of modules for 'import *'.

adm/util/register/trans.py
  - Newly added module.
    Contains a refactored function for the weight of a BR update.

adm/adm-fine-tune.py
  - Use refactored function with 'bs' option.

adm/util/register/__init__.py
  - Added new module to the list of modules for 'import *'.

allow all transitions from ground state of two-electron
systems to become optically thick in the Case B limit.  Previous code had been
designed for He I so expected only E1 transitions to be fast

allow all transitions from ground state of two-electron
systems to become optically thick in the Case B limit.  Previous code had been
designed for He I so expected only E1 transitions to be fast

adm/util/register/viz.py
  - bugfix in scatter plots of level lifetimes: Calculated theoretical
    values including TP corrections.  Now they are explicitly excluded.
    Fixed.

tsuite/auto/igm_perseus.in
  - Updated to CS from Si+2017 used for He-like Fe.

svn merge -r14231:14232 ^/trunk 

Partial update to test suite, following r14228.

Split hydro_energies.dat and helike_energies.dat into files in the Stout directory

add command No scattering intensity to not include
electron scattering as part of the reported total line intensity, due to
its very large Doppler width

helike_cs.cpp
  - Added data for Fe, due to Si+2017.

mesh.cpp - improve print statement
data/hydro_energies.dat - adjust energy of He II n=2 shell so not redundant with H I on Cloudy energy mesh

Implemented genreration of level lifetime scatter plots.
  - adm/adm-visualize.py
  - adm/util/register/viz.py

test/idata/test_tp.py
  - In unit tests of total_TP(), exercised ground level total TPs of 0.
    This is possible as of the previous commit, r14216.

util/idata/tp.py
  - Defined new Transitions.total_TP() method to compute the total TP in all
    transitions out of a level; this is essentially the inverse of the level
    lifetime.

test/idata/test_tp.py
  - Exercised new method in unit tests.
    Defined new dataset to use with new method; also used with other unit tests.

svn merge -r14156:14207 ^/trunk

adm/util/register/report.py
  - Refactored reporting functions to separate data preparation from table
    presentation.  This is a preparatory step for producing TeX tables with
    the same information.

adm/util/adas/extract.py
  - Switched back to ignoring Aul <= 1e-30, but keeping the associated
    collision strengths.

Added wavelength comparison plots.

adm/util/register/viz.py
  - Implemented wavelength comparison plots.  Each plot shows the difference
    between the wavelengths of a dataset and the experimental wavelengths.
    Plots are created for each of the theory and hybrid datasets, against each
    of the observed and calculated experimental data.

adm/adm-visualize.py
  - Added WL comparisons as the first kind of scatter plots created.
    The plots are generated only if experimental wavelength exist (as wls_*.json
    files in the experimental directory).

adm-fine-tune.py
  - bugfix in branching ratios: Theory transition indices were overwritten
    when levels flipped, ie, the lower level was above the upper level relative
    to the hybrid dataset.  Now using temporary variables for the indices.
    Fixed.
  - Reorganized script into separate sections for functions operating on
    energies and transitions.
  - Dropped reading of collision files.  They were initially read in to set the
    connectedness of the dataset, but, as of r14147, this is done upon generation
    of the hybrid dataset.

Computing term energy shifts that minimize the total TP updates for the term.

adm-shift-unreg-terms.py
  - Theoretical transitions are always read in; the nickname used depends on
    which command-line option was given.

util/register/shifts.py
  - Implemented the calculation of the energy shift that minimizes the total
    updates for a given term.  The updates are computed as the absolute log
    value, to guarantee that the increasing and decreasing factors are treated
    equally.
    Computed value offered as an option to shift the term by.
  - Modified printed list of terms to show at most 11 terms to the user (5 below
    the term in question, and 5 above).
  - Added final validation confirmation before the update is committed to memory.

util/register/convert.py
  - In predict_hybrid_level_energy(), allow for calculations of terms shifted by
    some amount, using the code for terms registered against experimental terms.

Merge from mainline r14094 through r14199. 

util/nist/extract.py
  - When generating the internal energies dataset, use the index of the level
    in the main dataset to insert it in the groups and terms datasets.
    The previous practice of using the level index itself is not rigorous, and
    could in theory cause problems, although that'd be very unlikely.

util/idata/state.py
  - Renamed class J() -> AngMom().
  - bugfix in AngMom(): Missing comma in list of required keywords led to
    implicit concatentation of strings, which caused copy() and all methods
    that use that list to fail.  Essentially, the code was trying to manipulate
    non-existent private variables, but it was doing nothing.  The test suite
    failed to detect this because the equality operator also operates on these
    keywords, so with no data to compare, it reported that objects were equal.
    Added the missing comma.  Fixed.
  - Added AngMom.pprt().
  - Recoded OneState.term_names_agree() and MixedState.Jtot().

test/idata/test_state.py
  - Updated calls involing new class name, AngMom().
  - bugfix: Putative bugfix in r14185 in a test of MixedState.term_names() was
    (likely) the result of the AngMom() bug involving the missing comma.
    Reverted.

util/nist/phtml.py
  - bugfix in Header.parse_header(): Only the last wavelength column read would
    have an uncertainty column set up.  Now columns are set up for both the
    observed and the Ritz wavelengths.  Fixed.
  - In DataParser.parse(): With the previous fix in place, only one wavelength
    uncertainty would be stored. Now the uncertainty value read is copied to
    both uncertainty columns.  The code was redesigned so that the assignment
    of data to columns takes place in one step instead of two.

util/nist/wl.py
  - Added new module.  Contents refactored from util.idata.wl.

util/idata/wl.py
  - Removed code refactored in util.nist.wl module.

test/idata/test_wl.py
  - Updated module imports.

util/nist/__init__.py
  - Added new module to list for 'import *'.

util/idata/wl.py
  - Added module to handle wavelength data.

test/idata/test_wl.py
  - Exercised new module classes in unit tests.

util/idata/__init__.py
  - Updated for 'import *'.

util/idata/tp.py
  - Refactored several methods of Transitions() into class TranList().
    Transitions() now derives from TranList().

test/idata/test_tp.py
  - Minor formatting changes.

util/idata/state.py
  - Defined class J() to handle angular momentum data.
    - OneState() methods Jtot() and statw() now methods of J().
  - Renamed SpTerm._ignore -> SpTerm._term_ignore. 
    - OneState() now derives from J, as well as from EConfig & SpTerm.

test/idata/test_state.py
  - Exercises new class in unit tests.
  - Fixed bug in a test of MixedState.term_names(), which failed to identify
    that compatible terms had been provided.

svn merge -r14180:14181 ^/trunk 

svn merge -r14177:14178 ^/trunk 

test/idata/test_state.py
  - Added unit tests for export_dict() and import_dict() methods of
    OneState() and MixedState().

util/idata/state.py
  - Defined class SpTerm() to handle spectroscopic term data.
  - Renamed OneState.term() -> OneState.spterm().

test/idata/test_state.py
  - Exercised new class methods in unit tests.

util/atoms/spterm.py
  - Minor docstring update.

util/idata/state.py
  - In OneState():
    - Demoted _jtot to an optional keyword, as there may be levels of uknown J.
    - Renamed private variable: _config -> _name
    - Renamed method: config_level() -> name()
 - In MixedState():
    - Renamed:
      - _config -> _names
      - input_config() -> input_names()
      - have_two_configs() -> have_two_names()
      - primary_config() -> primary_name()
      - other_config() -> other_name()
      - config_level() -> names()

adm-fine-tune.py
test/idata/test_nrg.py
test/idata/test_state.py
util/idata/nrg.py
util/register/terms.py
util/register/viz.py
  - Update calls to MixedState() methods.

source/dynamics.cpp
  - Restored timestep_next() to original coding involving only changes to the
    H density.  Permits cooling below 6,500 K.

scripts/adm/util/idata/state.py
  - Defined class EConfig() to handle electronic configurations with.
    Now OneState() uses an object of EConfig() to store, and to manage
    the electronic configuration.  The refactored methods have been removed
    from OneState().
    - The new class also provides methods to access the ground configuration
      subshells, as well as the number of electrons in the ion, used with
      filling any missing shells.  In addition, it comes with methods to copy,
      compare, and export / import data as dictionaries.
  - Refactored OneState() methods copy(), __eq__(), export_dict(), and
    import_dict() to use the base.Base methods.

scripts/adm/test/idata/test_state.py
  - Exercised new class in unit tests.
  - Promoted inner functions of unit_tests() to module functions.

scripts/adm/util/idata/nrg.py
  - Minor formatting changes.

svn merge -r14162:14163 ^/trunk

svn merge -r14158:14159 ^/trunk 

hack at lines 864-868 rt_escprob.cpp to include or exclude Pesc

svn merge -r14152:14153 ^/trunk

svn merge -r14149:14150 ^/trunk/source update from 2020RNAAS...4...78S

adm/adm-merge-tp.py
adm/adm-merge-coll.py
  - Now setting the connectedness of a dataset as soon as both the transitions
    and collisions datasets have been merged.  The stored energies dataset is
    updated.
  - bugfix: It was possible to operate on an unset value for the energies file
    nickname.  If unset, the merged energies dataset is now used by default.
    Fixed.
  - bugfix: Checks against the presence of the output file actually produced a
    new filename if the one of intereset already existed.  Now using the
    intended file name for the checks.  Fixed.

adm/adm-fine-tune.py
adm/adm-finalize.py
  - No longer setting the hybrid dataset connectedness after loading.

adm/adm-fine-tune.py
adm/adm-visualize.py
  - Removed calls to JoinedEnergies.populate_cterms().  The hybrid dataset now
    has its dataset of terms populated when created.  These calls compromise the
    existing one.  See r14139.

adm/util/idata/nrg.py
  - bugfix: In the calculation of the energy, validation of the supplied indices
    was using the supplied configuration and term.  A theoretical combination
    that has been replaced by an experimental term would therefore fail the test
    (done on the hybrid dataset).  In addition, two terms of different ancestry
    could also lead to a failure.  To avoid such cases, the cterms() dataset is
    searched for all matches of conf-term combinations, which are then used to
    validate the supplied indices.  Fixed.

adm/util/register/merge.py
  - bugfix: For 14140, I had disabled the copying of level and term data into
    the hybrid dataset to expedite testing, but left them in the commited code
    by accident.  Re-enabled.  Fixed.

adm/util/register/merge.py
  - Now complementing the level energy dataset with a term dataset.
    Both theoretical and experimental terms are added to that dataset,
    using their hybrid level indices.

adm/util/idata/nrg.py
  - Defined ConfTerm.set_indices() to set the indices of a term in one go.
  - Defined ConfTerms.insert_term() to insert a term (instance of ConfTerm())
    to the dataset.
  - In JoinedEnergies():
    - Now allocating a ConfTerms() dataset upon initialization, which is now
      removed from method populate_cterms().

adm/adm-sort-nrg.py
  - Partition the levels into terms before storing the sorted dataset.

adm/util/register/shifts.py
  - bugfix: When selecting terms to shift, terms were skipped if they were both
    shifted to match an experimental term, and shifted by an arbitrary energy.
    The contradictory demand was not met for registered terms (shifted to match
    an experimental term) and caused them to be presented for shifting by an
    arbitrary amount, undoing their registration.  Removed the test against an
    arbitrary shift.  Fixed.

adm/util/idata/nrg.py
  - bugfix in ConfTerms.shifted_with(): Call to ConfTerms.shifted() examines
    if the Term has been shifted by an arbitrary amount, not if it has been
    shifted with an experimental term.  This is an orthogonal use case, and
    the function could never give the list of experimental levels.
    Removed the calls to self.shifted().  Fixed.
  - Energies.register_level_print() now shows the original energy by default,
    but accepts an option to show the current value, and the uncertainty if
    it has been copied.  See r14128.

adm/util/register/terms.py
  - When showing the levels of a shifted term, show the updated energy, not
    the original.  (Update related to last change above, made in function
    check_term_order().)
  - When shifting a term, now copying the experimental uncertainty to the
    theoretical level, see r14128.  (Update in shift_term_interactive().)
  - When multiple experimental terms match the theoretical one, offer to
    shift against the first one that is not already shifted.  No check was
    previously done.  Related to r14105.  (Change in register_term_by_term().)

adm/util/idata/nrg.py
  - Updated ConfTerms.shifted_with() to accept a list of level indices to
    operate on.

adm/util/register/terms.py
  - Refactored code of level registration into function match_levels().
  - Refactored inner function check_term_order() as a module function.

adm/util/register/terms.py
  - bugfix: Attempt to return obsolete variable in do_shift_term().
    Variable name updated.  Fixed.

adm/adm-merge-coll.py
  - Added error messages if either of the indices is not found.
    The program aborts in such a case.

adm/util/atoms/shells.py
  - bugfix in fix_hex(): Doubly-excited levels with uncertain valence shell
    (appearing as 'n', e.g., in 'nd') could not processed, and led to crashes.
    Such levels have only one electron, so there is no hex number to fix for
    them.  They are now being explicitly skipped.  Fixed.
 
adm/test/atoms/test_shells.py
  - Added a unit test to exercise the new functionality.

adm/adm-merge-tp.py
  - bugfix: When searching for a matching experimental transition, no provision
    was made for levels being in different order between the two datasets.  This
    led to crashes for flipped levels, as the lower index was higher than the
    upper index.  The indices are now brought in order before querying the
    experimental dataset.  Fixed.

adm/util/stout/check.py
  - Minor formatting change.

adm/util/idata/nrg.py
  - Indent warnings with a blank string when printing level details for
    registration.

adm/util/register/levels.py
  - During synthesis of hybrid energy dataset, omit reporting levels that are
    matched with levels that are not in experimental groups.
    This could happen if the experimental level was removed from the group
    (because, e.g., it was in a degenate term), but the levels were matched
    during the term-by-term registration.
  - Minor updates to how registered levels are presented.  See below.

adm/util/idata/nrg.py
  - Modified Energies.register_level_print() and Groups.level_print() to
    allow passing a string with which to prepend the output for a level.
    The string is used with the first line of state information.  Subsequent
    lines use a blank string instead.
    This now includes the lines showing the energy uncertainty, and the notes
    to the energy value, if present.

adm/util/register/report.py
  - bugfix in function show_term_registration(): Experimental term energy and
    indices were requested using the theoretical configuration and term.
    That led to craches when ancestry terms were present in the configuration.
    Now using the experimental configuration and term.  Fixed.
    See also r14109.

adm/util/register/shifts.py
  - In list of bracketing terms, columns of printed configuration and term are
    now guaranteed to be aligned -- string sizes adjust to maximum in dataset.

adm/adm-export-stout.py
  - Removed (repeated) import of util.stout.fmt.

adm/util/register/shifts.py
  - bugfix: Bracketing failed for terms above the highest term matched/shifted.
    Redesigned how the bracketing is done to address such cases -- which can be
    quite common in ions with AI levels.  Relative splitting now available for
    terms that fall within the energy range of registered terms.  Fixed.
  - Removed package name from imports of local modules.

adm/util/register/convert.py
  - Added function predict_hybrid_term_energy(), which was extracted from 
    get_final_term_energy(), inner function of get_bracketing_terms_energies()
    in shifts.py.

adm/util/register/terms.py
  - Removed package name from imports of local modules.

adm/util/register/shifts.py
  - In get_bracketing_terms_energies():
    - bugfix in inner function get_final_term_energy(): Experimental term energy
      and indices were requested using the theoretical configuration and term.
      That led to craches when ancestry terms were present in the configuration.
      Now using the experimental configuration and term.  Fixed.
    - bugfix: When printing the list of terms, the theoretical configuration and
      term were used to query the experimental dataset.  ConfTerms.get_equiv()
      is now used to get a list of matching terms, which are then iterated over
      until a term that matches in indices is also found.  An error is issued if
      no match is produced.  Fixed.
    - Now reporting the relative energy shifts, in terms of the original energy.
  - bugfix in shift_unregistered_terms(): register.terms.do_shift_terms() call
    did not include relative energy shift.  Fixed.

adm/util/register/terms.py
  - bugfix: Relative energy change is w.r.t. to the original theoretical,
    but the experimental was used.  Fixed.

adm/util/register/terms.py
  - bugfix:  When multiple experimental terms matched, only the first one was
    processed, even if it were matched.  Now processing all matching terms.
    Fixed.
  - When shifting levels mark the experimental levels as shifted with their
    corresponding theoretical levels.  These are now shown on screen, and are
    especially useful when multiple experimental terms match the ADAS terms
    that appear the same because their ancestry is not preserved -- they help
    keeping track of level registration.
  - Removed an outdated dict that keeps track of which experimental terms have
    been matched from __process_terms().

adm/util/register/levels.py
  - Now printing energy differences between theory and experiment in bold.

adm/util/idata/nrg.py
  - When printing to screen:
    - Use bold text for headers of terms (prior to the levels being displayed).
    - Use red color for already-matched levels.
    - Print warnings in blue about which levels they are matched with.

adm/util/io/log.py
  - Added ability to print text using ANSI colors and decorations.

adm/util/idata/nrg.py
  - In Energy.register_level_print(), print the uncertainty on the level energy
    in a separate line, after the data for all configurations contributing to
    the level are reported.

adm/util/idata/nrg.py
  - In Energies.insert(): Allow for levels duplicate in terms of their
    configuration, term, and J, to be entered to the dataset -- but do issue a
    warning (as before).  The change is necessary for ADAS levels with missing
    ancestry terms to be processed.

adm/util/nist/extract.py
  - Removes superfluous keyword argument 'overwrite' from calls to save_json()
    of the IPs and References classes -- they both invoke util.io.json.save()
    with this argument set to True.

scripts/adm/util/adas/web.py
scripts/adm/util/atoms/splevel.py
scripts/adm/util/atoms/spterm.py
scripts/adm/util/atoms/trans.py
scripts/adm/util/cloudy/cloudy.py
scripts/adm/util/cloudy/lines.py
scripts/adm/util/cloudy/stout_refs.py
scripts/adm/util/idata/nrg.py
scripts/adm/util/idata/unified.py
scripts/adm/util/io/fs.py
scripts/adm/util/io/json.py
scripts/adm/util/io/nicks.py
scripts/adm/util/nist/extract.py
scripts/adm/util/ptable/element_data.py
scripts/adm/util/ptable/ions.py
scripts/adm/util/ptable/spectra.py
scripts/adm/util/register/convert.py
scripts/adm/util/register/levels.py
scripts/adm/util/register/nocfg.py
scripts/adm/util/register/report.py
scripts/adm/util/register/shifts.py
scripts/adm/util/register/terms.py
scripts/adm/util/register/viz.py
scripts/adm/util/stout/check.py
scripts/adm/util/stout/dirtree.py
scripts/adm/util/stout/files.py
scripts/adm/util/types/base.py
scripts/adm/util/ui/cli.py
scripts/adm/util/ui/dirs.py
scripts/adm/util/ui/pfile.py
  - Use relative imports throughout the util package.
    These modules were using absolute imports.

scripts/adm/util/adas/web.py
  - Removed executable property.

Deleted obsolete file:
  - fetch-NIST-IP.py

Updated manual:
  - adm/docs/manual/adm-macros.tex
  - adm/docs/manual/algorithm.tex
  - adm/docs/manual/appendix-ML.tex
  - adm/docs/manual/commands.tex
  - adm/docs/manual/examples.tex

Figures added:
  - adm/docs/manual/images/final-TP-updates-adas-error.pdf
  - adm/docs/manual/images/final-TP-updates-adas.pdf
  - adm/docs/manual/images/final-WL-updates-arbitrary_shift-respowr.pdf
  - adm/docs/manual/images/final-WL-updates-arbitrary_shift-scatter.pdf
  - adm/docs/manual/images/final-WL-updates-matched_levels-respowr.pdf
  - adm/docs/manual/images/final-WL-updates-matched_levels-scatter.pdf
  - adm/docs/manual/images/final-WL-updates-matched_term-respowr.pdf
  - adm/docs/manual/images/final-WL-updates-matched_term-scatter.pdf
  - adm/docs/manual/images/final-WL-updates-respowr.pdf
  - adm/docs/manual/images/final-WL-updates-scatter.pdf
  - adm/docs/manual/images/final-terms-p1of4.pdf
  - adm/docs/manual/images/final-terms-p2of4.pdf
  - adm/docs/manual/images/final-terms-p3of4.pdf
  - adm/docs/manual/images/final-terms-p4of4.pdf

adm/adm-export-json.py
adm/adm-export-stout.py
  - Added option to exclude unconnected levels from output datasets.
    These are defined as levels that partake in no radiative or collisional
    transitions.

adm/util/ui/cli.py
  - Defined function for command-line option to exclude unconnected levels.

adm/util/idata/nrg.py
  - Added option to LeanEnergies.copy() to exlude unconnected levels.

adm/util/nist/refs.py
  - In References.prt_to_file(), use util.stout.fmt.compose_comment_line() to
    write the line to the file -- it automatically prepends the line with a
    comment character, if appropriate.

adm/util/stout/fmt.py
  - As above, for write_comments_header().

adm/adm-visualize.py
  - Added option to produce registration plots of terms relative to each other.
  - Undid (unintentional) commenting-out of scatter plots.

adm/util/idata/nrg.py
  - Absorbed data field for energy, and method energy() into ConfTermBasic()
    from ConfTermSort().
  - bugfix in ConfTerms.cterm_by_indices(): Comparison of list of indices fails
    if they are out of order.  Now sorting indices before comparison.  Fixed.

adm/util/register/convert.py
  - Added function to identify the hybrid term that matches the given theory
    one.  Identification is done by virtue of the registered indices (see also
    changes in cterm_by_indices() above).

adm/util/register/viz.py
  - Added plots for the registration of terms relative to each other.
  - Renamed pre-existing functions treating term levels to contain '_FS_'.
  - Refactored code for adjusting term labels into function edit_label().

Changed order in argument lists of all plot-generating functions.
  - adm/adm-visualize.py
  - adm/util/register/viz.py

adm/util/register/viz.py
  - Prepended detailed-transition-update file names with dataset nickname.

adm/adm-visualize.py
  - Use the nickname of the dataset selected as the final dataset (stored in
    the JSON file, r14081) to read the correct theoretical energies dataset.

adm/util/register/viz.py
  - Added scatter plots for the individual transition subsets (according to
    whether both levels were (J,p)-matched, term-matched, or shifted by some
    amount -- introduced in r14077), and a resolving-power plot for the entire
    dataset.
    Now both scatter and resolving-power plots are generated for the dataset
    subsets, as well as for the whole.

adm/adm-visualize.py
  - Changed the order scatter plots are generated:
    Now WL plots are done after TP plots.

adm/util/register/viz.py
  - Now printing the output of gnuplot runs to the log file.
    - To do so, now using prun.run_command() to run gnuplot in the background,
      instead of prun.run_command_interactive().

adm/util/misc/prun.py
  - Guarantee that the command output is printed to file.
    - To do so, run_command() now uses subprocess.getoutput() instead of
      subprocess.check_output().
  - Both run_command() and run_command_interactive() flush the log before
    executing a subprocess.

adm/util/io/log.py
  - When a subprocess is executed, end_of_run() now prints only the command
    name. It used to be that if the entire command was given as one str, the
    entired command would be reported.
    - To do so, updated argument processing function (__proc_cline_args()) to
      split single-line commands into a list of arguments.

adm/docs/manual/images/adm-flowchart-export-stout-json.pdf
  - Updated Figure.

adm/util/register/viz.py
  - bugfix in energy update plots: Energy was scaled by a power of 10 more than
    intended.  Fixed.

adm/docs/manual/images/energy-updates.pdf
  - Deleted incorrect plot.

adm/docs/manual/images/sorted-energy-updates.pdf
  - Added new plot.

adm/docs/manual/examples.tex
  - Updated text to use correct plot of energy updates.

adm/adm-export-stout.py
  - Refactored code that computes precision to use with printed energy, TP, and
    collision strength values as methods of the relevant classes.

adm/util/idata/nrg.py
  - In Energies(), defined set_nrg_prec() and nrg_prec().

adm/util/idata/tp.py
  - In Transitions(), defined set_exp_prec() and  exp_prec().

adm/util/idata/coll.py
  - In Collisions(), defined set_exp_prec() and  exp_prec().

adm/util/types/base.py
  - In Base.__eq__(), fields with empty values (empty str or None) no longer
    trigger a False result if the field is missing in only one of the datasets.

adm/test/idata/test_coll.py
adm/test/idata/test_nrg.py
adm/test/idata/test_tp.py
  - Added unit tests to exercise new methods.

adm/test/idata/test_coll.py
  - Updated to r14065.

adm/util/nist/ip.py
  - Removed call to dict_remove_empty() from IonPot.export_dict().

adm/util/types/base.py
  - Removed definition of dict_remove_empty(). No longer needed.

adm/adm-view-json
  - bugfix: After turning into executable, script runs in a separate shell,
    and so is unaware of the ADM installation.  Modified to specify the ADM
    directory, where it can find the required util.io.json module.  Fixed.

adm/adm-view-json
  - Added (bash) script to view the contents of compact JSON files.

adm/util/io/json.py
  - Added function to load a JSON, and then pretty-print it.

adm/adm-export-json.py
  - It is now possible to exclude auto-ionization levels, and any transitions
    they are involved in, from the output datasets with the '--noai' CL option.

adm/util/idata/nrg.py
  - In LeanEnergies.copy(), enabled exclusion of auto-ionization levels.

adm/util/idata/tp.py
  - In LeanTransitions.copy(), enabled exclusion of transitions that involve
    excluded levels.

adm/util/idata/coll.py
  - In LeanCollisionBlock.copy(), and LeanCollisions.copy(), enabled exclusion
    of transitions that involve excluded levels.

adm/test/idata/test_nrg.py
adm/test/idata/test_tp.py
adm/test/idata/test_coll.py
  - Exercised exclusion capabilities in unit tests.

adm/adm-export-stout.py
  - Minor docstring updates.

adm/adm-export-stout.py
  - Added command-line option '--noai'.  It prints auto-ionization levels,
    and any transitions involving them, after the end-of-data line in the
    respective files.
  - Now loading the experimental IP file before preparing the energy data
    lines, instead of while preparing the energy comments.
  - Modified .tp output to print TP values in exp notation, with an adaptive
    precision.
  - bugfix in add_tp_accuracy(): Attempt was made to append non-existent
    variable to list of accuracies.  Now using the correct local variable.
    Fixed.
  - Defined prep_coll_datalines() to prepare the datalines to print, and
    the ones to print as excluded, in the output .coll file.
  - Slightly modified the format of the command output.

adm/util/ui/cli.py
  - Added parse_add_noai() to enter option '--noai' to the CLI.

adm/util/idata/tp.py
  - bugfix in JoinedTransitions.new_dataline(): stout.tp.compose_line() was
    invoked with the TP value cast as a str().  This caused the printed string
    to appear as a plain float, not in exponential notation.  Now using the
    float TP value directly, which produces output in exponential notation.
    Fixed.

adm/util/idata/coll.py
  - Defined BasicCollision.new_dataline(), with variable precision, to produce
    a dataline.
  - Defined CollisionBlock.sorted_order() to access the output of the sort()
    method.

adm/util/stout/tp.py
  - Modified compose_line():
    - To allow None values be given for the precision.
    - To work with both str and float TP values.

adm/util/stout/coll.py
  - Modified compose_dataline() to permit control over the precision of the
    collision parameter values.

adm/util/stout/files.py
  - In FileSections():
    - Defined add_excluded_dataline() to add an 'excluded' line of data.
    - Defined excluded_datalines() to access these lines.
    - Modified prepare_file_lines() to print these lines after an end-of-data
      line.

adm/util/stout/__init__.py
  - Added missing module check.py.

adm/util/stout/info.py
  - Updated module docstring.

adm/test/stout/test_files.py
  - Added unit tests for new FileSections() methods.

adm/util/stout/nrg.py
adm/util/stout/fmt.py
  - Moved get_energy_fmt_precision() to fmt.py as get_fmt_precision_floats().

adm/util/stout/tp.py
adm/util/stout/fmt.py
  - Moved get_tp_fmt_precision() to fmt.py as get_fmt_precision_exps().
    - Adapted to process lists of floats, as well.
    - Introduced get_fmt_precision_exp() to handle the processing of a single
      float value.

adm/test/stout/test_nrg.py
adm/test/stout/test_fmt.py
  - Moved tests for get_energy_fmt_precision() to test_fmt.py, using the new
    function name.

adm/test/stout/test_tp.py
adm/test/stout/test_fmt.py
  - Moved tests for get_tp_fmt_precision() to test_fmt.py, using the new
    function name.
  - Added unit tests for get_fmt_precision_exp().

adm/adm-fine-tune.py
  - Fixed typos in branching ratios sub-menu.

adm/adm-visualize.py
  - Added CLI option (--nick) for the nickname of a complete dataset.
    If a numerical label is present in the nickname, it is also applied in the
    selection of a (merged) theoretical dataset.

adm/util/register/viz.py
  - Output file names are now prepended by a string.

adm/adm-visualize.py
adm/adm-merge-tp.py
  - bugfix: Missing import of util.types.  Fixed.

Same nickname for merged JSON energy datasets in input and output directories.

adm/util/io/nicks.py
  - Removed nickname 'matched'.

adm/adm-merge-nrg.py
adm/adm-merge-tp.py
adm/adm-visualize.py
adm/adm-fine-tune.py
  - Now using nickname 'merged' instead of 'matched'.

adm/adm-ingest-data.py
  - bugfix: Old Energy() method name copy_level() used instead of new name
    copy().  Fixed.
  - Added space on log output after creation of energies_original.json.

adm/adas-fix-fnames.py
  - Modified to operate with a path, instead of with an ADM parameter file.

adm/adm-setup.py
adm/adm-fetch-nist.py
  - Reordered CLI options.

adm/adm-adas-to-stout.py
  - Added clarification in help message.

adm/util/nist/extract.py
  - Guard against creating a JSON file, when only the IP uncertainty is
    available.
  - In write_nrg() and write_nrg_json(), guard against not extracting
    ionization potential values.

adm/util/nist/files.py
  - Use fs.write_file() in save_original_data(), instead of custom code.

adm/docs/manual/algorithm.tex
  - Updated to current flow of operations.

adm/docs/manual/adm-macros.tex
  - Added definitions for missing scripts.

Updated flowcharts (removed "Fix filenames" box, and adjusted size):
  - adm/docs/manual/images/adm-flowchart-adas-acquisition.pdf
  - adm/docs/manual/images/adm-flowchart-nist-acquisition.pdf

Added new flowcharts:
  - adm/docs/manual/images/adm-flowchart-export-stout-json.pdf
  - adm/docs/manual/images/adm-flowchart-vertical-sections-setup-collision-json.pdf

adm/nist-extract-species.py
  - Import missing modules, not ported with the move from nist.extract.py.

adm/util/nist/extract.py
  - Enabled extraction of level energy uncertainties.
  - Storage of 'original' energy dataset as a JSON file now depends on requests
    for uncertainty values, and for creating a JSON file.  If the uncertainty
    column is not present in the HTML table output, no JSON file is produced.
  - Now sanitizing reported paths to newly created files.
  - Modified drive() to accept the directory for data storage, instead of
    evaluating it based on the given species.

adm/adm-fetch-nist.py
  - Added CLI option to fetch the level energy uncertainties, if available.
  - Now using the relative basepath including the top directory
    (e.g., 'nist/li/li_1/li_1') when calling the driver function to extract
    data from NIST.  No need to move into the top experimental directory
    anymore (that was a residual of running NistExtractor.py as a subprocess).
  - Also, now requesting to save 'original' JSON datasets, if fetching the
    uncertainties has also been requested, and the uncertainty column is there.

adm/util/nist/extract.py
adm/nist-extract-species.py
  - Moved path preparation function from util.stout.nist.extract.
  - Moved statements processing the output of prepare_path() from
    extract.drive() to the main program.

adm/util/idata/nrg.py
  - In BasicEnergy():
    - Added optional data field for energy uncertainty.
    - Updated init() to process the energy uncertainty and store it as float.
    - Defined method energy_unc() to access stored value.

adm/util/nist/nrg.py
  - In EnergyLevel(), added data field for energy uncertainty.

adm/util/nist/files.py
  - Now sanitizing reported paths to newly created files.

adm/test/nist/test_nrg.py
adm/test/idata/test_nrg.py
  - Updated unit tests to exercise energy uncertainty data fields.

adm/util/nist/extract.py
  - Converted all positional arguments, except the one for the species, to
    keyword arguments.

adm/adm-fetch-nist.py
  - Prepended all variables that store relevant command-line options with
    'fetch_'.
  - Eliminated function run().  Its contents (two statements) are now part of
    main().
  - Updated script docstring.

adm/adm-fetch-nist.py
adm/nist-extract-species.py
  - Updated call to nist.extract_drive().

adm/util/nist/extract.py
  - Defined write_nrg_json() to produce an 'original' energies JSON dataset.
    The dataset is converted into an internal Energies() dataset from the
    existing list of EnergyLevel()'s.

adm/util/idata/nrg.py
  - In BasicEnergyMeta():
    - Turned dataline into an optional data field.  It may now be undefined
      on calls to init().
  - Defined Energy.set() to synthesize an Energy() object from BasicEnergy()
    and BasicEnergyMeta() objects.

adm/util/types/base.py
  - In Base.export_dict(), do not write out keys with None values.

adm/util/nist/nrg.py
  - In EnergyLevel.init(): Changed attribute from note to notes, which is
    actually used in set().

adm/util/nist/extract.py
  - bugfix in write_nrg(): Use of unset EnergyLevel.note instead of .notes.
    Fixed.
  - List of notes now accumulated during the preparation of datalines, because
    a comprehension does not accommodate the list of strings stored in notes.

adm/test/nist/test_nrg.py
  - Updated unit tests to use EnergyLevel.notes instead of .note.
  - Added unit test to exercise list of two notes.

adm/adm-ingest-data.py
adm/util/stout/files.py
  - Removal of degenerate terms from the (J,p) groups is no longer done as part
    of reading a Stout file.  It is now done by the data-ingestion script, after
    the original energies have been written to a JSON file.

Created subpackage adm/util/chianti
containing only:
  - adm/util/chianti/chianti.py (moved from adm/util/chianti.py)

Created testing subpackage adm/test/chianti
containing only:
  - adm/test/chianti/test_chianti.py (moved from adm/test/test_chianti..py)

Created subpackage adm/test/io
containing:
  - adm/util/io/fs.py (moved from adm/util/fs.py)
  - adm/util/io/json.py (moved from adm/util/json.py)
  - adm/util/io/log.py (moved from adm/util/ui/log.py)
  - adm/util/io/nicks.py (moved from adm/util/ui/nicks.py)
  - adm/util/io/__init__.py (for 'import *')

Created testing subpackage adm/test/io
containing:
  - adm/test/io/test_fs.py (moved from adm/test/test_fs.py)
  - adm/test/io/test_json.py (moved from adm/test/test_json.py)
  - adm/test/io/test_log.py (moved from adm/test/ui/test_log.py)
  - adm/test/io/test_nicks.py (moved from adm/test/ui/test_nicks.py)

Updated function calls.

Broke apart adm/util/io/ui.py into:
  - adm/util/io/cli.py (command-line interface)
  - adm/util/io/upr.py (user prompt)

Renamed:
  - adm/test/io/test_ui.py -> adm/test/io/test_upr.py
  - adm/test/io/test_ui.py -> adm/test/io/test_upr.py

adm/adas-extract-species.py
adm/adas-fix-fnames.py
adm/adm-adas-to-stout.py
adm/adm-export-json.py
adm/adm-export-stout.py
adm/adm-fetch-nist.py
adm/adm-finalize.py
adm/adm-fine-tune.py
adm/adm-ingest-data.py
adm/adm-merge-coll.py
adm/adm-merge-nrg.py
adm/adm-merge-tp.py
adm/adm-query-adas.py
adm/adm-register-levels.py
adm/adm-register-nocfg.py
adm/adm-register-terms.py
adm/adm-registration-summary.py
adm/adm-setup.py
adm/adm-shift-unreg-terms.py
adm/adm-sort-nrg.py
adm/adm-visualize.py
adm/stout-data-summary.py
adm/util/adas/extract.py
adm/util/cloudy/lines.py
adm/util/cloudy/stout_refs.py
adm/util/io/__init__.py
adm/util/nist/tp.py
adm/util/register/levels.py
adm/util/register/shifts.py
adm/util/register/terms.py
fetch-NIST-IP.py
  - Updated calls to functions.

adm/util/fs.py
adm/util/misc.py
adm/util/json.py
  - Moved the JSON() class from util.fs into new module util.json as module
    functions.  Also moved JSON encoder class myEncoder() from util.misc. 

adm/util/__init__.py
  - Added util.json to list modules loaded with 'import *'.

adm/test/test_fs.py
adm/test/test_json.py
  - Extracted unit tests into new module test.test_json.

adm/open-adas-download.py
adm/util/cloudy/stout_refs.py
adm/util/idata/coll.py
adm/util/nist/ip.py
adm/util/nist/refs.py
adm/util/stout/meta.py
  - Updated calls to json functions.

Removed missing files:
  - adm/test/test_chtypes.py
  - adm/test/test_types.py

adm/util/cloudy/lines.py
adm/util/idata/state.py
adm/util/io/dirs.py
adm/util/io/pfile.py
adm/util/nist/nrg.py
adm/util/nist/tp.py
adm/util/register/viz.py
adm/util/stout/files.py
adm/util/stout/info.py
adm/util/stout/meta.py
adm/test/cloudy/test_lines.py
  - Switched deriving from chtypes.Valid() to base.Base().

adm/util/chtypes.py
  - Removed class Valid().

adm/test/stout/test_coll.py
  - Updated to r14017.

adm/util/nist/extract.py
  - Now assigning level indices after the levels have been sorted in energy.

adm/util/nist/nrg.py
  - In EnergyLevel():
    - Removed the level index from the interface of set().
    - Defined method set_index() to set the level index.

adm/util/nist/db.py
  - Added module containing parameter for NIST database name.

adm/util/nist/__init__.py
  - Included new module to list of modules loaded with 'import *'.

adm/util/nist/extract.py
adm/util/nist/files.py
adm/util/register/shifts.py
adm/nist-asd-ip.py
fetch-NIST-IP.py
  - Updated to use new parameter instead of bare string.

adm/util/nist/extract.py
  - In process_nrg_table():
    - bugfix: Notes on IP value no longer embedded in the energy value.  Now
      using parser method instead.  Fixed.
    - Extracting the IP or level refcodes now done only if requested.
  - In write_nrg() and write_tp():
    - Now using the refactored code in stout.nrg.get_energy_fmt_precision()
      and stout.tp.get_tp_fmt_precision() to get the energy/TP precision for
      the output file.
    - Now using the stout.files.FileSections() infrastructure for preparing
      the contents of the output file.
  - Now using stout.dirtree functions for the output Stout filenames, instead
    of the intermediate functions in nist.files.

adm/util/nist/files.py
  - Removed wrapper functions:
    - get_energy_filename()
    - get_transition_filename()
    - get_collisions_filename()

adm/util/nist/refs.py
  - Modified title of Stout file (sub)section from "References" to
    "Bibliography":
    - References.show()
    - References.prep_print_pub()

Removed calls to stout.fmt.compose_comment_line(), as it is handled by
stout.files.FileSections.prepare_file_lines():
    - adm.util.nist.notes.prepare_table()
    - adm.util.nist.tp.prepare_acc_table()
    - adm.util.nist.ip.IonPot.prep_for_stout()
    - adm.util.nist.ip.IonPots.prep_for_stout()
    - adm.util.nist.refs.References.prep_print_pub()

fix bug in unitary test

adm/open-adas-download.py
  - Do not download already existing files.

adm/adm-query-adas.py
adm/util/adas/web.py
  - Refactored function fetch_file_maybe() into module.

adm/open-adas-download.py
  - Added script to download the entire OpenADAS database.
    Apart from the data, for each species save a JSON file with the retrieved
    HTML table.

adm/util/fs.py
  - Modified JSON.save() to accept data in the form of a list of dicts,
    in addition to a dict.

adf042stout.py
NistExtractor/NistExtractor.py
  - Restore Matt's scripts to their versions on the trunk.

adm/adm-adas-extract.py
adm/util/adas/extract.py
  - Added Matt Lykin's adf042stout.py as an ADM script, and refactored its
    main functionality as a util.adas module.

adm/adm-adas-to-stout.py
  - Updated to use the new util.adas module, instead of running adf042stout.py
    as a subprocess.

adm/util/adas/__init__.py
  - Updated to include the new extract.py module to the list of modules for
    'import *'.

adm/util/nist/__init__.py
  - Added extract.py to the list of modules for 'import *'.

adm/util/atoms/splevel.py
  - In is_spec_level(), also accept missing and uncertain terms, in addition to
    unassigned terms (which equal the energy), as valid terms.
    This is the conjugate for writing blank terms with nist-extract.
  - In spec_level_nist_fmt(), allowed for blank terms.

adm/util/atoms/spname.py
  - In fmt_spname_output_noJ(), allowed for blank terms.
  - Removed leading and trailing spaces from the output of fmt_spname_output()
    and fmt_spname_output_noJ().

adm/test/atoms/test_splevel.py
adm/test/atoms/test_spname.py
  - Updated unit tests.  Added a few more to exercise levels with blank terms.

adm/util/atoms/jtot.py
  - Implemented the identification and processing of J values that are uncertain
    (1 or 2), or unresolved (1,2), or both (1, 2 or 3).
    'Mixed' J values can be used with any of the module functions.

adm/util/atoms/spterm.py
  - Updated are_terms_equiv() to be able to compare mixed terms.

adm/util/atoms/splevel.py
  - Updated module functions to be able to handle mixed J values:
    - are_term_Jtot_consistent()
    - convert_spec_level_to_real()
    - convert_spec_level_to_frac()

adm/util/atoms/spname.py
  - Updated parse_spname_into_ecl() to be able to handle mixed J values.

adm/test/atoms/test_jtot.py
adm/test/atoms/test_spterm.py
adm/test/atoms/test_splevel.py
adm/test/atoms/test_spname.py
  - Exercised new functions and new capabilities in unit tests.

adm/util/atoms/spterm.py
  - Renamed is_term_mixed() to the more appropriate is_term_missing().
  - Defined function is_term_mixed() to identify mixed terms of the form '1,3G'
    or '2P,4P'.
  - Defined function resolve_mixed_term() to convert a mixed term into a list of
    (LS) terms.
  - Updated is_term() to handle mixed terms.

adm/test/atoms/test_spterm.py
  - Updated function name in tests of is_term_mixed() to is_term_missing().
  - Added unit tests for new functions is_term_mixed() and resolve_mixed_term().
  - Exercised is_term() in unit tests with mixed terms.
  - Promoted inner functions of unit_tests() to module functions.

adm/util/atoms/orbitals.py
  - Expanded is_nprim() to be able to handle str input.  If the string can be
    converted to an positive integer with no fractional part, it is considered
    to be a valid principal quantum number.

adm/util/atoms/econfig.py
  - Expanded are_econfig_equiv(), econfig_has_ancestor(), and fix_econfig()
    to handle configurations that are principal quantum numbers; unassigned
    (e.g., '1234d'); or mixed (e.g., '2,3.sp').

adm/test/atoms/test_orbitals.py
adm/test/atoms/test_econfig.py
  - Added unit tests for the new capabilities of the modified functions.

adm/util/atoms/shells.py
  - Minor format change to error message.

adm/util/atoms/splevel.py
  - Removed unnecessary import of orbitals module.

ion_solver:
retourning to the population solver

iso_condensed:
	adapted to the population representation using bool variable lgpop
	obtained real ionization energies to get rid of negative source terms
	simplified indexing on Sourcebound and Sinkbound

iso_level:
	retaured iso_level with condensation

adm/util/nist/nrg.py
adm/test/nist/test_nrg.py
  - bugfix in level_parse_line(): extract_notes() now returns a list.
    Updated calls.  Fixed.

adm/util/nist/phtml.py
  - Modified extract_notes() to not bail after the first notes have been
    retrieved.

adm/util/atoms/econfig.py
  - Missed in previous commit.

adm/util/adas/adas.py
adm/util/adas/eissner.py
  - Update to using fmt_econfig() in econfig.py

adm/util/nist/phtml.py
  - In EnergyParser.parse(), if an ionization potential data line is read for a
    bare nucleus, use a single space, as its configuration.

adm/nist-asd-ip.py
  - Drop resetting the ion above configuration to 3 spaces, now that it is
    guaranteed to not be the empty string.

adm/test/nist/test_phtml.py
  - Exercised new functionality in unit test.

adm/test/nist/test_web.py
  - Updated column parameter names to current in phtml.py.

adm/util/nist/phtml.py
  - Modified extract_note() to return list of note codes, not a concatenated
    string.  Renamed to extract_notes().

adm/util/nist/nrg.py
  - Updated call to function.

adm/util/nist/notes.py
  - Updated docstring of parse_note().

adm/test/nist/test_phtml.py
  - Updated unit tests.

adm/util/nist/files.py
  - bugfix: Missing module re.  Now imported.  Fixed.

adm/util/io/log.py
  - In error(), removed the empty line before the error message.

adm/util/fs.py
  - In chdir(), removed the empty line after the message.

minor fixes

adm/util/nist/ip.py
  - In IonPots(), defined methods __len__() and __getitem__().

adm/test/nist/test_ip.py
  - Added unit tests for new methods.
  - Converted inner functions to module functions.

util/nist/refs.py
  - bugfix in fetch_ref_data(): Some authors field do not have the expected
    number of HTML anchors, and that caused a failure.  Now adapting to the
    number of returned fields.  Fixed.
  - bugfix in References.__contains__(): Code could use an uninitialized object.
    Now guarding against an empty reference dict.  Fixed.
  - bugfix in copy_ref(): Not copying a Reference() object if it already exists
    in the rederence dict prevented populating records with data from the master
    list of references.

test/nist/test_refs.py
  - Added a couple of units tests for is_ref_commented(), parse_commented_ref().

adm/util/nist/refs.py
  - In References():  The practice of telling save_json() and read_json() if the
    dataset is for more than one species (master references) was fragile.
    To use a master references dataset consistently with reading and writing
    JSON files, method set_master() has been defined. 
    - The is_master argument has been removed from save_json() and read_json().
    - It has been retained in pathname(), because as a class method it has no
      access to the object data.
    - Class-wide parameters have been defined to the JSON filenames for master
      and regular datasets.

adm/test/nist/test_refs.py
  - Updated unit tests.

corrected factor 2 in iso_sequences departure coefficients and populations derivation, removed calculation of departures from populations

adm/util/nist/ip.py
  - In IonPot, added optional field for notes on the IP value, and method note()
    to access it.  Updated other methods as needed.
  - In IonPots.add_IP(), added optional keyword for IP notes.

adm/test/nist/test_ip.py
  - Exercised new functionality in unit tests for both IonPot() and IonPots().

adm/util/fs.py
  - Changed default indentation in JSON files from 4 spaces to 2.

adm/util/nist/phtml.py
  - Renamed: skip_line() -> can_skip_line()

fetch-NIST-IP.py
NistExtractor/NistExtractor.py
adm/test/nist/test_phtml.py
adm/test/nist/test_web.py
  - Updated function calls.

adm/util/nist/web.py
  - Added options to set_NIST_lines_query() to more finely control which columns
    to request.
  - Modified query_NIST_levels_form() and query_NIST_lines_form() to report the
    exception error, and give a more informative error message before exiting,
    should the data download fail.

adm/util/misc.py
  - In download_page():
    - bugfix: Angstrom and micron symbols cannot be converted into UTF-8, and
      throw exceptions that cause the code to abort.  Unconvertable characters
      are now replaced with a unicode box.  Fixed.
    - Updated to throw more meaningful messages on Exceptions raised when
      getting data from a remote server, and when decoding data into UTF-8.

adm/test/nist/test_web.py
  - Exercised new options to query_NIST_lines() in unit tests.

adm/util/nist/extract.py
  - Read HTML tables from files, if present.
    Otherwise, do as before: download the data, and store them, if asked.

adm/util/nist/ip.py
adm/util/nist/notes.py
adm/util/nist/refs.py
  - In table output of notes, ionization potentials, and references, show title
    only if there are any data to be shown.

adf042stout.py
  - Added ADM path to list of paths, to enable running stand-alone.
  - In write_end_of_file(), now using stout.fmt.compose_comment_line() to
    format the comment line.

adm/adm-query-adas.py
adm/util/adas/adas.py
  - Moved definition of the 'comments_file' to the module to make it
    more widely accessible.

1) Changed name of the function from iso_departure to iso_departure_solver to
avoid ambiguity;

2) Finished coding iso_departure_solver implemented all checks and normalizations. Wrote header on iso.h

3) Checked and corrected Boltzmann factors implementation in iso_departure_solver

4) iso_solve would call now to iso_departure_solver

adm/util/nist/ip.py
  - Defined method IonPot.prep_for_stout(), as a wrapper around
    util.stout.fmt.prep_for_stout().
  - Defined method IonPots.prep_for_stout() to iterate over the IonPot data.
    Interface includes an option to only print data for the first IP entry.

adm/test/nist/test_ip.py
  - Exercised new methods in unit tests.

adm/util/nist/ip.py
  - In IonPot(), restored optional keywords, eliminated in r13947.

adm/util/stout/fmt.py
  - Refactored code to handle the writing of comment lines.
    - Defined use_comment_sentinel() to control starting a comment line with a
      sentinel.
    - Defined compose_comment_line() to write comment lines.  Now used by 
      prep_comments_header() and prep_IP_line().
  - compose_dataline() now strips whitespace from the output line.

adm/test/stout/test_fmt.py
  - Exercised new functions in unit tests.
  - bugfix: By rearranging in r13948, data needed by get_fields_from_dataline()
    were moved after it.  Fixed.

NistExtractor/NistExtractor.py
  - Now processing all ionization potential records, instead of reading just
    the first one.  Only the first one is printed in the .nrg file, but the
    entire dataset is saved in a JSON file (IP.json) in the output directory.

adm/util/nist/ip.py
  - Switched to using relative imports.
  - In IonPot():
    - Eliminated optional keywords.  Now they are all required.
    - Allowing for empty NIST refcodes, in case they are not present, or they
      are not picked up.
    - In export_dict(), omiting fields with empty or None values.
  - bugfix in IonPots.__next__(): Typo in data structure name over which the
    method iterates.  Fixed.

adm/util/base.py
  - Switched to using relative imports.
  - In Base.pprt(): Now skipping fields of value None.

adm/util/nist/ip.py
  - Added module to handle ionization potential values more efficiently.
    Currently, the module defines two classes.

adm/test/nist/test_ip.py
  - Exercised the new classes in unit tests.

adm/adm-export-json.py
  - Added utility to export a lean JSON dataset from the 'final' dataset.
    The output files use the Stout filenames as their base, e.g., ne_9.nrg.json.

adm/util/base.py
  - Added function to remove empty entries from a dictionary.

adm/util/idata/nrg.py
  - Defined LeanEnergyMeta.export_dict() to remove empty fields from the
    dictionary produced by the parent method.
  - Updated LeanEnergy.copy():
    - It now instructs the parent method to not copy optional keywords.
    - It now overwrites the state() object with the primary name of the level.
  - Defined LeanEnergy.export_dict() to store the metadata as a separate dict,
    so that they easily distinguishable in the output JSON record.
  - In LeanEnergies(), overrode methods save_json() and read_json() to enable
    saving under a different naming scheme.

adm/util/idata/tp.py
  - Added argument to BasicTransition.copy() to skip copying optional keywords.
  - bugfix in LeanTranMeta.copy(): The keywords (required and optional) of
    BasicTranMeta were copied, not of LeanTranMeta.  Changed variables.  Fixed.
  - Defined LeanEnergyMeta.export_dict() to remove empty fields from the
    dictionary produced by the parent method.
  - Updated LeanTransition.copy() to not copy the optional keywords in the call
    to the parent method.
  - Defined LeanTransition.export_dict() to store the metadata as a separate
    dict, so that they easily distinguishable in the output JSON record.
  - In LeanEnergies(), overrode methods save_json() and read_json() to enable
    saving under a different naming scheme.

adm/util/idata/coll.py
  - Defined LeanCollMeta.export_dict() to remove empty fields from the
    dictionary produced by the parent method.
  - Defined LeanCollision.export_dict() to store the metadata as a separate
    dict, so that they easily distinguishable in the output JSON record.
  - In LeanCollisions(), overrode methods save_json() and read_json() to enable
    saving under a different naming scheme.

adm/test/idata/test_nrg.py
  - Defined facility __compare_Energy_with_Lean() to compare Energy() and
    LeanEnergy() objects.  Used where needed, in lieu of repeating comparison
    statements.

adm/test/idata/test_tp.py
  - In test of LeanTranMeta.copy(), removed check on refcode, which is not
    stored with lean metadata.

adm/util/idata/coll.py
  - Partition Collision() into three classes from which it now derives:
    - BasicCollision(), which holds the main collision data.
    - BasicCollMeta(), which holds metadata.
    - BasicCollProc(), which holds processing info, if any.
    - Defined methods pprt(), copy(), __eq__(), export_dict(), and
      import_dict() on each of them, as well as on Collision(), which calls
      these methods.
  - Defined method unset_replaced(), now part of BasicCollProc() along with
    the set_replaced() and replaced().
  - CollisionBlock() and Collisions() now derive from base.Base.
  - bugfix in CollisionBlock.add_collision_record(): Missed replacing method
    Collision.init_dataline() with .init() in r13940.  Fixed.

adm/test/idata/test_coll.py
  - Exercised new classes in unit tests.

adm/util/base.py
  - In Base.pprt(), changed format to separate variable name from value with
    ':\t', instead of '\t'.

adm/util/idata/nrg.py
  - Minor variable name changes.

adm/util/idata/tp.py
  - Defined classes:
    - LeanTranMeta(), a lightweight version of BasicTranMeta(). 
    - LeanEnergy(), a composite of BasicEnergy() & LeanTranMeta(). 
    - LeanEnergies(), a lightweight version of Energies(), with data stored as
      LeanEnergy() records.

adm/test/idata/test_tp.py
  - Exercised new classes in unit tests.

adm/util/idata/nrg.py
  - In BasicEnergyMeta.dataline(), guarded return value with a check that the
    attribute exists.  If not, an empty string is returned.
    This is needed because LeanEnergyMeta inherits the method, and not storing
    the attribute, the call might lead to a crash.
  - bugfix in LeanEnergy.copy(): The input type validated to be BasicEnergy()
    and then used to populate a BasicEnergyMeta() object.  Now validating for
    an Energy() object, which can be used with both.  Fixed.

coded matrix inversion and conversion of eigenvector to extended array

filled matrix z and condensed into matrix c

adm/util/base.py
  - Added base class from which other can derive.  It contains the classmethods
    of Valid (valiate() and is_obj()), and methods for pprt(), copy(), __eq__(),
    import_dict(), and export_dict(), with code refactored from methods in
    idata.nrg.

adm/util/idata/nrg.py
  - The following classes now derive from Base(), instead of Valid():
    - BasicEnergy()
    - BasicMeta()
    - RegisterEnergy()
    - ConfTerm()
    - ConfTermSort()
    - ConfTerms()
    - Groups()
    - Energies()
  - Rewrote methods to use their Base() equivalents in classes:
    - BasicEnergy()
    - BasicMeta()
    - RegisterEnergy()
    - ConfTerm()
    - ConfTermSort()
  - In BasicEnergy.__init__() removed obsolete keyword argument 'meta'.

adm/util/__init__.py
  - Added 'base' module to the list of modules for 'import *'.

adm/util/idata/nrg.py
  - Refactored Energy() to inherit from three separate classes that partition
    it into distinct logical units:
    - BasicEnergy() encapsulates the fundamental data for energy level.
    - BasicMeta() holds metadata on the level, such as its original database
      name, any comments given on the Stout energy record, etc.
    - RegisterEnergy() holds only data relevant to registration.
    - Energy() methods have been partitioned between these three classes.
      A few notes:
      - Method name changes:
        - reset_energy() -> set_energy().  Now in BasicEnergy().
          A new reset_energy() has been introduced in BasicEnergy() that
          restores the level energy to its original value.
        - set_comment() -> reset_comment().  Now in BasicMeta().
        - dismissed() -> set_dismissed().  Now in RegisterEnergy().
        - toggle_ignore() -> toggle_ignored().  Now in RegisterEnergy().
      - Energy() overrides RegisterEnergy.unset_shifted(), which calls the new
        BasicEnergy.reset_energy() as well as invoking the RegisterEnergy()
        homonymous method.
      - Each class defines its own pprt(), copy(), __eq__(), export_dict() and
        import_dict().  The Energy() homonymous methods call all of these
      - The old BasicEnergy() attributes, which derived from the even older
        LeanEnergy() class have been removed.
  - In Energies.copy(), removed the keyword arguments 'lean' and 'meta'.
    The method now longer produces 'lean' copies of the data.

adm/util/idata/state.py
  - Removed explicit tab characters ('\t') from pprt() methods.
    Now using log.tab_right() and log.tab_left() to control indentation.

adm/test/idata/test_nrg.py
  - Exercised functionality of the new classes (BasicEnergy(), BasicMeta(),
    and RegisterEnergy()) in unit tests.
  - Removed tests on lean energy records and datasets.

adm/adm-ingest-data.py
adm/test/idata/test_nrg.py
  - Updated calls to new method name: Energy.set_dismissed().

adm/util/register/levels.py
adm/test/idata/test_nrg.py
  - Updated calls to new method name: Energy.toggle_ignored().

adm/adm-fine-tune.py
adm/util/register/terms.py
adm/test/idata/test_nrg.py
  - Updated calls to new method name: Energy.set_energy()

adm/util/chtypes.py
  - Modified functions that validate collections (list, tuple, etc) to use the
    relevant bool functions from the types.py module.
  - Defined method Valid.is_obj(). 

adm/test/test_chtypes.py
  - Added unit tests for validate_collection().

adm/util/types.py
adm/test/test_types.py
  - Renamed is_obj() -> is_obj_of().

adm/util/chtypes.py
adm/util/types.py
  - Added change log.
  - Removed unnecessary imports.

adm/util/types.py
  - Added Return type in docstrings of functions:
    - is_int()
    - is_int_pos()
    - is_int_notneg()
    - is_float()
    - is_float_pos()

adm/util/idata/nrg.py
  - Renamed class LeanEnergy() to BasicEnergy().
    - Renamed method ingest() to set().

adm/util/idata/nrg.py
adm/test/idata/test_nrg.py
  - Update calls to the new class and method names.

1) Iniciated a new file iso_departure.cpp where the matrix will be solved for departures.
   a) created source and sink terms from recombination and CX
   b) this file is not linked to the rest of the code yet and is under
      construction

2) Removed the divition by LTE populations as this will not give us the correct matrix. it need to be solved using departure coefficients

The populations are not sensitive enough to adapt Brocklehurst 1971 method for
departures, it could work however using a spline or other tipe of extrapolation if we are sure it works.

The best approach is to use departure coefficients as we know they need to asymptotically approach 1

adm/util/idata/unified.py
  - Modified set_connectedness() to admit a keyword argument that prints a title
    announcing the operation to take place.  Two empty lines are added at the
    end of the operation.

adm/adm-ingest-data.py
  - Removed the two empty lines after the call to set_connectedness().