pisa.scripts package

Submodules

pisa.scripts.add_flux_to_events_file module

Add neutrino fluxes (and neutrino weights(osc*flux*sim_weight) if needed) for each event.

pisa.scripts.add_flux_to_events_file.add_fluxes_to_file(data_file_path, flux_table, flux_name, outdir=None, label=None, overwrite=False)[source]

Add fluxes to PISA events file (e.g. for use by an mc stage)

Parameters:

  • data_file_path (string)

  • flux_table

  • flux_name

  • outdir (string or None) – If None, output is to the same directory as data_file_path

  • overwrite (bool, optional)

pisa.scripts.add_flux_to_events_file.main()[source]

Run add_fluxes_to_file function with arguments from command line

pisa.scripts.compare module

Compare two entities: Maps, map sets, pipelines, or distribution makers. One kind can be compared against another, so long as the resulting map(s) have equivalent names and binning. The result each entity specification is formatted into a MapSet and can be stored to disk so that e.g. re-running a DistributionMaker is unnecessary to reproduce the results.

pisa.scripts.compare.compare(outdir, ref, ref_label, test, test_label, asymm_max=None, asymm_min=None, combine=None, diff_max=None, diff_min=None, fract_diff_max=None, fract_diff_min=None, json=False, pdf=False, png=False, ref_abs=False, ref_param_selections=None, sum=None, test_abs=False, test_param_selections=None)[source]

Compare two entities. The result each entity specification is formatted into a MapSet and stored to disk, so that e.g. re-running a DistributionMaker is unnecessary to reproduce the results.

Parameters:

  • outdir (string) – Store output plots to this directory

  • ref (string or array of strings) – Pipeline settings config file that generates reference output, or a stored map or map set. Multiple pipelines, maps, or map sets are supported

  • ref_abs (bool) – Use the absolute value of the reference plot for comparisons

  • ref_label (string) – Label for reference

  • ref_param-selections (string) – Param selections to apply to ref pipeline config(s). Not applicable if ref specifies stored map or map sets

  • test (string or array of strings) – Pipeline settings config file that generates test output, or a stored map or map set. Multiple pipelines, maps, or map sets are supported

  • test_abs (bool) – Use the absolute value of the test plot for comparisons

  • test_label (string) – Label for test

  • test_param_selections (None or string) – Param selections to apply to test pipeline config(s). Not applicable if test specifies stored map or map sets

  • combine (None or string or array of strings) – Combine by wildcard string, where string globbing (a la command line) uses asterisk for any number of wildcard characters. Use single quotes such that asterisks do not get expanded by the shell. Multiple combine strings supported

  • sum (None or int) – Sum over (and hence remove) the specified axis or axes. I.e., project the map onto remaining (unspecified) axis or axes

  • json (bool) – Save output maps in compressed json (json.bz2) format

  • pdf (bool) – Save plots in PDF format. If neither this nor png is specified, no plots are produced

  • png (bool) – Save plots in PNG format. If neither this nor pdf is specfied, no plots are produced

  • diff_min (None or float) – Difference plot vmin; if you specify only one of diff_min or diff_max, symmetric limits are automatically used (min = -max)

  • diff_max (None or float) – Difference plot max; if you specify only one of diff_min or diff_max, symmetric limits are automatically used (min = -max)

  • fract_diff_min (None or float) – Fractional difference plot vmin; if you specify only one of fract_diff_min or fract_diff_max, symmetric limits are automatically used (min = -max)

  • fract_diff_max (None or float) – Fractional difference plot max; if you specify only one of fract_diff_min or fract_diff_max, symmetric limits are automatically used (min = -max)

  • asymm_min (None or float) – Asymmetry plot vmin; if you specify only one of asymm_min or asymm_max, symmetric limits are automatically used (min = -max)

  • asymm_max (None or float) – Fractional difference plot max; if you specify only one of asymm_min or asymm_max, symmetric limits are automatically used (min = -max)

Returns:

  • summary_stats (dict) – Dictionary containing a summary for each h Map processed

  • diff (MapSet) – MapSet of the difference - (Test - Ref)

  • fract_diff (MapSet) – MapSet of the fractional difference - (Test - Ref) / Ref

  • asymm (MapSet) – MapSet of the asymmetric fraction difference or pull - (Test - Ref) / sqrt(Ref)

pisa.scripts.compare.main()[source]

Function used from command-line calls (either python compare.py or via installer’s console scripts (see setup.py).

pisa.scripts.compare.parse_args()[source]

Parse command line arguments

pisa.scripts.convert_config_format module

Convert PISA config files from format used up until July 2017 to the new config file format.

pisa.scripts.convert_config_format.main()[source]

Do the conversion.

pisa.scripts.convert_config_format.parse_args(description='\nConvert PISA config files from format used up until July 2017 to the new config\nfile format.\n')[source]

Parse command line arguments

pisa.scripts.create_barr_sys_tables_mceq module

pisa.scripts.fit_hypersurfaces module

Fit a hypersurface to discrete systematics datasets – as specified by a fit config file – and produce a fit file usable by, e.g., the discr_sys.hypersurfaces service.

For more details, see pisa/utils/hypersurface.pu

A script for making plots from the fit results produced by this file can be found in the Fridge (private repository) at

fridge/analysis/common/scripts/plotting/plot_hypersurface_fits.py

See example fit config files in directory pisa_examples/resources/discr_sys/.

Config file syntax

“general” section

You must define a “general” section which must have at least options “sys_list” and “sys_func_list” and can optionally include options “units” and “combine_regex”. You can add more options to the “general” section, but they will be ignored by PISA unless they are explicitly referenced elsewhere in your config file. E.g., a “general” section with all three required and optional options (and no more)

[general]
sys_list = dom_eff, hole_ice, hole_ice_fwd
sys_func_list = linear, exponential, linear
units = dimensionless, dimensionless, dimensionless
combine_regex = ["nue.*_cc", "numu.*_cc", "nutau.*_cc", ".*_nc"]

If “units” is not specified, units default to “dimensionless” for all systematics. If specified, there must be the same number of comma-separated units strings (interpret-able by the Pint module) as there are options in the “sys_list”.

Note that “combine_regex” should be Python-evaulatable to a string or a sequence of strings. Old versions of this script allowed values like

combine_regex = nue.*_cc,numu.*_cc,nutau.*_cc,.*_nc

but unambiguously parsing such a single comma-separated string of one or more regexes (which themselves can contian commas) is probably impossible, so this syntax is deprecated (a warning is emitted if it is detected) and should be avoided.

“apply_to_all_sets” section

You can optionally include an “apply_to_all_sets” section in your config that, true to its name, defines options applied to all systematic sets sections in the file. E.g.,

[apply_to_all_sets]
pipeline_cfg = settings/pipeline/nutau_mc_baseline.cfg
remove [discr_sys.hypersurfaces] =
set [data.simple_data_loader] data_dict = {
    'true_energy': 'true_energy',
    'true_coszen': 'true_coszen',
    'reco_energy': 'reco_energy',
    'reco_coszen': 'reco_coszen',
    'pid': 'pid',
    'weighted_aeff': 'weighted_aeff',
    'dunkman_L5': 'dunkman_L5',
    }

The above sets “pipeline_cfg” for all discr sets, removes the “discr_sys.hypersurfaces” service, and redefines the “data_dict” in the “data.simple_data_loader” section. Any options defined in the discrete set sections of the config will start with this as their configuration. Details of the above syntax are described more below.

“nominal_set” and “sys_set” sections

All other required sections in the config file describe the discrete sets and must define a “pipeline_cfg” for that systematics set (whether explicitly in the section or via the “apply_to_all_sets” section). A systematics set section either starts with “nominal_set” (for one and only one set) or “sys_set” (for all remaining sets), followed by a colon and magnitudes of the values of each systematic specified in “general” section / “sys_list” option (in the same order as defined there). E.g., continuing the example config file defined in the above examples,

[nominal_set : 1.00 , 25 , 0.0]
set [data.simple_data_loader] events_file = /path/to/nominal_set.hdf5

[sys_set : 0.88 , 22 , 0.1]
set [data.simple_data_loader] events_file = /path/to/sys_set_1.hdf5

[sys_set : 1.12 , 28 , 0.2]
set [data.simple_data_loader] events_file = /path/to/sys_set_2.hdf5

and so forth. Note that all magnitudes must be specified in the same units specified in “general” section / “units” option, or if that option is not specified, all magnitudes must represent dimensionless quantities. Also note that the “pipeline_cfg” is already defined in the above “apply_to_all_sets” section, so each of these sys set sections simply swap out the events_file in that pipeline config file for the appropriate events file.

Syntax for modifying the specified “pipeline_cfg”

The following syntax is interpreted if specified in “apply_to_all_sets”, “sys_set”, and/or “nominal_set” sections. Note an “apply_to_all_sets” section must specify a “pipeline_cfg” for any of the following syntax to be used.

Define or redefine an option via the “set” keyword followed by the section in square brackets, the option, either equals (=) or colon (:), and finally the value to set the option to. E.g., “events_file” in config section “data.simple_data_loader” is set to “settings/pipeline/example.cfg” via

set [data.simple_data_loader] events_file = settings/pipeline/example.cfg

the section is created if it doesn’t exist and the option “events_file” is added to that section if it doesn’t already exist.

You can create a new section (if it doesn’t already exist) via the “set” keyword followed by the section in square brackets and either equals (=) or colon (:). Anything following the equals/colon is ignored. E.g. to add the “data.simple_data_loader” section (if it doesn’t already exist),

set [data.simple_data_loader] =

Notes on whitespace

Whitespace is ignored in section names and in lists, so the following are interpreted equivalently

[ sys_set : 0.88 , 22 , 0.1 ]
[sys_set:0.88,22,0.1]

Likewise, the following are all equivalent

sys_list = aa,bb,cc
sys_list = aa , bb , cc
pisa.scripts.fit_hypersurfaces.APPLY_ALL_SECTION_NAME = 'apply_to_all_sets'

section name that defines pipeline config / options for all discr sets

pisa.scripts.fit_hypersurfaces.COMBINE_REGEX_OPTION = 'combine_regex'

Option in general section to specify map(s) to combine before performing fit

pisa.scripts.fit_hypersurfaces.GENERAL_SECTION_NAME = 'general'

general section name

pisa.scripts.fit_hypersurfaces.REMOVE_OPTION_RE = re.compile('\\s*remove\\s*\\[(.*)\\]\\s*(\\S*.*)')

modifications to pipeline configs are specified by options following this pattern

pisa.scripts.fit_hypersurfaces.SET_OPTION_RE = re.compile('\\s*set\\s*\\[(.*)\\]\\s*(\\S*.*)')

defining a section and/or an option within a section in a pipeline configs is specified by following this pattern

pisa.scripts.fit_hypersurfaces.SYS_FUNC_LIST_OPTION = 'sys_func_list'

Option in general section to specify functional form of the discrete systematics

pisa.scripts.fit_hypersurfaces.SYS_LIST_OPTION = 'sys_list'

Option in general section to specify discrete systematics

pisa.scripts.fit_hypersurfaces.SYS_SET_OPTION = 'pipeline_cfg'

systematics set config file is specified by this option

pisa.scripts.fit_hypersurfaces.UNITS_OPTION = 'units'

Option in general section to specify units of discrete systematics

pisa.scripts.fit_hypersurfaces.UNITS_SPECIFIER = 'units.'

User is allowed to use e.g. <UNITS_SPECIFIER>meter or simply meter in UNITS_OPTION

pisa.scripts.fit_hypersurfaces.create_hypersurfaces(fit_cfg)[source]

Generate and store mapsets for different discrete systematics sets (with a single set characterised by a dedicated pipeline configuration)

Parameters:

fit_cfg (string) – Path to a fit config file

Returns:

hypersurfaces – Container with the fitted hypersurface for each map type

Return type:

OrderedDict

pisa.scripts.fit_hypersurfaces.load_and_modify_pipeline_cfg(fit_cfg, section)[source]

Load and modify the pipeline config file as specified in that section of the fit config.

Parameters:
Returns:

  • pipeline_cfg (pisa.utils.config_parser.PISAConfigParser) – pipeline config

  • pipeline_cfg_path (str) – path to the pipeline config as it is specified in the fit config

pisa.scripts.fit_hypersurfaces.main()[source]

Perform a hypersurface fit to discrete systematics sets.

pisa.scripts.fit_hypersurfaces.parse_args()[source]

Parse arguments from command line.

Returns:

args

Return type:

namespace

pisa.scripts.fit_hypersurfaces.parse_fit_config(fit_cfg)[source]

Perform sanity checks on and parse fit configuration file.

Parameters:

fit_cfg (str) – path to a fit configuration file

Returns:

  • fit_cfg (PISAConfigParser) – parsed fit configuration

  • sys_list (list of str) – parsed names of systematic parameters

  • sys_func_list (list of str) – parsed names of systematic parameter functional forms

  • units_list (list of str) – units corresponding to each discrete systematic

  • combine_regex (list of str) – each string is a regular expression for combining pipeline outputs; see pisa.core.map.MapSet.combine_regex() for details.

pisa.scripts.make_events_file module

Take simulated (and reconstructed) HDF5 file(s) (as converted from I3 by icecube.hdfwriter.I3HDFTableService) as input and writes out a simplified HDF5 file for use with PISA.

pisa.scripts.make_events_file.CMSQ_TO_MSQ = 0.0001

convert from centimeters^2 to meters^2

Type:

Conversion factor

pisa.scripts.make_events_file.main()[source]
pisa.scripts.make_events_file.makeEventsFile(data_files, detector, proc_ver, cut, outdir, run_settings=None, data_proc_params=None, join=None, cust_cuts=None, extract_fields=('true_energy', 'true_coszen', 'reco_energy', 'reco_coszen', 'one_weight', 'interaction_prob', 'pid'), output_fields=('true_energy', 'true_coszen', 'reco_energy', 'reco_coszen', 'weighted_aeff', 'pid'))[source]

Take the simulated and reconstructed HDF5 file(s) (as converted from I3 by icecube.hdfwriter.I3HDFTableService) as input and write out a simplified PISA-standard-format HDF5 file for use in aeff, reco, and/or PID stages.

Parameters:

  • data_files (dict) –

    File paths for finding data files for each run, formatted as:
    {

    <string run>: <list of file paths>, <string run>: <list of file paths>, … <string run>: <list of file paths>,

    }

  • detector (string) – Name of the detector (e.g. IceCube, DeepCore, PINGU, etc.) as found in e.g. mc_sim_run_settings.json and data_proc_params.json files.

  • proc_ver – Version of processing applied to the events, as found in e.g. data_proc_params.json.

  • cut – Name of a standard cut to use; must be specified in the relevant detector/processing version node of the data processing parameters (file from which the data_proc_params object was instantiated)

  • outdir – Directory path in which to store resulting files; will be generated if it does not already exist (including any parent directories that do not exist)

  • run_settings (string or MCSimRunSettings) – Resource location of mc_sim_run_settings.json or an MCSimRunSettings object instantiated therefrom.

  • data_proc_params (string or DataProcParams) – Resource location of data_proc_params.json or a DataProcParams object instantiated therefrom.

  • join –

    String specifying any flavor/interaction types (flavInts) to join together. Separate flavInts with commas (‘,’) and separate groups with semicolons (‘;’). E.g. an acceptable string is:

    ’numucc+numubarcc; nuall bar NC, nuall NC’

  • cust_cuts – dict with a single DataProcParams cut specification or list of same (see help for DataProcParams for detailed description of cut spec)

  • extract_fields (None or iterable of strings) – Field names to extract from source HDF5 file. If None, extract all fields.

  • output_fields (None or iterable of strings) – Fields to include in the generated PISA-standard-format events HDF5 file; note that if ‘weighted_aeff’ is not preent, effective area will not be computed. If None, all fields will be written.

Notes

Compute “weighted_aeff” field:

Within each int type (CC or NC), ngen should be added together; events recorded of that int type then get their one_weight divided by the total for that int type only to obtain the “weighted_aeff” for that event (even if int types are being grouped/joined together).

This has the effect that within a group, …

… and within an interaction type, effective area is a weighted average of that of the flavors being combined. E.g. for CC,

sum_{run x}sum_{flav y} (Aeff_{x,y} * ngen_{x,y})

Aeff_CC = —————————————————– ,

sum_{run x}sum_{flav y} (ngen_{x,y})

… and then across interaction types, the results of the above for each int type need to be summed together, i.e.:

Aeff_total = Aeff_CC + Aeff_NC

Note that each grouping of flavors is calculated with the above math completely independently from other flavor groupings specified.

See Justin Lanfranchi’s presentation on the PINGU Analysis call, 2015-10-21, for more details:

pisa.scripts.make_events_file.parse_args()[source]

Get command line arguments

pisa.scripts.make_events_file.powerLawIntegral(E0, E1, gamma)[source]

pisa.scripts.make_nufit_theta23_spline_priors module

Create splines to the NuFit delta-chi2 surfaces for theta23 and output them in a format that can be read by PISA to use as a prior on this parameter.

pisa.scripts.make_nufit_theta23_spline_priors.extract_vals(infile, string_of_interest)[source]
pisa.scripts.make_nufit_theta23_spline_priors.main()[source]

Create splines to the NuFit delta-chi2 surfaces for theta23 and output them in a format that can be read by PISA to use as a prior on this parameter.

pisa.scripts.make_nufit_theta23_spline_priors.make_prior_dict(f_io=None, f_no=None, f=None)[source]

pisa.scripts.test_flux_weights module

A set of tests on the flux weights calculated by PISA.

pisa.scripts.test_flux_weights.do_1d_2d_bartol_test(spline_dict, flux_dict, outdir, enpow=1)[source]
pisa.scripts.test_flux_weights.do_1d_2d_honda_test(spline_dict, flux_dict, legend_filename, save_name, outdir, enpow=1)[source]
pisa.scripts.test_flux_weights.do_1d_3d_honda_test(spline_dict, flux_dict, legend_filename, save_name, outdir, enpow=1)[source]
pisa.scripts.test_flux_weights.do_2d_2d_bartol_test(spline_dict, flux_dict, outdir, ip_checks, oversample, enpow=1)[source]
pisa.scripts.test_flux_weights.do_2d_2d_comparisons(honda_spline_dict, bartol_spline_dict, outdir, oversample, enpow=1)[source]
pisa.scripts.test_flux_weights.do_2d_2d_honda_test(spline_dict, flux_dict, outdir, ip_checks, oversample, save_name, title_filename, enpow=1)[source]
pisa.scripts.test_flux_weights.do_2d_3d_honda_test(spline_dict, flux_dict, outdir, oversample, save_name, title_filename, flav, flavtex, enpow=1)[source]
pisa.scripts.test_flux_weights.logplot(m, title, ax, clabel, cmap=<matplotlib.colors.LinearSegmentedColormap object>, logz=True, largelabels=False, medlabels=False)[source]

Simple plotting of a 2D histogram (map)

pisa.scripts.test_flux_weights.main()[source]
pisa.scripts.test_flux_weights.plot_1d_slices(xintvals, yintvals, xtabvals, ytabvals, xtabbins, xlabel, ylabel, xtext, ytext, text, table_name, save_name, log)[source]

Test function to show interpolation and tables overlaid in 1D slices. See main function for how to use this function.

Parameters:

  • xintvals (list) – A list of the x points where the spline was evaluated.

  • yintvals (list) – A list of the y points which the spline evaluated to.

  • xtabvals (list) – A list of the x points where the table is defined.

  • ytabvals (list) – A list of the y points where the table is defined.

  • xtabbins (list) – A list of the bin edges. Should have xtabvals as the bin centres.

  • xlabel (string) – A label for the x-axis of the plot.

  • ylabel (string) – A label for the y-axis of the plot.

  • xtext (float) – The position for the text label showing the slice along x.

  • ytext (float) – The position for the text label showing the slice along y.

  • text (string) – The text label showing the slice.

  • table_name (string) – The text label naming the tables used

  • save_name (string) – The place and name to save the plot.

  • log (bool) – A boolean to whether the axes should be made logarithmic. Will do both.

pisa.scripts.test_flux_weights.take_average(interp_map, oversampling)[source]

Module contents