Fiducia’s Documentation¶
Fiducia is an open source package for unfolding spectral information from filtered diode array diagnostics (such as Dante) using the cubic splines analysis method . This method simply assumes that the underlying spectrum is smoothly varying, and does not impose any other constraints on the shapes of spectrum. See below for instructions on how to install Fiducia, and for examples on how to run an analysis using Fiducia.
Installing Fiducia¶
Requirements¶
Fiducia require Python version 3.7 or newer. Fiducia also require the following openly available packages for installation:
Installation with pip¶
Official releases of Fiducia are published to pypi.org and can simply be pip installed like so:
pip install fiducia
Building and installing from source (for contributors)¶
Make sure you have python installed, preferably via Anaconda¶
Here is where you get Anaconda, and make sure to get the Python 3 version. https://www.anaconda.com/distribution/
Setup installation directory¶
Make a directory called “fiducia” in a sensible place on your system. Preferably in a directory where none of the higher level directory names have spaces in them.
Setup a virtual environment¶
If you have python installed via Anaconda, then create your virtual environment like this
conda create --name fiducia
Clone the repository using git¶
In the fiducia directory you created, run the following on the command line
git clone https://github.com/lanl/fiducia.git
Install requirements¶
pip install -r requirements.txt
Install fiducia¶
If you are a user then do
pip install .
If you wish to help in developing fiducia, then do
pip install -e .
Test if install was successful¶
Open a python and try doing import fiducia
. If all went well then you shouldn’t get any error messages.
Examples¶
General examples¶
General-purpose and introductory examples from Fiducia
Note
Click here to download the full example code
Test Example for Sphinx Docs¶
We create a test plot to see if sphinx gallery works
import numpy as np
import fiducia
import matplotlib.pyplot as plt
Generating some data
dataX = np.arange(10)
dataY = dataX ** 2
Plotting the data
plt.plot(dataX, dataY)
plt.xlabel('X')
plt.ylabel('Y')
plt.title('Example data')
plt.show()

Total running time of the script: ( 0 minutes 0.156 seconds)
How to Contribute¶
Visit our GitHub repository and look through the list of open issues to see how you can contribute. If you find a bug, or would like to see a feature enhancement, then open up an issue and describe it detail.
Acknowledging and Citing¶
If you use Fiducia for work/research presented in a publication (whether directly, or as a dependency to another package), we encourage the following acknowledgement:
This research made use of Fiducia, a community-developed Python package for analysis of filtered diode array signals.
and that you cite the following paper(s):
@article{barnak2020soft,
title={Soft x-ray spectrum unfold of K-edge filtered x-ray diode arrays using cubic splines},
author={Barnak, DH and Davies, JR and Knauer, JP and Kozlowski, Pawel Marek},
journal={Review of Scientific Instruments},
volume={91},
number={7},
pages={073102},
year={2020},
publisher={AIP Publishing LLC}
url={https://doi.org/10.1063/5.0002856}
}
Fiducia License (BSD 3-clause)¶
© 2020. Triad National Security, LLC. All rights reserved. This program was produced under U.S. Government contract 89233218CNA000001 for Los Alamos National Laboratory (LANL), which is operated by Triad National Security, LLC for the U.S. Department of Energy/National Nuclear Security Administration. All rights in the program are reserved by Triad National Security, LLC, and the U.S. Department of Energy/National Nuclear Security Administration. The Government is granted for itself and others acting on its behalf a nonexclusive, paid-up, irrevocable worldwide license in this material to reproduce, prepare derivative works, distribute copies to the public, perform publicly and display publicly, and to permit others to do so.
This program is open source under the BSD-3 License. Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
2.Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
3.Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Cubic Spline Matrices (fiducia.cspline)¶
Created on Fri Mar 8 09:41:36 2019
Functions for working with cubic spline equation in matrix form.
@author: Pawel M. Kozlowski
Functions¶
|
Convert photon energy value into normalized coordinates for a particular spline region. |
|
Given a normalized energy value and the bounds of a spline segment, return the un-normalized photon energy value. |
|
Returns the matrix M_y(t) for a given value of t in: |
|
Returns the matrix M_D(t) for a given value of t in: |
|
Construct matrix for converting from \(D_i\) to \(y_i\) vector. |
|
Given a DANTE detector response as a function of energy, convert the response to normalized photon energy, t, over a given spline segment, and return interpolated response values for a given value of t. |
|
This is the matrix corresponding to: |
|
This is the matrix corresponding to: |
|
Trap rule integration of the folding between our \(M_{y \chi}\) matrix and response function matrix, with respect to normalized photon energy, for each channel. |
|
Returns the bounds of each spline segment, given the spline knot points. |
|
Matrix representing the spectrally integrated folding of the detector response with a cubic spline interpolation of the x-ray spectrum. |
|
Get knot points \(y_i\) from measured DANTE signals \(S_d\). |
|
Reconstruct the inferred DANTE spectrum given the knot points \(y_i\) obtained from knotSolve(). |
Spline Uncertainty Propagation (fiducia.error)¶
Created on Tues June 16 13:48:21 2020
Utilities for calculating response uncertainty
@author: Myles Brophy
Functions¶
|
Monte Carlo simulation and statistics to determine cubic spline uncertainty. |
|
Modification of response.knotFind() |
|
Given a DANTE detector response as a function of energy, convert the response to normalized photon energy, t, over a given spline segment, and return interpolated response values for a given value of t. |
|
Calculate the variance when propogating uncertainties through |
|
Propagates uncertanity through |
|
Finds the cspline detector matrix, it`s inverse matrix and std matrix using Monte Carlo uncertainty propagation. |
Data Loading Utilities (fiducia.loader)¶
Created on Fri Mar 8 09:20:37 2019
Utilities for loading DANTE measurement and response function data.
@author: Pawel M. Kozlowski
Functions¶
|
Strip whitespace and rename DataFrame headers. |
|
Load DANTE measurement data from files given the channels and path to the directory containing the response function files. |
|
Load uncertainty percentages into a DataFrame. |
|
Loads DANTE measurement data from files given the channels and path to the directory containing the reduced and aligned DANTE data. |
|
Get DANTE signals from each channel at a particular time. |
|
Get time-integrated Dante signals for a specified time interval. |
|
Reads Dante .dat file and returns header info and channel signals as two separate pandas dataframes. |
Fiducia Main File (fiducia.main)¶
Created on Fri Jan 25 12:22:01 2019
FIDUCIA: Filtered Diode Unfolder (using) Cubic Spline Algorithm
DANTE spectrum deconvolver based on cubic splines method [1]. Translated from Dan Barnak’s Mathematica code.
DANTE channels are bounded by edge absorption feature (knot point) due to filter for the respective channel. Cubic splines representing the estimated spectrum are fitted in each spectral region bounded by knot points. The detector signal for each channel is then equal to the response function of the detector folded with the matrix representation of the cubic spline. A triadiagonal matrix representation of the cubic spline equation is used to make the problem numerically tractable. This way a matrix inversion can be used to solve for the unknown coefficients in the cubic spline equation, using the measured signals. These coefficients are then plugged back into the cubic spline equation over each interval (between knot points) to make a piecewise reconstruction of the x-ray spectrum at each time step.
References
Cubic spline deconvolution method [1] J. P. Knauer and N. C. Gindele. Temporal and spectral deconvolution of data from diamond, photoconductive devices. Rev. Sci. Instrum. 75, 3714 (2004) https://doi.org/10.1063/1.1785274
Error propagation for cubic spline deconvolution method [2] D. L. Fehl and F. Briggs. Verification of unfold error estimates in the unfold operator code. Rev. Sci. Instrum. 68, 890 (1997) https://doi.org/10.1063/1.1147713
Useful description of cubic spline matrix representation [3] http://mathworld.wolfram.com/CubicSpline.html
Paper comparing cubic splines unfolds to other methods [4] D. H. Barnak, J. R. Davies, J. P. Knauer, and P. M. Kozlowski. Soft x-ray spectrum unfold of K-edge filtered x-ray diode arrays using cubic splines. Submitted to Review of Scientific Instruments in 2020.
@author: Pawel M. Kozlowski
Functions¶
Takes the inferred spectrum and folds it with the instrument function to retrieve the forward propagated signal for each DANTE channel. |
|
|
Gets the inferred radiation temperature by calculating in from radiated power through the Stefan-Boltzmann Law. |
|
Gets the inferred total radiation power as a function of time. |
|
Given the response function file and the DANTE measurement data file, run cubic spline analysis to reconstruct spectrum for a given time. |
|
Given the response function file and the DANTE measurement data file, run cubic spline analysis to reconstruct spectrum for a given time. |
|
Attempt processing dante signals given dante data file and calibration files using sensible defaults. |
Miscellaneous Functions (fiducia.misc)¶
Created on Fri Mar 8 09:25:05 2019
Miscellaneous utilities
@author: Pawel M. Kozlowski
Functions¶
|
Find nearest value in array and return index, and value as a tuple. |
|
Check DataFrame compatibility for specified channels. |
Plot Defaults (fiducia.pltDefaults)¶
Created on Fri Oct 27 02:37:12 2017
Default plotting parameters
@author: Pawel M. Kozlowski
Functions¶
|
Generate a line plot with shaded region representing y-error bars. |
|
Generate a scatter plot with y-error bars. |
Raw Dante Data Processing (fiducia.rawProcess)¶
Created on Wed Mar 13 16:43:39 2019
- Utilities for processing raw DANTE data. Typical steps include:
attenuator correction
background shot subtraction
channel alignment (via e.g. peak finding)
temporal axis calibration
@author: Pawel M. Kozlowski
Functions¶
|
Given a header frame, return a list of channels with no scope. |
|
Given a header frame, return a list of channels with no XRD. |
|
Given a header frame, return a list of which dante channels were on for the shot. |
|
Given a headerFrame, returns a timesFrame containing an array of oscilloscope times for each channel and background shot in the headerFrame. |
|
Scales voltage (vertical) axis of dante signals based on information contained in the header. |
|
Give a Dante data frame containing measurement data and background shot data, remove the background from the data and return the corrected data as a dataframe. |
|
Reads given offset correction file (.xls) and applies offsets to dante measurement data given in dataframe. |
|
Given a header frame, return the attenuation factors applied to each channel. |
|
Given a Dante data frame and header frame, return a data frame with attenuation corrections applied to each channel. |
|
Calculates time averaged background for given data. |
|
Applies background correction to bring the signal down to zero, based on averaging the signal background over a section of time from earliest time contained in timesFrame to earliest time plus timeLength. |
|
Fit polynomial function to ends of the signal as an estimate of the background signal + hyesteresis. |
|
Determines locations and widths of peaks above the mean of the signal for each dante channel. |
|
|
|
Find the highest peak, and return list of peaks with the highest peak removed from the list. |
|
Select the N tallest peaks. |
|
|
|
|
|
Takes out put timesFrame and dataFrame from rawProcess.py functions and generates a measurementFrame that can be passed to analyzeStreak() and other main.py functions. |
|
Given a dante data file, an attenuators file, and an offsets file, reads the file and applies background correction, attenuation correction, and channel offset correction. |
|
Corrects for hysteresis by detecting edges of signal containing region and fitting a polynomial background to regions that do not belong to signal. |
|
Aligns dante signals based on peak finding. |
Dante Response Functions (fiducia.response)¶
Created on Fri Mar 8 09:23:02 2019
Utilities for working with DANTE response functions (e.g. plotting, locating edges).
@author: Pawel M. Kozlowski
Uncertainty Propagation for Common Operations (fiducia.stats)¶
Created on Tue Jul 21 12:11:12 2020
Common statistic operations
@author: Myles T. Brophy
Functions¶
|
Propagates variance (\(\sigma^2\)) through Simpson’s rule numerical integration. |
|
Error propogation for Trapezoidal rule integration using uniform or non-uniform grids. |
|
Propogates uncertainty for the gradient operator of an array of a given step size. |
|
Propogate uncertainty for the dot product of matrix a and 1D vector b. |
|
Propagate uncertainty for linear interpolation. |
Derivations for propagating uncertainty for some common operations.
Weighted Summation¶
Gien a vector \(X_i\) with N elements
A simplified version of this can be written as
Trap Rule Variance¶
For \(N\) steps of \(\Delta x_{k} = x_{k+1} - x_{k}\) where \(f(x_k) = y_k\), where each \(y_k\) is independent, an integral can be approximated as
To find the variance in the general case, use the last 3 equations.
The two first \(\operatorname{Var}\) terms are simple
where \(\sigma_k\) is the uncertainty of \(y_k\). Now for the Covariance of the two summations
Combine all this and the fourth equation to get
In the simple case of a uniform grid, where all \(\Delta x_k = \Delta x\) this can be expanded to
Gradient Variance¶
From the numpy.grad documentation, the gradient for discrete step sizes is approximated by
with the gradient at the first and the last data point being
for a list of \(x_i\) data points and \(h_d = x_{i+1} - x_i = h_i\) and \(h_s = x_i - x_{i-1} = h_{i-1}\). From this we can say that \(f(x_i) = y_i\) and \(f(x_i + h_i) = y_{i+1}\) and \(f(x_i - h_s) = y_{i-1}\). Using the variance of weighted sums where the weights are the \(h\) terms we get
where \(\sigma_i\) corresponds to the uncertainty in \(y_i\). Keep in mind that there are \(N\) \(y\) values and \(N-1\) \(h\) values because \(h_i\) is the difference between the \(N\) data points. Note that unlike in the Trap rule integration, the covariant term is 0 because no repeated uncertainty terms appear in the sum. At the borders \(i =0,N\) the variance is
Dot Product Variance¶
The dot product of vectors \(X,Y\) is defined as
Uncertainty propagation when multiplying two variables, \(f(u,v) = a u v\), with \(a\) as a constant, is given by
For the dot product we assume there is no covariance between \(X\) and \(Y\) because they are independent. This gets us
operatorname{Var}(X cdot Y) = sum_{i=1}^N (x_i sigma_{y_i})^2 + (y_i sigma_{x_i})^2
Linear interpolation¶
Linear interpolation of a point \(x_0 < x < x_1\) is given by
First thing we can do, to make this calculation easier, is assume that there is no uncertainty in the \(x_i\) terms. This is a short cut, but it`s all that’s required for the application in which this linear interpolation is being implemented. Starting with the numerator, we can use the weighted summation rules to say
where there is no covariant term, because we assume \(y_0\) and \(y_1\) and independent. Then, including the denominator as a constant (because we assume all \(x\) values have no uncertainty, we get a variance of
Visualization Utilities (fiducia.visualization)¶
Created on Fri Mar 8 10:49:17 2019
Utilities for visualizing DANTE data.
@author: Pawel M. Kozlowski
Functions¶
|
Plots response function curves with knot locations identified as vertical dashed lines. |
|
Given a dataframe of Dante channel data, plot all the signal traces onto a single plot. |
|
Plot streak of unfolded Dante spectra. |
|
Visualize dante signals as an image. |