I want to perform a GSA, but since my model is built on top of MimiFAIR, I have a custom setup for making Monte Carlo draws. In the end, I have vectors of all the parameter values and my outcomes of interest, but I do not have Distributions that I can apply to several of the parameters.
I see that GlobalSensitivityAnalysis.jl currently requires a SobolData, which takes distributions. I’m not too familiar with GSA, but shouldn’t the algorithm work directly from the Monte Carlo values? e.g. https://onlinelibrary.wiley.com/doi/full/10.1111/risa.12117
Hi @jrising sorry for the delay!
Sobol Analysis, at least in its implementation in the GlobalSensitivityAnalysis.jl package (based on Python’s SALib) assumes that you sample using the Sobol quasi-random sampling method (I depend on GitHub - JuliaMath/Sobol.jl: generation of Sobol low-discrepancy sequence (LDS) for the Julia language but have also implemented it myself). In addition, each GSA method in GlobalSensitivityAnalysis.jl, like Sobol Analysis, has (1) a sample function method and (2) an analyze function method, where analyze assumes certain characteristics about the sampled data it is passed. That’s the main reason that the requirement includes the Distributions, so that it can do the sampling within the call and select both (1) the shape and organization of the output array and (2) the sampling method.
When I’ve done this analysis for things like FaIR where I have presampled data, I create a Mimi.EmpiricalDistribution with uniform probabilities and pass that. If I have correlated parameter sets, I have abstracted away by using an Mimi.EmpiricalDistribution to select which set of calibrated to use, and then that just gets one set of indices.
On the latter I’ve recently been chatting with folks who have a fancier way of breaking apart parameters even when they belong to a calibrated set of samples, such that they each get their own index and I can explore sensitivity and interactions between, say, the parameters presampled for FaIR from AR6, but have not yet implemented that.
Let me know if that helps, would be happy to answer more questions if that didn’t hit what you need, or have a call since I’m starting to dive back into this as well. Another implementation of some of these methods in Julia is here: Sobol Method · GlobalSensitivity.jl though I haven’t explored the API etc. as much.
Thanks @lrennels, this is helpful. Since I do have correlated parameters, I started down your path but got concerned that it was imposing new assumptions that might not work as well with GSA. But I tried out GlobalSensitivity.jl: Global Sensitivity Analysis (GSA) · GlobalSensitivity.jl, and that seems to already have options for pre-sampled Monte Carlos. I’ve been able to get that working without much trouble.
Good to know! I may be in touch about that, as I’ve run into the same topic but haven’t quite had the time to update it within my own package.