As I understand, the migrate parameter in FUND is a 16-by-16 region array. In MimiFUND, its distribution is defined in defmcs.jl as a 256-element vector instead. When performing a trial, the .csv output is also a 256-element vector. How should I turn this vector back into an array for use in the model? If I iterate through the .csv output 16 elements at a time to recreate the array, is each 16-element chunk a new column or a new row of the array?
Hi @AlexandrePavlov, good question! I also want to let you know, since it seems you are using the mcd code, that there are some coming API changes and bug fixes in the next tagged release of Mimi as described here: PSA Simulation API
Looking at defmcs.jl it looks like the migrate parameter random variable is defined with the code:
migrate = [["USA", "USA"] => Gamma(101.990195135928,0.00990195135927852), ["USA", "CAN"] => Gamma(1.0,0.05), ["USA", "WEU"] => Gamma(1.0,0.05), ["USA", "JPK"] => Gamma(1.0,0.05), ["USA", "ANZ"] => Gamma(1.0,0.05), ["USA", "EEU"] => Gamma(1.0,0.05), ["USA", "FSU"] => Gamma(1.0,0.05), ["USA", "MDE"] => Gamma(2.0,0.04), ["USA", "CAM"] => Gamma(2.0,0.1), ["USA", "LAM"] => Gamma(2.0,0.1), ["USA", "SAS"] => Gamma(2.0,0.04), ["USA", "SEA"] => Gamma(2.0,0.04), ["USA", "CHI"] => Gamma(2.0,0.04), ["USA", "MAF"] => Gamma(2.0,0.01), ["USA", "SSA"] => Gamma(2.0,0.03), ["USA", "SIS"] => Gamma(2.0,0.15), ["CAN", "USA"] => Gamma(1.0,0.05), ["CAN", "CAN"] => Gamma(101.990195135928,0.00990195135927852), ["CAN", "WEU"] => Gamma(1.0,0.02), ["CAN", "JPK"] => Gamma(1.0,0.02), ["CAN", "ANZ"] => Gamma(1.0,0.02), ["CAN", "EEU"] => Gamma(1.0,0.02), ["CAN", "FSU"] => Gamma(1.0,0.02), ["CAN", "MDE"] => Gamma(2.0,0.01), ["CAN", "CAM"] => Gamma(1.0,0.05), ["CAN", "LAM"] => Gamma(1.0,0.05), ["CAN", "SAS"] => Gamma(2.0,0.01), ["CAN", "SEA"] => Gamma(2.0,0.01), ["CAN", "CHI"] => Gamma(2.0,0.01), ["CAN", "MAF"] => Gamma(1.0,0.005), ["CAN", "SSA"] => Gamma(2.0,0.005), ["CAN", "SIS"] => Gamma(2.0,0.1), ["WEU", "USA"] => Gamma(1.0,0.02), ["WEU", "CAN"] => Gamma(1.0,0.02), ["WEU", "WEU"] => Gamma(101.990195135928,0.00990195135927852), ["WEU", "JPK"] => Gamma(1.0,0.02), ["WEU", "ANZ"] => Gamma(1.0,0.02), ["WEU", "EEU"] => Gamma(1.0,0.02), ["WEU", "FSU"] => Gamma(1.0,0.02), ["WEU", "MDE"] => Gamma(2.0,0.04), ["WEU", "CAM"] => Gamma(1.0,0.05), ["WEU", "LAM"] => Gamma(1.0,0.05), ["WEU", "SAS"] => Gamma(2.0,0.04), ["WEU", "SEA"] => Gamma(2.0,0.02), ["WEU", "CHI"] => Gamma(2.0,0.02), ["WEU", "MAF"] => Gamma(2.0,0.09), ["WEU", "SSA"] => Gamma(2.0,0.06), ["WEU", "SIS"] => Gamma(2.0,0.15), ["JPK", "USA"] => Gamma(1.0,0.01), ["JPK", "CAN"] => Gamma(1.0,0.01), ["JPK", "WEU"] => Gamma(1.0,0.01), ["JPK", "JPK"] => Gamma(101.990195135928,0.00990195135927852), ["JPK", "ANZ"] => Gamma(1.0,0.01), ["JPK", "EEU"] => Gamma(1.0,0.001), ["JPK", "FSU"] => Gamma(1.0,0.001), ["JPK", "MDE"] => Gamma(1.0,0.005), ["JPK", "CAM"] => Gamma(1.0,0.01), ["JPK", "LAM"] => Gamma(1.0,0.01), ["JPK", "SAS"] => Gamma(2.0,0.005), ["JPK", "SEA"] => Gamma(2.0,0.01), ["JPK", "CHI"] => Gamma(2.0,0.01), ["JPK", "MAF"] => Gamma(1.0,0.001), ["JPK", "SSA"] => Gamma(1.0,0.001), ["JPK", "SIS"] => Gamma(2.0,0.05), ["ANZ", "USA"] => Gamma(1.0,0.02), ["ANZ", "CAN"] => Gamma(1.0,0.02), ["ANZ", "WEU"] => Gamma(1.0,0.02), ["ANZ", "JPK"] => Gamma(1.0,0.02), ["ANZ", "ANZ"] => Gamma(101.990195135928,0.00990195135927852), ["ANZ", "EEU"] => Gamma(1.0,0.01), ["ANZ", "FSU"] => Gamma(1.0,0.01), ["ANZ", "MDE"] => Gamma(2.0,0.01), ["ANZ", "CAM"] => Gamma(1.0,0.01), ["ANZ", "LAM"] => Gamma(1.0,0.01), ["ANZ", "SAS"] => Gamma(2.0,0.005), ["ANZ", "SEA"] => Gamma(2.0,0.02), ["ANZ", "CHI"] => Gamma(2.0,0.02), ["ANZ", "MAF"] => Gamma(1.0,0.005), ["ANZ", "SSA"] => Gamma(2.0,0.005), ["ANZ", "SIS"] => Gamma(2.0,0.15), ["EEU", "USA"] => Gamma(1.0,0.001), ["EEU", "CAN"] => Gamma(1.0,0.001), ["EEU", "WEU"] => Gamma(1.0,0.001), ["EEU", "JPK"] => Gamma(1.0,0.001), ["EEU", "ANZ"] => Gamma(1.0,0.001), ["EEU", "EEU"] => Gamma(101.990195135928,0.00990195135927852), ["EEU", "FSU"] => Gamma(1.0,0.05), ["EEU", "MDE"] => Gamma(1.0,0.001), ["EEU", "CAM"] => Gamma(1.0,0.001), ["EEU", "LAM"] => Gamma(1.0,0.001), ["EEU", "SAS"] => Gamma(1.0,0.001), ["EEU", "SEA"] => Gamma(1.0,0.001), ["EEU", "CHI"] => Gamma(1.0,0.001), ["EEU", "MAF"] => Gamma(1.0,0.001), ["EEU", "SSA"] => Gamma(1.0,0.001), ["EEU", "SIS"] => Gamma(1.0,0.001), ["FSU", "USA"] => Gamma(1.0,0.001), ["FSU", "CAN"] => Gamma(1.0,0.001), ["FSU", "WEU"] => Gamma(1.0,0.001), ["FSU", "JPK"] => Gamma(1.0,0.001), ["FSU", "ANZ"] => Gamma(1.0,0.001), ["FSU", "EEU"] => Gamma(1.0,0.001), ["FSU", "FSU"] => Gamma(101.990195135928,0.00990195135927852), ["FSU", "MDE"] => Gamma(1.0,0.001), ["FSU", "CAM"] => Gamma(1.0,0.001), ["FSU", "LAM"] => Gamma(1.0,0.001), ["FSU", "SAS"] => Gamma(1.0,0.001), ["FSU", "SEA"] => Gamma(1.0,0.001), ["FSU", "CHI"] => Gamma(1.0,0.001), ["FSU", "MAF"] => Gamma(1.0,0.001), ["FSU", "SSA"] => Gamma(1.0,0.001), ["FSU", "SIS"] => Gamma(1.0,0.001), ["MDE", "USA"] => Gamma(1.0,0.001), ["MDE", "CAN"] => Gamma(1.0,0.001), ["MDE", "WEU"] => Gamma(1.0,0.001), ["MDE", "JPK"] => Gamma(1.0,0.001), ["MDE", "ANZ"] => Gamma(1.0,0.001), ["MDE", "EEU"] => Gamma(1.0,0.001), ["MDE", "FSU"] => Gamma(1.0,0.001), ["MDE", "MDE"] => Gamma(82.987950057818,0.0109772228646444), ["MDE", "CAM"] => Gamma(1.0,0.001), ["MDE", "LAM"] => Gamma(1.0,0.001), ["MDE", "SAS"] => Gamma(1.0,0.001), ["MDE", "SEA"] => Gamma(1.0,0.001), ["MDE", "CHI"] => Gamma(1.0,0.001), ["MDE", "MAF"] => Gamma(1.0,0.001), ["MDE", "SSA"] => Gamma(1.0,0.001), ["MDE", "SIS"] => Gamma(1.0,0.001), ["CAM", "USA"] => Gamma(1.0,0.001), ["CAM", "CAN"] => Gamma(1.0,0.001), ["CAM", "WEU"] => Gamma(1.0,0.001), ["CAM", "JPK"] => Gamma(1.0,0.001), ["CAM", "ANZ"] => Gamma(1.0,0.001), ["CAM", "EEU"] => Gamma(1.0,0.001), ["CAM", "FSU"] => Gamma(1.0,0.001), ["CAM", "MDE"] => Gamma(1.0,0.001), ["CAM", "CAM"] => Gamma(82.987950057818,0.0109772228646444), ["CAM", "LAM"] => Gamma(1.0,0.005), ["CAM", "SAS"] => Gamma(1.0,0.001), ["CAM", "SEA"] => Gamma(1.0,0.001), ["CAM", "CHI"] => Gamma(1.0,0.001), ["CAM", "MAF"] => Gamma(1.0,0.001), ["CAM", "SSA"] => Gamma(1.0,0.001), ["CAM", "SIS"] => Gamma(2.0,0.1), ["LAM", "USA"] => Gamma(1.0,0.001), ["LAM", "CAN"] => Gamma(1.0,0.001), ["LAM", "WEU"] => Gamma(1.0,0.001), ["LAM", "JPK"] => Gamma(1.0,0.001), ["LAM", "ANZ"] => Gamma(1.0,0.001), ["LAM", "EEU"] => Gamma(1.0,0.001), ["LAM", "FSU"] => Gamma(1.0,0.001), ["LAM", "MDE"] => Gamma(1.0,0.001), ["LAM", "CAM"] => Gamma(1.0,0.005), ["LAM", "LAM"] => Gamma(82.987950057818,0.0109772228646444), ["LAM", "SAS"] => Gamma(1.0,0.001), ["LAM", "SEA"] => Gamma(1.0,0.001), ["LAM", "CHI"] => Gamma(1.0,0.001), ["LAM", "MAF"] => Gamma(1.0,0.001), ["LAM", "SSA"] => Gamma(1.0,0.001), ["LAM", "SIS"] => Gamma(2.0,0.1), ["SAS", "USA"] => Gamma(1.0,0.001), ["SAS", "CAN"] => Gamma(1.0,0.001), ["SAS", "WEU"] => Gamma(1.0,0.001), ["SAS", "JPK"] => Gamma(1.0,0.001), ["SAS", "ANZ"] => Gamma(1.0,0.001), ["SAS", "EEU"] => Gamma(1.0,0.001), ["SAS", "FSU"] => Gamma(1.0,0.001), ["SAS", "MDE"] => Gamma(1.0,0.001), ["SAS", "CAM"] => Gamma(1.0,0.001), ["SAS", "LAM"] => Gamma(1.0,0.001), ["SAS", "SAS"] => Gamma(82.987950057818,0.0109772228646444), ["SAS", "SEA"] => Gamma(1.0,0.005), ["SAS", "CHI"] => Gamma(1.0,0.001), ["SAS", "MAF"] => Gamma(1.0,0.001), ["SAS", "SSA"] => Gamma(1.0,0.001), ["SAS", "SIS"] => Gamma(2.0,0.1), ["SEA", "USA"] => Gamma(1.0,0.001), ["SEA", "CAN"] => Gamma(1.0,0.001), ["SEA", "WEU"] => Gamma(1.0,0.001), ["SEA", "JPK"] => Gamma(1.0,0.001), ["SEA", "ANZ"] => Gamma(1.0,0.001), ["SEA", "EEU"] => Gamma(1.0,0.001), ["SEA", "FSU"] => Gamma(1.0,0.001), ["SEA", "MDE"] => Gamma(1.0,0.001), ["SEA", "CAM"] => Gamma(1.0,0.001), ["SEA", "LAM"] => Gamma(1.0,0.001), ["SEA", "SAS"] => Gamma(1.0,0.005), ["SEA", "SEA"] => Gamma(82.987950057818,0.0109772228646444), ["SEA", "CHI"] => Gamma(1.0,0.005), ["SEA", "MAF"] => Gamma(1.0,0.001), ["SEA", "SSA"] => Gamma(1.0,0.001), ["SEA", "SIS"] => Gamma(2.0,0.1), ["CHI", "USA"] => Gamma(1.0,0.001), ["CHI", "CAN"] => Gamma(1.0,0.001), ["CHI", "WEU"] => Gamma(1.0,0.001), ["CHI", "JPK"] => Gamma(1.0,0.001), ["CHI", "ANZ"] => Gamma(1.0,0.001), ["CHI", "EEU"] => Gamma(1.0,0.001), ["CHI", "FSU"] => Gamma(1.0,0.001), ["CHI", "MDE"] => Gamma(1.0,0.001), ["CHI", "CAM"] => Gamma(1.0,0.001), ["CHI", "LAM"] => Gamma(1.0,0.001), ["CHI", "SAS"] => Gamma(1.0,0.001), ["CHI", "SEA"] => Gamma(1.0,0.001), ["CHI", "CHI"] => Gamma(82.987950057818,0.0109772228646444), ["CHI", "MAF"] => Gamma(1.0,0.001), ["CHI", "SSA"] => Gamma(1.0,0.001), ["CHI", "SIS"] => Gamma(1.0,0.001), ["MAF", "USA"] => Gamma(1.0,0.001), ["MAF", "CAN"] => Gamma(1.0,0.001), ["MAF", "WEU"] => Gamma(1.0,0.001), ["MAF", "JPK"] => Gamma(1.0,0.001), ["MAF", "ANZ"] => Gamma(1.0,0.001), ["MAF", "EEU"] => Gamma(1.0,0.001), ["MAF", "FSU"] => Gamma(1.0,0.001), ["MAF", "MDE"] => Gamma(1.0,0.001), ["MAF", "CAM"] => Gamma(1.0,0.001), ["MAF", "LAM"] => Gamma(1.0,0.001), ["MAF", "SAS"] => Gamma(1.0,0.001), ["MAF", "SEA"] => Gamma(1.0,0.001), ["MAF", "CHI"] => Gamma(1.0,0.001), ["MAF", "MAF"] => Gamma(82.987950057818,0.0109772228646444), ["MAF", "SSA"] => Gamma(1.0,0.005), ["MAF", "SIS"] => Gamma(1.0,0.001), ["SSA", "USA"] => Gamma(1.0,0.001), ["SSA", "CAN"] => Gamma(1.0,0.001), ["SSA", "WEU"] => Gamma(1.0,0.001), ["SSA", "JPK"] => Gamma(1.0,0.001), ["SSA", "ANZ"] => Gamma(1.0,0.001), ["SSA", "EEU"] => Gamma(1.0,0.001), ["SSA", "FSU"] => Gamma(1.0,0.001), ["SSA", "MDE"] => Gamma(1.0,0.001), ["SSA", "CAM"] => Gamma(1.0,0.001), ["SSA", "LAM"] => Gamma(1.0,0.001), ["SSA", "SAS"] => Gamma(1.0,0.001), ["SSA", "SEA"] => Gamma(1.0,0.001), ["SSA", "CHI"] => Gamma(1.0,0.001), ["SSA", "MAF"] => Gamma(1.0,0.001), ["SSA", "SSA"] => Gamma(82.987950057818,0.0109772228646444), ["SSA", "SIS"] => Gamma(1.0,0.001), ["SIS", "USA"] => Gamma(1.0,0.001), ["SIS", "CAN"] => Gamma(1.0,0.001), ["SIS", "WEU"] => Gamma(1.0,0.001), ["SIS", "JPK"] => Gamma(1.0,0.001), ["SIS", "ANZ"] => Gamma(1.0,0.001), ["SIS", "EEU"] => Gamma(1.0,0.001), ["SIS", "FSU"] => Gamma(1.0,0.001), ["SIS", "MDE"] => Gamma(1.0,0.001), ["SIS", "CAM"] => Gamma(1.0,0.001), ["SIS", "LAM"] => Gamma(1.0,0.001), ["SIS", "SAS"] => Gamma(1.0,0.001), ["SIS", "SEA"] => Gamma(1.0,0.001), ["SIS", "CHI"] => Gamma(1.0,0.001), ["SIS", "MAF"] => Gamma(1.0,0.001), ["SIS", "SSA"] => Gamma(1.0,0.001), ["SIS", "SIS"] => Gamma(1.0,0.1)]
which produces a 256 element array, the first 16 elements being pertaining to ["USA", region]
["USA", "CAN", "WEU", "JPK", "ANZ", "EEU", "FSU", "MDE", "CAM", "LAM", "SAS", "SEA", "CHI", "MAF", "SSA", "SIS"]
You can see this more clearly in the data\migrate.csv file which holds this same information in .csv format.
After digging into the FUND code a bit, I found the key details. When a parameter’s .csv file has 3 columns, and thus 2 indices, such as migrate.csv, the following code from prepparameters! runs to format the raw input.
This shows that if you iterate through the .csv output 16 elements at a time to recreate the array, each each 16-element chunk a new ROW of the array.
elseif column_count == 3
length_index1 = length(unique(p[:,1]))
length_index2 = length(unique(p[:,2]))
new_p = Array{Float64}(undef, length_index1, length_index2)
cur_1 = 1
cur_2 = 1
for j in 1:size(p,1)
new_p[cur_1,cur_2] = getbestguess(convertparametervalue(p[j,3]))
cur_2 += 1
if cur_2 > length_index2
cur_2 = 1
cur_1 += 1
end
end
parameters[param_name] = new_p
end
I don’t know off the top of my head if it gets flattened column first or row first, but if it’s helpful instead: the 256 migrate distribution definitions are also available in long format in the file “MimiFUND/data/migrate.csv” which are labeled with their two regions
Thank you @lrennels @ckingdon95, this is exactly the answer I was looking for. I was going to treat each 16-element chunk as a new column of the array, so it is a good thing I asked to make sure.
No problem, yes I also expected them to be column-major but I was not personally involved in writing FUND, and this turned out to be a FUND specific as opposed to Mimi specific question.