Hi team,
I would like to use MimiFUND with other input scenarios than the default ones (original SSP + versions for zero migration that I developed this summer).
How should I proceed ? Can I set the scenpgrowth, scenypcgrowth etc. to csv files of my choosing, just like I do with parameters?
Thanks!
Hélène
Hi
I have a very similar questions. I would like to use different Emission Scenarios when calculating the SCC for FUND (as well as DICE and RICE?). I my case it would suffice if I could simply use different SRES Scenarios, but the newer ones (RCP+SSP) would of course be fine as well
Is it possible to add that feature to the functions that calculates SCC for the respective IAMs.
Thanks Quirin
Hi @hbenveniste and @Quirin thank you for the question! @davidanthoff, @ckingdon95, and I will discuss during the week and get back to you as soon as we can!
We will discuss the possibility of adding in SSP scenarios to the existing functionality in a user-friendly way, but in the mean time, what @hbenveniste suggested is correct. You can modify the scenpgrowth and scenypcgrowth parameter values by changing their CSVs for now.
@ckingdon95 can you take a look at this and give me your reactions/post? I can edit this one too if you have reactions. I’m not a FUND expert so I’m not sure which CSV files would be involved, or if there is a need to use the scenconverter component? I tried to include general workflow info here but you might have some FUND specifics … but I think @Quirin wants to do this in several models so it’s going to change a bit by model.
@Quirin sorry for the delay on details. This is long but I wanted to be thorough, you can skip to the sections that are most useful to you!
I believe the scenario you are in is that you want to alter SSP scenarios, which we have not explicitly built in as a method in Julia. Thus, you need to change the related parameters themselves. If you add an Issue to Github we can consider trying to add this to our API, but it probabl wouldn’t be done for a few weeks.
Tldr; There are two methods discussed and I’ll give details on both in this post:
Use the development tools in Julia to download the code base and change the original CSV files of interest. This may be the cleanest way to do things if you are fully replacing the files, since you can cleanly imitate the FUND organization and keep country labels, handle cases where a parameter is a distribution, etc. but it is a bit more software involved. If this is an ongoing project, we should talk about how you can set up your own repository and commit your changes to Github while working on FUND, but for now I’ll show you the quick and dirty way.
Continue to use Jupyter notebook and the public API, especially in simple cases like you mentioned (multiplying existing matrixes by a factor rather than replacing them with new ones, for example to raise/lower the individual regional growth rate of per capita consumption for each time period by 50%.)
1. Replacing CSV Files
To do this, you will use the development tools in Julia. You are currently working in a Julia default environment, which you may not know! If you enter Pkg mode by typing ], you will see something like
(v1.2) pkg>
which shows that you are in the v1.1 environment. We’ll just stay in your default for for now. You want to do some development on this package locally, so you are going to type
(v1.2) pkg> dev MimiFUND
This is an alternative to add MimiFUND, which you used before. Doing this will put a local copy of the MimiFUND package onto your machine at your equivalent of /Users/lisarennels/.julia/dev if you are on a mac, or something slightly different on a PC. We can easily help you find it if it’s hidden or anything!
Now when you check that status of your environment with st you’ll see list of installed packages including a development version of Mimi, something like
(v1.2) pkg> st # st is short for status
[336ed68f] CSV v0.5.13
[e4e893b0] Mimi v0.9.5-DEV [`~/.julia/dev/Mimi`]
...
From now on, when you type using MimiFUND your machine will look to this local repository. You can open that in any editor to look at and change the CSV files or just work with the file system to replace the CSV files. As @ckingdon95 mentions above, I believe the ones you want to change are at least scenpgrowth and `scenypcgrowth, @hbenveniste may know too it seems.
Once you replace those files, you can run your code as usual in this environment and it will pick up those changed CSV files!
If at any point you want to return to the standard version of MimiFUND, you simply use the free command:
(v1.2) pkg> free MimiFUND
This will bring you back to using our version, but will NOT delete your local copy so that will be saved don’t worry!
2. Using set_param!
It is certainly possible to do this through Jupyter notebook. I will first present an example that multiplies the values of a parameter of interest by 2, and then generalize. First carry out your usual code as follows:
using MimiFUND
using Mimi
m = MimiFUND.get_model() # or m = getfund()
run(m)
Now, you can now access the set values for parameter paramname in component compname with bracket indexing. You can then multiply those by two, call set_param!, and then re-run the model with your new parameters.
values = m[:compname, :paramname]
set_param!(m, :compname, :paramname, values * 2)
run(m)
More generally, you could use set_param! to replace the parameter values with any values matrix, as long as it is the correct dimensions etc.
Important: Although this is not a problem in FUND, there is an important thing to note here. If a parameter is reused in more than one component (ie. has the exact same name in more than one component) and you call set_param(m, :compname, :paramname, values), the parameter values will be changed not only in compname, but also in all other components where that parameter is connected. We are soon adding an explicit flag and warning for this, but I wanted to give a heads up.
Don’t hesitate to respond to this with any further questions or clarifications, or email me to schedule a call! You can copy and paste code or error messages here too if it’s not proprietary.
This all looks very good and thorough to me! The only thing I would add is that I would highly encourage you to take the second approach. This will allow you to continue to use Jupyter, and not have to worry about modifying the actual MimiFUND package.
Below I’ve included an example in more detail of what this approach of modifying the model could look like. I’ve used the update_param! function instead of the set_param! function Lisa mentioned, which in this case does the same thing:
using DelimitedFiles
using MimiFUND
# Write a function for getting your modified version of a MimiFUND model using
# files in a specified data directory
function get_quirins_ssp_model(datadir)
m = MimiFUND.get_model() # start with the regular FUND model
# change the population growth parameter
ssp_pop_growth = readdlm(joinpath(datadir, "pgrowth.csv"), ',')
update_param!(m, :pgrowth, ssp_pop_growth)
# change the gdp per capita growth parameter
ssp_ypc_growth = readdlm(joinpath(datadir, "ypcgrowth.csv"), ',')
update_param!(m, :ypcgrowth, ssp_ypc_growth)
# Change the energy intensity improvement parameter
ssp_aeei_growth = readdlm(joinpath(datadir, "aeei.csv"), ',')
update_param!(m, :aeei, ssp_aeei_growth)
# Change the carbon intensity improvement parameter
ssp_acei_growth = readdlm(joinpath(datadir, "acei.csv"), ',')
update_param!(m, :acei, ssp_acei_growth)
# change any other input parameters you need to
...
return m
end
# Get your modified model from an SSP3 data directory, then calculate the SCC
m_ssp3 = get_quirins_ssp_model("data/SSP3")
scc_ssp3 = MimiFUND.compute_scco2(m_ssp3, year=2020, ...)
# Get your modified model from an SSP5 data directory, then calculate the SCC
m_ssp5 = get_quirins_ssp_model("data/SSP5")
scc_ssp5 = MimiFUND.compute_scco2(m_ssp5, year=2020, ...)
Extra notes on this approach:
readdlm function from the DelimitedFiles package. The first argument to readdlm is the file path to the data file, and the second argument is the character ',', which specifies that it is a comma-delimited file.last_year argument to when computing the SCC: MimiFUND.compute_scco2(m_ssp3, year=2020, last_year=2100)
Hi @ckingdon95, @lrennels,
thanks for these thorough answers! I’m trying to not have to create the scenpgrowth, scenypcgrowth etc. files manually (the SSP files are in population/GDP levels, not growth), and use the scenconverter component embedded in FUND (as suggested by @davidanthoff). When trying to run the adapted model, I keep getting the following error message:
ERROR: MissingException: Cannot get index; data is missing. You may have tried to access a value that has not yet been computed.
Do you have a sense of what could trigger this?
Thanks a lot!
Ok, so I actually think there was errors in the scenconverter component: scenaeei and scenacei were not defined for the first time step. Also, I don’t think we should have a - sign in the lines defining those variables (it is the case currently).
Instead, I propose this. What do you think?
function run_timestep(p, v, d, t)
if is_first(t)
for r in d.regions
v.pop0[r] = p.population[t, r]
v.gdp0[r] = p.income[t, r]
v.energint0[r] = p.energuse[t,r] / p.income[t, r]
v.emissint0[r] = p.emission[t, r] / p.energuse[t, r]
v.energint[t, r] = v.energint0[r]
v.emissint[t, r] = v.emissint0[r]
end
end
for r in d.regions
if t.t < 1050
v.scenpgrowth[t, r] = (p.population[t+1, r] / p.population[t, r] - 1.) * 100.
v.scenypcgrowth[t, r] = (p.income[t+1, r] / p.income[t, r] / (1 + 0.01 * v.scenpgrowth[t, r]) - 1.) * 100.
else
v.scenpgrowth[t, r] = v.scenpgrowth[t - 1, r]
v.scenypcgrowth[t, r] = v.scenypcgrowth[t - 1, r]
end
if t.t < 1050
v.energint[t, r] = p.energuse[t, r] / p.income[t,r]
v.emissint[t, r] = p.emission[t, r] / p.energuse[t, r]
v.energint[t + 1, r] = p.energuse[t + 1, r] / p.income[t + 1,r]
v.emissint[t + 1, r] = p.emission[t + 1, r] / p.energuse[t + 1, r]
v.scenaeei[t, r] = (v.energint[t + 1, r] / v.energint[t, r] - 1.) * 100.
v.scenacei[t, r] = (v.emissint[t + 1, r] / v.emissint[t, r] - 1.) * 100.
else
v.scenaeei[t, r] = v.scenaeei[t - 1, r]
v.scenacei[t, r] = v.scenacei[t - 1, r]
end
end
end
Ah ok, @ckingdon95 might know more about this function but I’ll certainly take a look. I think you’ve found the problem though, the response I had posted was:
This Stacktrace refers to line 31 of your ScenarioUncertaintyComponent.jl file. The check that is happening is whether you are trying to do a calculation with a value that has not yet been calculated. For example if you had some line like
v.a[t] = 2 * p.b[t]
But the b parameter, perhaps connected from another component, is not yet set so it’s missing. Can you post the defcomp from that file here? I can also go look at it on Github.
I’m not very familiar with the scenario uncertainty component in FUND, so any questions about using it and modifying it should probably go through @davidanthoff!