Is there a way to have other models added to the Mimi Registry or have them treated as packages (without being listed in your registry)? I currently have a case where multiple teams are using NICE (which builds off of RICE). There’s a second model in the works that builds off of NICE that multiple teams will also use.
So I’m going to end up with a case where I call RICE as a package in NICE… but for the new model NICE will have to be a git sub-module. Obviously not the end of the world, but my sense is this might be confusing to the non-coding members of the teams who struggle with multi-step processes to download the model code.
I’d think we could just move forward with registering NICE as a package, I’ll bring it up in our team tag up tomorrow morning and get back to you asap! The idea certainly is that more will be added over time, and I don’t think we’re imposing strict rules on them … the owner is responsible for their correctness etc.
We just discussed it and would be happy to just add NICE to our registry. We can handle the registry side, and you’ll just need to add a few files to your package. If you give @davidanthoff read/write access to NICE on Github and ping him, he’ll help you add anything it is missing. Then we can do the work for you on the registry side.
Note 1: The machinery to automatically update new versions of your package when you tag them is in flux, thus when you add your MimiNICE package, add MimiNICE will add the tagged version that David manually adds for you. That said, since you’ll be developing NICE one can just do addMimiNICE#master to add the version on the master branch, or use #tagname for a tag if you use those.
Note 2: You probably aren’t working with Julia environments much yet, since you haven’t been developing a package. Once the package is registered let me know and I’ll come by your desk and show you, it’s a simple one-time action but might as well be in person.
Thanks, this registry seems like an awesome feature! Of course I have a couple questions…
(1) Are there longer term plans to have some approach for people to register Mimi models that doesn’t involve giving David read/write access and requiring him to add some files?
(2) Is it possible for me to just mimic your registry on my end and add the necessary files myself? Then people could just do:
pkg> registry add https://github.com/frank_models.
pkg> add NICE
I’m trying to avoid cases where (i) I have a mixture of git submodules and Mimi model “packages” and (ii) David has to formally go and edit every single different model that I code up.
(3) Are you trying to standardize the format of the models that get included as packages as well as the functions that are exported? For instance I have a bunch of optimization functions I use for NICE. Would the NICE package just export the construct_nice() function, and users can figure out optimization on their own?
Always happy to answer questions!
(1) Yes, there are these long term plans. There are two primary tasks (1) adding files (2) adding the package to the registry. The former is pretty simple, and I plan to write a short tutorial explaining what must be done so that read/write access is not necessary … for this one we just wanted to keep your life simple 
. The latter will always need to be done by the owner of the registry, but adding the files and then requesting to be added is something we can have users do without our involvement in the future.
(2) In this case you would be managing your own personal registry, which wouldn’t be connected to the Mimi registry. That seems doable, if you wanted, but would require going through the steps to establish a registry which @davidanthoff could advise you on.
The described situation certainly makes sense for you to want to avoid, and would make our use of the registry less effective! I think we can establish a workflow where you would follow instructions on what files to add to your package, and then formally request to be added to the registry … and after that request we guarantee adding it within X days.
(3) You are welcome to export whatever functions you want, and we suggest putting instructions in the README or docs. At minimum it would be good to have a construct_nice() type function, but if you want to export more that’s totally up to you, you would just put that in the MimiNICE.jl file i.e.
export construct_nice(), optimize_nice() ...
The rule we’ve tried to follow is only putting functions In the export list that we don’t plan to break … but again it’s your package so it’s up to you!
My expectation is that the whole registering of new versions will be completely automatic in a few weeks, be quite simple and wont involve any manual work by me or anyone else. They are switching the general julia registry (i.e. where you get all your normal packages from) over to a new set of tools for registering new versions right now, and that stuff should also work for our own registry. Right now we should just give them a few weeks to switch over the general registry, and then we can start to use those new tools for our own registry as well.
I think I would generally advice against creating your own registry, it is kind of involved and the goal would really be to have the central MimiRegistry work for everyone. One thing we could do is just give folks like @FrankErrickson push access to MimiRegistry until the automatic tools are up and running, then he wouldn’t have to wait for anyone. But be warned, manually handling the registry is a bit of a pain
There is of course another way to distribute these packages, that skips registries entirely. You can just do ]add https://github.com/anthofflab/MimiDICE2013.jl and it will add the master version of that package. If you add tags to your github repo, you can also do something like ]add https://github.com/anthofflab/MimiDICE2013.jl#v1.2.0 and that should work if there is a tag called v1.2.0 in the MimiDICE2013 repo. The only real downside of that is that a) it is a bit more typing, and b) you can’t have other packages depend on MimiDICE2013 in this case. But I think both situations might be ok for @FrankErrickson’s situation?
I second everything @lrennels wrote about exporting functions. In addition, we are also looking into how we can standardize some of these things a bit more across models, but I think that would probably be optional going forward if we come up with something good.
My expectation is that the whole registering of new versions will be completely automatic in a few weeks, be quite simple and wont involve any manual work by me or anyone else.
@davidanthoff @lrennels Thanks! This all sounds good to me and I think will solve most of my issues. I mostly wanted a sense of where this package registry was going, and was hoping to avoid ending up being forced to rely on a mix of sub-modules and Mimi model packages a few months down the line. It sounds like this won’t be an issue soon, and I think the registry would benefit from being in a central spot with lots of models (e.g. it seems unnecessary for me to have my own registry). I can wait a few weeks on any of this stuff, so no rush on my end.
In terms of standardizing these packages, I’d suggest every package having at least a construct_model() function. You can export any other functions you want to, but that would always guarantee that a user can just build and run the base version of the model using familiar syntax. If there’s a way to ensure that models were coded up in a consistent manner that’d be great, but might be asking for too much…