key, as and conflict#
in a dict, it is easy to change a model by overwriting its entry such as:
import numpy as np
from pprint import pprint
import modeldag
# a default model with -10<a<0 and 100<b<110
initial_model = {"a": {"func": np.random.uniform, "kwargs": {"low": -10, "high":0}},
"b": {"func": np.random.uniform, "kwargs": {"low": 100, "high":110}},
}
# update the model by changing its a entry
update_model = {"a": {"func": np.random.uniform, "kwargs": {"low": 50, "high":60}}}
# such that the final model has "b" from the initial and "a" from the update:
model = initial_model | update_model
pprint(model)
{'a': {'func': <built-in method uniform of numpy.random.mtrand.RandomState object at 0x107872d40>,
'kwargs': {'high': 60, 'low': 50}},
'b': {'func': <built-in method uniform of numpy.random.mtrand.RandomState object at 0x107872d40>,
'kwargs': {'high': 110, 'low': 100}}}
but with the “as” options, this can but lost.
Fortunately, ModelDAG checks this while loading the model.
There is however several cases or interest
Single as=name case#
This is the simplest case, key of the updating dict does not match with that from the initial dict, but the as makes it so.
# a default model with -10<a<0 and 100<b<110
initial_model = {"a": {"func": np.random.uniform, "kwargs": {"low": -10, "high":0}},
"b": {"func": np.random.uniform, "kwargs": {"low": 100, "high":110}},
}
# update the model by changing its a entry
update_model = {"new_a": {"func": np.random.uniform,
"kwargs": {"low": 50, "high":60},
"as":"a" # this will lead to 'a' in the final dataframe, so effectively overwriting 'a'
}
}
# such that the final model has "b" from the initial and "a" from the update:
model = initial_model | update_model
pprint(model)
{'a': {'func': <built-in method uniform of numpy.random.mtrand.RandomState object at 0x107872d40>,
'kwargs': {'high': 0, 'low': -10}},
'b': {'func': <built-in method uniform of numpy.random.mtrand.RandomState object at 0x107872d40>,
'kwargs': {'high': 110, 'low': 100}},
'new_a': {'as': 'a',
'func': <built-in method uniform of numpy.random.mtrand.RandomState object at 0x107872d40>,
'kwargs': {'high': 60, 'low': 50}}}
There seem to be a conflict, but modeldag has tools that checks in ‘as’ to clean the input model
It knows here as “new_a” should be assumed as ‘a’ and therefore behaves as such
dag = modeldag.ModelDAG(model)
pprint(dag.model, sort_dicts=False)
{'b': {'func': <built-in method uniform of numpy.random.mtrand.RandomState object at 0x107872d40>,
'kwargs': {'low': 100, 'high': 110}},
'new_a': {'func': <built-in method uniform of numpy.random.mtrand.RandomState object at 0x107872d40>,
'kwargs': {'low': 50, 'high': 60},
'as': 'a'}}
dag.draw(2)
| b | a | |
|---|---|---|
| 0 | 106.309783 | 57.430408 |
| 1 | 102.372642 | 57.531347 |
as=list cases#
Things are most complex with as is used to specify that several entries as used. and again several cases exist
joined draw overwrites former keys#
def joined_draw(size, alpha=3., beta=1.5, **kwargs):
""" """
a = np.random.uniform(size=size, **kwargs)
b = a*alpha + beta
return a, b
# a default model with -10<a<0 and 100<b<110
initial_model = {"a": {"func": np.random.uniform, "kwargs": {"low": -10, "high":0}},
"b": {"func": np.random.uniform, "kwargs": {"low": 100, "high":110}},
}
# Now, new model makes that a and b are drawn simultaneously.
update_model = {"a_and_b": {"func": joined_draw,
"kwargs": {"low":0, "high":2},
"as": ["a", "b"]}
}
model = initial_model | update_model
pprint(model, sort_dicts=False)
{'a': {'func': <built-in method uniform of numpy.random.mtrand.RandomState object at 0x107872d40>,
'kwargs': {'low': -10, 'high': 0}},
'b': {'func': <built-in method uniform of numpy.random.mtrand.RandomState object at 0x107872d40>,
'kwargs': {'low': 100, 'high': 110}},
'a_and_b': {'func': <function joined_draw at 0x1345722a0>,
'kwargs': {'low': 0, 'high': 2},
'as': ['a', 'b']}}
In that case: the solution is simple, ModelDAG knows it has to overwrite both a and b with the new a_and_b
dag = modeldag.ModelDAG(model)
dag.model
{'a_and_b': {'func': <function __main__.joined_draw(size, alpha=3.0, beta=1.5, **kwargs)>,
'kwargs': {'low': 0, 'high': 2},
'as': ['a', 'b']}}
Say now that only b is updated by the joined draw, but now a
# Now, new model makes that a and b are drawn simultaneously.
update_model = {"b_and_c": {"func": joined_draw,
"kwargs": {"low":0, "high":2},
"as": ["b","c"]}
}
Same, it is easy for ModelDAG to know what to do:
ais left unchangedbis replaced by the b_and_c draw.
dag = modeldag.ModelDAG( initial_model | update_model )
dag.model
{'a': {'func': <function RandomState.uniform>,
'kwargs': {'low': -10, 'high': 0}},
'b_and_c': {'func': <function __main__.joined_draw(size, alpha=3.0, beta=1.5, **kwargs)>,
'kwargs': {'low': 0, 'high': 2},
'as': ['b', 'c']}}
new draw overwrites former as=list#
Say you and up with a model dict that has no obvious solutions, for instance
# a default model with -10<a<0 and 100<b<110
complex_model = {"a": {"func": np.random.uniform,
"kwargs": {"low": -10, "high":0}},
"a_and_b": {"func": joined_draw,
"kwargs": {"low":0, "high":2},
"as": ["a", "b"]},
"b": {"func": np.random.uniform, "kwargs": {"low": 100, "high":110}},
}
here a_and_b is expected to replace a. That is ok.
But after, b wants to replace existing b drawn as part of a_and_b. But then what about a ? Since both a and b are supposed to be drawn together, it does not make sense to just replace b.
In such a case, ModelDAG will raise a ValueError but default
dag = modeldag.ModelDAG( complex_model )
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In[67], line 1
----> 1 dag = modeldag.ModelDAG( complex_model )
File ~/miniforge3/envs/ztfdc/lib/python3.11/site-packages/modeldag/modeldag.py:69, in ModelDAG.__init__(self, model, obj, as_conflict)
47 def __init__(self, model={}, obj=None, as_conflict="raise"):
48 """
49
50 Parameters
(...)
67 instance
68 """
---> 69 self.set_model( model, as_conflict=as_conflict)
70 self.obj = obj
File ~/miniforge3/envs/ztfdc/lib/python3.11/site-packages/modeldag/modeldag.py:168, in ModelDAG.set_model(self, model, as_conflict)
166 """ sets the model to the instance (inplace) applying basic validation. """
167 from .tools import _get_valid_model_
--> 168 self._model = _get_valid_model_(model, as_conflict=as_conflict)
File ~/miniforge3/envs/ztfdc/lib/python3.11/site-packages/modeldag/tools.py:93, in _get_valid_model_(model, as_conflict)
88 value["kwargs"] = {}
90 #
91 # one of the new as or key overwrites known key or former as.
92 #
---> 93 key_to_pop = _as_to_key_to_pop_(key, past_as, conflict=as_conflict)
94 if key_to_pop is not None:
95 _ = out_model.pop(key_to_pop)
File ~/miniforge3/envs/ztfdc/lib/python3.11/site-packages/modeldag/tools.py:60, in _as_to_key_to_pop_(key_or_as, past_as, conflict)
58 # crashes
59 elif conflict == "raise":
---> 60 raise ValueError(f"new {key_or_as=} cannot replace that from {this_as['input_key']} ('as': {this_as['as_orig']})")
61 # ignores
62 elif conflict in ["warn", "skip"]:
ValueError: new key_or_as='b' cannot replace that from a_and_b ('as': ['a', 'b'])
You can however force it to accept this by specifying as_conflict=’warn’ or ‘skip’.
dag = modeldag.ModelDAG( complex_model, as_conflict="warn") # use skip to ignore the warning.
/Users/rigault/miniforge3/envs/ztfdc/lib/python3.11/site-packages/modeldag/tools.py:65: UserWarning: new key_or_as='b' cannot replace that from a_and_b ('as': this_as['as_orig']). This is skiped. Potentially leads to conflict.
warnings.warn(f"new {key_or_as=} cannot replace that from {this_as['input_key']} ('as': this_as['as_orig']). This is skiped. Potentially leads to conflict.")
dag.draw(3)
| a | b | |
|---|---|---|
| 0 | 1.651262 | 104.747112 |
| 1 | 0.028661 | 106.084687 |
| 2 | 1.160684 | 103.223104 |
In that case b will be overwritten and a unchanged.