Improve error messages for errors in the config file
This is a first pass at improving error messages in config files as discussed in #16 (closed).
In general, error messages are simplified--it doesn't print the full schema, and it makes a stronger effort to narrow in on the most relevant error. For example, in @jcury's first example of forgetting to defined the learned_params
property, the error will now read just:
error in config "/path/to/config.yaml": 'learned_params' is a required property
Furthermore, it adds a mechanism whereby we can provide custom error messages directly in the schema. The exact uses for this are something I need to explore further. But for example if the configuration for a parameter is invalid, we can provide a custom error message explaining how to configure a parameter, maybe even link to the documentation: See !49 (comment 425446) for an example.
This is handled directly in the schema, so there is no need to catch specific ValidationErrors anywhere a config file is loaded and try to handle them manually.
This is still a WIP though. I know it still has some problems, e.g. it does not work properly when handling oneOf/allOf/anyOf, though I think I know how to fix it... Update: The latest version fixes this issue and others; it's more complicated in a way, but more robust.
This makes some other improvements, again with a focused on learned_params
. For example this will return a more
useful error message now:
>>> Config({'param': {'type': 'classification'}}, schema='param-set', validate=True)
Before:
jsonschema.exceptions.ValidationError: 'regression was expected'
...big dump of schema...
After:
ConfigError: error in config at 'param': 'classes' is a required property
This could also serve as a starting point to improving #27.