Adding data_type to DataSpecs
Done so far :
-
Adding a DataTypeField
-
Adding data_type
toDataSpecs
-
Adding a data_type
property toInMemoryDataset
as well as anexpose_data_class
boolean to its__init__
Rationale:
- Tried subclassing _InMemoryDataset, for Jax specfically, but this does not work conceptually:
- Its not about a dataset in jax, its about a dataset used by a jax model that need to expose data_type
- I could actually add support for jax numpy, but that is a different issue
Open questions:
- I need the
data.dtype
to be unique for the intented use-case, but if I want to remain general that might not always be the case. So I've elected for now to force the dtype to be unique and raise value errors inDataTypeField.is_valid
and inMemoryDataset.data_type
.