Mentions légales du service

Skip to content

Adding data_type to DataSpecs

BIGAUD Nathan requested to merge jax-dataset into develop

Done so far :

  • Adding a DataTypeField
  • Adding data_type to DataSpecs
  • Adding a data_type property to InMemoryDataset as well as an expose_data_class boolean to its __init__

Rationale:

  • Tried subclassing _InMemoryDataset, for Jax specfically, but this does not work conceptually:
    • Its not about a dataset in jax, its about a dataset used by a jax model that need to expose data_type
    • I could actually add support for jax numpy, but that is a different issue

Open questions:

  • I need the data.dtype to be unique for the intented use-case, but if I want to remain general that might not always be the case. So I've elected for now to force the dtype to be unique and raise value errors in DataTypeField.is_valid and in MemoryDataset.data_type.

Merge request reports