citrine.resources.ingestion module

exception citrine.resources.ingestion.IngestionException(*, uid: UUID | None = None, errors: Iterable[IngestionErrorTrace])

Bases: CitrineException

[ALPHA] An exception that contains details of a failed ingestion.

classmethod from_api_error(source: ApiError) → IngestionException: [ALPHA] Build an IngestionException from an ApiError.

classmethod from_status(source: IngestionStatus) → IngestionException: [ALPHA] Build an IngestionException from an IngestionStatus.

with_traceback(): Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

args

errors = None: List[IngestionErrorTrace]

status = None

uid = None: Optional[UUID]

class citrine.resources.ingestion.FailedIngestion(errors: Iterable[IngestionErrorTrace])

Bases: Ingestion

[ALPHA] Object to fill in when building an ingest fails.

access_control_dict() → dict: Return an access control entity representation of this resource. Internal use only.

classmethod build(data: dict) → Self: Build an instance of this object from given data.

build_objects(*, build_table: bool = False, delete_dataset_contents: bool = False, delete_templates: bool = True) → IngestionStatus: [ALPHA] Satisfy the required interface for a failed ingestion.

build_objects_async(*, build_table: bool = False, delete_dataset_contents: bool = False, delete_templates: bool = True) → JobSubmissionResponse: [ALPHA] Satisfy the required interface for a failed ingestion.

dump() → dict: Dump this instance.

poll_for_job_completion(job: JobSubmissionResponse, *, timeout: float | None = None, polling_delay: float | None = None) → IngestionStatus: [ALPHA] Satisfy the required interface for a failed ingestion.

status() → IngestionStatus

[ALPHA] Retrieve the status of the ingestion from platform.

Returns:: The result of the ingestion attempt
Return type:: IngestionStatus

dataset_id = None

project_id = None

raise_errors = True

session = None

uid = None

Unique uuid4 identifier of this ingestion.

Type:: UUID

class citrine.resources.ingestion.Ingestion

Bases: Resource[Ingestion]

[ALPHA] A job that uploads new information to the platform.

Datasets are the basic unit of access control. A user with read access to a dataset can view every object in that dataset. A user with write access to a dataset can create, update, and delete objects in the dataset.

access_control_dict() → dict: Return an access control entity representation of this resource. Internal use only.

classmethod build(data: dict) → Self: Build an instance of this object from given data.

build_objects(*, build_table: bool = False, delete_dataset_contents: bool = False, delete_templates: bool = True, timeout: float | None = None, polling_delay: float | None = None) → IngestionStatus

[ALPHA] Perform a complete ingestion operation, from start to finish.

Initiates an ingestion operation, polls the server to determine when the job has finished, and returns the outcome.

Parameters:

build_table (bool) – Whether to build a table immediately after ingestion. Default : False
delete_dataset_contents (bool) – Whether to delete objects prior to generating new gemd objects. Default: False.
delete_templates (bool) – Whether to delete all objects and templates (as opposed to not deleting templates) when delete_dataset_contents is True. Default: True
timeout (Optional[float]) – Amount of time to wait on the job (in seconds) before giving up. Note that this number has no effect on the underlying job itself, which can also time out server-side.
polling_delay (Optional[float]) – How long to delay between each polling retry attempt.

Returns:

The object for the submitted job

Return type:

JobSubmissionResponse

build_objects_async(*, build_table: bool = False, delete_dataset_contents: bool = False, delete_templates: bool = True) → JobSubmissionResponse

[ALPHA] Begin an async ingestion operation.

Parameters:

build_table (bool) – Whether to build a table immediately after ingestion. Default : False
delete_dataset_contents (bool) – Whether to delete objects prior to generating new gemd objects. Default: False.
delete_templates (bool) – Whether to delete all objects and templates (as opposed to not deleting templates) when delete_dataset_contents is True. Default: True

Returns:

The object for the submitted job

Return type:

JobSubmissionResponse

dump() → dict: Dump this instance.

poll_for_job_completion(job: JobSubmissionResponse, *, timeout: float | None = None, polling_delay: float | None = None) → IngestionStatus

[ALPHA] Repeatedly ask server if a job associated with this ingestion has completed.

Parameters:

job (JobSubmissionResponse) – The job to check on
timeout – Amount of time to wait on the job (in seconds) before giving up. Note that this number has no effect on the underlying job itself, which can also time out server-side.
polling_delay – How long to delay between each polling retry attempt.

Returns:

A string representation of the status

Return type:

IngestionStatus

status() → IngestionStatus

[ALPHA] Retrieve the status of the ingestion from platform.

Returns:: The result of the ingestion attempt
Return type:: IngestionStatus

dataset_id = None

project_id = None

raise_errors = True

session = None

uid = None

Unique uuid4 identifier of this ingestion.

Type:: UUID

class citrine.resources.ingestion.IngestionCollection(project_id: UUID, dataset_id: UUID, session: Session)

Bases: Collection[Ingestion]

[ALPHA] Represents the collection of all ingestion operations.

Parameters:

project_id (UUID) – Unique ID of the project this dataset collection belongs to.
session (Session) – The Citrine session used to connect to the database.

build(data: dict) → Ingestion: Build an instance of an Ingestion.

build_from_file_links(file_links: Iterable[FileLink], *, raise_errors: bool = True) → Ingestion

[ALPHA] Create an on-platform ingestion event based on the passed FileLink objects.

Parameters:

file_links (Iterable[FileLink]) – The files to ingest.
raise_errors (bool) – Whether ingestion errors raise exceptions (vs. simply reported in the results). Default: True

delete(model: Ingestion) → Ingestion: Cannot delete an Ingestion.

get(uid: UUID | str) → ResourceType: Get a particular element of the collection.

list(*, per_page: int = 100) → Iterator[Ingestion]: Cannot list Ingestions.

register(model: Ingestion) → Ingestion: Cannot register an Ingestion.

update(model: Ingestion) → Ingestion: Cannot update an Ingestion.

class citrine.resources.ingestion.IngestionErrorFamily(value)

Bases: BaseEnumeration

[ALPHA] What class of ingest error was encountered.

DATA = 'data'

FILE = 'file'

STRUCTURE = 'structure'

UNKNOWN = 'unknown'

class citrine.resources.ingestion.IngestionErrorLevel(value)

Bases: BaseEnumeration

[ALPHA] Severity of the issue encountered.

ERROR = 'error'

INFO = 'info'

WARNING = 'warning'

class citrine.resources.ingestion.IngestionErrorTrace(msg, level=IngestionErrorLevel.ERROR, *, family=IngestionErrorFamily.UNKNOWN, error_type=IngestionErrorType.UNKNOWN_ERROR, dataset_file_id=None, file_version_uuid=None, row_number=None, column_number=None)

Bases: Resource[IngestionErrorTrace]

[ALPHA] Detailed information about an ingestion issue.

access_control_dict() → dict: Return an access control entity representation of this resource. Internal use only.

classmethod build(data: dict) → Self: Build an instance of this object from given data.

dump() → dict: Dump this instance.

classmethod from_validation_error(source: ValidationError) → IngestionErrorTrace: [ALPHA] Generate an IngestionErrorTrace from a ValidationError.

column_number = None

dataset_file_id = None

error_type = None

family = None

file_version_uuid = None

level = None

msg = None

row_number = None

class citrine.resources.ingestion.IngestionErrorType(value)

Bases: BaseEnumeration

[ALPHA] What ingest error was encountered.

CATEGORICAL_OUTSIDE_BOUNDS_ERROR = 'categorical_outside_bounds_error'

COLLIDING_ID = 'colliding_id'

DUPLICATED_MATERIAL = 'duplicated_material'

FILE_EXTENSION_NOT_SUPPORTED = 'file_extension_not_supported'

INTEGER_OUTSIDE_BOUNDS_ERROR = 'integer_outside_bounds_error'

INVALID_BASIS_ON_ASPECT = 'invalid_basis_on_aspect'

INVALID_DUPLICATE_NAME = 'invalid_duplicate_name'

INVALID_FRACTION_ON_ASPECT = 'invalid_fraction_on_aspect'

INVALID_PROCESS_REFERENCE = 'invalid_process_reference'

INVALID_TYPE_HINT_ON_ASPECT = 'invalid_type_hint_on_aspect'

INVALID_UNITS_ON_ASPECT = 'invalid_units_on_aspect'

MISSING_ASPECT_TYPE = 'missing_aspect_type'

MISSING_RAW_FOR_INGREDIENT = 'missing_raw_for_ingredient'

MISSING_TYPE_HEADER = 'missing_type_header'

REAL_OUTSIDE_BOUNDS_ERROR = 'real_outside_bounds_error'

REGISTERING_OBJECTS_ERROR = 'registering_objects_error'

UNKNOWN_ERROR = 'unknown_error'

class citrine.resources.ingestion.IngestionStatus(*, uid: UUID | None = None, status: IngestionStatusType = IngestionStatusType.INGESTION_CREATED, errors: Iterable[IngestionErrorTrace])

Bases: Resource[IngestionStatus]

[ALPHA] An object that represents the outcome of an ingestion event.

access_control_dict() → dict: Return an access control entity representation of this resource. Internal use only.

classmethod build(data: dict) → Self: Build an instance of this object from given data.

dump() → dict: Dump this instance.

classmethod from_exception(exception: IngestionException) → IngestionStatus: [ALPHA] Build an IngestionStatus from an IngestionException.

errors = None: List[IngestionErrorTrace]

status = None: IngestionStatusType

property success: bool: Whether the Ingestion operation was error-free.

uid = None: UUID

class citrine.resources.ingestion.IngestionStatusType(value)

Bases: BaseEnumeration

[ALPHA] State of the ingestion process.

INGESTION_CREATED = 'ingestion_created'