graviti.operation.frame#

Definitions of different operations on a DataFrame.

Module Contents#

Classes#

DataFrameOperation

This class defines the basic method of the operation on a DataFrame.

DataOperation

This class defines the basic method of the data operation on a DataFrame.

AddData

This class defines the operation that add data to a DataFrame.

UpdateSchema

This class defines the operation that update the schema of a DataFrame.

UpdateData

This class defines the operation that updates the data of a DataFrame.

DeleteData

This class defines the operation that delete the data of a DataFrame.
class graviti.operation.frame.DataFrameOperation[source]#

This class defines the basic method of the operation on a DataFrame.

get_file_count(self)[source]#

Get the file amount to be uploaded.

Returns

The file amount to be uploaded.

Return type

int

get_data_count(self)[source]#

Get the data amount to be uploaded.

Returns

The data amount to be uploaded.

Return type

int

abstract execute(self, dataset, *, draft_number, sheet, jobs, data_pbar, file_pbar)[source]#

Execute the OpenAPI create sheet.

Parameters

  • dataset (graviti.manager.Dataset) – The Dataset instance.

  • draft_number (int) – The draft number.

  • sheet (str) – The sheet name.

  • jobs (int) – The number of the max workers in multi-thread operation.

  • data_pbar (tqdm.tqdm) – The process bar for uploading structured data.

  • file_pbar (tqdm.tqdm) – The process bar for uploading binary files.

Raises

NotImplementedError – The method of the base class should not be called.

Return type

None

class graviti.operation.frame.DataOperation(data)[source]#

Bases: DataFrameOperation

This class defines the basic method of the data operation on a DataFrame.

Parameters

data (graviti.dataframe.DataFrame) –

get_file_count(self)[source]#

Get the file amount to be uploaded.

Returns

The file amount to be uploaded.

Return type

int

get_data_count(self)[source]#

Get the data amount to be uploaded.

Returns

The data amount to be uploaded.

Return type

int

class graviti.operation.frame.AddData(data)[source]#

Bases: DataOperation

This class defines the operation that add data to a DataFrame.

Parameters

data (graviti.dataframe.DataFrame) – The data to be added.

execute(self, dataset, *, draft_number, sheet, jobs, data_pbar, file_pbar)[source]#

Execute the OpenAPI add data.

Parameters

  • dataset (graviti.manager.Dataset) – The Dataset instance.

  • draft_number (int) – The draft number.

  • sheet (str) – The sheet name.

  • jobs (int) – The number of the max workers in multi-thread operation.

  • data_pbar (tqdm.tqdm) – The process bar for uploading structured data.

  • file_pbar (tqdm.tqdm) – The process bar for uploading binary files.

Return type

None

class graviti.operation.frame.UpdateSchema(schema, data)[source]#

Bases: DataFrameOperation

This class defines the operation that update the schema of a DataFrame.

Parameters

  • schema (graviti.portex.record) – New portex schema after updated.

  • data (graviti.dataframe.DataFrame) –

execute(self, dataset, *, draft_number, sheet, jobs, data_pbar, file_pbar)[source]#

Execute the OpenAPI update schema.

Parameters

  • dataset (graviti.manager.Dataset) – The Dataset instance.

  • draft_number (int) – The draft number.

  • sheet (str) – The sheet name.

  • jobs (int) – The number of the max workers in multi-thread operation.

  • data_pbar (tqdm.tqdm) – The process bar for uploading structured data.

  • file_pbar (tqdm.tqdm) – The process bar for uploading binary files.

Return type

None

class graviti.operation.frame.UpdateData(data)[source]#

Bases: DataOperation

This class defines the operation that updates the data of a DataFrame.

Parameters

data (graviti.dataframe.DataFrame) – The data for updating.

execute(self, dataset, *, draft_number, sheet, jobs, data_pbar, file_pbar)[source]#

Execute the OpenAPI add data.

Parameters

  • dataset (graviti.manager.Dataset) – The Dataset instance.

  • draft_number (int) – The draft number.

  • sheet (str) – The sheet name.

  • jobs (int) – The number of the max workers in multi-thread operation.

  • data_pbar (tqdm.tqdm) – The process bar for uploading structured data.

  • file_pbar (tqdm.tqdm) – The process bar for uploading binary files.

Return type

None

class graviti.operation.frame.DeleteData(record_keys)[source]#

Bases: DataFrameOperation

This class defines the operation that delete the data of a DataFrame.

Parameters

record_keys (List[str]) – The record keys of the data to be deleted.

execute(self, dataset, *, draft_number, sheet, jobs, data_pbar, file_pbar)[source]#

Execute the OpenAPI delete data.

Parameters

  • dataset (graviti.manager.Dataset) – The Dataset instance.

  • draft_number (int) – The draft number.

  • sheet (str) – The sheet name.

  • jobs (int) – The number of the max workers in multi-thread operation.

  • data_pbar (tqdm.tqdm) – The process bar for uploading structured data.

  • file_pbar (tqdm.tqdm) – The process bar for uploading binary files.

Return type

None