graviti.manager.policy#

The implementation of the dataset policy.

Module Contents#

Classes#

ObjectPolicyManager

The basic structure of the object policy of the dataset.

OSSObjectPolicyManager

The basic structure of the object policy of the dataset stored in OSS.

AZUREObjectPolicyManager

The basic structure of the object policy of the dataset stored in AZURE.

S3ObjectPolicyManager

The basic structure of the object policy of the dataset stored in S3.
class graviti.manager.policy.ObjectPolicyManager(dataset)[source]#

The basic structure of the object policy of the dataset.

Parameters

dataset (graviti.manager.Dataset) – Class Dataset instance.

property prefix(self)[source]#

Return the prefix of the put policy.

Returns

The prefix of the put policy.

Return type

str

abstract get_object(self, key, _allow_retry=True)[source]#

Get the object from graviti.

Parameters

  • key (str) – The key of the file.

  • _allow_retry (bool) – Whether requesting the get policy again is allowed.

Raises

NotImplementedError – The method of the base class should not be called.

Return type

graviti.utility.UserResponse

abstract put_object(self, key, path, _allow_retry=True)[source]#

Put the object to OSS.

Parameters

  • key (str) – The key of the file.

  • path (pathlib.Path) – The path of the file.

  • _allow_retry (bool) – Whether requesting the put policy again is allowed.

Raises

NotImplementedError – The method of the base class should not be called.

Return type

None

class graviti.manager.policy.OSSObjectPolicyManager(dataset)[source]#

Bases: ObjectPolicyManager

The basic structure of the object policy of the dataset stored in OSS.

Parameters

dataset (graviti.manager.Dataset) –

get_object(self, key, _allow_retry=True)[source]#

Get the object from OSS.

Parameters

  • key (str) – The key of the file.

  • _allow_retry (bool) – Whether requesting the get policy again is allowed.

Raises

ResponseError – If post response error.

Returns

The response of OSS get object API.

Return type

graviti.utility.UserResponse

put_object(self, key, path, _allow_retry=True)[source]#

Put the object to OSS.

Parameters

  • key (str) – The key of the file.

  • path (pathlib.Path) – The path of the file.

  • _allow_retry (bool) – Whether requesting the put policy again is allowed.

Raises

ResponseError – If post response error.

Return type

None

class graviti.manager.policy.AZUREObjectPolicyManager(dataset)[source]#

Bases: ObjectPolicyManager

The basic structure of the object policy of the dataset stored in AZURE.

Parameters

dataset (graviti.manager.Dataset) –

get_object(self, key, _allow_retry=True)[source]#

Get the object from AZURE.

Parameters

  • key (str) – The key of the file.

  • _allow_retry (bool) – Whether requesting the get policy again is allowed.

Raises

ResponseError – If post response error.

Returns

The response of AZURE get object API.

Return type

graviti.utility.UserResponse

put_object(self, key, path, _allow_retry=True)[source]#

Put the object to AZURE.

Parameters

  • key (str) – The key of the file.

  • path (pathlib.Path) – The path of the file.

  • _allow_retry (bool) – Whether requesting the put policy again is allowed.

Raises

ResponseError – If post response error.

Return type

None

class graviti.manager.policy.S3ObjectPolicyManager(dataset)[source]#

Bases: ObjectPolicyManager

The basic structure of the object policy of the dataset stored in S3.

Parameters

dataset (graviti.manager.Dataset) –

get_object(self, key, _allow_retry=True)[source]#

Get the object from S3.

Parameters

  • key (str) – The key of the file.

  • _allow_retry (bool) – Whether requesting the get policy again is allowed.

Raises

ResponseError – If post response error.

Returns

The response of S3 get object API.

Return type

graviti.utility.UserResponse

put_object(self, key, path, _allow_retry=True)[source]#

Put the object to OSS.

Parameters

  • key (str) – The key of the file.

  • path (pathlib.Path) – The path of the file.

  • _allow_retry (bool) – Whether requesting the put policy again is allowed.

Raises

ResponseError – If post response error.

Return type

None