graviti.utility.pyarrow#

The PyArrow extension types and arrays to represent PortexType.

Module Contents#

Classes#

ExtensionBase

The base class for the PyArrow representation of PortexType types.

BuiltinExtension

This class defines the PyArrow representation of PortexBuiltinType type.

ExternalExtension

This class defines the PyArrow representation of PortexExternalType type.

GravitiExtension

This class defines the PyArrow representation of PortexGravitiType type.

RemoteFileArray

This class defines the PyArrow representation of RemoteFileArray.

RemoteFileType

This class defines the PyArrow representation of RemoteFileType type.
class graviti.utility.pyarrow.ExtensionBase(name, storage_type, *, nullable=False, **kwargs)[source]#

Bases: pyarrow.ExtensionType

The base class for the PyArrow representation of PortexType types.

Parameters

  • name (str) – The PortexType name.

  • storage_type (pyarrow.lib.DataType) – The corresponding PyArrow type.

  • nullable (bool) – Whether the PortexType is nullable.

  • kwargs (Any) – Other keyword arguments to init the PortexType.

class graviti.utility.pyarrow.BuiltinExtension(name, storage_type, *, nullable=False, **kwargs)[source]#

Bases: ExtensionBase

This class defines the PyArrow representation of PortexBuiltinType type.

Parameters

  • name (str) –

  • storage_type (pyarrow.lib.DataType) –

  • nullable (bool) –

  • kwargs (Any) –

class graviti.utility.pyarrow.ExternalExtension(name, storage_type, url, revision, *, nullable=False, **kwargs)[source]#

Bases: ExtensionBase

This class defines the PyArrow representation of PortexExternalType type.

Parameters

  • name (str) – The PortexExternalType name.

  • storage_type (pyarrow.lib.DataType) – The corresponding PyArrow type.

  • url (str) – The repo url which contains the PortexExternalType.

  • revision (str) – The repo revision which contains the PortexExternalType.

  • nullable (bool) – Whether the PortexExternalType is nullable.

  • kwargs (Any) – Other keyword arguments to init the PortexExternalType.

class graviti.utility.pyarrow.GravitiExtension(name, storage_type, *, nullable=False, **kwargs)[source]#

Bases: ExtensionBase

This class defines the PyArrow representation of PortexGravitiType type.

Parameters

  • name (str) –

  • storage_type (pyarrow.lib.DataType) –

  • nullable (bool) –

  • kwargs (Any) –

class graviti.utility.pyarrow.RemoteFileArray[source]#

Bases: pyarrow.ExtensionArray

This class defines the PyArrow representation of RemoteFileArray.

class graviti.utility.pyarrow.RemoteFileType[source]#

Bases: GravitiExtension

This class defines the PyArrow representation of RemoteFileType type.