graviti.dataframe.frame

The implementation of the Graviti DataFrame.

Module Contents

Classes

DataFrame

Two-dimensional, size-mutable, potentially heterogeneous tabular data.

Attributes

RECORD_KEY
APPLY_KEY

graviti.dataframe.frame.RECORD_KEY = __record_key

graviti.dataframe.frame.APPLY_KEY = apply_result

class graviti.dataframe.frame.DataFrame

Bases: graviti.dataframe.container.Container

Two-dimensional, size-mutable, potentially heterogeneous tabular data.

Parameters

• data – The data that needs to be stored in DataFrame.

• schema – The schema of the DataFrame. If None, will be inferred from data.

• columns – Column labels to use for resulting frame when data does not have them, defaulting to RangeIndex(0, 1, 2, …, n). If data contains column labels, will perform column selection instead.

Examples

Constructing DataFrame from list.

>>> df = DataFrame(
...     [
...         {"filename": "a.jpg", "box2ds": {"x": 1, "y": 1}},
...         {"filename": "b.jpg", "box2ds": {"x": 2, "y": 2}},
...         {"filename": "c.jpg", "box2ds": {"x": 3, "y": 3}},
...     ]
... )
>>> df
    filename box2ds
             x      y
0   a.jpg    1      1
1   b.jpg    2      2
2   c.jpg    3      3

classmethod from_pyarrow(cls, array, schema=None)

Create DataFrame with pyarrow struct array.

Parameters

• array (pyarrow.StructArray) – The input pyarrow struct array.

• schema (Optional[graviti.portex.PortexRecordBase]) – The schema of the DataFrame.

• cls (Type[_T]) –

Raises

TypeError – When the given schema is mismatched with the pyarrow array type.

Returns

The loaded DataFrame instance.

Return type

_T

property iloc(self)

Purely integer-location based indexing for selection by position.

Allowed inputs are:

• An integer, e.g. 5.

• A list or array of integers, e.g. [4, 3, 0].

• A slice object with ints, e.g. 1:7.

• A boolean array of the same length as the axis being sliced.

Returns

The instance of the ILocIndexer.

Return type

graviti.dataframe.indexing.DataFrameILocIndexer

Examples

>>> df = DataFrame({"col1": [1, 2], "col2": [3, 4]})
>>> df.iloc[0]
col1    1
col2    3
Name: 0, dtype: int64
>>> df.iloc[[0]]
   col1  col2
0     1     3

property loc(self)

Access a group of rows and columns by indexes or a boolean array.

Allowed inputs are:

• A single index, e.g. 5.

• A list or array of indexes, e.g. [4, 3, 0].

• A slice object with indexes, e.g. 1:7.

• A boolean array of the same length as the axis being sliced.

Returns

The instance of the LocIndexer.

Return type

graviti.dataframe.indexing.DataFrameLocIndexer

Examples

>>> df = DataFrame({"col1": [1, 2], "col2": [3, 4]})
>>> df.loc[0]
col1    1
col2    3
Name: 0, dtype: int64
>>> df.loc[[0]]
   col1  col2
0     1     3

property shape(self)

Return a tuple representing the dimensionality of the DataFrame.

Returns

Shape of the DataFrame.

Return type

Tuple[int, int]

Examples

>>> df = DataFrame(
...     [
...         {"filename": "a.jpg", "box2ds": {"x": 1, "y": 1}},
...         {"filename": "b.jpg", "box2ds": {"x": 2, "y": 2}},
...         {"filename": "c.jpg", "box2ds": {"x": 3, "y": 3}},
...     ]
... )
>>> df
    filename box2ds
             x      y
0   a.jpg    1      1
1   b.jpg    2      2
2   c.jpg    3      3
>>> df.shape
(3, 2)

copy(self)

Get a copy of the dataframe.

Returns

A copy of the dataframe.

Parameters

self (_T) –

Return type

_T

extend(self, values)

Extend Sequence object or DataFrame to itself row by row.

Parameters

values (Union[Iterable[Dict[str, Any]], DataFrame]) – A sequence object or DataFrame.

Raises

TypeError – When the given Dataframe mismatched with the self schema.

Return type

None

Examples

>>> df = DataFrame([
...     {"filename": "a.jpg", "box2ds": {"x": 1, "y": 1}},
...     {"filename": "b.jpg", "box2ds": {"x": 2, "y": 2}},
... ])

Extended by another list.

>>> df.extend([{"filename": "c.jpg", "box2ds": {"x": 3, "y": 3}}])
>>> df
    filename box2ds
             x      y
0   a.jpg    1      1
1   b.jpg    2      2
2   c.jpg    3      3

Extended by another DataFrame.

>>> df2 = DataFrame([{"filename": "d.jpg", "box2ds": {"x": 4 "y": 4}}])
>>> df.extend(df2)
>>> df
    filename box2ds
             x      y
0   a.jpg    1      1
1   b.jpg    2      2
2   d.jpg    4      4

to_pylist(self)

Convert the DataFrame to a python list.

Returns

The python list representing the DataFrame.

Return type

List[Dict[str, Any]]

query(self, func)

Query the columns of a DataFrame with a lambda function.

Parameters

func (Callable[[Any], Any]) – The query function.

Returns

The query result DataFrame.

Raises

TypeError – When the DataFrame is not in a Commit.

Return type

DataFrame

Examples

>>> df = DataFrame([
...     {"filename": "a.jpg", "box2ds": {"x": 1, "y": 1}},
...     {"filename": "b.jpg", "box2ds": {"x": 2, "y": 2}},
... ])
>>> df.query(lambda x: x["filename"] == "a.jpg")
    filename box2ds
             x      y
0   a.jpg    1      1

apply(self, func)

Apply a function to the DataFrame row by row.

Parameters

func (Callable[[Any], Any]) – Function to apply to each row.

Returns

The apply result DataFrame or Series.

Raises

TypeError – When the DataFrame is not in a Commit.

Return type

graviti.dataframe.container.Container

Examples

>>> df = DataFrame([
...     {"filename": "a.jpg", "box2ds": {"x": 1, "y": 1}},
...     {"filename": "b.jpg", "box2ds": {"x": 2, "y": 2}},
... ])
>>> df.apply(lambda x: x["box2ds"]["x"] + 1)
    filename box2ds
             x      y
0   a.jpg    2      1
1   b.jpg    3      2