ml.dmlc.xgboost4j.java.spark.rapids

Class GpuColumnBatch

java.lang.Object

ml.dmlc.xgboost4j.java.spark.rapids.GpuColumnBatch

public class GpuColumnBatch
extends Object

Constructor Summary

Constructors

Constructor and Description
GpuColumnBatch(ai.rapids.cudf.Table table, org.apache.spark.sql.types.StructType schema)

Method Summary

Modifier and Type	Method and Description
long	getColumn(int index)
ai.rapids.cudf.ColumnVector	getColumnVector(int index)
ai.rapids.cudf.ColumnVector	getColumnVectorInitHost(int index)
int	getIntInColumn(int dataIndex, int colIndex, int defVal)
int	getNumColumns()
long	getNumRows()
static ai.rapids.cudf.DType	getRapidsType(org.apache.spark.sql.types.DataType type)
org.apache.spark.sql.types.StructType	getSchema()
int	groupAndAggregateOnColumnsHost(int groupIdx, int weightIdx, int prevTailGid, List<Integer> groupInfo, List<Float> weightInfo) Group data by column "groupIndex", and do "count" aggregation on column "groupIndex", while do aggregation similar to "average" on column "weightIdx", but require values in a group are equal to each other, then merge the results with "groupInfo" and "weightInfo" separately.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GpuColumnBatch

public GpuColumnBatch(ai.rapids.cudf.Table table,
                      org.apache.spark.sql.types.StructType schema)

Method Detail

getSchema

public org.apache.spark.sql.types.StructType getSchema()

getNumRows

public long getNumRows()

getNumColumns

public int getNumColumns()

getColumnVector

public ai.rapids.cudf.ColumnVector getColumnVector(int index)

getColumn

public long getColumn(int index)

getColumnVectorInitHost

public ai.rapids.cudf.ColumnVector getColumnVectorInitHost(int index)

getIntInColumn

public int getIntInColumn(int dataIndex,
                          int colIndex,
                          int defVal)

groupAndAggregateOnColumnsHost

public int groupAndAggregateOnColumnsHost(int groupIdx,
                                          int weightIdx,
                                          int prevTailGid,
                                          List<Integer> groupInfo,
                                          List<Float> weightInfo)

Group data by column "groupIndex", and do "count" aggregation on column "groupIndex", while do aggregation similar to "average" on column "weightIdx", but require values in a group are equal to each other, then merge the results with "groupInfo" and "weightInfo" separately. This is used to calculate group and weight info, and support chunk loading.

Parameters:

groupIdx - The index of column to group by.

weightIdx - The index of column where to get a value in each group.

prevTailGid - The group id of last group in prevGroupInfo.

groupInfo - Group information calculated from earlier batches.

weightInfo - Weight information calculated from earlier batches.

Returns:

The group id of last group in current column batch.

getRapidsType

public static ai.rapids.cudf.DType getRapidsType(org.apache.spark.sql.types.DataType type)