Class MatrixObject
- java.lang.Object
-
- org.apache.sysds.runtime.instructions.cp.Data
-
- org.apache.sysds.runtime.controlprogram.caching.CacheableData<MatrixBlock>
-
- org.apache.sysds.runtime.controlprogram.caching.MatrixObject
-
- All Implemented Interfaces:
Serializable
public class MatrixObject extends CacheableData<MatrixBlock>
Represents a matrix in control program. This class contains method to read matrices from HDFS and convert them to a specific format/representation. It is also able to write several formats/representation of matrices to HDFS. IMPORTANT: Preserve one-to-one correspondence betweenMatrixObjectandMatrixBlockobjects, for cache purposes. Do not change aMatrixBlockobject without informing itsMatrixObjectobject.- See Also:
- Serialized Form
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static classMatrixObject.UpdateType-
Nested classes/interfaces inherited from class org.apache.sysds.runtime.controlprogram.caching.CacheableData
CacheableData.CacheStatus
-
-
Field Summary
-
Fields inherited from class org.apache.sysds.runtime.controlprogram.caching.CacheableData
cacheEvictionLocalFilePath, cacheEvictionLocalFilePrefix, CACHING_ASYNC_FILECLEANUP, CACHING_ASYNC_SERIALIZE, CACHING_BUFFER_PAGECACHE, CACHING_BUFFER_POLICY, CACHING_COUNTER_GROUP_NAME, CACHING_EVICTION_FILEEXTENSION, CACHING_THRESHOLD, CACHING_WRITE_CACHE_ON_READ
-
-
Constructor Summary
Constructors Constructor Description MatrixObject(Types.ValueType vt, String file)Constructor that takes the value type and the HDFS filename.MatrixObject(Types.ValueType vt, String file, MetaData mtd)Constructor that takes the value type, HDFS filename and associated metadata.MatrixObject(Types.ValueType vt, String file, MetaData mtd, MatrixBlock data)Constructor that takes the value type, HDFS filename and associated metadata and a MatrixBlock used for creation after serializationMatrixObject(MatrixObject mo)Copy constructor that copies meta data but NO data.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description longgetBlocksize()longgetNnz()StringgetPartitionFileName(IndexRange pred, int blen)ParForProgramBlock.PDataPartitionFormatgetPartitionFormat()intgetPartitionSize()doublegetSparsity()MatrixObject.UpdateTypegetUpdateType()booleanisDiag()booleanisMarked()booleanisPartitioned()MatrixBlockreadMatrixPartition(IndexRange pred)NOTE: for reading matrix partitions, we could cache (in its real sense) the read block with soft references (no need for eviction, as partitioning only applied for read-only matrices).voidrefreshMetaData()Make the matrix metadata consistent with the in-memory matrix datavoidsetDiag(boolean diag)voidsetInMemoryPartition(MatrixBlock block)voidsetMarkForLinCache(boolean mark)voidsetPartitioned(ParForProgramBlock.PDataPartitionFormat format, int n)voidsetUpdateType(MatrixObject.UpdateType flag)StringtoString()voidunsetPartitioned()voidupdateDataCharacteristics(DataCharacteristics dc)-
Methods inherited from class org.apache.sysds.runtime.controlprogram.caching.CacheableData
acquireModify, acquireRead, acquireReadAndRelease, addBroadcastSize, cleanupCacheDir, cleanupCacheDir, clearData, clearData, disableCaching, enableCaching, enableCleanup, exportData, exportData, exportData, exportData, exportData, freeEvictedBlob, getBroadcastHandle, getBroadcastSize, getCacheLineage, getCompressedSize, getDataCharacteristics, getDataSize, getDebugName, getDim, getFedMapping, getFileFormatProperties, getFileName, getGPUObject, getMetaData, getNumColumns, getNumRows, getRDDHandle, getStatus, getUniqueID, hasValidLineage, initCaching, initCaching, isBelowCachingThreshold, isCached, isCachingActive, isCleanupEnabled, isCompressed, isDirty, isFederated, isFederated, isFederatedExcept, isHDFSFileExists, isPendingRDDOps, moveData, release, removeGPUObject, removeMetaData, setBroadcastHandle, setCacheLineage, setCompressedSize, setDirty, setEmptyStatus, setFedMapping, setFileFormatProperties, setFileName, setGPUObject, setHDFSFileExists, setMetaData, setRDDHandle
-
Methods inherited from class org.apache.sysds.runtime.instructions.cp.Data
getDataType, getPrivacyConstraint, getValueType, setPrivacyConstraints
-
-
-
-
Constructor Detail
-
MatrixObject
public MatrixObject(Types.ValueType vt, String file)
Constructor that takes the value type and the HDFS filename.- Parameters:
vt- value typefile- file name
-
MatrixObject
public MatrixObject(Types.ValueType vt, String file, MetaData mtd)
Constructor that takes the value type, HDFS filename and associated metadata.- Parameters:
vt- value typefile- file namemtd- metadata
-
MatrixObject
public MatrixObject(Types.ValueType vt, String file, MetaData mtd, MatrixBlock data)
Constructor that takes the value type, HDFS filename and associated metadata and a MatrixBlock used for creation after serialization- Parameters:
vt- value typefile- file namemtd- metadatadata- matrix block data
-
MatrixObject
public MatrixObject(MatrixObject mo)
Copy constructor that copies meta data but NO data.- Parameters:
mo- matrix object
-
-
Method Detail
-
setUpdateType
public void setUpdateType(MatrixObject.UpdateType flag)
-
getUpdateType
public MatrixObject.UpdateType getUpdateType()
-
isDiag
public boolean isDiag()
-
setDiag
public void setDiag(boolean diag)
-
setMarkForLinCache
public void setMarkForLinCache(boolean mark)
-
isMarked
public boolean isMarked()
-
updateDataCharacteristics
public void updateDataCharacteristics(DataCharacteristics dc)
- Overrides:
updateDataCharacteristicsin classData
-
refreshMetaData
public void refreshMetaData()
Make the matrix metadata consistent with the in-memory matrix data- Specified by:
refreshMetaDatain classCacheableData<MatrixBlock>
-
getBlocksize
public long getBlocksize()
- Overrides:
getBlocksizein classCacheableData<MatrixBlock>
-
getNnz
public long getNnz()
-
getSparsity
public double getSparsity()
-
setPartitioned
public void setPartitioned(ParForProgramBlock.PDataPartitionFormat format, int n)
-
unsetPartitioned
public void unsetPartitioned()
-
isPartitioned
public boolean isPartitioned()
-
getPartitionFormat
public ParForProgramBlock.PDataPartitionFormat getPartitionFormat()
-
getPartitionSize
public int getPartitionSize()
-
setInMemoryPartition
public void setInMemoryPartition(MatrixBlock block)
-
readMatrixPartition
public MatrixBlock readMatrixPartition(IndexRange pred)
NOTE: for reading matrix partitions, we could cache (in its real sense) the read block with soft references (no need for eviction, as partitioning only applied for read-only matrices). However, since we currently only support row- and column-wise partitioning caching is not applied yet. This could be changed once we also support column-block-wise and row-block-wise. Furthermore, as we reject to partition vectors and support only full row or column indexing, no metadata (apart from the partition flag) is required.- Parameters:
pred- index range- Returns:
- matrix block
-
getPartitionFileName
public String getPartitionFileName(IndexRange pred, int blen)
-
toString
public String toString()
- Overrides:
toStringin classCacheableData<MatrixBlock>
-
-