:topic: Data Structure Overview .. index:: triple: DREAM3D-NX; Concepts; DataStructure DREAM3D-NX Data Structure ########################### Overview ======== DREAM3D-NX uses an *abstract*, *hierarchical* data structure that has roots in the concepts of combinatorial topology and common methods for describing mesh structures. The generalized data structure follows a tree structure with the following possible node types: - DataStructure: The top level object that holds a tree of *DataObjects*. - DataGroup: This *DataObject* can hold 1 or more *DataObjects* - AttributeMatrix: This is a special kind of *DataObject* that only holds *DataArray* objects where all *DataArray* objects must have the same number of Tuples. The AttributeMatrix is an exact corollary to the original AttributeMatrix from the previous version of DREAM3D-NX - DataArray: This *DataObject* holds the actual raw data values that are used within DREAM3D-NX. This can be anything from simple gray scale image data to multidimensional data. DREAM3D-NX utilizes a *flat* data storage approach, such that even multidimensional data is allocated into a contiguous section of memory. This approach enables faster and more efficient compute times. - Geometry: This is another special *DataGroup* that represents any one of the geometric meshes that are supported in DREAM3D-NX. The supported geometric meshes are the following: - ImageData: - Rectilinear Grid: - Vertex - Edge - Triangle - Quad - Hexahedron - Tetrahedral For most DREAM3D-NX functions, the user is given control over how to name the various objects within the *Data Structure* tree with most filters providing reasonable defaults. Geometry ======== A special kind of object is known as the **Geometry**, which defines the spatial organization of the data. The **Data Container** objects are responsible for holding **Geometries**. In general, this means that the distinguishing characteristic between two **Data Containers** is their associated **Geometry** objects. **Geometries** are generally classified by the *topology* of their unit **Element**, which is the smallest building block of the geometry. **Elements** can have 4 different dimensions: 0 (points), 1 (lines), 2 (polygons), and 3 (polyhedra). In the DREAM3D-NX ontology, these are generally referred to as **Vertices**, **Edges**, **Faces**, and **Cells**. .. figure:: Images/vefc.png :alt: Unit Element Types Unit Element Types Data attributes can be associated with any of these element types, as long as that association makes sense for the given element topology. For example, a triangulated surface mesh can have **Vertex**, **Edge**, and **Face** data associated with it, but not **Cell** data, since a triangle has a 2-dimensional topology. To help the user keep track of the **Geometry** type associated with a particular **Data Container**, DREAM3D-NX sets default names of **Data Containers** to be prefixed with the **Geometry** type when applicable. For example, a **Data Container** containing an **Image Geometry** would have a default name **ImageDataContainer**. Of course, you can `rename <../../Filters/RenameDataObjectFilter/index.html>`__ **Data Containers** if you prefer a different data scheme. Currently Implemented Geometries ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +-------------+----------+-------------------+--------------------------------------------------------------------------------------------------------------------------------------+ | Name | Topology | Associated Data | Description | +=============+==========+===================+======================================================================================================================================+ | Vertex | 0 | Vertex | A collection of points, commonly referred to as a point cloud | +-------------+----------+-------------------+--------------------------------------------------------------------------------------------------------------------------------------+ | Edge | 1 | Edge/Vertex | A collection of edges defined by two vertices, forming *lines* | +-------------+----------+-------------------+--------------------------------------------------------------------------------------------------------------------------------------+ | Triangle | 2 | Face/Edge/Vertex | A collection of triangles; one type of *surface mesh* | +-------------+----------+-------------------+--------------------------------------------------------------------------------------------------------------------------------------+ | Quad | 2 | Face/Edge/Vertex | A collection of quadrilaterals; one type of *surface mesh* | +-------------+----------+-------------------+--------------------------------------------------------------------------------------------------------------------------------------+ | Image | 3 | Cell | A *structured*, *rectilinear* grid; this **Geometry** is composed of *pixels* (in 2D) or *voxels* (in 3D) with constant resolution | +-------------+----------+-------------------+--------------------------------------------------------------------------------------------------------------------------------------+ | Tetrahedral | 3 | Cell | A collection of tetrahedra; one type of *volume mesh* | +-------------+----------+-------------------+--------------------------------------------------------------------------------------------------------------------------------------+ | Hexahedron | 3 | Cell | A collection of hexahedra; one type of *volume mesh* | +-------------+----------+-------------------+--------------------------------------------------------------------------------------------------------------------------------------+ Attribute Matrices & Data Hierarchy ===================================== Data is organized in DREAM3D-NX using **Attribute Matrix** objects. These objects themselves do not store data; instead, they store the last level in the object hierarchy, called **Attribute Arrays**, which are the low-level containers for contiguous regions of data. There are a variety of different types of **Attribute Matrices**: - **Element**: Attribute data associated directly with the features of the **Geometry** objects. Types of **Element Attribute Matrices** include: - **Vertex**: Attribute data associated with vertices - **Edge**: Attribute data associated with edges - **Face**: Attribute data associated with polygons - **Cell**: Attribute data associated with polyhedra - **Feature**: Attribute data associated with *collections* of **Elements** - **Ensemble**: Attribute data associated with *collections* of **Features** The four different types of **Element Attribute Matrix** correspond to the four possible levels of dimensional topology for a given **Geometry**. For example, a **Triangle Geometry** may have a **Vertex Attribute Matrix** that stores an **Attribute Array** that defines a scalar at each vertex, and a **Face Attribute Matrix** that defines a vector on every triangle. This example points out one of the advantages of the DREAM3D-NX data structure: data with any amount of dimensionality can be stored efficiently thanks to the generic *object-attribute* associations. Note as well that the **Attribute Matrix** itself defines the *tuple dimensions* for a given kind of data, whereas the **Attribute Arrays** in that **Attribute Matrix** define the *component dimensions* of the raw data. For example, consider a **Triangle Geometry** with 20 triangles. A **Face Attribute Matrix** for this **Geometry** will have 20 *tuples*; i.e., the number of triangles. An associated scalar **Attribute Array** in this **Face Attribute Matrix** will also have a *tuple* length of 20, since 20 x 1 = 20. But an **Attribute Array** that defines a vector (such as a normal for each triangle) would have 3 components. This vector array is still associated with the triangles, so it is stored in the **Face Attribute Matrix**. The total number of *elements* for the underlying array in this case is now 60, since 20 x 3 = 60. .. figure:: Images/AttributeMatrix.png :alt: Attribute Matrix Layout Attribute Matrix Layout A given **Data Container** should only have at most one type of each of the different varieties of **Element Attribute Matrix**. This arises because a given **Geometry** should only have one way to organize its unit elements in space (e.g., the number of triangles in a **Triangle Geometry** remains fixed). If the number of any components in a **Geometry** change, then that defines a new **Geometry** that should belong in a new **Data Container** with new associated **Attribute Matrices**. Note that if the elements of a **Geometry** merely *move* (e.g., via a mesh smoothing process), then there is no need to instantiate new **Attribute Matrices**, since the underlying *tuple* dimensions have remained constant. The underlying **Elements** of a given **Geometry** can be *grouped* together to form collections of **Elements** via some classification rule. A common example is *segmenting* a grayscale image by assigning all pixels with less than a certain grayscale value to class 1, and all others to class 2. This procedure introduces a sense of *hierarchy* to a data stream. DREAM3D-NX refers to these collections of **Elements** as **Features**. These **Features** can then have data associated with them through a **Feature Attribute Matrix**. Any kind of **Element** may be grouped into **Features**, thus there are many ways to group said **Elements**. Therefore, **Data Containers** can generally contain as many **Feature Attribute Matrices** as are necessary. To continue the hierarchy scale, DREAM3D-NX allows for **Features** to be grouped together to form **Ensembles**. Again, these **Ensembles** can have associated **Ensemble Attribute Matrices** to store data, and there may be many **Ensemble Attribute Matrices** in a given **Data Container**. In principle, it is even possible to have **Ensembles** of **Ensembles**! In this manner, the DREAM3D-NX data structure allows data associations to bridge across multiple length scales. Both **Features** and **Ensembles** must be made up of some specific **Element** type at the lowest level. DREAM3D-NX currently requires that any set of **Features** (and thus **Ensembles** as well) must be composed of the *same* unit **Element** type. For example, **Features** can be made up of a collection of **Triangles**, but not both **Edges** and **Triangles**. Of course, it is possible to have a set of **Features** made up of **Edges** only and a set of **Features** made up of **Triangles** only within the same **Triangle Geometry**. To help the user keep track of these distinctions, DREAM3D-NX names **Feature/Ensemble Attribute Matrices** with a prefix that identifies the unit **Element** building block by default. For example, a set of **Features** made up of **Cells** would have a default **Attribute Matrix** name of **CellFeatureData**. Of course, you can `rename <../../Filters/RenameDataObjectFilter/index.html>`__ **Attribute Matrices** if you prefer a different data scheme. A key concept behind **Features** and **Ensembles** is the existence of a *map* that ties the entries in a **Feature/Ensemble Attribute Matrix** to one level lower in the hierarchy. The values in this map are commonly referred to as *Ids*. For example, consider an **Image Geometry** in 2D that has a set of 10 **Features** within it, defined by 10 groups of pixels. The **Attribute Matrix** associated with this set of **Features** will have *11* tuples. Why 11 and not 10? DREAM3D-NX begins numbering **Features** and **Ensembles** at 1, and reserves the *0* value for special use. Therefore, the number of tuples for a given **Feature/Ensemble Attribute Matrix** is always one larger than the actual number of **Features/Ensembles**. Since the **Features** are groups of pixels (a kind of **Element**), DREAM3D-NX associates each pixel with a particular **Feature**. These *Feature Id* values correspond to integers that sit on the pixel **Elements**, and allow DREAM3D-NX to index into the **Feature Attribute Matrix** by knowing the *Feature Id* at one level lower in the hierarchy. The same concept applies to *Ensemble Ids* sitting at the **Feature** level. These *map* associations enable DREAM3D-NX to efficiently *link* between the various hierarchy scales, allowing for connections and correlations to be assessed. .. figure:: Images/Elements_Features_Ensembles.png :alt: Object-Attribute Associations Object-Attribute Associations For a detailed discussion of the DREAM3D-NX data structure and its associated applications to the analysis of microstructure data, please consult this `IMMI paper `__.