No. Row group?

[mengjie09]

According to the lance file layout, the current lance V2 cancels the concept of row group. What is the relationship between DataFragment and row group in the code?
The DataFragment concept describes how to express different numbers of rows in different columns of the same row. Is this function implemented?

[mengjie09]

According to the lance file layout, the current lance V2 cancels the concept of row group. What is the relationship between DataFragment and row group in the code?
The DataFragment concept describes how to express different numbers of rows in different columns of the same row. Is this function implemented?

[wjones127]

DataFragment is a table-level concept. It has a fixed number of rows. When you first write data, it typically corresponds to a single data file. This is different than a row group. Row groups are inside files; as in, there are multiple row groups in a file. But Lance V2 doesn't have row groups.

The layout of data fragments is described here: https://lancedb.github.io/lance/format.html#fragments

[mengjie09]

Thank you. Here's another question. If lance supports different number of rows for different columns, and DataFragment needs to have the same number of rows, how is this DataFragment represented? Is this expressed in one DataFragment, or different DataFragments?

[wjones127]

If lance supports different number of rows for different columns

Each file must have the same number of rows per column. No row groups means there isn't a smaller unit that is required to have the same number of rows per column.