Variable-length list arrays have a view counterpart type similarly to how Variable-length binary arrays have a view(Variable-length binary view arrays (German Strings))
The ListView<T> contains:
- a validity bitmap
- an offsets buffer
- a child array
- an additional buffer to represent the size of the list views
The following image shows the ListView<Int8> array for [[12, -7, 25], null, [0, -127, 127, 50], [], [50, 12]]
ListView arrays have some peculiar characteristics:
- Offsets are not guaranteed to increase like in the standard variable-length list array layout: the elements that make up the first list in the array are located at the end of the child array and the elements that make up the third list element are at the beginning of the child array.
- Values in the child array can be shared between more than one list element. The final list element utilizes the values 50 and 12, which are also part of the other two lists.
The list-view layout allows for more efficient representations of lists that contain overlapping elements, along with easier parallelization by facilitating out-of-order creation and processing, but it requires an additional buffer of values to represent the sizes. As a result, for list arrays that are not defined out-of-order or do not contain overlapping list elements, this representation would be less efficient than just using the standard list data type. As with the variable-length binary view array layout, this one was also inspired by representations utilized by the open source Velox and DuckDB engines.