BloGroonga

2022-03-29

Groonga 12.0.2 has been released

Groonga 12.0.2 has been released!

How to install: Install

Changes

Here are important changes in this release:

Improvements

  • logical_range_filter Added support for reducing reference immediately after processing a shard.

    Groonga had reduced reference all shards when the finish of logical_range_filter until now. Groonga reduces reference immediately after processing a shard by this feature. The usage of memory may reduce while logical_range_filter executes by this feature.

    This feature is only valid for the reference count mode. We can valid the reference count mode by setting GRN_ENABLE_REFERENCE_COUNT=yes.

    Normally, Groonga keep objects(tables and column and index, and so on) that Groonga opened even once on memory. However, if we open many objects, Groonga uses much memory. In the reference count mode release objects that are not referenced anywhere from memory. The usage of memory of Groonga may reduce by this.

  • We increased the stability of the feature of recovering on crashes.

    This feature is experimental and it is disabled by default. Therefore, the following improvements are no influence on ordinary users.

    • We fixed a bug that the index was broken when Groonga crashed.
    • We fixed a bug that might remain a lock.
    • We fixed a bug that Groonga crashed while it was recovering the crash.

  • Improved performance for mmap if anonymous mmap available.[GitHub:MariaDB/server#1999][Suggested by David CARLIER]

    The performance of Groonga is improved a bit by this improvement.

  • Indexing Added support for the static index construction against the following types of columns.

    • The non-reference vector column with weight
    • The reference vector column with weight
    • The reference scalar column

    These columns have not supported the static index construction until now. Therefore, the time of making the index has longed even if we set the index against these columns after we loaded data into them. By this improvement, the time of making the index is short in this case.

  • column_create Added new flags MISSING_* and INVALID_*.

    We added the following new flags for column_create.

    • MISSING_ADD
    • MISSING_IGNORE

    • MISSING_NIL

    • INVALID_ERROR
    • INVALID_WARN
    • INVALID_IGNORE

    Normally, if the data column is a reference data column and the nonexistent key is specified, a new record for the nonexistent key is newly created automatically.

    The behavior that Groonga adds the key automatically into the column of reference destination is useful in the search like tag search. Because Groonga adds data automatically when we load data.

    However, this behavior is inconvenient if we need the other data except for the key. Because a record that only has the key exists.

    We can change this behavior by using flags that are added in this release.

    • MISSING_ADD: This is the default value. This is the same behavior as the current.

      If the data column is a reference data column and the nonexistent key is specified, a new record for the nonexistent key is newly created automatically.

    • MISSING_IGNORE:

      If the data column is a reference data column and the nonexistent key is specified, the nonexistent key is ignored. If the reference data column is a scalar column, the value is GRN_ID_NIL. If the reference data column is a vector column, the element is just ignored as below ::

      ["existent1", "nonexistent", "existent2"] -> ["existent1" "existent2"]

    • MISSING_NIL:

      If the data column is a reference data column and the nonexistent key is specified, the nonexistent key in a scalar column and a vector column is treated as GRN_ID_NIL ::

      ["existent1", "nonexistent", "existent2"] -> ["existent1", "" (GRN_ID_NIL), "existent2"]

    • INVALID_ERROR: This is the default value. This is the same behavior as the current except an error response of a vector column case.

      If we set the invalid value (e.g.: XXX for UInt8 scalar column), the set operation is treated as an error. If the data column is a scalar column, load reports an error in log and response. If the data column is a vector column, load reports an error in log but doesn't report an error in response. This is an incompatible change.

    • INVALID_WARN:

      If we set the invalid value (e.g.: XXX for UInt8 scalar column), a warning message is logged and the set operation is ignored. If the target data column is a reference vector data column, MISSING_IGNORE and MISSING_NIL are used to determine the behavior.

    • INVALID_IGNORE:

      If we set the invalid value (e.g.: XXX for UInt8 scalar column), the set operation is ignored. If the target data column is a reference vector data column, MISSING_IGNORE and MISSING_NIL are used to determine the behavior.

  • dump column_list Added support for MISSING_* and INVALID_* flags.

    These commands doesn't show MISSING_ADD and INVALID_ERROR flags to keep backward compatibility. Because these flags show the default behavior.

  • schema Added support for MISSING_* and INVALID_* flags.

    MISSING_AND and INVALID_ERROR flags aren't shown in flags to keep backward compatibility. However, new missing and invalid keys are added to each column.

  • We provided the package of Amazon Linux 2.

  • [Windows] Dropped support for building with Visual Studio 2017.

    Because we could not use windows-2016 image on GitHub Actions.

Known Issues

  • Currently, Groonga has a bug that there is possible that data is corrupt when we execute many additions, delete, and update data to vector column.

  • *< and *> only valid when we use query() the right side of filter condition. If we specify as below, *< and *> work as &&.

    • 'content @ "Groonga" *< content @ "Mroonga"'

  • Groonga may not return records that should match caused by GRN_II_CURSOR_SET_MIN_ENABLE.

Conclusion

Please refert to the following news for more details.

News Release 12.0.2

Let's search by Groonga!