News - 16 series#
Release 16.1.1 - 2026-09-23#
Improvements#
[logical_select] Added support for parallel execution by n_workers#
logical_select executes the following processes in parallel when --n_workers is -1 or 2 or more:
Searching each shard
Each key of
--drilldownIndependent
--drilldowns[${LABEL}]
“Independent” means that a drilldown doesn’t refer to the result of another drilldown by --drilldowns[${LABEL}].table.
A drilldown that refers to the result of another drilldown is executed after the referred drilldown is finished.
Therefore, the degree of parallelism is reduced if drilldowns have dependencies.
For example, logical_select with --n_workers 2 against 20 shards that have 4,000,000 records in total is 1.5x faster than --n_workers 1 on a 20-core machine.
Note that the following behaviors are changed when logical_select is executed in parallel:
All shards are processed even if one of them fails. The first error is reported.
Query log entries of each shard such as
filter(N)[SHARD]andselect(N)[SHARD]are logged in finished order not in shard order.
Query log entries of drilldowns are also improved.
drilldown(N) and drilldowns[${LABEL}](N) show their keys and the filter operation of a drilldown shows its prefix such as drilldowns[${LABEL}].filter(N).
[logical_select] Added shard[${LABEL}].table#
logical_select finds actual tables automatically by ${LOGICAL_TABLE}_${YYYYMMDD} naming rule.
So you couldn’t use logical_select for actual tables that don’t follow the naming rule.
You can specify actual tables explicitly by --shard[${LABEL}].table:
logical_select \
--shard_key timestamp \
--shard[1].table Logs_host1 \
--shard[2].table Logs_host2
${LABEL} is just a label.
It’s used only for sorting the specified actual tables.
You don’t need to specify --logical_table when you specify one or more --shard[${LABEL}].table.
Note that a label isn’t used for choosing shards to be searched with --min and --max even when it looks like YYYYMM or YYYYMMDD.
All of the specified actual tables are searched.
[logical_select] Added support for keys other than shard_key in sort_keys#
Only the same key as shard_key was supported.
You can use other keys now.
Records of all search target shards are sorted as one result set.
Note that you can’t use a vector column in sort_keys with multiple shards.
scorer_tf_idf should use the number of records in all shards but it uses the number of records in each shard.
So its score isn’t accurate with multiple shards.
The cost of sorting depends on offset + limit because each shard sorts the top offset + limit records.
Sorting with a large offset is slow.
[logical_select] Added support for slices[${LABEL}]#
You can get multiple search results with one logical_select as with select.
Note that only some of slice related parameters in select are supported. See logical_select for supported parameters.
[logical_select] Added support for query_flags#
It’s the same as query_flags of select.
[logical_select] Added support for columns[${NAME}].stage=output#
A dynamic column of the output stage is created only for output target records.
So it’s faster than the filtered stage when you use it only in output_columns.
[BFloat16] Changed the conversion from Float32 to BFloat16 to round to nearest even#
We truncated the lower 16 bits of Float32.
Round to nearest even is used now.
It’s the standard conversion that is used by Intel AVX512-BF16, Arm, TensorFlow and PyTorch.
Truncation has a bias and may convert NaN to Inf.
Note that a converted value may be different from the value that is converted by 16.1.0 or earlier.
Deprecated BFloat16 conversion functions#
They are just casts now. Use a cast instead.
grn::numeric::bfloat16_to_float32(): Usestatic_cast<float>(value)instead.grn::numeric::to_bfloat16(): Usestatic_cast<grn_bfloat16>(value)instead.grn_bfloat16_to_float32(): Use(float)valueinstead.grn_float32_to_bfloat16(): Use(grn_bfloat16)valueinstead.
[WASI] Added libgroonga.wasm that exports the C API#
This is an experimental feature. Currently, this feature is still not stable.
You can use it from JavaScript and so on.
groonga-wasm-${VERSION}.tar.gz is available as a release asset.
Headers aren’t provided because a user of libgroonga.wasm doesn’t use a C compiler.
groonga.pc isn’t provided because libgroonga.wasm can’t be linked.
The groonga command isn’t provided for WASI yet.
libgroonga.mjs is also provided as a convenience wrapper of the C API in libgroonga.wasm for JavaScript.
grn_init() doesn’t ignore SIGPIPE anymore#
This is a behavior change for programs that embed libgroonga.
Programs that write to pipes must handle SIGPIPE by themselves.
Fixes#
[logical_select] Fixed a bug that a persistent column specified by drilldown_calc_target was closed after grouping#
A persistent column is shared with other contexts. So this may cause a crash.
[logical_select] Fixed a bug that records was duplicated in command version 3#
records was output multiple times in one result set.
Fixed a bug that a floating point number was treated as an inverted boolean value#
0.0 was treated as true and a non 0.0 value was treated as false with Float, Float32 and BFloat16.
So !float_column, float_column && ... and ... || float_column in filter, between and load --ifexists worked incorrectly.
[load] Fixed a bug that a weight for a BFloat16 weight vector column was rounded twice#
A weight in JSON was converted to Float32 and then converted to BFloat16.
The result may be different from the directly converted value.
For example, 1.003906251 was 1.0 by the twice conversion but it’s 1.0078125 by the direct conversion.
Fixed a bug that distance_cosine() and distance_inner_product() sometimes returned an unexpected value#
The SIMD implementations read data after the end of a vector in some cases. The result was normally correct because the read data was normally zero. But it was incorrect when the read data was left by the previous use of the buffer.
[autotools] Fixed a bug that out-of-source builds fail#
The build directory for a generated file wasn’t created.
Release 16.1.0 - 2026-08-28#
Fixes#
Fixed a crash bug when allocation of a new segment fails#
This issue may occur when updating the index in an environment with insufficient memory or when the limit on the number of memory mappings has been reached.
Groonga outputs an error message when it cannot allocate memory while updating the index. However, the error message refers to an uninitialized value when it is generated.
This error message is generated only when Groonga cannot allocate memory due to insufficient memory or when the limit on the number of memory mappings has been reached. Therefore, this issue does not occur in an environment with sufficient memory.
Release 16.0.9 - 2026-08-07#
Improvements#
[benchmark] Remove incorrect G_GNUC_CONST usage#
GH-2857 Reported by correctmost.
GH-2858 Patched by correctmost.
Changed for compatibility with recent GCC versions. See https://gitlab.gnome.org/GNOME/glib/-/commit/016829559 for details.
[string_truncate] Added new function string_truncate()#
string_truncate() truncates a string to at most the specified number of characters.
If the string is truncated, the tail of the kept characters is replaced with an omission mark so that the result, including the omission mark, is length characters long, in the same way as Ruby on Rails’ String#truncate.
Execution example:
plugin_register functions/string
table_create Memos TABLE_HASH_KEY ShortText
load --table Memos
[
{"_key": "Groonga is a full text search engine"}
]
select Memos \
--output_columns '_key, string_truncate(_key, 15)'
# [
# [
# 0,
# 1337566253.89858,
# 0.000355720520019531
# ],
# [
# [
# [
# 1
# ],
# [
# [
# "_key",
# "ShortText"
# ],
# [
# "string_truncate",
# null
# ]
# ],
# [
# "Groonga is a full text search engine",
# "Groonga is a..."
# ]
# ]
# ]
# ]
Added new function binary_length()#
binary_length() returns the size of the data in a column in bytes.
You can use it to check the data size of each column as shown below.
plugin_register functions/binary
table_create Memos TABLE_HASH_KEY ShortText
load --table Memos
[
{"_key": "ぐるんが"},
{"_key": "Groonga"}
]
select Memos \
--output_columns '_key, binary_length(_key)'
#[
# [
# 0,
# 1786079749.354771,
# 0.0005497932434082031
# ],
# [
# [
# [
# 2
# ],
# [
# [
# "_key",
# "ShortText"
# ],
# [
# "binary_length",
# null
# ]
# ],
# [
# "ぐるんが",
# 12
# ],
# [
# "Groonga",
# 7
# ]
# ]
# ]
#]
[normalizers] Added support for Unicode 17.0.0#
Fixes#
Fixed a bug where input was unintentionally tokenized while ignoring whitespace when using multiple normalizers#
GH-2853 Reported by askdkc.
For example, the following text incorrectly matches the query “sayuri” cause of this issue:
nanakusa yurine konnyaku
However, the text does not contain “sayuri”.
This issue occurs only when using multiple normalizers as shown below:
table_create Normalizations TABLE_PAT_KEY ShortText
column_create Normalizations normalized COLUMN_SCALAR ShortText
load --table Normalizations
[
{"_key": "Anything", "normalized": "Whatever"}
]
table_create EntriesChained TABLE_NO_KEY
column_create EntriesChained content COLUMN_SCALAR Text
table_create TermsChained TABLE_PAT_KEY ShortText \
--default_tokenizer 'TokenNgram("unify_alphabet", false)' \
--normalizers 'NormalizerNFKC150, NormalizerTable("normalized", "Normalizations.normalized")'
column_create TermsChained content_index COLUMN_INDEX|WITH_POSITION EntriesChained content
load --table EntriesChained
[
{"content": "nanakusa yurine konnyaku"}
]
select EntriesChained \
--match_columns content \
--query sayuri \
--output_columns _id,content
[
[
0,
1783649374.310488,
0.0007901191711425781
],
[
[
[
1
],
[
[
"_id",
"UInt32"
],
[
"content",
"Text"
]
],
[
1,
"nanakusa yurine konnyaku"
]
]
]
]
Thanks#
askdkc
correctmost
Release 16.0.8 - 2026-07-13#
Improvements#
[between] Added support for vector type value as the 1st parameter#
Note that
betweendoes not use an index for a vector value yet.
In the following example, between returns true if any of the elements is included in the specified range.
Therefore, we can use between to search records that have at least one value in the range.
Execution example:
table_create Products TABLE_HASH_KEY ShortText
# [[0,1337566253.89858,0.000355720520019531],true]
column_create Products prices COLUMN_VECTOR Int32
# [[0,1337566253.89858,0.000355720520019531],true]
load --table Products
[
{"_key": "A", "prices": [17, 170, 1700]},
{"_key": "B", "prices": [18, 180, 1800]},
{"_key": "C", "prices": [19, 190]},
{"_key": "D", "prices": [20]},
{"_key": "E", "prices": [21, 210, 2100]}
]
# [[0,1337566253.89858,0.000355720520019531],5]
select Products --filter 'between(prices, 18, "include", 20, "exclude")'
# [
# [
# 0,
# 1337566253.89858,
# 0.000355720520019531
# ],
# [
# [
# [
# 2
# ],
# [
# [
# "_id",
# "UInt32"
# ],
# [
# "_key",
# "ShortText"
# ],
# [
# "prices",
# "Int32"
# ]
# ],
# [
# 2,
# "B",
# [
# 18,
# 180,
# 1800
# ]
# ],
# [
# 3,
# "C",
# [
# 19,
# 190
# ]
# ]
# ]
# ]
# ]
Dropped support for Debian 12 (bookworm)#
It reached EOL on 2026-06-10.
Fixes#
Fixed a crash bug when we specify language_model_knn(...) as sort_keys#
For example, if you specify language_model_knn(text, "male child", {"k" : 2 }) as sort_keys, Groonga crashes due to a double free that occurs while processing the { "k" : 2 } option in the sort routine.
Fixed a crash bug when we use language_model_knn() with empty value#
Groonga crashes when language_model_knn() is used with empty value as shown below:
plugin_register language_model/knn
[[0,0.0,0.0],true]
table_create Data TABLE_NO_KEY
[[0,0.0,0.0],true]
column_create Data text COLUMN_SCALAR ShortText
[[0,0.0,0.0],true]
column_create Data rabitq_code COLUMN_SCALAR ShortBinary
[[0,0.0,0.0],true]
load --table Data
[
{"text": "I am a boy."},
{"text": ""},
{"text": "This is an apple."}
]
[[0,0.0,0.0],3]
table_create RaBitQ TABLE_HASH_KEY ShortBinary \
--default_tokenizer 'TokenLanguageModelKNN("model", \
"hf:///groonga/all-MiniLM-L6-v2-Q4_K_M-GGUF", \
"code_column", "rabitq_code")'
[[0,0.0,0.0],true]
column_create RaBitQ data_text COLUMN_INDEX Data text
[[0,0.0,0.0],true]
select Data --filter 'language_model_knn(text, "male child")' --output_columns text
Experimental features#
This is an experimental feature. Currently, this feature is still not stable.
[json_extract] Added support for json_extract()#
json_extract() extracts values from a JSON by a JSONPath expression.
The extracted values keep their JSON types.
For example, a string in JSON is extracted as a text and an integer in JSON is extracted as an integer.
Therefore, we can use json_extract() for both full text search against strings and range search against numbers as shown below:
table_create Data TABLE_NO_KEY
# [[0,1337566253.89858,0.000355720520019531],true]
column_create Data value COLUMN_SCALAR JSON
# [[0,1337566253.89858,0.000355720520019531],true]
load --table Data
[
{"value": "{\"value\": [[1, 10], [100]]}"},
{"value": "{\"value\": [[2], [20, 200]]}"},
{"value": "{\"value\": [[-1, -10], [-100]]}"}
]
select Data --filter 'between(json_extract(value, "$.value[*][*]"), 10, 20)'
# [
# [
# 0,
# 1337566253.89858,
# 0.000355720520019531
# ],
# [
# [
# [
# 2
# ],
# [
# [
# "_id",
# "UInt32"
# ],
# [
# "value",
# "JSON"
# ]
# ],
# [
# 1,
# {
# "value": [
# [
# 1,
# 10
# ],
# [
# 100
# ]
# ]
# }
# ],
# [
# 2,
# {
# "value": [
# [
# 2
# ],
# [
# 20,
# 200
# ]
# ]
# }
# ]
# ]
# ]
# ]
[ExtractorJSON] Added support for ExtractorJSON#
This is an experimental feature. Currently, this feature is still not stable.
ExtractorJSON extracts values from JSON data by a JSONPath expression as well as json_extract().
We can use this extractor to index only the values you need from JSON without indexing the whole JSON text.
When you attach ExtractorJSON to a lexicon, the lexicon indexes the extracted values. The original JSON is kept in the data column.
The following example indexes integers in a JSON column. The lexicon key type is Int32 because the extracted values are integers. The index is used automatically when the JSON column is loaded, so you can search the original records by the extracted values:
table_create Data TABLE_NO_KEY
# [[0,1337566253.89858,0.000355720520019531],true]
column_create Data value COLUMN_SCALAR JSON
# [[0,1337566253.89858,0.000355720520019531],true]
table_create Numbers TABLE_PAT_KEY Int32 \
--extractors 'ExtractorJSON("path", "$.value[*][*]")'
# [[0,1337566253.89858,0.000355720520019531],true]
column_create Numbers data_value COLUMN_INDEX Data value
# [[0,1337566253.89858,0.000355720520019531],true]
load --table Data
[
{"value": "{\"value\": [[1, 10], [100]]}"},
{"value": "{\"value\": [[2], [20, 200]]}"},
{"value": "{\"value\": [[-1, -10], [-100]]}"}
]
# [[0,1337566253.89858,0.000355720520019531],3]
select Data --filter 'between(Numbers.data_value, 10, 20)'
# [
# [
# 0,
# 1337566253.89858,
# 0.000355720520019531
# ],
# [
# [
# [
# 2
# ],
# [
# [
# "_id",
# "UInt32"
# ],
# [
# "value",
# "JSON"
# ]
# ],
# [
# 1,
# {
# "value": [
# [
# 1,
# 10
# ],
# [
# 100
# ]
# ]
# }
# ],
# [
# 2,
# {
# "value": [
# [
# 2
# ],
# [
# 20,
# 200
# ]
# ]
# }
# ]
# ]
# ]
# ]
Release 16.0.5 - 2026-05-22#
Fixes#
Fixed a bug where Groonga for Windows failed to run due to a missing required DLL#
The required DLL (msvcp140_atomic_wait.dll) for Groonga for Windows was missing from the groonga-16.0.2-x64-vs2022-with-vcruntime.zip.
This issue only affects Groonga 16.0.2 for Windows.
Release 16.0.2 - 2026-05-07#
Improvements#
Added support for Ubuntu 26.04 LTS#
Fixes#
Fixed the ODR(One Definition Rule) violation#
GH-2787 Reported by Nicolas PARLANT.
The LTO(Link-Time Optimization) build was failing because grn_tokenizer_query had two different definitions.
This change fixes the ODR violation, allowing the LTO process to complete successfully.
Fixed missing files in benchmark directory of source archive#
GH-2793 Reported by Nicolas PARLANT.
The missing files are as follows:
groonga/benchmark/CMakeLists.txt
groonga/benchmark/bench-distance.c
groonga/benchmark/geo-distance-summary.rb
Fixed a bug where duplicate “tools” directories were created#
GH-2798 Reported by Kentaro Hayashi.
Fixed an issue where duplicate “tools” directories were created as below when installing the groonga-tools package on AlmaLinux and Amazon Linux:
/usr/share/groonga/tools/tools/
Thanks#
Nicolas PARLANT
Kentaro Hayashi
Release 16.0.1 - 2026-03-30#
Improvements#
[language_model_vectorize] Added prefix option#
You can now add a prefix to the input text.
This is useful for models that require a prefix, similar to the passage_prefix and query_prefix options of TokenLanguageModelKNN.
language_model_vectorize("hf:///groonga/multilingual-e5-base-Q4_K_M-GGUF", \
"male child", \
{"prefix": "query: "})
[object_list] Added normalizer information to the output#
object_list command now includes a normalizers field in the output for each table.
Fixes#
[HTTP] Fixed a false error for chunked HTTP requests#
Fixed a bug where valid chunked HTTP requests could fail with an error under certain conditions.
This issue could only occur when a chunked request body was split across multiple receives in a very specific pattern, so most users would never encounter it.
If it did occur during a load command, the load would only be partially completed and would need to be re-executed.
No index corruption occurs due to this bug.
Fixed a bug that caused a crash due to using the wrong free function#
Fixed a bug where grn_obj_unlink() was used instead of GRN_OBJ_FIN() for bulk objects.
This could cause a crash in some cases.
Reported by Daniel Black
Experimental features#
These features are still experimental and unstable. We must not use these features in production.
[Data types] Added support for arrays and objects in JSON type#
The JSON type now supports arrays and objects, including nested ones.
All JSON value types are now supported.
Added support for OpenZL compression and decompression for scalar columns and Float32 vector columns#
OpenZL compression and decompression are now supported for scalar columns and Float32 vector columns. OpenZL can achieve better compression ratios than Zstandard.
Examples of compression:
The compression of float32 array results for the following data:
The number of array elements is 40,960.
The number of records is 10,000.
Results:
Compress with OpenZL: 1.3GB
Compress with Zstandard: 1.5GB
No compress: 1.6GB
Added extractor support#
Extractors are a new type of module in Groonga that extract plain text from structured data before tokenization.
The built-in ExtractorHTML extractor strips HTML tags and decodes HTML entities from a value, leaving only the text content.
Here is an example of using ExtractorHTML with the extract command:
extract \
--extractors 'ExtractorHTML' \
--value "<html><body>He<ll>o</body></html>"
[[0,0.0,0.0],{"extracted":"He<ll>o"}]
Thanks#
Daniel Black
Release 16.0.0 - 2026-02-09#
This is our annual major release! This release doesn’t have any backward incompatible changes! So you can upgrade Groonga without migrating your existing databases. You can still use your existing databases as-is.
Fixes#
Fixed a bug that causes overflow in the key of TABLE_DAT_KEY tables#
If you set a value of 4096 byte against a key, a TABLE_DAT_KEY table may be broken.