Groonga 7.1.0 has been released
Groonga 7.1.0 has been released!
How to install: Install
Changes
Here are important changes in this release:
-
load Improved the
load's query-log format. -
logical_count Improved the
logical_count's query-log format. -
logical_select Improve the
logical_select's query-log format. -
delete Improved the
delete's query-log format. -
Supported vector for drilldown calc target.
-
[bulk] Reduced the number of
realloc(). -
Added new function
index_column_source_records.
load Improved the load's query-log format
Added detail below items in the load's query-log.
- outputs number of loaded records.
- outputs number of error records and columns.
- outputs number of total records.
Specifically, below.
2017-12-29 15:23:47.049299|0x7ffe8af29a50|:000000001209848 load(3): [1][2][3]
2017-12-29 15:23:47.049311|0x7ffe8af29a50|<000000001221494 rc=-22
The number in () of after the load is number of loaded records. The number in first [] is number of error columns. The number in second [] is number of error records. The number in third [] is number of total records.
logical_countImproved the logical_count's query-log format.
Added detail below items in the logical_count's query-log.
- outputs number of count.
Specifically, below.
2017-12-29 15:25:06.068077|0x7fffedde8460|:000000001276405 count(2)
2017-12-29 15:25:06.068107|0x7fffedde8460|<000000001305264 rc=0
The number in () of after the count is number of count.
logical_select Improve the logical_select's query-log format.
Added detail below items in the logical_select's query-log.
- log N outputs.
- outputs plain drilldown.
- outputs labeled drilldown.
Specifically, below.
2017-12-29 15:19:53.703472|0x7ffe0ce4e650|:000000001372833 filter(1)
2017-12-29 15:19:53.703499|0x7ffe0ce4e650|:000000001397623 select(1)[Logs_20170315]
2017-12-29 15:19:53.703796|0x7ffe0ce4e650|:000000001695440 filter(2)
2017-12-29 15:19:53.703813|0x7ffe0ce4e650|:000000001711123 select(2)[Logs_20170316]
2017-12-29 15:19:53.704024|0x7ffe0ce4e650|:000000001923225 filter(2)
2017-12-29 15:19:53.704040|0x7ffe0ce4e650|:000000001937931 select(2)[Logs_20170317]
2017-12-29 15:19:53.704198|0x7ffe0ce4e650|:000000002096788 output(5)
2017-12-29 15:19:53.704354|0x7ffe0ce4e650|<000000002253133 rc=0
The number in () of after the select is number of matched records or result of plain drilldown, result of labeled drilldown. These number's meaning is differernt by executed query. These numbers is displayed each shard. The above example have three shard. So, Three selects are displyed. At the trailing [], the table name searched is displayed.
deleteImproved the delete's query-log format.
Added detail below items in the delete's query-log.
- outputs number of deleted and error records.
- outputs number of rest number of records.
The number in () of after the delete is number of deleted records. The number in first [] is number of error records. The number in second [] is rest number of records.
Supported vector for drilldown calc target
You can drilldown against vector columns.
As below, you can specify vecotr column in drilldown_calc_target. So, you can get min and max and sum, average with elements of vector columns.
table_create Tags TABLE_PAT_KEY ShortText
table_create Memos TABLE_HASH_KEY ShortText
column_create Memos tag COLUMN_SCALAR Tags
column_create Memos scores COLUMN_VECTOR Int64
load --table Memos
[
{"_key": "Groonga1", "tag": "Groonga", "scores": [10, 29]},
{"_key": "Groonga2", "tag": "Groonga", "scores": [20]},
{"_key": "Groonga3", "tag": "Groonga", "scores": [60, 71]},
{"_key": "Mroonga1", "tag": "Mroonga", "scores": [61, 62, 63]},
{"_key": "Mroonga2", "tag": "Mroonga", "scores": [24, 20, 16]},
{"_key": "Mroonga3", "tag": "Mroonga", "scores": [8, 5, 2]},
{"_key": "Rroonga1", "tag": "Rroonga", "scores": [3]},
{"_key": "Rroonga2", "tag": "Rroonga", "scores": [-9, 0, 9]},
{"_key": "Rroonga3", "tag": "Rroonga", "scores": [0]}
]
When you execute below query against the above table, you can get min and max and sum, average in the below groups.
- Group with Groonga in tag.
- Group with Mroonga in tag.
- Group with Rroonga in tag.
select Memos \
--limit 0 \
--drilldowns[tag].keys tag \
--drilldowns[tag].calc_types 'MAX, MIN, SUM, AVG' \
--drilldowns[tag].calc_target scores \
--drilldowns[tag].output_columns _key,_max,_min,_sum,_avg
It is the value of _key, _max, _min, _sum, _avg from left.
["Groonga", 71, 10, 190, 38.0],
["Mroonga", 63, 2, 261, 29.0],
["Rroonga", 9, -1, 3, 0.6],
[bulk] Reduced the number of realloc()
It improves performance for large output case on Windows. For example, it causes 100x faster for 100MB over output.
Added new function index_column_source_records
As below, this function gets source records of index column.
plugin_register functions/index_column
table_create Memos TABLE_HASH_KEY ShortText
table_create Terms TABLE_PAT_KEY ShortText \
--default_tokenizer TokenBigram \
--normalizer NormalizerAuto
column_create Terms index COLUMN_INDEX|WITH_POSITION Memos _key
load --table Memos
[
{"_key": "Groonga is a fast full text search engine."},
{"_key": "Mroonga is a MySQL storage engine based on Groonga."},
{"_key": "Rroonga is a Ruby bindings for Groonga."}
]
When you execute query in the below against the above table, you can get record which has token in registered in the Terms table.
select Terms \
--limit -1 \
--sort_keys _id \
--columns[index_records].stage output \
--columns[index_records].type Memos \
--columns[index_records].flags COLUMN_VECTOR \
--columns[index_records].value 'index_column_source_records("index")' \
--output_columns '_id, _key, index_records'
The rightmost value is the result of index_column_source_records.
[ 1, "groonga", [ "Groonga is a fast full text search engine.", "Mroonga is a MySQL storage engine based on Groonga.", "Rroonga is a Ruby bindings for Groonga." ] ],
[ 2, "is", [ "Groonga is a fast full text search engine.", "Mroonga is a MySQL storage engine based on Groonga.", "Rroonga is a Ruby bindings for Groonga." ] ],
(Abbreviation)
Conclusion
See Release 7.1.0 2017-12-29 about detailed changes since 7.0.9
Let's search by Groonga!