BloGroonga

2016-08-29

Groonga 6.0.8 has been released!

Groonga 6.0.8 has been released!

How to install in each environment: Install

Changes

Here are important changes in this release:

  • Limitation about the maximum number record is relaxed
  • Supported valid lexicon check on setting index sources
  • Changed to report error when column value set is failed

Limitation about the maximum number record is relaxed

A table had the following limitations.

  • The maximum one key size: 4KiB
  • The maximum total size of keys: 4GiB
  • The maximum number of records: 268,435,455 (more than 268 million)

In the past, the maximum number of records is announced as 268 million.

With recent research, it turns out that there is a capability to store more records than ever.

Here is the updated limitation about the maximum number of records:

Keep in mind that these limitations may vary depending on conditions.

For example, you need to use small size type for key to store many records. Because the maximum total size of keys limitation is exceeded before the maximum number of records limitation is exceeded. If you use UInt64 (8byte) type and store 2^29 records, total key size is 4GiB (= 8 * (2^29)). You can't add more records. You need to choose decreasing key size (e.g. UInt32) or using KEY_LARGE and TABLE_HASH_KEY to store more records.

Supported valid lexicon check on setting index sources

In this release, invalid lexicon is treated as an error.

Let's try sample schema which contains an invalid case.

table_create Tags TABLE_HASH_KEY ShortText

table_create Sites TABLE_HASH_KEY ShortText
column_create Sites tags COLUMN_VECTOR Tags

table_create TagsIndex TABLE_HASH_KEY ShortText
column_create TagsIndex sites_tags COLUMN_INDEX Sites tags

In the previous versions of Groonga, there is no error report.

$ groonga -n testdb/db < sample.grn
[[0,1472192178.94475,0.006346225738525391],true]
[[0,1472192178.951148,0.006434917449951172],true]
[[0,1472192178.957618,0.009174346923828125],true]
[[0,1472192178.966841,0.006294727325439453],true]
[[0,1472192178.973179,0.01138973236083984],true]

With Groonga 6.0.8, valid lexicon check reports an invalid error.

$ groonga -n testdb/db < sample.grn
[[0,1472192179.028527,0.00267481803894043],true]
[[0,1472192179.031259,0.002692937850952148],true]
[[0,1472192179.033985,0.004066228866577148],true]
[[0,1472192179.03809,0.002942562103271484],true]
[[-22,1472192179.041083,0.01088976860046387,"[column][index][source] index table must equal to source type: <TagsIndex> -> <Tags>: index-column:<TagsIndex.sites_tags> sourc",[["grn_obj_set_info_source_invalid_lexicon_error","db.c",8418]]],false]

Changed to report error when column value set is failed

In this release, when column value set is failed, it is reported as error.

Here is the sample data and schema. age column stores the value of Int32. but it try to load string value.

table_create Users TABLE_NO_KEY
column_create Users age COLUMN_SCALAR Int32

load --table Users
[
{"age": "invalid number!"}
]

In the previous versions, there is no error.

$ groonga testdb/db < invalid.test
[[0,1472195681.942045,0.006659030914306641],true]
[[0,1472195681.948765,0.007688283920288086],true]
[[0,1472195681.956529,0.002389430999755859],1]
[[0,1472195681.958963,0.0009322166442871094],[[[1],[["_id","UInt32"],["age","Int32"]],[1,0]]]]

With Groonga 6.0.8, it fails to load.

$ groonga -n testdb/db < invalid.test
[[0,1472195948.997128,0.005559444427490234],true]
[[0,1472195949.002747,0.007435083389282227],true]
[[-22,1472195949.01023,0.000843048095703125,"<Users.age>: failed to cast to <Int32>: <\"invalid number!\">",[["grn_obj_set_value_column_fix_size","db.c",7520]]],1]
[[0,1472195949.011117,0.000377655029296875],[[[1],[["_id","UInt32"],["age","Int32"]],[1,0]]]]

You can notice mismatch of data types when loading.

Conclusions

Please refer to Release 6.0.8 - 2016-08-29 about detailed changes since 6.0.7.

Then, let's go all out to search by Groonga!