Groonga 4.1.1 has been released

BloGroonga

2015-01-29

Groonga 4.1.1 has been released

Groonga 4.1.1 has been released!

How to install: Install

Changes

In this release, there are following topics:

Improvements
- Supported drilldown with sum up feature
- Supported groonga-httpd with stream dump processing
Experimental improvements
- Supported more compact default database size on Windows
- Supported Groonga with jemalloc to improve performance in multi-threaded usage

Improvement - Supported drilldown with sum up feature

In this release, drilldown is extended. This section describes about only drilldown with named label usage. It supports drilldown with SUM, MIN, MAX or AVG by each group. See calc_types about detail of each operator.

Try to pass following parameter to select command:

--drilldown[${LABEL}].calc_types=(SUM,MIN,MAX or AVG) \
--drilldown[${LABEL}].calc_target=(COLUMN)

Here is the simple example which use MAX.

table_create Tags TABLE_PAT_KEY ShortText

table_create Memos TABLE_HASH_KEY ShortText
column_create Memos tag COLUMN_SCALAR Tags
column_create Memos priority COLUMN_SCALAR Int64

load --table Memos
[
{"_key": "Groonga1", "tag": "Groonga", "priority": 10},
{"_key": "Groonga2", "tag": "Groonga", "priority": 20},
{"_key": "Groonga3", "tag": "Groonga", "priority": 40},
{"_key": "Mroonga1", "tag": "Mroonga", "priority": 50},
{"_key": "Mroonga2", "tag": "Mroonga", "priority": 25},
{"_key": "Mroonga3", "tag": "Mroonga", "priority": 10},
{"_key": "Rroonga1", "tag": "Rroonga", "priority": 25},
{"_key": "Rroonga2", "tag": "Rroonga", "priority": -25},
{"_key": "Rroonga3", "tag": "Rroonga", "priority": 0}
]

select Memos \
  --limit 0 \
  --drilldown[tag].keys tag \
  --drilldown[tag].calc_types MAX \
  --drilldown[tag].calc_target priority \
  --drilldown[tag].output_columns _key,_max

Here is the drilldown results.

{
  "tag": [
    [
      3
    ],
    [
      [
        "_key",
        "ShortText"
      ],
      [
        "_max",
        "Int64"
      ]
    ],
    [
      "Groonga",
      40
    ],
    [
      "Mroonga",
      50
    ],
    [
      "Rroonga",
      25
    ]
  ]
}

The data is grouped by tag, there are three groups - "Groonga", "Mroonga" and "Rroonga". Each result contains the max value in group. Note that calculated value is accessed by pseudo columns such as _max and _min in output_columns.

If you want to get not only max value, but also min value, specify multiple types for calc_types. Here is the example which gets max value and min value:

--drilldown[tag].calc_types MAX,MIN

Don't forget to specify pseudo column name "_min".

--drilldown[tag].output_columns _key,_max,_min

Then, you can get max value and min value in each group.

See document about calc_types details.

Supported groonga-httpd with stream dump processing

In this release, groonga-httpd supported dump command with stream processing. By this change, it will not happen to exhaust memory by executing dump command.

Experimental improvement - Supported more compact default database size on Windows

In this release, more compact default database size on Windows is supported.

In the previous versions, Groonga uses more larger default database size (128MiB) on Windows. The size of database increases every you create new column.

It is not efficient, now Groonga supports to expand database size gradually as same as other GNU/Linux environment.

To enable this feature, set GRN_IO_VERSION=1 as environment variable. By default, it is not enable because it is in experimental stage.

Please give us feedback how it improves disk usage. It is important to enable by default.

Experimental improvement - Supported Groonga with jemalloc to improve performance in multi-threaded usage

In this release, you can build Groonga with jemalloc support. This change will improve multi-threaded performance. jemalloc support is suitable for groonga command or Mroonga because they are adobt multi-threaded model.

Note that it is no impact to groonga-httpd because it use multi-process model.

If you dislike to build Groonga on your own, try LD_PRELOAD environment variable.

Here is the example command:

LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1 groonga

Please give us feedback how it affects performance.

Conclusion

See Release 4.1.1 2015-01-29 about detailed changes since 4.1.0.

Let's search by Groonga!

BloGroonga