BloGroonga

2017-05-29

Groonga 7.0.3 has been released

Groonga 7.0.3 has been released!

How to install: Install

Changes

Here are important changes in this release:

  • [select] Add document about Full text search with specific index name
  • [load][dump] Supported Apache Arrow

[select] Add document about Full text search with specific index name

In this release, documentation about full text search with specific index name is added. For example, you can learn how to specify index name for --match_columns and what is better way to specify index name instead of data column name in this section.

[load][dump] Supported Apache Arrow

In this release, experimental API for Apache Arrow format has been supported.

As this is experimental feature, you can not use casually it right now, but sooner or later, Apache Arrow support will be improved and Groonga user can use this feature more easily.

Conclusion

See Release 7.0.3 2017-05-29 about detailed changes since 7.0.2.

Let's search by Groonga!

2017-04-29

PGroonga (fast full text search module for PostgreSQL) 1.2.0 has been released

PGroonga (píːzí:lúnɡά) (fast full text search module for PostgreSQL) 1.2.0 has been released!

See PGroonga 1.0.0 release announce about PGroonga.

Highlight

Here are highlights of this release:

  • Improve replication

  • Support token filters and plugins

  • Add auto PGroonga index recovery

  • Prepare for painless 2.0.0 upgrade

Improve replication

This release improves replication related features.

PGroonga supports replication for column that has NULL.

PGroonga adds a data synchronization point. PGroonga synchronizes replication data on not only SELECT but also INSERT.

Ubuntu 17.04 and CentOS 6 packages are replication ready.

See the official replication document for details.

Support token filters and plugins

You can specify token filters and plugins on creating a PGroonga index by CREATE INDEX USING pgroonga. You can use token_filters and plugins options for them.

You can tune full text search precision with this changes. For example, you can use a stemming token filter for English text. See the above documents for details.

Add auto PGroonga index recovery

PGroonga index may be broken when you force to kill PostgreSQL.

You can check PGroonga indexes and recover broken PGroonga indexes on startup by pgroonga_check module. If you may force to kill PostgreSQL, you may need to consider this module.

Prepare for painless 2.0.0 upgrade

PGroonga 2.0.0 will provide more flexible search features. But it will introduce incompatible changes.

If there are incompatibilities, upgrade will be painful. So we want to reduce the pain as much as possible.

Since this release, we start to backport 2.0.0 API to 1.x. With the work, you can use 2.0.0 API with 1.x. It means that you can migrate your application step by step.

In this release, the following 2.0.0 operators are backported:

You should use &@ and &? for newly written codes. If you use them, your codes will work with 2.0.0 as well. %% and @@ are deprecated.

2.0.0 will be released soon. We will provide 1.x compatible operators in 2.0.0 for painless upgrade.

How to upgrade

This version is compatible with 1.0 or later. You can upgrade by steps in "Compatible case" in Upgrade document.

Conclusion

New PGroonga version has been released.

See also release note for all changes.

Try PGroonga when you want to perform fast full text search against all languages on PostgreSQL!

2017-04-29

Groonga 7.0.2 has been released

Groonga 7.0.2 has been released!

How to install: Install

This release contains a fix to loose data by reindex command, please upgrade Groonga!

Changes

Here are important changes in this release:

  • Fixed a bug that data is lost by reindex
  • Supported Ubuntu 17.04 (Zesty Zapus)
  • Supported persistent cache
  • Added function to use an existing table as condition patterns.

Fixed a bug that data is lost by reindex

Since Groonga 5.1.0, reindex command is added, but there is a possibility that data is lost by this command.

Here is the sample to reproduce this bug.

> table_create User TABLE_HASH_KEY ShortText
[[0,1490948112.512871,0.004843950271606445],true]
> table_create Comment TABLE_NO_KEY
[[0,1490948112.517823,0.004474639892578125],true]
> column_create Comment user COLUMN_SCALAR User
[[0,1490948112.522444,0.00609898567199707],true]
> column_create User comment COLUMN_INDEX Comment user
[[0,1490948112.528664,0.009341239929199219],true]
> load --table Comment
> [
> {"user":"Mike"}
> ]
[[0,1490948112.538067,0.0004284381866455078],1]

> select User
[[0,1490948112.538533,0.0002899169921875],[[[1],[["_id","UInt32"],["_key","ShortText"],["comment","UInt32"]],[1,"Mike",1]]]]
> select Comment
[[0,1490948112.53886,4.744529724121094e-05],[[[1],[["_id","UInt32"],["user","User"]],[1,"Mike"]]]]

> reindex
[[0,1490948112.538933,0.02556586265563965],true]

> select User
[[0,1490948112.564538,5.91278076171875e-05],[[[0],[["_id","UInt32"],["_key","ShortText"],["comment","UInt32"]]]]]
> select Comment
[[0,1490948112.564621,3.457069396972656e-05],[[[1],[["_id","UInt32"],["user","User"]],[1,""]]]]

In above sample, the data of User table and Comment are lost after execution of reindex command.

Supported Ubuntu 17.04 (Zesty Zapus)

In this release, Ubuntu 17.04 (Zesty Zapus) is supported!

On the other hand, Ubuntu 12.04 reached EOL at April 28, 2017. So, it was dropped from supported platform.

Supported persistent cache

In this release, persistent cache feature is supported.

The cache of Groonga is lost after restarting, but by using this feature, there is no need to reconstruct cache from a scratch.

If you use groonga command, use --cache-base-path option.

This option specifies where to save persistent cache.

If you use groonga-httpd, use groonga_cache_base_path directive. Here is the sample configuration about groonga_cache_base_path.

http {
  ...
  groonga_cache_base_path /dev/shm/groonga-httpd-cache;
  ...
}

Persistent cache configuration is shared among workers. Thus the number of cache limit will be also shared in the future.

Here are some caution to use this feature:

  • Use base path on memory file system for performance reason.
  • Cache limit configuration is not supported yet.
  • Remove cache file when groonga has crashed.

Added function to use an existing table as condition patterns

In this release, ``in_records` function is added to use an existing table as condition patterns. Each record in the existing table is treated as a condition pattern.

By using in_records function, you may be able to reduce multiple queries to only one query by in_records.

Conclusion

See Release 7.0.2 2017-04-29 about detailed changes since 7.0.1.

Let's search by Groonga!

2017-03-29

Groonga 7.0.1 has been released

Groonga 7.0.1 has been released!

How to install: Install

Changes

Here are important changes in this release:

  • Dropped CentOS 5 support
  • Fixed a problem about dynamically updated index size
  • [experimental] Supported GRN_ORDER_BY_ESTIMATED_SIZE_ENABLE

Dropped CentOS 5 support

CentOS 5 will reach EOL at March 31, 2017. So, Groonga packages on CentOS 5 won't be shipped anymore.

We recommends to migrate CentOS 5 environment to CentOS 6 or later.

Fixed a problem about dynamically updated index size

Since Groonga 6.1.4 release, there is a problem about dynamically updated index size was increased.

Here is the example how Groonga database size is increased:

Database size comparison between dynamically updated index and statically updated index

This problem was fixed in this release!

[experimental] Supported GRN_ORDER_BY_ESTIMATED_SIZE_ENABLE

In this release, GRN_ORDER_BY_ESTIMATED_SIZE_ENABLE environment variable was supported.

This variable controls whether query optimization which is based on estimated size is applied or not. This feature is disabled by default.

Set GRN_ORDER_BY_ESTIMATED_SIZE_ENABLE=yes if you want to try it.

Conclusion

See Release 7.0.1 2017-03-29 about detailed changes since 7.0.0.

Let's search by Groonga!

2017-02-09

Groonga 7.0.0 has been released

Groonga 7.0.0 has been released! This is a major version up. But It keeps backward compatibility. You can upgrade to 7.0.0 without rebuilding database.

How to install: Install

Changes

Here are important changes in this release:

  • Removed needless metadata updates on DB. It reduces the case that database lock remains even though select command is executed.
  • lock_clear Changed to clear metadata lock by lock_clear against DB.

  • Supported groonga-token-filter-stem package which provides stemming feature by TokenFilterStem token filter on CentOS 7.
  • load Supported --output_errors yes option. If you specify "yes", you can get errors for each load failed record.

@naoa_y contributed many patches for this release, Thanks!

Conclusion

See Release 7.0.0 2017-02-09 about detailed changes since 6.1.5.

Let's search by Groonga!