PGroonga (fast full text search module for PostgreSQL) 1.1.9 has been released

BloGroonga

2016-11-30

PGroonga (fast full text search module for PostgreSQL) 1.1.9 has been released

PGroonga (píːzí:lúnɡά) (fast full text search module for PostgreSQL) 1.1.9 has been released!

See PGroonga 1.0.0 release announce about PGroonga.

Benchmark

We have published benchmark result with Wikipedia data to the official site.

Here are compared modules:

textsearch in PostgreSQL
pg_trgm in contrib
pg_bigm - third party module

English Wikipedia data are used for textsearch and pg_trgm because they don't support Asian languages such as Japanese.

Japanese Wikipedia data are used for pg_bigm because it supports Asian languages such as Japanese.

PGroonga, textsearch and pg_trgm

This section describes about benchmark result summary with PGroonga, textsearch and pg_trgm. See PGroonga versus textsearch and pg_trgm for details.

Here is search time benchmark result.

pg_trgm is very slower than PGroonga and textsearch as the following. (Bar length means elapsed time in ms. It means that shorter bar means more faster.)

Here is the graph without pg_trgm. PGroonga and textsearch have similar performance. "Groonga" is a case that search by Groonga directly by pgroonga.command function. Groonga is the full text search engine used by PGroonga. Groonga is 10x faster than PGroonga and textsearch.

Here is index creation time benchmark.

PGroonga is the most fastest module as the following. PGroonga is about 2x faster than textsearch and about 30% faster than pg_trgm.

Index creation time by PGroonga, textsearch and pg_trgm

PGroonga and pg_bigm

This section describes about benchmark result summary with PGroonga and pg_bigm. See PGroonga versus pg_bigm for details.

Here is search time benchmark result.

pg_bigm is very slower than PGroonga when query contains 3 or more characters. (Bar length means elapsed time in ms. It means that shorter bar means more faster.)

Here is index creation time benchmark.

PGroonga is about 75% faster than pg_bigm as the following.

Index creation time by PGroonga and pg_bigm

Changes

Here are changes since 1.1.1. We recommend you to upgrade to 1.1.9 because some search related problems are fixed and new features such as replication support and Zstandard support are implemented. See links in the list for details.

[Experimental] Supported WAL. PGroonga isn't crash safe but supports replication. PostgreSQL 9.6 or later and MessagePack are needed.
[Windows] Fixed a bug that SELECT may be crashed.
Supported Windows-1252 encoding.
[Windows] Fixed a bug that CREATE EXTENSION pgroonga is failed.
[Windows] Upgraded bundled Groonga to 6.1.1.
[Windows] Supported packages for both PostgreSQL 9.5.5 and 9.6.1.
Supported integer in IN.
Fixed a bug that some records may be found after UPDATE or DELETE. (Reported by yongxianggao-chanjet)
Fixed a memory leak of pgroonga.score function (Reported by Takahashi)
Fixed a bug that PGroonga indexes created in non default tablespace are removed on VACUUM. (Reported by yongxianggao-chanjet)
Supported tablespace.
Disabled index only scan when there are one or more large record.
[Ubuntu] Dropped Wily Werewolf (15.10) support.
[Ubuntu] Supported Yakkety Yak (16.10).
Supported text data compression by Zstandard.
Supported text data compression by zlib.
Supported PGroonga version logging on startup.
Added pgroonga_tuple_is_alive Groonga function. You can remove dead tuples in PostgreSQL by this Groonga function. You can use this Groonga function with groonga.command function.
Reduced memory usage on CREATE INDEX.
Supported backtrace logging on crash. (It doesn't work on Windows.)
Supported logging with PID.
Supported optimization when the following functions are called with constant arguments:
- pgroonga.table_name function
- pgroonga.snippet_html function
- pgroonga.highlight_html function
- pgroonga.match_positions_byte function
- pgroonga.match_positions_character function
- pgroonga.query_extract_keywords function
Added pgroonga.escape function. It escapes script syntax value.
Added pgroonga.query_escape function. It escapes query syntax value.
Added pgroonga.command_escape_escape function. It escaped Groonga command argument value.
Changed the return type of pgroonga.table_name function to text from cstring.
Added a new call style to pgroonga.command function. You can pass Groonga command arguments as an array. If you specify arguments an array, arguments are escaped automatically. It prevents Groonga command injection security risk.
Supported query log. Use pgroonga.query_log_path parameter to control query log location and enabling/disabling query log.

How to upgrade

This version is compatible with 1.1.1 or later. Upgrade by steps in "Compatible case" in Upgrade document.

Announce

There is PGroonga session in PGConf.ASIA 2016 DAY 2. In this session, not only difference between PGroonga and other modules but also how to implement replication by generic WAL API added since PostgreSQL 9.6 are described. The slide for the session has been published:

PGroonga - Make PostgreSQL fast full text search platform for all languages!

If you're interested in PGroonga, please go to the session.

Conclusion

New PGroonga version has been released.

Try PGroonga when you want to perform fast full text search against all languages on PostgreSQL!

BloGroonga