BloGroonga

2012-10-29

Groonga fulltext search engine meetup

We hold a event groonga fulltext search engine session next month!

  • Location: VOYAGE GROUP, Inc. Shibuya First Place Bldg. 8F Seminar room Shinsen-cho 8-16, Shibuya, Tokyo Google Maps
  • Date: 2012/11/29 19:00 - 21:00 JST (Open: 18:30, Party starts 21:00)
  • Reservation: via ATND event page

See ATND event page about event details.

Now, we are looking for the speakers! Deadline for submission is Nov 04 23:59, 2012 (JST). There is a benefit for the speaker that fee of party is free.

NOTE: ATND is the web site which support to announce event and keep track of who's attending.

Groonga 2.0.8 has been released

Groonga 2.0.8 has been released!

How to install: Install

There are two topics for this release.

  • Supported custom query expansion by plugin
  • Supported Ubuntu 12.10 (Quantal Quetzal)

Supported custom query expansion by plugin

This release began to support custom query expansion by plugin.

Groonga already supports query expansion, but it requires table which stores synonyms beforehand.

Here is the more concrete example.

Schema definition:

  table_create Memos TABLE_NO_KEY
  column_create Memos content COLUMN_SCALAR ShortText
  table_create Lexicon TABLE_PAT_KEY ShortText --default_tokenizer TokenBigram
  column_create Lexicon diary_content COLUMN_INDEX|WITH_POSITION Memos content
  table_create Synonyms TABLE_PAT_KEY ShortText
  column_create Synonyms words COLUMN_SCALAR ShortText

Sample data:

  load --table Memos
  [
  ["content"],
  ["Start groonga!"],
  ["Start mroonga!"],
  ["Start rroonga!"],
  ["Start Ruby!"],
  ["Learning Ruby and groonga..."],
  ]

  load --table Synonyms
  [
  ["_key", "words"],
  ["groonga", "(groonga OR rroonga OR mroonga)"]
  ]

If you want to search 'groonga' with expanded query 'rroonga' and 'mroonga', you need to setup synonyms table 'Synonyms' at first. Then, you can search 'groonga' with automatically expanded keyword 'rroonga' and 'mroonga' from content column of Memos table.

Here is the query to search 'groonga' with query expansion.

  > select --table Memos --query_expansion Synonyms.words --match_columns content --query groonga

  [[0,1350898793.34538,0.00174403190612793],
   [
    [[4],
     [
      ["_id","UInt32"],["content","ShortText"]],
      [1,"Start groonga!"],
      [5,"Learning Ruby and groonga..."],
      [3,"Start rroonga!"],
      [2,"Start mroonga!"]
    ]
   ]
  ]

On the other hand, custom query expansion by plugin has supported. As proof of concept, example plugin QueryExpanderTSV is bundled in this release.

Here is the more concrete example how to use QueryExpanderTSV.

Using QueryExpanderTSV, you doesn't need to setup table for synonyms anymore. There are two differences between builtin query expansion and QueryExpanderTSV.

  • Describe tab separated text file
  • Specify plugin name to --query_expansion

Here is the example about tab separated file with synonyms.

  groonga[TAB]groonga[TAB]rroonga[TAB]mroonga

Here is the query to search 'groonga' with QueryExpanderTSV.

  > select --table Memos --query_expansion QueryExpanderTSV --match_columns content --query groonga

This query is equivalent to following:

  --query "((groonga) OR (rroonga) OR (mroonga))"

The synonyms template file is installed as /etc/groonga/synonyms.tsv. You can customize this file on your purpose.

If you want to put somewhere else, set GRN_QUERY_EXPANDER_TSV_SYNONYMS_FILE environment variable.

Supported Ubuntu 12.10 (Quantal Quetzal)

Ubuntu 12.10 Quantal Quetzal had been released Oct 18, 2012.

This release began to support Ubuntu 12.10 by providing deb packages.

Conclusion

See Release 2.0.8 2012/10/29 about detailed changes since 2.0.7.

Let's search by groonga!