"First Elasticsearch" starting with a python client

Overview

It is a memo that summarizes the trial from the installation of elasticsearch to the point before operation. The client uses a python client. With this, you can add and search data using python, so it will be easier to process the data.

Environmental setting

Elasticsearch installation

Since the environment was debian, bring the deb package and install it.

% wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.3.1.deb
% sudo dpkg -i elasticsearch-1.3.1.deb

Add the location of JAVA_HOME in your environment to the directory where you are looking for java in /etc/init.d/elasticsearch (added / usr / loca / java).

# The first existing directory is used for JAVA_HOME (if JAVA_HOME is not defined in $DEFAULT)
JDK_DIRS="/usr/local/java /usr/lib/jvm/java-7-oracle /usr/lib/jvm/java-7-openjdk /usr/lib/jvm/java-7-openjdk-amd64/ /usr/lib/jvm/java-7-openjdk-armhf /usr/lib/jvm/java-7-openjdk-i386/ /usr/lib/jvm/default-java"

Start-up

Start-up

% sudo /etc/init.d/elasticsearch start

Verification

Try accessing http: // localhost: 9200 and confirm that the following is returned. Access the address from a browser to check it, or access it from the command line as shown below.

% curl -XGET http://localhost:9200/                                                       

response

{
  "status" : 200,
  "name" : "White Rabbit",
  "version" : {
    "number" : "1.3.1",
    "build_hash" : "2de6dc5268c32fb49b205233c138d93aaf772015",
    "build_timestamp" : "2014-07-28T14:45:15Z",
    "build_snapshot" : false,
    "lucene_version" : "4.9"
  },
  "tagline" : "You Know, for Search"
}

Try using plugin

Install plugin

Since it is inconvenient if there is nothing, the standard elasticsearch-head is included. This is enough for the minimum. To install, just hit the plugin command that comes with elasticsearch. Nice!

If you can access http: // localhost: 9200 / _plugin / head, you are successful.

Installation

% /usr/share/elasticsearch/bin/plugin -install mobz/elasticsearch-head

The screen looks like this.

use python client

Official: elasticsearch-py Documentation: Python Elasticsearch Client

Prepare a python environment

I have python 2.6.6 in my calculator. Since I have never used python, I decided to put pip for package management and virtualenv for switching the environment by referring to the googled page.

Reference: Pip installation became easier before I knew it

Quoted from "Pip installation became easier before I knew it"

curl -kL https://raw.github.com/pypa/pip/master/contrib/get-pip.py | python
pip install virtualenv virtualenvwrapper
vi ~/.bashrc
#Add the following 3 lines
export WORKON_HOME=$HOME/.virtualenvs
export PROJECT_HOME=$HOME/work
source /path/to/your/virtualenvwrapper.sh

Install python client and launch interactively

Installation

Installation

pip install elasticsearch

Start-up

• I want to speak Japanese in the terminal, so start python in interactive mode while setting LANG
• import elasticsearch
• Prepare an instance of elasticsearch

Start-up

% LANG=ja_JP.UTF8 python
Python 2.6.6 (r266:84292, Dec 27 2010, 00:02:40)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from elasticsearch import Elasticsearch
>>> es = Elasticsearch("localhost:9200")
>>> es
<Elasticsearch([{'host': 'localhost', 'port': 9200}])>

Add document to index

API:elasticsearch.Elasticsearch.index

Add the following data. index and doc_type are required items. If the specified index does not exist, a new index will be created. If you do not specify the id, it will be numbered appropriately.

• index: fruit (required item)
• doc_type: test (required)
• id: 1
• body (required item)
• name:apple, color:red

add to

>>> es.index(index="fruit", doc_type="test", id=1, body={"name":"apple", "color":"red"})
{u'_type': u'test', u'_id': u'1', u'created': True, u'_version': 1, u'_index': u'fruit'}

How the index was created

Added documentation

Try various operations

Change / Add

Overwrite if the same id is specified.

Overwrite

>>> es.index(index="fruit", doc_type="test", id=1, body={"name":"apple", "color":"green"})
{u'_type': u'test', u'_id': u'1', u'created': False, u'_version': 2, u'_index': u'fruit'}

If id is not specified, the numbers will be assigned appropriately. In the following, id = dnMiX8ufSiiZC_c8KwykuQ.

If id is not specified

>>> es.index(index="fruit", doc_type="test", body={"name":"Apple", "color":"red"})
{u'_type': u'test', u'_id': u'dnMiX8ufSiiZC_c8KwykuQ', u'created': True, u'_version': 1, u'_index': u'fruit'}

Data acquisition

Premise

The following data is set in the index called fruit. The document type is set to test.

When id can be specified

Specified by id, return value is apple,Expect green

>>> res = es.get(index="fruit", doc_type="_all", id="MBEGluC5S-OzNdGoDYavGg")
>>> print json.dumps(res, indent=4)
{
    "_type": "test",
    "_source": {
        "color": "green",
        "name": "apple"
    },
    "_index": "fruit",
    "_version": 1,
    "found": true,
    "_id": "MBEGluC5S-OzNdGoDYavGg"
}

When you want to specify in a query

Bring them all

>>> res = es.search(index="fruit", body={"query": {"match_all": {}}})
>>> print json.dumps(res, indent=4)
{
    "hits": {
        "hits": [
            {
                "_score": 1.0,
                "_type": "test",
                "_id": "3MH8LiCNSkOgZMwx_kNebw",
                "_source": {
                    "color": "red",
                    "name": "apple"
                },
                "_index": "fruit"
            },
            {
                "_score": 1.0,
                "_type": "test",
                "_id": "mz1wlxRUSSWvCuIIh6k4OQ",
                "_source": {
                    "color": "orange",
                    "name": "orange"
                },
                "_index": "fruit"
            },
            {
                "_score": 1.0,
                "_type": "test",
                "_id": "9qsreGQTTMSIsMzlEe0H0A",
                "_source": {
                    "color": "red",
                    "name": "\u308a\u3093\u3054"
                },
                "_index": "fruit"
            },
            {
                "_score": 1.0,
                "_type": "test",
                "_id": "MBEGluC5S-OzNdGoDYavGg",
                "_source": {
                    "color": "green",
                    "name": "apple"
                },
                "_index": "fruit"
            },
            {
                "_score": 1.0,
                "_type": "test",
                "_id": "YXAo8TfrQbeF3JQpW6dakw",
                "_source": {
                    "color": "yellow",
                    "name": "banana"
                },
                "_index": "fruit"
            }
        ],
        "total": 5,
        "max_score": 1.0
    },
    "_shards": {
        "successful": 5,
        "failed": 0,
        "total": 5
    },
    "took": 3,
    "timed_out": false
}

Search with conditions

Try searching for one with color = red.

>>> res = es.search(index="fruit", body={"query": {"match": {"color":"red"}}})
>>> print json.dumps(res, indent=2 , ensure_ascii=False)
{
  "hits": {
    "hits": [
      {
        "_score": 0.30685282000000003,
        "_type": "test",
        "_id": "3MH8LiCNSkOgZMwx_kNebw",
        "_source": {
          "color": "red",
          "name": "apple"
        },
        "_index": "fruit"
      },
      {
        "_score": 0.30685282000000003,
        "_type": "test",
        "_id": "9qsreGQTTMSIsMzlEe0H0A",
        "_source": {
          "color": "red",
          "name": "Apple"
        },
        "_index": "fruit"
      }
    ],
    "total": 2,
    "max_score": 0.30685282000000003
  },
  "_shards": {
    "successful": 5,
    "failed": 0,
    "total": 5
  },
  "took": 2,
  "timed_out": false
}

Delete index

The fruit index disappears cleanly.

>>> es.indices.delete(index="fruit")
{u'acknowledged': True}