[PYTHON] Elasticsearch installation and basic operation for ubuntu

Elasticsearch installation and basic operation

Installation (on Ubuntu 18.04)

ElasticSearch Install

install Java $ sudo apt install openjdk-11-jdk

install Elasticsearch $ wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add - $ echo "deb https://artifacts.elastic.co/packages/6.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-6.x.list $ sudo apt update $ sudo apt install elasticsearch

$ sudo systemctl start elasticsearch $ sudo systemctl status elasticsearch

Verification $ curl http://127.0.0.1:9200

Kibana Install $ apt update $ apt install kibana

$ systemctl start kibana

Verification access http://localhost:5601

kuromoji Install $ sudo /usr/share/elasticsearch/bin/elasticsearch-plugin install analysis-kuromoji

CRUD edition

Create an index called book and create one document.

PUT /book/_doc/1
{
	"title":"Business Python Super Primer",
	"author":"Seigo Nakajima",
	"publisherName":"Nikkei BP",
	"salesDate":"June 7, 2019",
	"itemPrice":"2592"
}
Automatically generated without specifying a document ID

PUT /book/_doc
{
	"title":"Business Python Super Primer",
	"author":"Seigo Nakajima",
	"publisherName":"Nikkei BP",
	"salesDate":"June 7, 2019",
	"itemPrice":"2592"
}
* Document ID will be 20-digit random alphanumeric characters
Get documentation

GET /book/_doc/1
Update documentation

POST /book/_update/1
{
  "doc": {
    "itemPrice": 1292
  }
}
Delete document

DELETE /book/_doc/1
Delete index

GET /book/_mapping
Check the mapping (like a schema)

GET /book/_mapping
Create an index

PUT /book
Create index by specifying mapping

PUT /book
{
  "mappings": {
    "properties": {
      "title": { "type": "keyword" },
      "author": { "type": "keyword" },
      "publisherName": { "type": "keyword" },
      "isbn" : { "type": "keyword" },
      "itemCaption" : { "type": "text" },
      "itePrice" : { "type": "long"}
    }
  }
}
Create multiple documents at once

POST /book/_bulk 
{"index": {"_id": 1}} 
{ "title": "Let Python do the boring things", "author": "Al Sweigart/Aizo Aikawa","publisherName":"O'Reilly Japan", "isbn":" 9784873117782", "itemCaption":"Tasks such as renaming files and updating spreadsheet data often occur in daily work. There is no problem if you only fix one or two, but when it becomes tens or hundreds, it is uncontrollable. It would be much easier to have a computer take over such a simple repetitive task. In this book, you will learn how to create a Python3 program that can handle processes that would take an enormous amount of time by hand in an instant. The target audience is non-programmers. If you master the basics in this book, even inexperienced programming users will be able to create convenient programs that can easily perform troublesome simple tasks. In addition, you can improve your skills in automatically processing similar tasks by solving the exercises at the end of the chapter.", "itemPrice": 3996 }
{"index": {"_id": 2}} 
{ "title":"Business Python Super Primer","author":"Seigo Nakajima","publisherName":"Nikkei BP","isbn":" 9784296102136","itemCaption":"You can learn basic programming skills that are indispensable for business! Explains Python, a language of interest with artificial intelligence, from the beginning. Step by step from writing to execution procedure. Steadily master essential syntax such as conditional branching and repetition. Automatic acquisition of online information, the basis of web scraping. Experience machine learning that recognizes handwritten characters from scratch.","itemPrice": 2592} 
{"index": {"_id": 3}} 
{ "title":"Getting Started Python","author":"Bill Rubanovic/Yasushi Saito","publisherName":"O'Reilly Japan"," isbn":" 9784873117386","itemCaption":"It's been a quarter of a century since Python was born. Python's popularity is skyrocketing in a variety of areas, including data science, web development, and security. Even in the field of programming education, the adoption of Python instead of C is increasing. This book is an introduction to Python, written for those who are new to programming. There is no particular knowledge to assume. From the basics of programming and Python to applications such as web, database, network, and concurrency, we will explain Python programming in an easy-to-understand manner.","itemPrice": 3996}
{"index": {"_id": 4}} 
{ "title":"Introduction to new and clear Python","author":"Nobuhiro Shibata","publisherName":"SB creative","isbn":"9784815601522","itemCaption":"With 299 appropriate sample programs and 165 easy-to-understand charts, you can systematically and steadily learn the basics of Python grammar and programming.","itemPrice": 2808}
{"index": {"_id": 5}} 
{ "title":"Self-study programmer From the basics of the Python language to the way you work","author":"Corey Arsov/Takayuki Shimizukawa"," publisherName":"Nikkei BP"," isbn":"9784822292270"," itemCaption":"Object-oriented, shell, regular expressions, package management, version control, data structures, algorithms, how to find and do jobs. From the basics of the Python language to the way of working, you can understand the knowledge and know-how necessary to work as a programmer in one book.","itemPrice": 2376}
Add item to document

POST /book/_update/5
{
  "doc": {
    "discountRate": 0
  }
}

Search

Search all documents

GET /book/_searh

GET book/_search
{
    "query": {
    "match_all": {}
  }
}
Search for documents that contain the specified word

GET book/_search
{
  "query": {
    "match": {
      "itemCaption": "programming"
    }
  }
}
Search documents by and condition

GET book/_search
{
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "itemCaption": "programming"
          }
        },
        {
          "match": {
            "itemCaption": "getting started"
          }
        }
      ]
    }
  }
}
not search (does not contain a word)

GET book/_search
{
  "query": {
    "bool": {
      "must_not": [
        {
          "match": {
            "itemCaption": "getting started"
          }
        }
      ]
    }
  }
}
or search

GET book/_search
{
  "query": {
    "bool": {
      "should": [
        {
          "match": {
            "itemCaption": "programming"
          }
        },
        {
          "match": {
            "itemCaption": "getting started"
          }
        }
      ]
    }
  }
}
Search by weighting specific words

GET book/_search
{
  "query": {
    "bool": {
      "should": [
        {
          "match": {
            "itemCaption": {
              "query": "programming"
            }
          }
        },
        {
          "match": {
            "itemCaption": {
              "query": "getting started",
              "boost": 5
            }
          }
        }
      ]
    }
  }
}
Highlight search

GET book/_search
{
  "query": {
    "match": {
      "itemCaption": "programming"
    }
  },
  "highlight": {
    "fields": {
      "itemCaption": {}
    }
  }
}
Narrow down search and sort by range

GET book/_search
{
  "query": {
    "range": {
      "itemPrice": {
        "lte": 3000
      }
    }
  },
  "sort": [
    {
      "itePrice": {
        "order": "asc"
      }
    }
  ]
}
Create statistical data using aggregation

Aggregate the number of items for each price range of data including programming


GET book/_search
{
  "query": {
    "match": {
      "itemCaption": "programming"
    }
  }, 
  "aggregations": {
    "itemPrice_bucket": {
      "range": {
        "field": "itemPrice",
        "ranges": [
          {
            "key": "0-1500", 
            "from": 0,
            "to": 1500
          },
          {
            "key": "1501-3000",
            "from": 1501,
            "to": 3000
          },
          {
            "key": "3000-4500",
            "from": 3001,
            "to": 4500
          }
        ]
      }
    }
  }
}

Recommended Posts

Elasticsearch installation and basic operation for ubuntu
Python installation and basic grammar
Django installation and operation check
Python (Python 3.7.7) installation and basic grammar
PySpark 1.5.2 + Elasticsearch 2.1.0 Installation procedure and execution
Ubuntu 20.04 LTS installation method (for external drive)
Python basic operation 3rd: Object-oriented and class
Linux operation for beginners Basic command summary
BESS Development Memo # 01: BESS Installation and Basic Usage
[Explanation for beginners] TensorFlow basic syntax and concept
From ROS for Windows installation to operation check
[Python / Chrome] Basic settings and operations for scraping
Basic operation of Python Pandas Series and Dataframe (1)
Organizing basic procedures for data analysis and statistical processing (4)
Organizing basic procedures for data analysis and statistical processing (2)
Installing Python 3 on Mac and checking basic operation Part 1
Web system construction (super basic) ③: DB server construction and basic operation
Dealing with pip and related installation errors on Ubuntu 18.04
Web system construction (super basic) ②: AP server construction and basic operation
Python basic course (2 Python installation)
jupyter and pandas installation
Basic operation of pandas
FX_tool for Hython Basic02
Basic operation of Pandas
ubuntu20.04 + Geth installation procedure
MongoDB installation (Ubuntu 18 LTS)
FX_tool for Hython Basic01
Installation procedure for Python and Ansible with a specific version
Summary of Hash (Dictionary) operation support for Ruby and Python