[PYTHON] Lösung, wenn der elastische Suchkurator den DistributionNotFound-Fehler in EC2 auslöst

Was ist Kurator (Elasticsearch-Kurator)?

Elasticsearch Index-Wartungstool, das offiziell von Elasticsearch entwickelt wurde.

Fehler und Lösungen

Installation des Kurators

$ python --version
Python 2.6.9

$ pip --version
pip 1.5.6 from /usr/lib/python2.6/site-packages (python 2.6)

# 2014/Die neueste Version ab 12 ist v2.0.2
#Bis zur Installation ohne Probleme abgeschlossen
$ sudo pip install elasticsearch-curator==2.0.2

DistributionNotFound-Fehler

Ein Fehler trat auf, als ich versuchte, den Vorgang nach Abschluss der Installation zu überprüfen.

$ curator -v
Traceback (most recent call last):
  File "/usr/bin/curator", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 2655, in <module>
    working_set.require(__requires__)
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 648, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 546, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: elasticsearch>=1.0.0,<2.0.0

Der Inhalt der Fehlermeldung ist, dass das Elasticsearch-Modul von Python nicht gefunden werden kann, aber zusammen als abhängiges Modul von Elasticsearch-Curator installiert werden kann.

$ pip list | grep elasticsearch
elasticsearch (1.2.0)
elasticsearch-curator (2.0.2)

Ähnliche Probleme wurden bei Github und Google Group gemeldet, aber keines scheint behoben worden zu sein ...

Tatsächlich habe ich auch versucht, das Elasticsearch-Modul neu zu installieren, aber es hat nicht funktioniert.

Lösung

Als letztes Mittel unter https://github.com/elasticsearch/curator/issues/56#issuecomment-51628636 erwähnt

At worst, you'll have to run it from /Library/Python/2.7/site-packages/curator/curator.py (which is where pip installs it on mine).

Versuchen.

(Laut https://github.com/elasticsearch/curator/issues/56#issuecomment-56980489 können Sie im v2-System ** curator_script.py ** anstelle von curator.py verwenden.)

Zuerst im ** - Trockenlauf ** -Modus

$ python /usr/lib/python2.6/site-packages/curator/curator_script.py -v
curator_script.py 2.0.2

# DRY RUN
$ python /usr/lib/python2.6/site-packages/curator/curator_script.py --host localhost --port 9200 --logfile /tmp/curator.log --loglevel DEBUG --dry-run delete --older-than 30

$ less /tmp/curator.log
2014-12-16 04:00:58,990 INFO      Job starting...
2014-12-16 04:00:58,990 INFO      DRY RUN MODE.  No changes will be made.
2014-12-16 04:00:59,187 DEBUG     Detected Elasticsearch version 1.4.1
2014-12-16 04:00:59,187 DEBUG     Setting default timestring for days to %Y.%m.%d
2014-12-16 04:00:59,187 DEBUG     Matching indices with pattern: logstash-%Y.%m.%d
2014-12-16 04:00:59,187 DEBUG     argdict = {'url_prefix': '', 'func': <function delete at 0x262c758>, 'prefix': 'logstash-', 'log_level': 'DEBUG', 'timestring': '%Y.%m.%d', 'dry_run': True, 'exclude_pattern': None, 'logformat': 'Default', 'auth': None, 'ssl': False, 'host': 'localhost', 'command': 'delete', 'time_unit': 'days', 'timeout': 30, 'debug': False, 'disk_space': None, 'log_file': '/tmp/curator.log', 'master_only': False, 'port': 9200, 'older_than': 30, 'suffix': ''}
2014-12-16 04:00:59,188 INFO      DRY RUN: Deleting indices...
2014-12-16 04:00:59,394 INFO      logstash-2014.12.11 is within the threshold period (30 days).
2014-12-16 04:00:59,420 INFO      logstash-2014.12.12 is within the threshold period (30 days).
2014-12-16 04:00:59,420 INFO      logstash-2014.12.13 is within the threshold period (30 days).
2014-12-16 04:00:59,420 INFO      logstash-2014.12.14 is within the threshold period (30 days).
2014-12-16 04:00:59,420 INFO      logstash-2014.12.15 is within the threshold period (30 days).
2014-12-16 04:00:59,420 INFO      logstash-2014.12.16 is within the threshold period (30 days).
2014-12-16 04:00:59,421 INFO      DRY RUN: Speficied indices deleted.
2014-12-16 04:00:59,421 INFO      Done in 0:00:00.516716.

Da es gut zu sein scheint, führen Sie es für den tatsächlichen elastischen Suchindex aus ...

Bloom-Befehl

$ python /usr/lib/python2.6/site-packages/curator/curator_script.py --host localhost --port 9200 --logfile /tmp/curator.log --loglevel DEBUG bloom --older-than 3

$ less /tmp/curator.log
2014-12-17 01:00:04,630 INFO      Job starting...
2014-12-17 01:00:04,646 DEBUG     Detected Elasticsearch version 1.4.1
2014-12-17 01:00:04,646 DEBUG     Setting default timestring for days to %Y.%m.%d
2014-12-17 01:00:04,646 DEBUG     Matching indices with pattern: logstash-%Y.%m.%d
2014-12-17 01:00:04,646 DEBUG     argdict = {'url_prefix': '', 'prefix': 'logstash-', 'log_level': 'DEBUG', 'timestring': '%Y.%m.%d', 'dry_run': False, 'exclude_pattern': None, 'logformat': 'Default', 'auth': None, 'ssl': False, 'host': 'localhost', 'command': 'bloom', 'time_unit': 'days', 'timeout': 30, 'debug': False, 'func': <function bloom at 0xc49668>, 'log_file': '/tmp/curator.log', 'master_only': False, 'port': 9200, 'older_than': 3, 'suffix': ''}
2014-12-17 01:00:04,647 INFO      Disabling the bloom filter cache for indices...
2014-12-17 01:00:04,895 INFO      disable_bloom_filter operation succeeded on logstash-2014.12.11
2014-12-17 01:00:05,106 INFO      disable_bloom_filter operation succeeded on logstash-2014.12.12
2014-12-17 01:00:05,428 INFO      disable_bloom_filter operation succeeded on logstash-2014.12.13
2014-12-17 01:00:05,888 INFO      disable_bloom_filter operation succeeded on logstash-2014.12.14
2014-12-17 01:00:05,888 INFO      logstash-2014.12.15 is within the threshold period (3 days).
2014-12-17 01:00:05,888 INFO      logstash-2014.12.16 is within the threshold period (3 days).
2014-12-17 01:00:05,888 INFO      logstash-2014.12.17 is within the threshold period (3 days).
2014-12-17 01:00:05,888 INFO      Disabled bloom filter cache for specified indices.
2014-12-17 01:00:05,889 INFO      Done in 0:00:01.415160.

Befehl schließen

$ python /usr/lib/python2.6/site-packages/curator/curator_script.py --host localhost --port 9200 --logfile /tmp/curator.log --loglevel DEBUG close --older-than 4

$ less /tmp/curator.log
2014-12-17 01:00:06,946 INFO      Job starting...
2014-12-17 01:00:07,024 DEBUG     Detected Elasticsearch version 1.4.1
2014-12-17 01:00:07,024 DEBUG     Setting default timestring for days to %Y.%m.%d
2014-12-17 01:00:07,024 DEBUG     Matching indices with pattern: logstash-%Y.%m.%d
2014-12-17 01:00:07,025 DEBUG     argdict = {'url_prefix': '', 'prefix': 'logstash-', 'log_level': 'DEBUG', 'timestring': '%Y.%m.%d', 'dry_run': False, 'exclude_pattern': None, 'logformat': 'Default', 'auth': None, 'ssl': False, 'host': 'localhost', 'command': 'close', 'time_unit': 'days', 'timeout': 30, 'debug': False, 'func': <function close at 0x2b0c6e0>, 'log_file': '/tmp/curator.log', 'master_only': False, 'port': 9200, 'older_than': 4, 'suffix': ''}
2014-12-17 01:00:07,025 INFO      Closing indices...
2014-12-17 01:00:08,457 INFO      close_index operation succeeded on logstash-2014.12.11
2014-12-17 01:00:08,841 INFO      close_index operation succeeded on logstash-2014.12.12
2014-12-17 01:00:09,225 INFO      close_index operation succeeded on logstash-2014.12.13
2014-12-17 01:00:09,225 INFO      logstash-2014.12.14 is within the threshold period (4 days).
2014-12-17 01:00:09,226 INFO      logstash-2014.12.15 is within the threshold period (4 days).
2014-12-17 01:00:09,226 INFO      logstash-2014.12.16 is within the threshold period (4 days).
2014-12-17 01:00:09,226 INFO      logstash-2014.12.17 is within the threshold period (4 days).
2014-12-17 01:00:09,226 INFO      Closed specified indices.
2014-12-17 01:00:09,226 INFO      Done in 0:00:02.293359.

Befehl löschen

$ python /usr/lib/python2.6/site-packages/curator/curator_script.py --host localhost --port 9200 --logfile /tmp/curator.log --loglevel DEBUG delete --older-than 5

$ less /tmp/curator.log
2014-12-17 01:00:10,375 INFO      Job starting...
2014-12-17 01:00:10,382 DEBUG     Detected Elasticsearch version 1.4.1
2014-12-17 01:00:10,382 DEBUG     Setting default timestring for days to %Y.%m.%d
2014-12-17 01:00:10,382 DEBUG     Matching indices with pattern: logstash-%Y.%m.%d
2014-12-17 01:00:10,382 DEBUG     argdict = {'url_prefix': '', 'func': <function delete at 0x2578758>, 'prefix': 'logstash-', 'log_level': 'DEBUG', 'timestring': '%Y.%m.%d', 'dry_run': False, 'exclude_pattern': None, 'logformat': 'Default', 'auth': None, 'ssl': False, 'host': 'localhost', 'command': 'delete', 'time_unit': 'days', 'timeout': 30, 'debug': False, 'disk_space': None, 'log_file': '/tmp/curator.log', 'master_only': False, 'port': 9200, 'older_than': 5, 'suffix': ''}
2014-12-17 01:00:10,382 INFO      Deleting indices...
2014-12-17 01:00:11,632 INFO      delete_index operation succeeded on logstash-2014.12.11
2014-12-17 01:00:11,727 INFO      delete_index operation succeeded on logstash-2014.12.12
2014-12-17 01:00:11,727 INFO      logstash-2014.12.13 is within the threshold period (5 days).
2014-12-17 01:00:11,727 INFO      logstash-2014.12.14 is within the threshold period (5 days).
2014-12-17 01:00:11,727 INFO      logstash-2014.12.15 is within the threshold period (5 days).
2014-12-17 01:00:11,728 INFO      logstash-2014.12.16 is within the threshold period (5 days).
2014-12-17 01:00:11,728 INFO      logstash-2014.12.17 is within the threshold period (5 days).
2014-12-17 01:00:11,728 INFO      Speficied indices deleted.
2014-12-17 01:00:11,728 INFO      Done in 0:00:01.437909.

Es ist fertig! ٩ (๑´3`๑) ۶

Bonus

Es ist eine gute Idee, den Alias festzulegen, damit auch der Speicherort der Python-Site-Pakete dynamisch bestimmt wird.

alias curator="python "`python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"`/curator/curator_script.py

Recommended Posts

Lösung, wenn der elastische Suchkurator den DistributionNotFound-Fehler in EC2 auslöst
Lösung, wenn Heroku bei Verwendung von Selen abstürzt