Try converting cloudmonkey CLI to python3 -1

Unfortunately, Python 2 is also available in the CLI tool CloudStack cloudmonkey CLI that makes AWS and Cloudn available as APIs. So, let's make it work with Python3 for studying.

** Not successful yet, using as a memo. ** **

Python 3 conversion

After a little research, it seems that there are generally two patterns, "Polyglot" that will continue to work with python2, and "Port" that does not support Python2 after that.

Direction I'm not using python2, so I'll use ** Port **.

Documents about Python3 conversion

diveintopython

Python 3 conversion tool

It seems that the transition from 2 to 3 is common, and I quickly found some tools. 2to3 Futurize Six

Work history

Create a working directory and move to it

$ mkdir porting
$ cd porting

Prepare a working environment

$ pyvenv ven
$ source ven/bin/activate

The documentation required the following packages, so install

readline
requests
Pygments
prettytable
 
argcomplete
$ pip install --upgrade readline requests Pygments prettytable argcomplete

$ pip freeze

argcomplete==1.4.1 prettytable==0.7.2 Pygments==2.1.3 readline==6.2.4.1 requests==2.11.1


Install future to add to them and port
$ pip install future

##Clone the code and move there
$ git clone https://github.com/apache/cloudstack-cloudmonkey.git
cd cloudstack-cloudmonkey

Contents is like this

```sh
$ ls

CHANGES.md        LICENSE           NOTICE            cloudmonkey       docs              setup.cfg
Dockerfile        Makefile          README.md         config.docker     performrelease.sh setup.py

##I will install it immediately

$ python setup.py develop
  File "setup.py", line 50
    print "If you're upgrading, run the following to enable parameter completion:"
                                                                                 ^
SyntaxError: Missing parentheses in call to 'print'

Well, it will fail. I will reread the document later, futurize--stage1 futurize --It seems that when you run stage2, it will be rewritten to python3 code. Setup so that it can be installed first.Convert py to pytho3 code

$ futurize --stage1 -w setup.py

RefactoringTool: Skipping optional fixer: idioms
RefactoringTool: Skipping optional fixer: ws_comma
RefactoringTool: Refactored setup.py
--- setup.py   	(original)
+++ setup.py   	(refactored)
@@ -1,3 +1,4 @@
+from __future__ import print_function
 # Licensed to the Apache Software Foundation (ASF) under one
 # or more contributor license agreements.  See the NOTICE file
 # distributed with this work for additional information
@@ -47,10 +48,10 @@
         requires.append('readline')

 # Upgrade notes for 5.3.0
-print "If you're upgrading, run the following to enable parameter completion:"
-print "  cloudmonkey sync"
-print "  cloudmonkey set paramcompletion true"
-print "Parameter completion may fail, if the above is not run!"
+print("If you're upgrading, run the following to enable parameter completion:")
+print("  cloudmonkey sync")
+print("  cloudmonkey set paramcompletion true")
+print("Parameter completion may fail, if the above is not run!")

 setup(
     name = 'cloudmonkey',
RefactoringTool: Files that were modified:
RefactoringTool: setup.py

$ futurize --stage2 -w setup.py

RefactoringTool: Refactored setup.py
--- setup.py   	(original)
+++ setup.py   	(refactored)
@@ -1,4 +1,5 @@
 from __future__ import print_function
+from builtins import str
 # Licensed to the Apache Software Foundation (ASF) under one
 # or more contributor license agreements.  See the NOTICE file
 # distributed with this work for additional information
RefactoringTool: Files that were modified:
RefactoringTool: setup.py

I finished the work, so I ran the installation $ python setup.py develop

Traceback (most recent call last):
  File "setup.py", line 29, in <module>
    from cloudmonkey import __version__, __description__
  File "/Users/kentaro/porting/cloudstack-cloudmonkey/cloudmonkey/__init__.py", line 22
    except ImportError, e:
                      ^
SyntaxError: invalid syntax

init.py got an error and ended, this time__init__.Try converting py. $ futurize --stage1 -w cloudmonkey/init.py

RefactoringTool: Skipping optional fixer: idioms
RefactoringTool: Skipping optional fixer: ws_comma
RefactoringTool: Refactored cloudmonkey/__init__.py
--- cloudmonkey/__init__.py    	(original)
+++ cloudmonkey/__init__.py    	(refactored)
@@ -1,3 +1,5 @@
+from __future__ import print_function
+from __future__ import absolute_import
 # Licensed to the Apache Software Foundation (ASF) under one
 # or more contributor license agreements.  See the NOTICE file
 # distributed with this work for additional information
@@ -16,8 +18,8 @@
 # under the License.

 try:
-    from config import __version__, __description__
-    from config import __maintainer__, __maintaineremail__
-    from config import __project__, __projecturl__, __projectemail__
-except ImportError, e:
-    print e
+    from .config import __version__, __description__
+    from .config import __maintainer__, __maintaineremail__
+    from .config import __project__, __projecturl__, __projectemail__
+except ImportError as e:
+    print(e)
RefactoringTool: Files that were modified:
RefactoringTool: cloudmonkey/__init__.py

$ futurize --stage2 -w cloudmonkey/init.py

RefactoringTool: No files need to be modified.

Try installing it again. $ python setup.py develop

Traceback (most recent call last):
  File "setup.py", line 29, in <module>
    from cloudmonkey import __version__, __description__
  File "/Users/kentaro/porting/cloudstack-cloudmonkey/cloudmonkey/__init__.py", line 21, in <module>
    from .config import __version__, __description__
  File "/Users/kentaro/porting/cloudstack-cloudmonkey/cloudmonkey/config.py", line 33
    except ImportError, e:
                      ^
SyntaxError: invalid syntax

This time config.py fails, exits, config.Try converting py. $ futurize --stage1 -w cloudmonkey/config.py

RefactoringTool: Skipping optional fixer: idioms
RefactoringTool: Skipping optional fixer: ws_comma
RefactoringTool: Refactored cloudmonkey/config.py
--- cloudmonkey/config.py      	(original)
+++ cloudmonkey/config.py      	(refactored)
@@ -16,6 +16,8 @@
 # specific language governing permissions and limitations
 # under the License.

+from __future__ import print_function
+from functools import reduce
 __version__ = "5.3.3"
 __description__ = "Command Line Interface for Apache CloudStack"
 __maintainer__ = "The Apache CloudStack Team"
@@ -30,8 +32,8 @@

     from ConfigParser import ConfigParser
     from os.path import expanduser
-except ImportError, e:
-    print "ImportError", e
+except ImportError as e:
+    print("ImportError", e)

 param_type = ['boolean', 'date', 'float', 'integer', 'short', 'list',
               'long', 'object', 'map', 'string', 'tzdate', 'uuid']
@@ -84,18 +86,18 @@
         try:
             with open(config_file, 'r') as cfg:
                 config.readfp(cfg)
-        except IOError, e:
-            print "Error: config_file not found", e
+        except IOError as e:
+            print("Error: config_file not found", e)

     profile = None
     try:
         profile = get_attr('profile')
-    except AttributeError, e:
+    except AttributeError as e:
         pass
     if profile is None or profile == '':
         profile = default_profile_name
     if profile in mandatory_sections:
-        print "Server profile name cannot be '%s'" % profile
+        print("Server profile name cannot be '%s'" % profile)
         sys.exit(1)

     has_profile_changed = False
@@ -117,8 +119,8 @@
                 else:
                     for key in config_fields[section].keys():
                         config.set(section, key, config_fields[section][key])
-            except ValueError, e:
-                print "Server profile name cannot be", profile
+            except ValueError as e:
+                print("Server profile name cannot be", profile)
                 sys.exit(1)
         if section in mandatory_sections:
             section_keys = config_fields[section].keys()
@@ -150,13 +152,13 @@
         try:
             with open(config_file, 'r') as cfg:
                 config.readfp(cfg)
-        except IOError, e:
-            print "Error: config_file not found", e
+        except IOError as e:
+            print("Error: config_file not found", e)
     else:
         config = write_config(get_attr, config_file)
-        print "Welcome! Use the `set` command to configure options"
-        print "Config file:", config_file
-        print "After setting up, run the `sync` command to sync apis\n"
+        print("Welcome! Use the `set` command to configure options")
+        print("Config file:", config_file)
+        print("After setting up, run the `sync` command to sync apis\n")

     missing_keys = []
     if config.has_option('core', 'profile'):
@@ -166,19 +168,19 @@
         profile = default_profile_name

     if profile is None or profile == '' or profile in mandatory_sections:
-        print "Server profile cannot be", profile
+        print("Server profile cannot be", profile)
         sys.exit(1)

     set_attr("profile_names", filter(lambda x: x != "core" and x != "ui",
                                      config.sections()))

     if not config.has_section(profile):
-        print ("Selected profile (%s) does not exist," +
-               " using default values") % profile
+        print(("Selected profile (%s) does not exist," +
+               " using default values") % profile)
         try:
             config.add_section(profile)
-        except ValueError, e:
-            print "Server profile name cannot be", profile
+        except ValueError as e:
+            print("Server profile name cannot be", profile)
             sys.exit(1)
         for key in default_profile.keys():
             config.set(profile, key, default_profile[key])
@@ -191,7 +193,7 @@
         for key in section_keys:
             try:
                 set_attr(key, config.get(section, key))
-            except Exception, e:
+            except Exception as e:
                 if section in mandatory_sections:
                     set_attr(key, config_fields[section][key])
                 else:
@@ -202,8 +204,8 @@
                 set_attr(key, get_attr('prompt').strip() + " ")

     if len(missing_keys) > 0:
-        print "Missing configuration was set using default values for keys:"
-        print "`%s` in %s" % (', '.join(missing_keys), config_file)
+        print("Missing configuration was set using default values for keys:")
+        print("`%s` in %s" % (', '.join(missing_keys), config_file))
         write_config(get_attr, config_file)

     return config_options
RefactoringTool: Files that were modified:
RefactoringTool: cloudmonkey/config.py

$ futurize --stage2 -w cloudmonkey/config.py

RefactoringTool: Refactored cloudmonkey/config.py
--- cloudmonkey/config.py      	(original)
+++ cloudmonkey/config.py      	(refactored)
@@ -17,6 +17,8 @@
 # under the License.

 from __future__ import print_function
+from future import standard_library
+standard_library.install_aliases()
 from functools import reduce
 __version__ = "5.3.3"
 __description__ = "Command Line Interface for Apache CloudStack"
@@ -30,7 +32,7 @@
     import os
     import sys

-    from ConfigParser import ConfigParser
+    from configparser import ConfigParser
     from os.path import expanduser
 except ImportError as e:
     print("ImportError", e)
@@ -114,18 +116,18 @@
             try:
                 config.add_section(section)
                 if section not in mandatory_sections:
-                    for key in default_profile.keys():
+                    for key in list(default_profile.keys()):
                         config.set(section, key, default_profile[key])
                 else:
-                    for key in config_fields[section].keys():
+                    for key in list(config_fields[section].keys()):
                         config.set(section, key, config_fields[section][key])
             except ValueError as e:
                 print("Server profile name cannot be", profile)
                 sys.exit(1)
         if section in mandatory_sections:
-            section_keys = config_fields[section].keys()
+            section_keys = list(config_fields[section].keys())
         else:
-            section_keys = default_profile.keys()
+            section_keys = list(default_profile.keys())
         for key in section_keys:
             try:
                 if not (has_profile_changed and section == profile):
@@ -143,9 +145,8 @@
     if not os.path.exists(config_dir):
         os.makedirs(config_dir)

-    config_options = reduce(lambda x, y: x + y, map(lambda x:
-                            config_fields[x].keys(), config_fields.keys()))
-    config_options += default_profile.keys()
+    config_options = reduce(lambda x, y: x + y, [list(config_fields[x].keys()) for x in list(config_fields.keys())])
+    config_options += list(default_profile.keys())

     config = ConfigParser()
     if os.path.exists(config_file):
@@ -171,8 +172,7 @@
         print("Server profile cannot be", profile)
         sys.exit(1)

-    set_attr("profile_names", filter(lambda x: x != "core" and x != "ui",
-                                     config.sections()))
+    set_attr("profile_names", [x for x in config.sections() if x != "core" and x != "ui"])

     if not config.has_section(profile):
         print(("Selected profile (%s) does not exist," +
@@ -182,14 +182,14 @@
         except ValueError as e:
             print("Server profile name cannot be", profile)
             sys.exit(1)
-        for key in default_profile.keys():
+        for key in list(default_profile.keys()):
             config.set(profile, key, default_profile[key])

     for section in (mandatory_sections + [profile]):
         if section in mandatory_sections:
-            section_keys = config_fields[section].keys()
+            section_keys = list(config_fields[section].keys())
         else:
-            section_keys = default_profile.keys()
+            section_keys = list(default_profile.keys())
         for key in section_keys:
             try:
                 set_attr(key, config.get(section, key))
RefactoringTool: Files that were modified:
RefactoringTool: cloudmonkey/config.py

Try to install again $ python setup.py develop

If you're upgrading, run the following to enable parameter completion:
  cloudmonkey sync
  cloudmonkey set paramcompletion true
Parameter completion may fail, if the above is not run!
running develop
running egg_info
creating cloudmonkey.egg-info
writing dependency_links to cloudmonkey.egg-info/dependency_links.txt
writing cloudmonkey.egg-info/PKG-INFO
writing entry points to cloudmonkey.egg-info/entry_points.txt
writing top-level names to cloudmonkey.egg-info/top_level.txt
writing requirements to cloudmonkey.egg-info/requires.txt
writing manifest file 'cloudmonkey.egg-info/SOURCES.txt'
reading manifest file 'cloudmonkey.egg-info/SOURCES.txt'
writing manifest file 'cloudmonkey.egg-info/SOURCES.txt'
running build_ext
Creating /Users/kentaro/porting/ven/lib/python3.5/site-packages/cloudmonkey.egg-link (link to .)
Adding cloudmonkey 5.3.3 to easy-install.pth file
Installing cloudmonkey script to /Users/kentaro/porting/ven/bin

Installed /Users/kentaro/porting/cloudstack-cloudmonkey
Processing dependencies for cloudmonkey==5.3.3
Searching for requests-toolbelt
Reading https://pypi.python.org/simple/requests-toolbelt/
Best match: requests-toolbelt 0.7.0
Downloading https://pypi.python.org/packages/59/78/1d391d30ebf74079a8e4de6ab66fdca5362903ef2df64496f4697e9bb626/requests-toolbelt-0.7.0.tar.gz#md5=bfe2009905f460f4764c32cfbbf4205f
Processing requests-toolbelt-0.7.0.tar.gz
Writing /var/folders/vl/nj7qbspd7hlgll3chjfgvl400000gn/T/easy_install-l6qjdcgd/requests-toolbelt-0.7.0/setup.cfg
Running requests-toolbelt-0.7.0/setup.py -q bdist_egg --dist-dir /var/folders/vl/nj7qbspd7hlgll3chjfgvl400000gn/T/easy_install-l6qjdcgd/requests-toolbelt-0.7.0/egg-dist-tmp-xbz2yomj
no previously-included directories found matching 'docs/_build'
warning: no previously-included files matching '*.py[cdo]' found anywhere in distribution
warning: no previously-included files matching '__pycache__' found anywhere in distribution
warning: no previously-included files matching '*.so' found anywhere in distribution
warning: no previously-included files matching '*.pyd' found anywhere in distribution
zip_safe flag not set; analyzing archive contents...
Copying requests_toolbelt-0.7.0-py3.5.egg to /Users/kentaro/porting/ven/lib/python3.5/site-packages
Adding requests-toolbelt 0.7.0 to easy-install.pth file

Installed /Users/kentaro/porting/ven/lib/python3.5/site-packages/requests_toolbelt-0.7.0-py3.5.egg
Searching for dicttoxml
Reading https://pypi.python.org/simple/dicttoxml/
Best match: dicttoxml 1.7.4
Downloading https://pypi.python.org/packages/74/36/534db111db9e7610a41641a1f6669a964aacaf51858f466de264cc8dcdd9/dicttoxml-1.7.4.tar.gz#md5=ec5643a048cf32dad3c28db236b923e4
Processing dicttoxml-1.7.4.tar.gz
Writing /var/folders/vl/nj7qbspd7hlgll3chjfgvl400000gn/T/easy_install-3yyvt_ku/dicttoxml-1.7.4/setup.cfg
Running dicttoxml-1.7.4/setup.py -q bdist_egg --dist-dir /var/folders/vl/nj7qbspd7hlgll3chjfgvl400000gn/T/easy_install-3yyvt_ku/dicttoxml-1.7.4/egg-dist-tmp-1oj02owk
zip_safe flag not set; analyzing archive contents...
Copying dicttoxml-1.7.4-py3.5.egg to /Users/kentaro/porting/ven/lib/python3.5/site-packages
Adding dicttoxml 1.7.4 to easy-install.pth file

Installed /Users/kentaro/porting/ven/lib/python3.5/site-packages/dicttoxml-1.7.4-py3.5.egg
Searching for requests==2.11.1
Best match: requests 2.11.1
Adding requests 2.11.1 to easy-install.pth file

Using /Users/kentaro/porting/ven/lib/python3.5/site-packages
Searching for prettytable==0.7.2
Best match: prettytable 0.7.2
Adding prettytable 0.7.2 to easy-install.pth file

Using /Users/kentaro/porting/ven/lib/python3.5/site-packages
Searching for argcomplete==1.4.1
Best match: argcomplete 1.4.1
Adding argcomplete 1.4.1 to easy-install.pth file

Using /Users/kentaro/porting/ven/lib/python3.5/site-packages
Searching for Pygments==2.1.3
Best match: Pygments 2.1.3
Adding Pygments 2.1.3 to easy-install.pth file
Installing pygmentize script to /Users/kentaro/porting/ven/bin

Using /Users/kentaro/porting/ven/lib/python3.5/site-packages
Finished processing dependencies for cloudmonkey==5.3.3

The installation was successful.

$ cloudmonkey

Traceback (most recent call last):
  File "/Users/kentaro/porting/ven/bin/cloudmonkey", line 9, in <module>
    load_entry_point('cloudmonkey', 'console_scripts', 'cloudmonkey')()
  File "/Users/kentaro/porting/ven/lib/python3.5/site-packages/pkg_resources/__init__.py", line 542, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/Users/kentaro/porting/ven/lib/python3.5/site-packages/pkg_resources/__init__.py", line 2569, in load_entry_point
    return ep.load()
  File "/Users/kentaro/porting/ven/lib/python3.5/site-packages/pkg_resources/__init__.py", line 2229, in load
    return self.resolve()
  File "/Users/kentaro/porting/ven/lib/python3.5/site-packages/pkg_resources/__init__.py", line 2235, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/Users/kentaro/porting/cloudstack-cloudmonkey/cloudmonkey/cloudmonkey.py", line 48
    except ImportError, e:
                      ^
SyntaxError: invalid syntax

But it ended with an error.

That's all for today.

Recommended Posts

Try converting cloudmonkey CLI to python3 -1
Try to understand Python self
Try to operate Facebook with Python
Try python
Try to calculate Trace in Python
Try to reproduce color film with Python
Try logging in to qiita with Python
Try to operate Excel using Python (Xlwings)
Try converting to tidy data with pandas
Python amateurs try to summarize the list ①
Updated to Python 2.7.9
Try converting latitude / longitude and world coordinates to each other with python
Python> try: / except:
"Backport" to python 2
First steps to try Google CloudVision in Python
Try to implement Oni Maitsuji Miserable in python
Try sending Metrics to datadog via python, DogStatsD
Try to calculate a statistical problem in Python
3.14 π day, so try to output in Python
Try auto to automatically price Enums in Python 3.6
Try to solve the Python class inheritance problem
Try to draw a life curve with python
Try to make a "cryptanalysis" cipher with Python
Try to automatically generate Python documents with Sphinx
Try to calculate RPN in Python (for beginners)
Python mock to try AWS IoT Device Shadow
Try to make a dihedral group with Python
Try to detect fish with python + OpenCV2.4 (unfinished)
Try to solve the programming challenge book with python3
[First API] Try to get Qiita articles with Python
Python beginners try adding basic auth to Django admin
How to install Python
[Python] Try to read the cool answer to the FizzBuzz problem
python try ~ except ~ else
Changes from Python 3.0 to Python 3.5
Changes from Python 2 to Python 3.0
Try to improve your own intro quiz in Python
(Python) Try to develop a web application using Django
Try to solve the internship assignment problem with Python
Rewrite Python2 code to Python3 (2to3)
Try to operate DB with Python and visualize with d3
How to install python
python decorator to retry
Python --Notes when converting from str type to int type
Introduction to OpenCV (python)-(2)
Try Debian + Python 3.4 + django1.7 ...
Try to implement yolact
Try gRPC in Python
Note to daemonize python
Introducing Python 2.7 to CentOS 6.6
Try to automate pdf format report creation with Python
Try to extract high frequency words using NLTK (python)
Connect python to mysql
Try 9 slices in Python
[Python MinMaxScaler] Normalize to 0 ~ 1
Try using Tweepy [Python2.7]
Python try / except notes
Try to make it using GUI and PyQt in Python
Try to get the function list of Python> os package
Try to display various information useful for debugging with python
Try to operate an Excel file using Python (Pandas / XlsxWriter) ①