Python 3 Migration Process

Upgrade Application Python 2 to Python 3 With Migration Process

Introduction

This article will briefly guide you through the process you need to know to migrate your application code from Python 2 to Python 3. Throughout this document I divide it into three parts: one is migration using tool here we will use Python-Modernize, then part two we will manually fix some code where the existing code does not work well with python 3. Finally, part three we will update our existing dependencies version in order for it to work with Python 3.

After the end of this document, you will be able to understand and do the migration process to work with python 3 yourself.

Part I — Setup Development Environment

Before you begin, make sure you have set up the development environment already.

Install Python version 3.8

Throughout the process we will be working with python 3. So make sure you have installed it ready.

$ sudo apt update
$ sudo apt install software-properties-common
$ sudo add-apt-repository ppa:deadsnakes/ppa
$ sudo apt install python3.8
$ python3.8 --version

Clone your project and activate virtual environment with python 3

1. Clone the application project from git hub repository.
2. Outside Project Directory, Create virtual environment with python 3:

$ virtualenv -p python3 venv3
or
$ python3 -m venv venv

3. Activate virtual environment and Pip install requirement.txt

$ pip install -r requirements.txt

Running Migration with Python-Modernize

Python-Modernize tool helps us to automate common most of Python 2 code to work compatible with Python 3 where we would manually do a lot of stuff without it, to understand more detail please refer to its document here.

Following steps show you how to install python-modernize and use it to automate your code:

1. Fire up the terminal and enter:

$ pip install modernize

2. Now inside root of project folder, type command line

$ python-modernize -w .

it will take a few moments to complete.

Part II — Manual Fix code

Tool helps us automate some of the common code issues in Python 2 to support with Python 3, however, there are some more we need to manually do.

TabError: inconsistent use of tabs and spaces in indentation

Using mix tabs and spaces is fine in Python 2 but not in Python 3. There are many files with this problem, to fix this we use autopep8

$ pip install autopep8
$ autopep8 -i app/*.py

More details about autopep8

SyntaxError: from __future__ imports must occur at the beginning of the file

To fix this issue, open file the error file in import statement block, move from __future__ import print_function to the top of every import statement.

NameError: name ‘reload’ is not defined

To fix this problem, navigate to file Python and change in import statement from

reload(sys)
sys.setdefaultencoding('utf8')

To

if sys.version_info[0] == 2:
    reload(sys)
    sys.setdefaultencoding('utf8')
else:
    import importlib
    importlib.reload(sys)

ModuleNotFoundError: No module named ‘StringIO’

There are several files with this issue, to fix this issue, open a file and change from

import StringIO

To

try:
    import StringIO ## for Python 2
except ImportError:
    from io import StringIO ## for Python 3

AttributeError: ‘bytes’ object has no attribute ‘encode’

This issue happen when variable of string call method encode of UTF-8, In the Python 3 string is Unicode by default. Let’s an error example

# Python 2 is work fine but it will error on Python 3
generate_key_hex  = os.urandom(24).encode('hex')

To

generate_key_hex  = os.urandom(24)

Fix try/except

Open file Python in exception block of try/except, we need to convert exception variable to string like this

try:
....
except Exception as e:
    flash(e.message)

To

try:
....
except Exception as e:
    flash(str(e))

Do the same for every occurrence try/except instances and update them like above.

AttributeError(“type object ‘_io.StringIO’ has no attribute ‘StringIO’”,

Fix this by open file mkt/Upload/exportimport.py find and change like variable like this

StrIO = StringIO.StringIO()

To

from io import StringIO ## for Python 3
StrIO = StringIO()

ImportError: cannot import name secure_filename from werkzeug

This error happened after you process updating dependencies as below, so it is good to fix it before it occurs.

Change from

from werkzeug import secure_filename

To

from werkzeug.utils import secure_filename

Division issue

You will got the error as “TypeError: integer argument expected, got float”.

Explain: In Python 3, `/` return float so for it to return int, we use `//` instead. Fix: to fix this issue, open your file and update function or code to double // as following:

Part III — Update Dependency

In Python 3, you need to update some existing dependencies otherwise you will meet an error.

Following command line to update dependencies:

$ pip install -U SQLAlchemy
$ pip install -U WTForms==2.2.1
$ pip install -U Flask
$ pip install -U Flask-SQLAlchemy

After this update, you need to replace old style of sqlalchemy import from flask.ext.module to flask_module

For example:

from flask.ext.login import LoginManager, login_user, logout_user, current_user, login_required

To

from flask_login import LoginManager, login_user, logout_user, current_user, login_required

Using pip-ugrader tool

Instead of manually upgrading every package dependencies manually, you can use pip-upgrader tool to help you upgrade dependencies to the latest version. First you need to install pip-upgrader tool from github repository here.

After that, please these following steps to update our requirements dependencies:

$ pip install pip-upgrader

cd to directory that has file requirements.txt and run command:

$ pip-upgrade

Then, you should see output in terminal as this:

requirements.txt
1/22: Flask ... upgrade available: 0.10.1 ==> 1.1.2 (uploaded on 2020-04-03 17:17:53)
2/22: Flask-SQLAlchemy ... upgrade available: 2.0 ==> 2.4.4 (uploaded on 2020-07-14 16:42:15)
3/22: Werkzeug ... upgrade available: 0.10.4 ==> 1.0.1 (uploaded on 2020-03-31 18:03:34)
17/22: flask-json-multidict ... up to date: 1.0.0
18/22: beautifulsoup4 ... upgrade available: 4.6.0 ==> 4.9.1 (uploaded on 2020-05-17 18:24:02)
....
20/22: flask-accept ... upgrade available: 0.0.5 ==> 0.0.6 (uploaded on 2018-06-27 14:30:56)
21/22: twilio ... upgrade available: 6.16.4 ==> 6.44.0 (uploaded on 2020-07-08 19:01:02)
22/22: Flask-Mail ... up to date: 0.9.1
Available upgrades:
+------+--------------------+-----------------+----------------+---------------------+
| No.  | Package            | Current version | Latest version | Release date        |
+------+--------------------+-----------------+----------------+---------------------+
|  1   |  Flask             | 0.10.1          | 1.1.2          | 2020-04-03 17:17:53 |
|  2   |  Flask-SQLAlchemy  | 2.0             | 2.4.4          | 2020-07-14 16:42:15 |
|  3   |  Werkzeug          | 0.10.4          | 1.0.1          | 2020-03-31 18:03:34 |
|  4   |  Flask-Login       | 0.2.11          | 0.5.0          | 2020-02-09 16:55:42 |
|  5   |  Jinja2            | 2.7.3           | 2.11.2         | 2020-04-13 16:16:44 |
|  6   |  Python-dateutil   | 2.4.2           | 2.8.1          | 2019-11-03 05:42:01 |
|  7   |  SQLAlchemy        | 0.9.9           | 1.3.18         | 2020-06-25 19:04:05 |
|  8   |  WTForms           | 2.0.2           | 2.3.1          | 2020-04-22 16:34:30 |
2020-07-09 19:25:33 |
|  18  |  flask-accept      | 0.0.5            | 0.0.6          | 2018-06-27 14:30:56 |
|  19  |  twilio            | 6.16.4          | 6.44.0         | 2020-07-08 19:01:02 |
+------+--------------------+-----------------+----------------+---------------------+

Please choose which packages should be upgraded. Choices: "all", "q"
(quit), "x" (exit) or "1 2 3"

Type “all” and press enter to start updating all dependencies in file requirements.txt.

Having completed all above steps, your project should be using the latest version of dependencies as well as in your project requirements.txt.

IV. Conclusion

If you followed all of the steps correctly, you should be able to run your Project in Python 3, here we use Python version 3.8.3. you might experience some other errors unexpectedly after upgraded.

Finally I’m sure, the information above will be useful for you. But there are hundreds of other creative people whom it might be useful for as well.

So Clap! Clap! :)👏🏼