ECE398BD: Installing Python

In this course, we will be making use of Python 2 and several freely available packages for big data. Assignments will be distributed as Jupyter Notebooks (formerly known as IPython Notebooks).

With a little care, the code you will be writing in this course can be valid Python 2 and Python 3.

Python on EWS Machines

The EWS machines ship with Enthought Canopy. To load Python and start the IPython/Jupyter Notebook environment, open up a terminal and run

module load canopy/1.6.1
jupyter notebook

After a few seconds, your web browser should open to the IPython/Jupyter Notebook environment. If it does not, look for output in the terminal for a statement like “The IPython Notebook is running at: (address)” and visit address in your browser (typically http://localhost:8888).

You can also use this at home via FastX. Note that there are occasionally file locking issues with IPython on network shares (or possibly other reasons), so it may be preferable to install Canopy/Anaconda on your home computer rather than use the EWS machines. You can also install Anaconda to your home directory.

The /1.6.1 refers to the version of Canopy loaded; at time of writing, the default is 1.4.1, and 1.6.1 is the latest available version.

Python at Home

We recommend you either install Enthought Canopy with the free academic license or Continuum Anaconda with Python 2 (do not install the Python 3 version or miniconda – they are not suitable for this course; if you have the Python 3 version installed, see the Anaconda documentation on how to setup a Python 2 environment). These Python distributions contain most of what you will need for the course. You may need you to manually install some packages using the built-in package manager (detailed below).

If you are a *nix or Mac OS X user, note that your system likely ships with a Python install. We still recommend you install and use Canopy or Anaconda, as the system install of Python will not come with many of the packages we will need for this course and may change depending on operating system updates.

Once installed, you can open a terminal and run (*nix/Mac OS X users)

jupyter notebook

Windows users can find a launcher for IPython/Jupyter notebooks in their Start menu (or equivalent). If you do not have “Jupyter Notebook” in your start menu, you can open the Anaconda Command Prompt (or Canopy Command Prompt) and run

jupyter notebook

My personal preference is for Anaconda over Canopy, but both are fine for this course.

Converting IPython notebooks to an Older Version

Assuming you have a current version of Canopy/Anaconda, you may want to use your notebook with an older version (e.g. Canopy 1.4.1 on the EWS machines).

jupyter nbconvert --to notebook --nbformat 2 mynotebook.ipynb

will make a copy of your notebook (called mynotebook.v2.ipynb) compatible with the old version on the EWS machines. Note that this conversion is not guaranteed to work/it is at your own risk.

Installing Packages

Depending on your installation, you may need to install some packages. Canopy uses the command

enpkg (package name)

Anaconda uses the command

conda install (package name)

Most Python distributions also have the pip package installer. You can install packages by

pip install (package name)

globally (i.e. for the whole system) or

pip install (package name) --user

for the current user. Note that you may need a development toolchain (e.g. XCode on Mac OS X or the “Development Tools” option on many Linux distributions or appropriate version of Microsoft Visual Studio and/or MinGW on Microsoft Windows) to use pip.

All three methods (enpkg, conda, pip) will resolve dependencies. If you are using Canopy, start by using enpkg and failing that, use pip. If you are using Anaconda, start by using conda and failing that, use pip. It is preferable to use enpkg/conda, since the packages are often built with things you may not have at home, e.g. Intel MKL support, which can boost performance.

Note that not all packages are pre-packaged for all platforms (or will build on all platforms). If you need such a package, you can use a virtual machine or a computer with the appropriate operating system.

Warnings in your code

Depending on what version of Python you are using (and what version of the corresponding packages), you may see some warnings. Some of these are due to unsafe practices or deprecated behavior. Unlike languages like C, Python's warnings are generally quite readable. Your code should give minimal warnings (e.g. maybe one or two about variables being clobbered, or an option or class being renamed in a future release).

In general, you should try to resolve warnings in your code. They are there for a reason.

If you insist on using deprecated behavior, note that you can disable or turn off certain warnings with code like:

import warnings

warnings.filterwarnings('ignore’)

To repeat, it is highly not recommended to do this. The only acceptable case for this course to use this (or similar) is when your code runs without warning on the EWS machines, but your submitted notebook was run with a newer version of Python (+ packages) and thus induces warnings.

Python Resources

Lab 1 provides you a basic tutorial to some of Python's features.

The following resources may be useful:

And a few links to write code concisely: