Skip to content

python

Jump to bottom
snow-stone edited this page Jul 13, 2018 · 7 revisions

Table of Contents

sources

python object oriented
python course

basic types

Sequence Types — str, unicode, list, tuple, bytearray, buffer, xrange

  1. str : string, written in single or double quotes
  2. unicode : unicode string, much like string
  3. list : constructed by square brackets, its element seperated by comma
  4. tuple : constructed by the comma operator (not within square brackets), with or without enclosing parentheses, but an empty tuple must have the enclosing parentheses, such as a, b, c or (). A single item tuple must have a trailing comma, such as (d,).
  5. bytearray : created with the built-in function bytearray()
  6. buffer : Buffer objects are not directly supported by Python syntax, but can be created by calling the built-in function buffer()
  7. xrange : similar to buffers in that there is no specific syntax to create them, but they are created using the xrange() function

Operators on list

>>> myList = ["The", "earth", "revolves", "around", "sun"] # initialize a list with double quotes
>>> myList
 ['The', 'earth', 'revolves', 'around', 'sun'] # output are single quoted
>>> myList = myList + ["sure"]  # need a list to perform operator+ on list objects
>>> myList
 ['The', 'earth', 'revolves', 'around', 'sun', 'sure'] # list + list = list

Sequence Types -- dict
A mapping object maps hashable values to arbitrary objects. Mappings are mutable objects (in c++ there are also mutable objects, what are they?). There is currently only one standard mapping type, the dictionary.
Dictionaries can be created by placing a comma-separated list of key: value pairs within braces, for example: {'jack': 4098, 'sjoerd': 4127} or {4098: 'jack', 4127: 'sjoerd'}, or by the dict constructor.

ipython

run a script

$ ipython
In [1]: run script.py
# and it is interactive !
In [2]: print somevariable # to verify

package info check

>>> import scipy    # hit the first package found if multi-verison
>>> print scipy.__path__
>>> print scipy.__version__

debug

$ python -t script.py
# This will warn you if you have mixed tabs and spaces

$ cat -A script.py
# On *nix systems, you can see where the tabs

Choose a programming editor not a word editor. (vim > gedit)

Manage packages

pip

pip install scipy --user will install python packages at $HOME/.local/lib/python2.7/site-packages.

remove

rm -rf $HOME/.local/lib/python2.7/site-packages scipy* Clean way to manage package. It is then preferable to work on only one version at a time always keep it clean.

update

pip install scipy --user -U

python3

corresponds to pip3... well packages will be at $HOME/.local/lib/python3.4/site-packages

no-binary

When you need Python.H or headers like that meaning you need the source package to recompile (in pre-compiled version you will not get headers. Ex : pip install --user h5py), you use pip install --no-binary=h5py h5py --user.
And you need mpi and hdf5 libraries (which are written in C/C++/Fortran but here needing to be wrappered by python) to be in the current environment because this is re-compiling, meaning that variable CPATH indicates where to look for headers and that LD_LIBRARY_PATH indicates where there are *.so etc. [One detail here, you will need definitely headers like mpi.h or hdf5.h, so when install these packages by using package manager yum. You will need to install -devel packages]

Anaconda

conda

# List all packages and versions installed in active environment
$ conda list
# List the history of each change to the current environment
$ conda list --revisions

spyder

some error

When encounter some complicated not very comprehensive errors, consider restart session for spyder. This worked for me one time on Error TypeError: 'str' object is not callable

3d plot and animation

Plot on spyder cannot be rotated (for 2d plot, there is no such possibility while it is possible in 3D case. Ex: 3d countour plot) because it is supposed to output in the window (which is a console) so not a real environment for figure display. Well when running this script outside spyder, you'll get a window just for figure display and this can be rotated if it's a 3d plot. It can be animated if it is an animation. For the exact same reason, spyder cannot do animation inside. It will just output one image.

print to pdf

File -> Print -> Print to File (PDF) [So all syntax will be highlighted]

Modules

module search path

sys.path 和 $PYTHONPATH

>>> import sys
>>> sys.path.append('somedir')

如果当前目录下有写一个fibo.py,直接

>>>　import fibo
# 或者
>>> from fibo import *  # all names except those begin with underscore

>>> dir(fibo)  # 显示module里面的names

install and use module Geodesics

python doc
paper Bessel on 地测线

matplotlib

interactive or not

Question: Can I use matplotlib interactively ?
Quick answer: Yes, as long as you enable "interactive".

$ python
>>> execfile('somescript.py')  # doing things in this way, if it is a script to plot. Command is not in your hands
#   This is the way you may want to proceed
>>> import matplotlib.pyplot as plt
>>> plt.ion()
>>> plt(x,x)      # with x pre-defined. After plot, you have the right to proceed.
>>> plt(x,2*x)    # Right after the first figure (x,x) the second is automatically added to the same figure which means "this is working interactively/dynamically".

$ ipython
In [1]: run somescript.py # same, if it is a script to plot. Command is not in your hands after execution.

In [2]: %matplotlib       # this is the cure

Backend

From my understanding : this is where we define how to output figure.

  1. If proceeding a lot of figures, you don't want to do it interactively and you may be working on a machine without graphic card. fig.save() will throw error Cannot display... on this machine. What do you need to do then?
    => add plt.use('Agg')
  2. jupyter plotting jupyter will plot the figure in the notebook inline when being indicated %matplotlib inline

scatter

出来的图不仅鼠标的位置能指出坐标，鼠标按键还可以(interactively)输出object，所以可以在scatter图已经话画出来的情况下选两个点画条线，选两个点算个斜率并加在散点图上去via annoation. （实验室的TP_matplotlib里面issue就有这一项）

NOTE

  1. when plotting multiple figures : plt.show() can cause some fig BEFORE the current one not to have the proper axis label.
    Explanation : plt is a global name meaning it is the same for all figures. Use it carefully. Actually I don't need it when using spyder <= figures just come out in console not pop out in another window.
  2. spyder cannot have animation as far as I know : system monitor. Need to run this without spyder.
  3. plot with no-filled markers : markerfacecolor='none'

sources

Resources from Nicolas P. Rougier

No module named basemap
solved by $ conda install basemap

Plot csv data with mask basic-plotting-in-python

When saving a fig with legend outside the drawing, savefig will not include the legend but in spyder window you can save image as. This will do the trick. Or see Placing the legend (bbox_to_anchor)

trick

# print in latex mode and variable values using ax.legend
from matplotlib import pyplot as plt
import matplotlib.mlab as mlab

fig,ax = plt.subplots()
mu = np.mean(variable)
sigma = np.std(variable)
y = mlab.normpdf(bins, mu, sigma)
ax.plot(bins, y, '--', color='blue', linewidth=5, label='Gauss fit '+r'$\mu=%.4f, \sigma=%.4f$'%(mu, sigma))
ax.legend(...)

MY NOTES

NotToDo

  1. module name cannot be imported as 10D_5Dforced. Do not begin with number -> syntax error

__main__

what does if "__name__" == "__main__" do

I assume that this is implying that even if you have commanded to execute some function in the module but when A is just importing the module and use let's-say other functions, using this __main__ technique the script will ignore the execution that you have written only executing the ones that are used by A.

keyword global

Want to use this functionality taking global parameters like sizeLabel for plot in different modules.
Solution1:

# In scripts using variable sizeLabel but never initialize it
import settings # where in settings.py just put the simple "sizeLabel = 15"

# replace every "sizeLabel" by "settings.sizeLabel"

Solution2:

# at the head of script
sizeLabel = 15

# functions          # whenever you want to use sizeLabel
# main()             # just write "global sizeLabel" in the respective function !

system

# this will do "echo $HOME" in a bash env. 
# When lancing the script in Spyder. Nothing is printed.
# When lancing like in terminal "python *.py" there will be $HOME echoed.
os.system("echo $HOME")
 
# retrieve return values from bash
# this will get you the hostname of the current machine
hostname = os.popen("hostname").read().strip() 
print hostname

list

# get things(iteratable but not a list) into a list
# for example a numpy.array
l = np.linspace(0.005,0.035,num=10)
listStr = [str(x) for x in l]       # here is a list of str

# read several files and put them seperately in a list of data named : data
def readLines(location,baseName,i):
    data=np.genfromtxt(
                        location
                        +baseName
                        +str(i)
                        +'_U'
                        +'.xy'
                        ,delimiter=' ')
    return data

Nb_lines=160
data=[]
for i in range(Nb_lines):
    #data[i] = readLines('../postProcessing/sets/1/','line',i)  ## list index out of range
    data.append(readLines('../postProcessing/sets/0.5/','line',i))

numpy.ndarray

Operators for accessing element of a ndarray : []. Like a list

# dimension
data=np.genfromtxt('fileName',delimiter=' ') # returns a numpy.ndarray. File is 2 dimensional : 2darray
data.shape # returns dimension of data : data.shape[0]=nb of lines; data.shape[1]=nb of colons
data[N] # will indicate then the N-th line of the 2D data
data[:,0] # the first colon of the 2D data
data[:3,0] # first 3 lines of the first colon : as a 1D array

# initialization
yPlus=np.zeros(length)  
mean=np.zeros(***.shape) # some primitive type : list has no attribute "shape".

# sort
data1=data[data[:,1].argsort()] # sort by data's 2nd colon and return a 2darray

# change of coordinates
# original (coordinate x): x[0]   0 |---------> 1   x[-1]
# objectif (coordinate r): r[-1]  1 <---------| 0   r[0]

# intuitively we have
# x = 1-r => r = -x+1    BUT!!! The result will not be what we expect above in python language !!!
# f(r) = f(1-x)
# explanation :

# x is an uniform serie from 0 to 1   
# 均匀非常重要，如果不均匀即使对称，那么即使x倒序，y(与正序相对应)不变，画出来的ax.plot(x[::-1],y)会完全错位。例如x=[-2, -1, 0, 0.5, 1, 1.5, 2]，不均匀在这里是关键
#    math(but not true)    python             or just look at x[0]
# or x   = [0, 1]          [0,  1]               x[0] = 0
#   -x   = [-1,0]          [0, -1]              -x[0] = -0 = 0
# r=-x+1 = [0, 1]          [1,  0]         r[0]=-x[0] + 1 = 1     What we are expecting is r[0]=0 !
# r=r[::-1]                [0,  1]

# corollary : if we don't do the last inversion operation.
#             r will be a decreasing "interval". When calculating integrals
#             delta(r) = delta(r_n) - delta(r_{n-1}) which will be negative
#             For each x in [0, 1], f(x)>0. integrate.simps(f,x) > 0
#             whereas                       integrate.simps(f,r) < 0

# conclusion : an interval is not a array. The convention is
#              set to [a,b] where a =< b in the sense of intervals
#  => interval [0, -1] doesn't exist it changes automatically to [-1, 0] 
#              换元仅仅需要一一对应的变换

# sqrt
# var is a numpy array
import numpy as np
np.sqrt(var) # is good
var.sqrt()   # is bad. Why?? object.method() does not work? (need sqrt to be implemented as sqrt(self) ??) or is it linked with the way I imported numpy?

mpi4py

$ which mpiexec
~/bin/anaconda2/bin/mpiexec

# mpi4py_test1.py
from mpi4py import MPI
comm = MPI.COMM_WORLD
rank = MPI.COMM_WORLD.Get_rank()
size = MPI.COMM_WORLD.Get_size() 
print(size)
#
$ mpirun -n 4 python mpi4py_test1.py 
4
4
4
4

Clone this wiki locally