Can not import matplotlib when launching from non-ascii path by mdboom · Pull Request #3487 · matplotlib/matplotlib

mdboom · 2014-09-12T15:33:08Z

Can not import matplotlib when launching from non-ascii path. Matplotlib crashes with a UnicodeDecodeError. As far as I can see it is caused by calling os.getcwd and not os.getcwdu when building the path to matplotlibrc.

I could find 4 instances of getcwd in the code base:

__init__.py(748): fname = os.path.join(os.getcwd(), 'matplotlibrc')
backends\backend_gtk.py(820): if not path: path = os.getcwd() + os.sep
backends\backend_gtk3.py(645): if not path: path = os.getcwd() + os.sep
sphinxext\plot_directive.py(451): pwd = os.getcwd()

Example non-ascii path:

In [1]: pwd
Out[1]: u'C:\\python\\external\\\xe5\xe4\xf6'

In [2]: import matplotlib
---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)
<ipython-input-2-82be63b7783c> in <module>()
----> 1 import matplotlib

C:\Python27-64\lib\site-packages\matplotlib\__init__.py in <module>()
   1046
   1047 # this is the instance used by the matplotlib classes
-> 1048 rcParams = rc_params()
   1049
   1050 if rcParams['examples.directory']:

C:\Python27-64\lib\site-packages\matplotlib\__init__.py in rc_params(fail_on_error)
    895     default matplotlib rc file.
    896     """
--> 897     fname = matplotlib_fname()
    898     if not os.path.exists(fname):
    899         # this should never happen, default in mpl-data should always be found

C:\Python27-64\lib\site-packages\matplotlib\__init__.py in matplotlib_fname()
    746       system-defined copy.
    747     """
--> 748     fname = os.path.join(os.getcwd(), 'matplotlibrc')
    749     if os.path.exists(fname):
    750         return fname

C:\Python27-64\lib\ntpath.pyc in join(a, *p)
    106                     path += b
    107                 else:
--> 108                     path += "\\" + b
    109             else:
    110                 # path is not empty and does not end with a backslash,

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe5 in position 19: ordinal not in range(128)

Ex: ascii path:

In [1]: pwd
Out[1]: u'C:\\python\\external'

In [2]: import matplotlib

In [3]: matplotlib.v
matplotlib.validate_backend matplotlib.verbose

In [3]: matplotlib.__v
matplotlib.__version__        matplotlib.__version__numpy__

In [3]: matplotlib.__version__
Out[3]: u'1.4.0'

mdboom · 2014-09-08T13:47:01Z

Since this is kind of Windows-specific, maybe you wouldn't mind testing a theory for me. Does changing these calls to os.getcwdu() help?

jstenar · 2014-09-09T06:35:34Z

Yes that fixes my problem.

ifmihai · 2014-09-12T15:10:10Z

Since upgrading to 1.4.0, I get the same exception.
I'm on Mac, Mavericks, latest version. anaconda bundle.

I'm not even sure it's about matplotlib, but it crashes when it gets to font_manager

This is the relevant output for the exception:

---> 18 import matplotlib.pyplot as plt
     19
     20 from copy import deepcopy

/Users/ifmichael/anaconda/lib/python2.7/site-packages/matplotlib/pyplot.py in <module>()
     25
     26 import matplotlib
---> 27 import matplotlib.colorbar
     28 from matplotlib import style
     29 from matplotlib import _pylab_helpers, interactive

/Users/ifmichael/anaconda/lib/python2.7/site-packages/matplotlib/colorbar.py in <module>()
     32 import matplotlib.artist as martist
     33 import matplotlib.cbook as cbook
---> 34 import matplotlib.collections as collections
     35 import matplotlib.colors as colors
     36 import matplotlib.contour as contour

/Users/ifmichael/anaconda/lib/python2.7/site-packages/matplotlib/collections.py in <module>()
     25 import matplotlib.artist as artist
     26 from matplotlib.artist import allow_rasterization
---> 27 import matplotlib.backend_bases as backend_bases
     28 import matplotlib.path as mpath
     29 from matplotlib import _path

/Users/ifmichael/anaconda/lib/python2.7/site-packages/matplotlib/backend_bases.py in <module>()
     54
     55 import matplotlib.tight_bbox as tight_bbox
---> 56 import matplotlib.textpath as textpath
     57 from matplotlib.path import Path
     58 from matplotlib.cbook import mplDeprecation

/Users/ifmichael/anaconda/lib/python2.7/site-packages/matplotlib/textpath.py in <module>()
     17 from matplotlib.path import Path
     18 from matplotlib import rcParams
---> 19 import matplotlib.font_manager as font_manager
     20 from matplotlib.ft2font import FT2Font, KERNING_DEFAULT, LOAD_NO_HINTING
     21 from matplotlib.ft2font import LOAD_TARGET_LIGHT

/Users/ifmichael/anaconda/lib/python2.7/site-packages/matplotlib/font_manager.py in <module>()
   1410                 verbose.report("Using fontManager instance from %s" % _fmcache)
   1411         except:
-> 1412             _rebuild()
   1413     else:
   1414         _rebuild()

/Users/ifmichael/anaconda/lib/python2.7/site-packages/matplotlib/font_manager.py in _rebuild()
   1395     def _rebuild():
   1396         global fontManager
-> 1397         fontManager = FontManager()
   1398         if _fmcache:
   1399             pickle_dump(fontManager, _fmcache)

/Users/ifmichael/anaconda/lib/python2.7/site-packages/matplotlib/font_manager.py in __init__(self, size, weight)
   1035         #  Load TrueType fonts and create font dictionary.
   1036
-> 1037         self.ttffiles = findSystemFonts(paths) + findSystemFonts()
   1038         self.defaultFamily = {
   1039             'ttf': 'Bitstream Vera Sans',

/Users/ifmichael/anaconda/lib/python2.7/site-packages/matplotlib/font_manager.py in findSystemFonts(fontpaths, fontext)
    320                     fontfiles[f] = 1
    321
--> 322             for f in get_fontconfig_fonts(fontext):
    323                 fontfiles[f] = 1
    324

/Users/ifmichael/anaconda/lib/python2.7/site-packages/matplotlib/font_manager.py in get_fontconfig_fonts(fontext)
    272         pipe = subprocess.Popen(['fc-list', '--format=%{file}\\n'],
    273                                 stdout=subprocess.PIPE,
--> 274                                 stderr=subprocess.PIPE)
    275         output = pipe.communicate()[0]
    276     except (OSError, IOError):

/Users/ifmichael/anaconda/lib/python2.7/subprocess.pyc in __init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags)
    708                                 p2cread, p2cwrite,
    709                                 c2pread, c2pwrite,
--> 710                                 errread, errwrite)
    711         except Exception:
    712             # Preserve original exception in case os.close raises.

/Users/ifmichael/anaconda/lib/python2.7/subprocess.pyc in _execute_child(self, args, executable, preexec_fn, close_fds, cwd, env, universal_newlines, startupinfo, creationflags, shell, to_close, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite)
   1325                         raise
   1326                 child_exception = pickle.loads(data)
-> 1327                 raise child_exception
   1328
   1329

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0: ordinal not in range(128)

tacaswell · 2014-09-12T15:23:24Z

@ifmihai That is in a different section of code (text lookup rather than rcparam file finding), can you open a new issue please?

mdboom · 2014-09-12T15:29:18Z

I'm going to attach a PR with a fix for the original issue here, and I agree with @tacaswell that @ifmihai's issue is probably something separate.

cimarronm · 2014-09-12T16:54:35Z

That won't work for python3 since the function doesn't exist as it doesn't need any special unicode return. I don't think six has any compatibility function for this but what I've seen done for this problem in ipython is

try:
    getcwdu = os.getcwdu
except AttributeError:
    getcwdu = os.getcwd

and then use getcwdu function

mdboom · 2014-09-12T16:57:23Z

@cimarronm: Good catch. Will update.

ifmihai · 2014-09-14T13:30:33Z

Is it still necessary to create another issue?

For now i downgraded to 1.3.1 and everything got back to normal.

tacaswell · 2014-09-14T13:49:46Z

@ifmihai Yes, it makes it easier for us to keep track of the bugs in the library.

BUG : use getcwdu to deal with unicode in user home-path in python 2

tacaswell added this to the v1.4.x milestone Sep 8, 2014

mdboom self-assigned this Sep 12, 2014

Allow running from non-ascii directory on Windows

0a41d83

mdboom force-pushed the use-getcwdu branch from 6ebd113 to 49d0b00 Compare September 12, 2014 16:58

tacaswell mentioned this pull request Sep 14, 2014

import error when non-ascii characters are present in cwd or user name (windows) #3516

Closed

tacaswell modified the milestones: v1.4.1, v1.4.x Sep 14, 2014

ifmihai mentioned this pull request Sep 14, 2014

Issue with non-ascii paths in font look up #3517

Closed

tacaswell added a commit that referenced this pull request Sep 17, 2014

Merge pull request #3487 from mdboom/use-getcwdu

1b11e2f

BUG : use getcwdu to deal with unicode in user home-path in python 2

tacaswell merged commit 1b11e2f into matplotlib:v1.4.x Sep 17, 2014

mdboom mentioned this pull request Sep 18, 2014

BUG : skip looking up expanduser if unicode error #3532

Closed

mdboom deleted the use-getcwdu branch March 3, 2015 18:44

blitzmann mentioned this pull request Mar 28, 2017

matplotlib failing to import when running from Unicode path pyfa-org/Pyfa#1075

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Can not import matplotlib when launching from non-ascii path #3487

Can not import matplotlib when launching from non-ascii path #3487
tacaswell merged 1 commit intomatplotlib:v1.4.xfrom
mdboom:use-getcwdu

mdboom commented Sep 12, 2014

Uh oh!

mdboom commented Sep 8, 2014

Uh oh!

jstenar commented Sep 9, 2014

Uh oh!

ifmihai commented Sep 12, 2014

Uh oh!

tacaswell commented Sep 12, 2014

Uh oh!

mdboom commented Sep 12, 2014

Uh oh!

cimarronm commented Sep 12, 2014

Uh oh!

mdboom commented Sep 12, 2014

Uh oh!

ifmihai commented Sep 14, 2014

Uh oh!

tacaswell commented Sep 14, 2014

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

mdboom commented Sep 12, 2014

Uh oh!

mdboom commented Sep 8, 2014

Uh oh!

jstenar commented Sep 9, 2014

Uh oh!

ifmihai commented Sep 12, 2014

Uh oh!

tacaswell commented Sep 12, 2014

Uh oh!

mdboom commented Sep 12, 2014

Uh oh!

cimarronm commented Sep 12, 2014

Uh oh!

mdboom commented Sep 12, 2014

Uh oh!

ifmihai commented Sep 14, 2014

Uh oh!

tacaswell commented Sep 14, 2014

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants