GCPトレーニング「Big Data & ML Fundamentals Lab 2: Interact with Google Cloud Storage v1.2 」でのエラー対応

カテゴリ: IT Tech

タグ: Homebrew 技術メモ

Author: f_prg

Twitter: f_prg

GCPトレーニング「Big Data & ML Fundamentals Lab 2: Interact with Google Cloud Storage v1.2 」でのエラー対応

上記のエラーが出てトレーニングが進まないのでその対応方法を調べた

python transform.pyを実行するとエラーする

google1117239_student@instance-1:~/training-data-analyst/CPB100/lab2b$ python transform.py
Traceback (most recent call last):
  File "transform.py", line 13, in <module>
    import numpy as np
ImportError: No module named numpy

numpyパッケージの解決までの準備

google1117239_student@instance-1:~/training-data-analyst/CPB100/lab2b$ curl -O https://bootstrap.pypa.io/ez_setup.py
google1117239_student@instance-1:~/training-data-analyst/CPB100/lab2b$ sudo python ./ez_setup.py
google1117239_student@instance-1:~/training-data-analyst/CPB100/lab2b$ sudo easy_install pip

numpyパッケージのインストール

google1117239_student@instance-1:~/training-data-analyst/CPB100/lab2b$ sudo pip install numpy
Collecting numpy
  Downloading https://files.pythonhosted.org/packages/40/c5/f1ed15dd931d6667b40f1ab1c2fe1f26805fc2b6c3e25e45664f838
de9d0/numpy-1.15.2-cp27-cp27mu-manylinux1_x86_64.whl (13.8MB)
    100% |████████████████████████████████| 13.8MB 1.7MB/s 
Installing collected packages: numpy
Successfully installed numpy-1.15.2

matplotlibパッケージでエラー

google1117239_student@instance-1:~/training-data-analyst/CPB100/lab2b$ python transform.py 
Traceback (most recent call last):
  File "transform.py", line 14, in <module>
    import su as mpl
ImportError: No module named matplotlib

matplotlibパッケージをインストール

pipではインストールできない

google1117239_student@instance-1:~/training-data-analyst/CPB100/lab2b$ sudo pip install matplotlib

パッケージでインストール

google1117239_student@instance-1:~/training-data-analyst/CPB100/lab2b$ sudo apt-get install -y python-subprocess32
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  python-subprocess32
0 upgraded, 1 newly installed, 0 to remove and 1 not upgraded.
Need to get 28.9 kB of archives.
After this operation, 119 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian stretch/main amd64 python-subprocess32 amd64 3.2.7-2 [28.9 kB]
Fetched 28.9 kB in 0s (503 kB/s)         
Selecting previously unselected package python-subprocess32.
(Reading database ... 42570 files and directories currently installed.)
Preparing to unpack .../python-subprocess32_3.2.7-2_amd64.deb ...
Unpacking python-subprocess32 (3.2.7-2) ...
Setting up python-subprocess32 (3.2.7-2) ...
google1117239_student@instance-1:~/training-data-analyst/CPB100/lab2b$ sudo pip install matplotlib
Collecting matplotlib
  Using cached https://files.pythonhosted.org/packages/59/08/04933377dc4500e3698e93f9113dc3624874e0914f4c85767ecb5b
389084/matplotlib-2.2.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: numpy>=1.7.1 in /usr/local/lib/python2.7/dist-packages (from matplotlib) (1.15.2)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python2.7/dist-packages (from matplotlib) (0.10.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python2.7/dist-packages (from matplotlib) (1.0.1
)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python2.7/dist-packages (from matplotlib) (2.
7.3)
Requirement already satisfied: six>=1.10 in /usr/lib/python2.7/dist-packages (from matplotlib) (1.10.0)
Requirement already satisfied: pytz in /usr/local/lib/python2.7/dist-packages (from matplotlib) (2018.5)
Requirement already satisfied: subprocess32 in /usr/lib/python2.7/dist-packages (from matplotlib) (3.2.7)
Collecting backports.functools-lru-cache (from matplotlib)
  Using cached https://files.pythonhosted.org/packages/03/8e/2424c0e65c4a066e28f539364deee49b6451f8fcd4f718fefa50cc
3dcf48/backports.functools_lru_cache-1.5-py2.py3-none-any.whl
Collecting pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 (from matplotlib)
  Using cached https://files.pythonhosted.org/packages/42/47/e6d51aef3d0393f7d343592d63a73beee2a8d3d69c22b053e252c6
cfacd5/pyparsing-2.2.1-py2.py3-none-any.whl
Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages/setuptools-33.1.1-py2.7.egg (fr
om kiwisolver>=1.0.1->matplotlib) (33.1.1)
Installing collected packages: backports.functools-lru-cache, pyparsing, matplotlib
Successfully installed backports.functools-lru-cache-1.5 matplotlib-2.2.3 pyparsing-2.2.1

basemapパッケージでエラー

google1117239_student@instance-1:~/training-data-analyst/CPB100/lab2b$ python transform.py 
Traceback (most recent call last):
  File "transform.py", line 17, in <module>
    from mpl_toolkits.basemap import Basemap
ImportError: No module named basemap

basemapをインストール

sudo apt install python-mpltoolkits.basemap

vimで__init__.pyのメソッドを編集する

google1117239_student@instance-1:~/training-data-analyst/CPB100/lab2b$ sudo vim /usr/lib/python2.7/dist-packages/mp
l_toolkits/basemap/__init__.py
fill_color = ax.get_axis_bgcolor()
　↓
fill_color = ax.get_fc()

参考はこちらを。 https://github.com/matplotlib/matplotlib/issues/7808/ https://qiita.com/nigo1973/items/5408d3b4a5f69591f835

transform.pyを実行できるようになる

google1117239_student@instance-1:~/training-data-analyst/CPB100/lab2b$ python transform.py 
{'lat': 69.5363, 'timestamp': '2018-09-24T13:17:10.233Z', 'magnitude': 2.2, 'lon': -144.0685}
/usr/lib/python2.7/dist-packages/mpl_toolkits/basemap/__init__.py:3260: MatplotlibDeprecationWarning: The ishold fu
nction was deprecated in version 2.0.
  b = ax.ishold()
/usr/lib/python2.7/dist-packages/mpl_toolkits/basemap/__init__.py:3269: MatplotlibDeprecationWarning: axes.hold is 
deprecated.
    See the API Changes document (http://matplotlib.org/api/api_changes.html)
    for more details.
  ax.hold(b)
google1117239_student@instance-1:~/training-data-analyst/CPB100/lab2b$ ls -al
total 1416
drwxr-xr-x 3 google1117239_student google1117239_student   4096 Sep 24 13:26 .
drwxr-xr-x 8 google1117239_student google1117239_student   4096 Sep 24 12:42 ..
-rw-r--r-- 1 google1117239_student google1117239_student    637 Sep 24 12:42 commands.sh
-rw-r--r-- 1 google1117239_student google1117239_student 370625 Sep 24 12:35 earthquakes.csv
-rw-r--r-- 1 google1117239_student google1117239_student    751 Sep 24 12:42 earthquakes.htm
-rw-r--r-- 1 google1117239_student google1117239_student 321866 Sep 24 13:26 earthquakes.png
-rw-r--r-- 1 google1117239_student google1117239_student  12537 Sep 24 12:45 ez_setup.py
-rwxr-xr-x 1 google1117239_student google1117239_student    759 Sep 24 12:42 ingest.sh
-rwxr-xr-x 1 google1117239_student google1117239_student    707 Sep 24 12:42 install_missing.sh
drwxr-xr-x 2 google1117239_student google1117239_student   4096 Sep 24 12:42 scheduled
-rw-r--r-- 1 google1117239_student google1117239_student 702770 Sep 24 12:45 setuptools-33.1.1.zip
-rwxr-xr-x 1 google1117239_student google1117239_student   3058 Sep 24 12:42 transform.py
google1117239_student@instance-1:~/training-data-analyst/CPB100/lab2b$ 
google1117239_student@instance-1:~/training-data-analyst/CPB100/lab2b$ gsutil cp earthquakes.* gs://<YOUR-BUCKET>/e
arthquakes/
-bash: YOUR-BUCKET: No such file or directory
google1117239_student@instance-1:~/training-data-analyst/CPB100/lab2b$ gsutil cp earthquakes.* gs://qwiklabs-gcp-3c
9c336ef09395ac/earthquakes/
Copying file://earthquakes.csv [Content-Type=text/csv]...
Copying file://earthquakes.htm [Content-Type=text/html]...                      
Copying file://earthquakes.png [Content-Type=image/png]...                      
\ [3 files][677.0 KiB/677.0 KiB]                                                
Operation completed over 3 objects/677.0 KiB.                                    
google1117239_student@instance-1:~/training-data-analyst/CPB100/lab2b$ gsutil acl ch -u AllUsers:R gs://qwiklabs-gc
p-3c9c336ef09395ac/earthquakes/*
Updated ACL on gs://qwiklabs-gcp-3c9c336ef09395ac/earthquakes/earthquakes.csv
Updated ACL on gs://qwiklabs-gcp-3c9c336ef09395ac/earthquakes/earthquakes.htm
Updated ACL on gs://qwiklabs-gcp-3c9c336ef09395ac/earthquakes/earthquakes.png

最後に

動画の9:30ごろから編集する流れがあったのでちょい困る。。。 悩んで対応しなくてもよかった。