Wednesday, February 1, 2017

Python import error for sklearn.model_selection 0.17.1

ImportError: No module named 'sklearn.model_selection'

Before doing the embarrassing things I did below, read this:
Setting Jupyter kernel to match Python conda environment


Change to conda environment you need to update (tensorflow_gpu in may case).

List packages that have "scikit" in them.

(tensorflow_gpu) $ conda list | grep scikit


Nothing, nada.


$ conda install scikit-learn
Fetching package metadata .........
Solving package specifications: .
Package plan for installation in environment /Users/ukilucas/anaconda3/envs/tensorflow_gpu:
The following NEW packages will be INSTALLED:
    mkl:          2017.0.1-0            numpy:        1.11.3-py35_0         scikit-learn: 0.18.1-np111py35_1    scipy:        0.18.1-np111py35_1




(tensorflow_gpu)  $ conda list | grep scikit

scikit-learn              0.18.1              np111py35_1 



import sklearn
print (sklearn.__version__)
from sklearn.model_selection import train_test_split
0.18.1

You do not have to re-start Jupyter Notebook
that is all!

And here is my very long and embarrassing story:


Check the version of the package

import sklearn
print (sklearn.__version__)
0.17.1

Install the latest package

$ conda install scikit-learn

installs:
scikit-learn-0.18.1

However, already running jupyter notebook does not see the change.

Making sure I am in the right  (tensorflow) environment:

$ source activate tensorflow
$ conda install scikit-learn


installs:

    mkl:          2017.0.1-0        
    numpy:        1.11.3-py35_0     
    scikit-learn: 0.18.1-np111py35_1
    scipy:        0.18.1-np111py35_1



Still, Jupyter Notebook does not see scikit-learn 0.18.1

Restarting jupyter notebook from the same tensorflow environment


(tensorflow) uki@Uki ~ $ jupyter notebook ~/dev/

Still, Jupyter Notebook does NOT see scikit-learn 0.18.1



(tensorflow) uki@Uki ~ $ conda update --all
Fetching package metadata .......
Solving package specifications: ..........

# All requested packages already installed.
# packages in environment at /Users/ukilucas/anaconda3/envs/tensorflow:
#
mkl                       2017.0.1                      0  
numpy                     1.11.3                   py35_0  
openssl                   1.0.2k                        0  
pip                       9.0.1                    py35_1  
python                    3.5.2                         0  
readline                  6.2                           2  
scikit-learn              0.18.1              np111py35_1  
scipy                     0.18.1              np111py35_1  
setuptools                27.2.0                   py35_0  
sqlite                    3.13.0                        0  
tk                        8.5.18                        0  
wheel                     0.29.0                   py35_0  
xz                        5.2.2                         1  
zlib                      1.2.8                         3  

(tensorflow) uki@UkiPEsMcBookPro 192.168.1.77 09:03 ~ $ 


Jupyter Kernels

Ok, so there is a clear distinction of the conda env (in my case tensorflow) from which I am starting Jupyter Notebook and what the kernels see.

I tried to switch kernels Python [conda root] and Python [default], NO difference: sklearn.__version__ = 0.17.1


(py35) ~ $ conda list
# packages in environment at /Users/ukilucas/anaconda3/envs/py35:
#
cycler                    0.10.0                   py35_0  
freetype                  2.5.5                         2  
icu                       54.1                          0  
libpng                    1.6.27                        0  
matplotlib                2.0.0               np111py35_0  
mkl                       2017.0.1                      0  
mock                      2.0.0                    py35_0  
numpy                     1.11.3                   py35_0  
olefile                   0.44                     
openssl                   1.0.2k                        0  
pandas                    0.19.2              np111py35_1  
pbr                       1.10.0                   py35_0  
Pillow                    4.0.0                    
pip                       9.0.1                    py35_1  
protobuf                  3.1.0                    py35_0    conda-forge
pyparsing                 2.1.4                    py35_0  
pyqt                      5.6.0                    py35_2  
python                    3.5.2                         0  
python-dateutil           2.6.0                    py35_0  
pytz                      2016.10                  py35_0  
qt                        5.6.2                         0  
readline                  6.2                           2  
scikit-learn              0.18.1                   
scikit-learn              0.18.1              np111py35_1  
scipy                     0.18.1              np111py35_1  
seaborn                   0.7.1                    py35_0  
setuptools                27.2.0                   py35_0  
sip                       4.18                     py35_0  
six                       1.10.0                   py35_0  
sklearn                   0.0                      
sqlite                    3.13.0                        0  
tensorflow                0.12.1                   py35_1    conda-forge
tk                        8.5.18                        0  
wheel                     0.29.0                   py35_0  
xz                        5.2.2                         1  
zlib                      1.2.8                         3  

(py35) ~ $ jupyter notebook ~/dev/

NO difference: sklearn.__version__ = 0.17.1
Jupyter Notebook does NOT see scikit-learn 0.18.1



Removing Old Packages




$ conda listscikit-learn 0.18.1
sklearn 0.0
(py35) ~ $ conda remove scikit-learn
(py35) ~ $ pip uninstall scikit-learn
(py35) ~ $ pip uninstall sklearn
(py35) ~ $ conda install scikit-learn
(py35) ~ $ jupyter notebook ~/dev/


Did not help!


Check which Python Executable is being used


import sys

sys.executable
'/Users/ukilucas/anaconda3/bin/python'


$ conda list
# packages in environment at /Users/ukilucas/anaconda3/envs/py35:

Deleting Old and Reinstalling Jupyter Notebook

from jupyter_core.paths import jupyter_data_dir 
print(jupyter_data_dir())

/Users/ukilucas/Library/Jupyter

Deleting :
- /Users/ukilucas/Library/Jupyter
- ~/.ipython
- ~/.jupyter
(py35)  ~ $ jupyter --version
4.2.0

(py35)  ~ $ jupyter notebook --generate-config
Writing default config to: /Users/ukilucas/.jupyter/jupyter_notebook_config.py
(py35)  dev $ jupyter notebook
[I 09:52:59.160 NotebookApp] [nb_conda_kernels] enabled, 2 kernels found



Did not HELP!!



Removing Anaconda3 and Jupyter - fresh installs

$ rm -r ~/Library/Jupyter
$ rm -r ~/.ipython
$ rm -r ~/.jupyter
$ rm -r ~/anaconda3/


Installation (Mac OSX)


Downloads $ wget https://repo.continuum.io/archive/Anaconda3-4.3.0-MacOSX-x86_64.sh

Note the new Python is 3.6, but for TensorFlow I will need 3.5

bash Anaconda3-4.3.0-MacOSX-x86_64.sh


Anaconda3 will now be installed into this location:

/Users/ukilucas/anaconda3

Refresh Bash Terminal

$ source ~/.bash_profile
conda info --envs

root                  *  /Users/ukilucas/anaconda3
conda list
...
python                    3.6.0 
...


Create a new enviroment for Python 3.5

TensorFlow Installation instructions
# Python 3.5
$ conda create -n tensorflow python=3.5


At this time GPU Mac version is available only via pip


# Mac OS X, GPU enabled, Python 3.4 or 3.5:
(tensorflow)$ export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-0.12.1-py3-none-any.whl

$ sudo  pip3 install --ignore-installed --upgrade $TF_BINARY_URL

Summary

Full delete of Anaconda3 and Jupyter and fresh re-installation fixed the problem.