Monday, March 27, 2017

Python: Passing *args and **kwargs into a function

# *args is the usual notation for optional, non-keyword arguments
# **kwargs is the usual notation for optional, keyword arguments
def test_var_args(regular_argument, *args, **kwargs):
    print ("- regular argument:", regular_argument)
    for arg in args:
        print ("- unnamed arguments arg:", arg)
    for key in kwargs:
        print ("- named arguments: %s: %s" % (key, kwargs[key]))
test_var_args(1, "two", 3, four=4, five="five")


- regular argument: 1
- unnamed arguments arg: two
- unnamed arguments arg: 3
- named arguments: four: 4
- named arguments: five: five

Wednesday, March 22, 2017

Python bubble sort

Since I had to learn python for my machine learning and self-driving classes and I had to review the basics. Not to waste my efforts, I am creating a new guide, you can see more in:

https://github.com/UkiDLucas/python_zen_interviews

def bubble_sort(the_list: list, verbose: int=0):
    """
    author: @UkiDLcuas
    This function changes the provided list.
    The list can contain integers, decimal numbers, strings, etc.
    The list is sorted in ascending order (first to last).
    The function does not return anything.
    """
    if verbose > 0:
        iteration = 0
   
    # count remining bubbles ( aka step backwards)
    start = len(the_list)-1 # end, zero-based list
    stop = 0 # beginning
    step = -1 # backwards
    for remaining_bubbles in range(start, stop, step):
        for i in range(remaining_bubbles):
            if verbose > 0:
                iteration = iteration + 1
                print("iteration", iteration, "remaining_bubbles", remaining_bubbles, "index", i)
                print("  ", the_list)
                print("    comparing if is", the_list[i], "bigger than", the_list[i+1])
            if the_list[i] > the_list[i+1]:
                # swap
                temp = the_list[i+1] # temp placehoder for the value to be moved
                the_list[i+1] = the_list[i] # bubble up
                the_list[i] = temp # bubble down
    if verbose > 0:
        print("*** finished", len(the_list), "element list in ", iteration, "iterations")

Thursday, March 16, 2017

AWS

Notes on how I use Amazon AWS EC2 instances for Convolutional Deep Neural Networks Machine Learning, using powerful GPU CUDA configurations.

I have moved this post to:

https://ukidlucas.github.io/posts/AWS.html

Sunday, March 5, 2017

Ubuntu: installing TensorFlow for NVidia (CUDA) GPU


I am setting TensorFlow on:
  • Ubuntu 16.04 LTS 64-bit
  • 16 GiB RAM
  • AMD Athlon(tm) II X4 640 Processor × 4 
  • GeForce GTX 1050 Ti/PCIe/SSE2

Check if you have NVidia CUDA GPU



uki@uki-p6710f:~$  lspci | grep -i nvidia
01:00.0 VGA compatible controller: NVIDIA Corporation Device 1c82 (rev a1)
01:00.1 Audio device: NVIDIA Corporation Device 0fb9 (rev a1)







Check the name of your OS




uki@uki-p6710f:~$ uname -m && cat /etc/*release
x86_64
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.1 LTS"
NAME="Ubuntu"
VERSION="16.04.1 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.1 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial


Check C compiler



uki@uki-p6710f:~$ gcc --version
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.4) 5.4.0 20160609
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Ubuntu Headers 



uki@uki-p6710f:~$ sudo apt-get install linux-headers-$(uname -r)
[sudo] password for uki:
Reading package lists... Done
Building dependency tree
Reading state information... Done
linux-headers-4.4.0-62-generic is already the newest version (4.4.0-62.83).
linux-headers-4.4.0-62-generic set to manually installed.
The following packages were automatically installed and are no longer required:
  linux-headers-4.4.0-31 linux-headers-4.4.0-31-generic linux-image-4.4.0-31-generic linux-image-extra-4.4.0-31-generic
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 89 not upgraded.

Download newest CUDA installer (1.4GB)


https://developer.nvidia.com/cuda-downloads

https://developer.nvidia.com/compute/cuda/8.0/Prod2/local_installers/cuda_8.0.61_375.26_linux-run

Execute CUDA installer

 cd ~/Downloads/
uki@uki-p6710f:~/Downloads$ ls -alt

Mar  5 14:54 cuda_8.0.61_375.26_linux.run
Feb  2 09:53 NVIDIA-Linux-x86_64-375.10.run

$ sudo sh cuda_8.0.61_375.26_linux.run

Set environment variables

uki@uki-p6710f:~$ nano ~/.bashrc

##### CUDA 
export PATH=/usr/local/cuda-8.0/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64



uki@uki-p6710f:~$ echo $LD_LIBRARY_PATH

/usr/local/cuda-8.0/lib64

Install TensorFlow via pip3 (Python 3.5)


uki@uki-p6710f:~$ python --version
Python 3.5.2 :: Anaconda 4.3.0 (64-bit)

$ sudo apt install python3-pip


$  conda info --envs
# conda environments:
#
tensorflow               /home/uki/anaconda3/envs/tensorflow
root                  *  /home/uki/anaconda3


$ conda env create -f /Users/ukilucas/dev/uki.guru/conda_enviroment_GPU.yml

$source activate tensorflow


Setting Jupyter kernel to match Python conda environment


http://ukitech.blogspot.com/2017/02/kernel.html

Saturday, February 18, 2017

Removing unused Jupyter notebook kernels on Mac

Since I am experimenting with various Python versions 2 vs 3.5 vs 3.6 and packages, e.g. TensorFlow I often end up with conda environments and corresponding Jupyter kernels that I do not use.

Removing Kernels



$ cd ~/Library/Jupyter/kernels 
$ rm -r tensorflow_gpu/

See my post on:

Installing Jupyter kernels to match Conda environments


http://ukitech.blogspot.com/2017/02/kernel.html

Developing Alexa skills using flask-ask Python framework

Install Flask-Ask for Alexa


$ pip install flask-ask

Start your favorite Python dev editor


$ jupyter notebook

Write code


To avoid repeating myself in 2 places, here is my code on GitHub:

https://raw.githubusercontent.com/UkiDLucas/uki.guru/master/Alexa/skill_001.py

https://github.com/UkiDLucas/uki.guru/blob/master/Alexa/templates.yaml


Test code running locally via HTTPS using ngrok


$ python skill_001.py  * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit) * Restarting with stat * Debugger is active! * Debugger pin code: 275-052-815





https://ngrok.com/

- unzip
- put in Applications folder
- click to open
- you should get a message "ngrok was blocked from opening because it is not from an identified developer"
- got to System Preferences -> Security & Privacy
- click "Open Anyway"
- run from Terminal:



$ /Applications/ngrok http 5000

You will need the HTTPS URL form ngrok


ngrok by @inconshreveable                                                                                                                                                (Ctrl+C to quit)                                                                                                                                                                                         Session Status                online                                                                                                                                                     Version                       2.1.18                                                                                                                                                     Region                        United States (us)                                                                                                                                         Web Interface                 http://127.0.0.1:4040                                                                                                                                      Forwarding                    http://908cd95a.ngrok.io -> localhost:5000                                                                                                                 Forwarding                    https://908cd95a.ngrok.io -> localhost:5000                                                                                                                                                                                                                                                                                                         Connections                   ttl     opn     rt1     rt5     p50     p90                                                                                                                                              0       0       0.00    0.00    0.00    0.00 


Log in to Amazon Apps and Services


Make sure you use the same email and you have set up your Alexa Echo device with.

https://developer.amazon.com/edw/home.html#/


Alexa Skills Kit
Easily add new skills to Alexa




Thursday, February 16, 2017

Installing TensorFlow 1.0 as conda environment with yml definition file on MacBook Pro with NVidia GPU support



My conda environment yml file on GitHub

https://github.com/UkiDLucas/uki.guru/blob/master/conda_enviroment_GPU.yml

Execute yml file, set environment




$ conda env create --file .../conda_environment.yml
$ conda info --envs  
$ source activate py352_tf_gpu




Update jupyter notebook (iPython) kernel with new environment we created



$ python -m ipykernel install --user --name py352_tf_gpu --display-name "conda env py352_tf_gpu"$ jupyter notebook

Test run in jupyter notebook


You can see that jupyter notebook recognize GPU




You can see execution on MacBook Pro (late 2013 with NVidia). Both CPU and GPU are measured.



Wednesday, February 15, 2017

Google TensorFlow 1.0 - Mac with Nvidia GPU installation and testing in jupyter notebook


TensorFlow 1.0 is promising more stable GPU build for Mac, let's test it.


Make sure you have CUDA installed


$ echo $DYLD_LIBRARY_PATH/usr/local/cuda/lib:


http://docs.nvidia.com/cuda/cuda-installation-guide-mac-os-x/#axzz4Ynrb42hi



$ echo $CUDA_HOME/Developer/NVIDIA/CUDA-8.0/

https://developer.nvidia.com/cudnn


$ sudo easy_install pip

Set variable with TensorFlow 1.0 binary



export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-1.0.0-py3-none-any.whl
$ echo $TF_BINARY_URL https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-1.0.0-py3-none-any.whl



Use pip to install TensorFlow




$ sudo pip3 install --upgrade  $TF_BINARY_URL Password:The directory '/Users/ukilucas/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.The directory '/Users/ukilucas/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.Collecting tensorflow-gpu==1.0.0 from https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-1.0.0-py3-none-any.whl  Downloading https://storage.googleapis.com/tensorflow/mac/gpu/tensorflow_gpu-1.0.0-py3-none-any.whl (89.0MB)    100% |████████████████████████████████| 89.0MB 8.8kB/s Requirement already up-to-date: numpy>=1.11.0 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from tensorflow-gpu==1.0.0)Requirement already up-to-date: wheel>=0.26 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from tensorflow-gpu==1.0.0)Requirement already up-to-date: six>=1.10.0 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from tensorflow-gpu==1.0.0)Requirement already up-to-date: protobuf>=3.1.0 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from tensorflow-gpu==1.0.0)Collecting setuptools (from protobuf>=3.1.0->tensorflow-gpu==1.0.0)  Downloading setuptools-34.2.0-py2.py3-none-any.whl (389kB)    100% |████████████████████████████████| 399kB 2.0MB/s Requirement already up-to-date: appdirs>=1.4.0 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from setuptools->protobuf>=3.1.0->tensorflow-gpu==1.0.0)Requirement already up-to-date: packaging>=16.8 in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from setuptools->protobuf>=3.1.0->tensorflow-gpu==1.0.0)Requirement already up-to-date: pyparsing in /Users/ukilucas/anaconda3/envs/tensorflow_gpu/lib/python3.5/site-packages (from packaging>=16.8->setuptools->protobuf>=3.1.0->tensorflow-gpu==1.0.0)Installing collected packages: tensorflow-gpu, setuptools  Found existing installation: tensorflow-gpu 0.12.1    Uninstalling tensorflow-gpu-0.12.1:      Successfully uninstalled tensorflow-gpu-0.12.1  Found existing installation: setuptools 34.1.1    Uninstalling setuptools-34.1.1:      Successfully uninstalled setuptools-34.1.1Successfully installed setuptools-34.2.0 tensorflow-gpu-1.0.0(tensorflow_gpu) uki@UkiPEsMcBookPro 192.168.1.24 19:44 dev $ 



Start jupyter notebook in the right (same) environment



(tensorflow_gpu) $ jupyter notebook

Try in jupyter notebook


import tensorflow as tf
import time
from tensorflow.python.client import device_lib

def get_available_CPU_GPU():
   devices = device_lib.list_local_devices()
   #return [x.name for x in devices if x.device_type == 'CPU']
   return [x.name for x in devices ]

print(get_available_CPU_GPU())
['/cpu:0', '/gpu:0']


start = timeit.timeit()
print ("starting")
with tf.device('/gpu:0'):
    # [1.0, 2.0, 3.0, 4.0, 5.0, 6.0]
    a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
    b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
    c = tf.matmul(a, b)
# Creates a session with log_device_placement set to True.
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
# Runs the op.
print (sess.run(c))
end = timeit.timeit()
print ("elapsed", end - start)
starting
[[ 22. 28.]
[ 49. 64.]]
elapsed -0.003607149003073573

It is a win so far, time will show if it is usable.

Reference:

https://www.tensorflow.org/install/install_mac