Hello World of Machine Learning using TensorFlow

0 Shares

Lets learn the basic ‘Hello World’ in machine learning. We will use TensorFlow, an end-to-end open source machine learning platform deployed on google cloud platform.

The basics view of machine learning.

Consider building applications in the traditional manner as represented in the following diagram:

You express rules in a programming language. These act on data and your program provides answers. In the case of the activity detection, the rules (the code you wrote to define types of activities) acted upon the data (the person’s movement speed) in order to find an answer — the return value from the function for determining the activity status of the user (whether they were walking, running, biking, etc.). The process for detecting this activity status via machine learning is very similar — only the axes are different:

Instead of trying to define the rules and express them in a programming language, you provide the answers (typically called labels) along with the data, and the machine then infers the rules that determine the relationship between the answers and the data. For example, activity detection scenario might look like this in a machine learning context:

You gather lots of data, and label it to effectively say “This is what walking looks like”, “This is what running looks like” etc. Then, the computer can infer the rules that determine, from the data, what the distinct patterns that denote a particular activity are.

Beyond being an alternative method to programming this scenario, this also gives you the ability to open up new scenarios, such as the golfing one that may not have been possible under the rules-based traditional programming approach.

In traditional programming your code compiles into a binary that is typically called a program. In machine learning, the item you create from the data and labels is called a model.

Consider the result of this to be a model, which at runtime is used like this:

You pass the model some data, and the model uses the rules it inferred from the training to come up with a prediction — i.e. “That data looks like walking”, “That data looks like biking” etc.

Lets build a very simple ‘Hello World’ model which has most of the building blocks that can be used in any machine learning scenario. We will do the following.

Create VM on google cloud platform and set up the Python development environment
Create a script for a Machine Learned Model
Train your Neural Network
Test your model

Goto google cloud platform console

Set up VM to carry out the lab tasks.

Select Navigation menu > Compute Engine, and then click VM instances.
In the Create an instance dialog with a name mllearn, in the right pane, set the Machine type to n1-standard1. Leave all other fields at the default values.
Click Create.

You new VM is listed in the VM instance list, you see a green check your VM has been successfully created.

Click SSH to the right of your new VM instance to connect to the console of the VM via SSH.

Install the Python development environment on your system

Check if your Python environment is already configured:

python3 --version
pip3 --version
virtualenv --version

The output shows that Python 3.5.3 is already in the environment, but not the pip package manager, and Virtualenv.

2. Install the pip package manager, and Virtualenv:

sudo apt update

sudo apt install python3-pip -y

sudo pip3 install -U virtualenv  # system-wide install

3. Confirm the pip package manager and Virtualenv is installed:

Create a virtual environment

Create a new virtual environment. Create a directory (./venv) and then choose a Python interpreter to put in the directory:

virtualenv --system-site-packages -p python3 ./venv

2. Activate the virtual environment:

source ./venv/bin/activate 

(venv) student-02-f4b8fb7e3cd3@mllearn:~$

3. Upgrade pip to install packages within a virtual environment without affecting the host system setup.

pip install --upgrade pip

4. View the packages installed within the virtual environment

pip list

Install the TensorFlow pip package

Install the TensorFlow package: It will take a while to install.

pip install --upgrade tensorflow

2. Verify the install

python -c "import warnings;warnings.simplefilter(action='ignore', category=FutureWarning);import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([1000, 1000])))"

We will see something like this at the end.

Tensor("Sum:0", shape=(), dtype=float32)

Now TensorFlow is now installed!

Create your first machine learning models

Consider the following sets of numbers. Can you see the relationship between them?

X:	-1	0	1	2	3	4
Y:	-2	1	4	7	10	13

As we look at them we might notice that the X value is increasing by 1 as you read left to right, and the corresponding Y value is increasing by 3. So you probably think Y=3X plus or minus something. Then we’d probably look at the zero on X and see that Y = 1, and we come up with the relationship Y=3X+1.

That’s almost exactly how we would use code to train a model, known as a neural network, to spot the patterns between these items of data!

Looking at the code used to do it, we use data to train the neural network! By feeding it with a set of Xs and a set of Ys, it should be able to figure out the relationships.

Step through creating the script piece by piece.

Create and open the file model.py.

nano model.py

Add the following code to imports:

TensorFlow and call it tf for ease of use
A library called numpy, which helps represent the data as lists
keras, the framework for defining a neural network as a set of sequential layers
Add code to suppress deprecation warnings to make the output easier to follow.

import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)

import tensorflow as tf
import numpy as np
from tensorflow import keras

from tensorflow.python.util import deprecation
deprecation._PRINT_DEPRECATION_WARNINGS = False

Leave model.py open in nano for the next section.

Define and compile the neural network

Next, create the simplest possible neural network. It has 1 layer, and that layer has 1 neuron, and the input shape is 1 value.

Add the following code to model.py:

model = tf.keras.Sequential([keras.layers.Dense(units=1, input_shape=[1])])

Next, write the code to compile your neural network. When we do, we must specify 2 functions, a loss and an optimizer.

If we’ve seen lots of math for machine learning, here’s where we would usually use it, but in this case it’s nicely encapsulated in functions . Step through what’s happening:

We know that in the function, the relationship between the numbers is y=3x+1.
When the computer tries to ‘learn’ that, it makes a guess…maybe y=10x+10. The loss function measures the guessed answers against the known correct answers and measures how well or how badly it did.
Next, the model uses the optimizer function to make another guess. Based on the loss function’s result, it will try to minimize the loss. At this point maybe it will come up with something like y=5x+5. While this is still pretty bad, it’s closer to the correct result (i.e. the loss is lower).
The model repeats this for the number of epochs you specify.
But first, we add code to model.py to tell it to use mean squared error for the loss and stochastic gradient descent (sgd) for the optimizer. We don’t need to understand the math for these yet, but we can see that they work

Over time we will learn the different and appropriate loss and optimizer functions for different scenarios.

2. Add the following code to model.py:

model.compile(optimizer='sgd', loss='mean_squared_error')

Leave model.py open in nano for the next section.

Providing the data

As you can see, the relationship between these is that Y=3X+1, so where X = -1, Y=-2 etc. etc.

A python library called numpy provides lots of array type data structures that are a defacto standard way of feeding in data. To use these, specify the values as an array in numpy using np.array\[\]. Add the following code to model.py:

xs = np.array([-1.0, 0.0, 1.0, 2.0, 3.0, 4.0], dtype=float)
ys = np.array([-2.0, 1.0, 4.0, 7.0, 10.0, 13.0], dtype=float)

model.py, contains all of the code needed to define the neural network. Now lets add code to train the neural network to infer the patterns between these numbers and use those to create a model.

Training the neural network

To train the neural network to ‘learn’ the relationship between the Xs and Ys, we step it through a loop: make a guess, measure how good or bad it is (aka the loss), use the optimizer to make another guess, etc. It will do it for the number of epochs you specify, say 500.

Add the following code to model.py:

model.fit(xs, ys, epochs=500)

Final model.py should be like this

import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)
import tensorflow as tf
import numpy as np
from tensorflow import keras

from tensorflow.python.util import deprecation
deprecation._PRINT_DEPRECATION_WARNINGS = False

model = tf.keras.Sequential([keras.layers.Dense(units=1, input_shape=[1])])

model.compile(optimizer='sgd', loss='mean_squared_error')

xs = np.array([-1.0, 0.0, 1.0, 2.0, 3.0, 4.0], dtype=float)
ys = np.array([-2.0, 1.0, 4.0, 7.0, 10.0, 13.0], dtype=float)

model.fit(xs, ys, epochs=500)

Press Ctrl+x to close nano, then press Y to save the model.py script, then Enter to confirm the script name.

Run your script

python model.py

Look at the output, which may be slightly different. Notice that the script prints out the loss for each epoch. Scroll through the epochs, the loss value is quite large for the first few epochs, but gets smaller with each step. For example:

And by the time the training is done, the loss is extremely small, showing that our model is doing a great job of inferring the relationship between the numbers:

Using the model

We now have a model that has been trained to learn the relationship between X and Y. We can use the model.predict method to figure out the Y for a previously unknown X. So, for example, if X = 10, what do you think Y will be?

Open model.py:

nano model.py

2. Add the following code to the end of the script:

print(model.predict([10.0]))

3. Press Ctrl+x, Y, then Enter to save and close model.py.

4. Take a guess about the Y value, then run your script:

python model.py

The Y value is listed after the epochs.

We might have thought Y=31, right? But in the example output above, it ended up being a little over (31.005917). Why?

Neural networks deal with probabilities, so given the data that we fed in to it, the neural network calculated a very high probability that the relationship between X and Y is Y=3X+1, but with only 6 data points we can’t know for sure. As a result, the result for 10 is very close to 31, but not necessarily 31.

As you work with neural networks, we will see this pattern recurring. You will almost always deal with probabilities, not certainties, and will do a little bit of coding to figure out what the result is based on the probabilities, particularly when it comes to classification.

This concludes the session for Hello World of Machine Learning.

0 Shares