I want to be able to read and process a CSV with Tensorflow/Keras. My first test will consist of reading a very small CSV representing the trut table for an EX-NOR gate:
EX-NOR Gate
Input A Input B Output 0 0 1 0 1 0 1 0 0 1 1 1
The CSV file, ex-nor.csv will look like this:
Input A,Input B,Output
0,0,1
0,1,0
1,0,0
1,1,1
I want to read the file into a NumPy variable, create a Tensorflow neural network model with 2 inputs and one output, then train the model on the data from the CSV.
First things first: I have to create the file somewhere that is easily accessible to my Jupyter notebook. I think the best thing to do here is to create a list in Python3, then write the list to a CSV file. This should do the trick:
tt = [[0, 0, 1], [0, 1, 0], [1, 0, 0], [1, 1, 1]]
with open('data/ex-nor.csv', 'w') as csvfile:
writer = csv.writer(csvfile)
writer.writerow(["Input A", "Input B", "Output"])
for row in tt:
writer.writerow(row)
Let's check and see if we actually wrote a file:
data = []
with open('data/ex-nor.csv') as csvfile:
reader = csv.reader(csvfile)
for row in reader:
data.append(row)
print(data)
Which gives us:
[['Input A', 'Input B', 'Output'], ['0', '0', '1'], ['0', '1', '0'], ['1', '0', '0'], ['1', '1', '1']]
OK. We have a file. Now, how do we read it with Tensorflow/Keras/NumPy?
First, I have a look at the basic_classification Tensorflow tutorial. The first thing I notice is that we end up with 2 separate variables for the training set: train_images and train_labels. This is different from the configuration of the CSV file that I created, where each row contains both the inputs and the output. To make my CSV file similar in configuration to the tensorflow variables, I would want to create 2 CSV files, one with the data and one with the outputs. Nevertheless, I start working with the CSV file that I already have.
Let's try this:
inputs = list(map(lambda x: list(x[0:-1]), data))
outputs = list(map(lambda x: x[-1], data))
[inputs, outputs]
Which gives us this:
[[[0.0, 0.0], [0.0, 1.0], [1.0, 0.0], [1.0, 1.0]],
[1.0, 0.0, 0.0, 1.0]
]
But, that doesn't work because we no longer have a NumPy array.
This almost works:
inputs = data[:, [0, 1]]
outputs = data[:, [2]]
[inputs, outputs]
However, it gives us this:
[array([[0., 0.],
[0., 1.],
[1., 0.],
[1., 1.]]),
array([[1.],
[0.],
[0.],
[1.]])]
Notice how the second array (the labels array) is a list of lists, when it needs to be a list of integers. How to I take a single column from the data array and convert it into an array of integers?
Here's the final answer:
inputs = data[:, [0, 1]]
outputs = data[:, 2]
[inputs, outputs]
Which gives me this:
[array([[0., 0.],
[0., 1.],
[1., 0.],
[1., 1.]]), array([1., 0., 0., 1.])]
In sum:
import numpy as np
import matplotlib.pyplot as plt
import csv
import os
data = np.genfromtxt('data/ex-nor.csv', delimiter=',', skip_header=1)
inputs = data[:, [0, 1]]
outputs = data[:, 2]
Here's a start:
model = Sequential([
Dense(2, input_shape=(2,)),
Dense(5, activation=tf.nn.relu),
Dense(2, activation=tf.nn.softmax)
])
Then, I compile the model:
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
But then, when I try to train the neural network, our kernel crashes:
model.fit(inputs, outputs, epochs=10)
What do to? Could it be that the labels array (the outputs array) needs to be of type uint? Nope. It turns out that the system was complaining about not being able to allocate memory. The system logged the complain in the screen where I started the notebook.
This happens sometimes with this system and I'm not sure yet of how or why it happens. Anyway, it's easy to get around this: just stop the notebook in the screen where you started it (with Control-C), then start the notebook again.
Now, the training doesn't crash the model anymore, but the accuracy doesn't change a lot and won't go above 25. Perhaps I need to increase the epochs. I tried 10 first, then 100, then 1000. That still didn't bring the accuracy above 75%. So, I finally try 10000, and that seems to have done the trick.
After some trial and error, I see that 500 epochs does it (with a lot of epochs to spare) the vast majority of the times.
Let's try some predictions:
predictions = model.predict(inputs)
[predictions, list(map(lambda x: np.argmax(x), predictions))]
And the result:
[array([[0.09247182, 0.9075282 ],
[0.8656213 , 0.13437864],
[0.9077665 , 0.09223349],
[0.10185426, 0.8981457 ]], dtype=float32),
[1, 0, 0, 1]]
Wow! Just wow! Perfect.
In sum:
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Activation
import numpy as np
model = Sequential([
Dense(2, input_shape=(2,)),
Dense(5, activation=tf.nn.relu),
Dense(2, activation=tf.nn.softmax)
])
model.compile(
optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
model.fit(inputs, outputs, epochs=500)
predictions = model.predict(inputs)
[
predictions,
list(map(lambda x: np.argmax(x), predictions))
]
Here's a summary of all the code I used to create a CSV file, read it into a couple of NumPy arrays, create a neural network, train it with the NumPy arrays, and verify that training went well.
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Activation
import numpy as np
import csv
import os
# Create the CSV with data to train a neural network
# with an EX-NOR truth table.
tt = [[0, 0, 1], [0, 1, 0], [1, 0, 0], [1, 1, 1]]
with open('data/ex-nor.csv', 'w') as csvfile:
writer = csv.writer(csvfile)
writer.writerow(["Input A", "Input B", "Output"])
for row in tt:
writer.writerow(row)
# Read the CSV file into a NumPy array
data = np.genfromtxt('data/ex-nor.csv', delimiter=',', skip_header=1)
# Reshape the data into 2 arrays, inputs and outputs. The outputs
# array will contain the outputs that we expect the neural network
# to give when presented with the associated inputs.
inputs = data[:, [0, 1]]
outputs = data[:, 2]
# Create the neural network
model = Sequential([
Dense(2, input_shape=(2,)),
Dense(5, activation=tf.nn.relu),
Dense(2, activation=tf.nn.softmax)
])
# Compile the neural network
model.compile(
optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
# Train the neural network
model.fit(inputs, outputs, epochs=500)
# Get some predictions
predictions = model.predict(inputs)
[predictions, list(map(lambda x: np.argmax(x), predictions))]
Output:
[
array([[0.09247182, 0.9075282 ],
[0.8656213 , 0.13437864],
[0.9077665 , 0.09223349],
[0.10185426, 0.8981457 ]],
dtype=float32),
[1, 0, 0, 1]
]