Ugly alternatives to - Generator Expressions

print("URL for PEP - 289 = https://www.python.org/dev/peps/pep-0289/")

URL for PEP - 289 = https://www.python.org/dev/peps/pep-0289/

In [2]:

sum([x*x for x in range(10)]) # Source -  PEP - 289

Out[2]:

285

In [3]:

#So whats ->  x in range(10):
for x in range(10): # as seen below the variable - x , will take all values from 0-9 
    print(x)

0
1
2
3
4
5
6
7
8
9

In [5]:

mylist = [] # Init empty list 
for x in range(10): 
    mylist.append(x) # Append all ints from 0-9 

In [8]:

# Multiply Consecutive Elements of LIST - mylist 
#
print("myList = ",mylist)
n = 2
sqr_of_nums = []
for e in range(len(mylist)):
    sqr_of_nums.append(mylist[e]*mylist[e])
print("List of Product's of Consecutive Nums = ",sqr_of_nums)
print("SUM of Product's of Consecutive Nums = ",sum(sqr_of_nums))

myList =  [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
List of Product's of Consecutive Nums =  [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
SUM of Product's of Consecutive Nums =  285

In [ ]:

Tensorflow - for beginners

 

 

Tensorflow - Notebook -1

Index :

1. Sys config checks .
2. What are Sessions.
3. What are Variables.
4. What are Tensors. What are Tensors.
5. ....

---

GeometrySymbols$GeometrySymbols$

---

In [2]:

import sys
print(sys.executable)
print(sys.version)
print(sys.version_info)

/home/dhankar/anaconda2/envs/tensor/bin/python
3.6.2 |Continuum Analytics, Inc.| (default, Jul 20 2017, 13:51:32) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
sys.version_info(major=3, minor=6, micro=2, releaselevel='final', serial=0)

Source_1 - https://github.com/aymericdamien/TensorFlow-Examples/blob/master/notebooks/1_Introduction/basic_operations.ipynb

In [2]:

import tensorflow as tf

In [4]:

#Quick hacks ...

# Simple hello world using TensorFlow

# Create a Constant op
# The op is added as a node to the default graph.
#
# The value returned by the constructor represents the output
# of the Constant op.

hello = tf.constant('Hello, TensorFlow!')

In [5]:

# Start tf session
sess = tf.Session()

In [6]:

# Run graph
print(sess.run(hello))

b'Hello, TensorFlow!'

In [17]:

# Basic constant operations
# The value returned by the constructor represents the output
# of the Constant op.
a = tf.constant(21)
b = tf.constant(13)
#
print(type(a))
# <class 'tensorflow.python.framework.ops.Tensor'>

<class 'tensorflow.python.framework.ops.Tensor'>

The SCALAR Tensor's == a = tf.constant(21)

• a rank 0 tensor; a scalar with shape [],

In [19]:

# Launch the default graph.
with tf.Session() as sess:
    print("a: sess.run(a) b:_______TYPE_______",type(sess.run(b)))
    print("b: sess.run(b) b:__________________",sess.run(b))
    print("Addition with constants:___________",sess.run(a+b))
    print("Multiplication with constants:_____",sess.run(a*b))
    print("Subtraction with constants_________",sess.run(a-b))

print(type(sess))   ## <class 'tensorflow.python.client.session.Session'> 

a: sess.run(a) b:_______TYPE_______ <class 'numpy.int32'>
a: sess.run(a) b:__________________ 13
Addition with constants:___________ 34
Multiplication with constants:_____ 273
Subtraction with constants_________ 8
<class 'tensorflow.python.client.session.Session'>

In [21]:

# Add , Divide - basic Op's 

x = tf.add(a, b, name="add")
y = tf.div(a, b, name="divide")
print(type(x))
print(type(y))


# x and y are   == # <class 'tensorflow.python.framework.ops.Tensor'>
# same as the - #b = tf.constant(13) ...
#

<class 'tensorflow.python.framework.ops.Tensor'>
<class 'tensorflow.python.framework.ops.Tensor'>

In [20]:

# Closing the session.
sess.close()
print(type(sess))

<class 'tensorflow.python.client.session.Session'>

In [10]:

# Create Graphs over and above the DEFAULT Graph 

graph1 = tf.Graph()
print(type(graph1))

graph2 = tf.Graph()
print(type(graph2))

<class 'tensorflow.python.framework.ops.Graph'>
<class 'tensorflow.python.framework.ops.Graph'>

---

Official Tutorials for TF

• Replicating verbatim for pedagogy , with minor changes :-
• Source - https://www.tensorflow.org/programmers_guide/low_level_intro

---

In [3]:

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import numpy as np
import tensorflow as tf

Tensor Values

• The central unit of data in TensorFlow is the tensor.
• A tensor consists of a set of primitive values shaped into an array of any number of dimensions.
• A tensor's rank is its number of dimensions, while its shape is a tuple of integers specifying the array's length along each dimension.
• TensorFlow uses numpy arrays to represent tensor values.

---

Here are some examples of tensor values:

3. # a rank 0 tensor; a scalar with shape [],
[1., 2., 3.] # a rank 1 tensor; a vector with shape [3]
[[1., 2., 3.], [4., 5., 6.]] # a rank 2 tensor; a matrix with shape [2, 3]
[[[1., 2., 3.]], [[7., 8., 9.]]] # a rank 3 tensor with shape [2, 1, 3]

Creating a Variable

---

• The best way to create a variable is to call the tf.get_variable function. This function requires you to specify the Variable's name.
• This name will be used by other replicas to access the same variable, as well as to name this variable's value when checkpointing and exporting models.
• tf.get_variable also allows you to reuse a previously created variable of the same name, making it easy to define models which reuse layers.

---

To create a variable with tf.get_variable, simply provide the name and shape..

---

• When you train a model, you use variables to hold and update parameters.
• Variables are in-memory buffers containing tensors.
• They must be explicitly initialized and can be saved to disk during and after training.
• You can later restore saved values to exercise or analyze the model.
• A tf.Variable represents a tensor whose value can be changed by running ops on it.
• Unlike tf.Tensor objects, a tf.Variable exists outside the context of a single session.run call.
• Internally, a tf.Variable stores a persistent tensor.
• Specific ops allow you to read and modify the values of this tensor.
• These modifications are visible across multiple tf.Sessions, so multiple workers can see the same values for a tf.Variable.

Current Version - r1.6 - https://www.tensorflow.org/programmers_guide/variables

Older version - r1.0 - https://www.tensorflow.org/versions/r1.0/programmers_guide/variables

In [4]:

my_variable = tf.get_variable("my_variable", [1, 2, 3])
print(type(my_variable))

<class 'tensorflow.python.ops.variables.Variable'>

• This creates a variable named "my_variable" which is a three-dimensional tensor with shape [1, 2, 3].
• This variable will, by default, have the dtype tf.float32 and its initial value will be randomized via tf.glorot_uniform_initializer.
• You may optionally specify the dtype and initializer to tf.get_variable.

---

For example:

Pedagogy ---

• Seen below the "traceback" when we run the Code Chunk twice or more .
• We have already created a named variable - the code chunk re-run tries to create the same variable again.
• This is checked and Error thrown by - /anaconda2/envs/tensor/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py in _get_single_variable

cool

In [6]:

#my_int_variable = tf.get_variable("my_int_variable", [1, 2, 3], dtype=tf.int32,initializer=tf.zeros_initializer)

#print(type(my_int_variable))

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-6-90d7e657800c> in <module>()
      1 my_int_variable = tf.get_variable("my_int_variable", [1, 2, 3], dtype=tf.int32,
----> 2   initializer=tf.zeros_initializer)
      3 print(type(my_int_variable))

~/anaconda2/envs/tensor/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py in get_variable(name, shape, dtype, initializer, regularizer, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter, constraint)
   1260       partitioner=partitioner, validate_shape=validate_shape,
   1261       use_resource=use_resource, custom_getter=custom_getter,
-> 1262       constraint=constraint)
   1263 get_variable_or_local_docstring = (
   1264     """%s

~/anaconda2/envs/tensor/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py in get_variable(self, var_store, name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter, constraint)
   1095           partitioner=partitioner, validate_shape=validate_shape,
   1096           use_resource=use_resource, custom_getter=custom_getter,
-> 1097           constraint=constraint)
   1098 
   1099   def _get_partitioned_variable(self,

~/anaconda2/envs/tensor/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py in get_variable(self, name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter, constraint)
    433           caching_device=caching_device, partitioner=partitioner,
    434           validate_shape=validate_shape, use_resource=use_resource,
--> 435           constraint=constraint)
    436 
    437   def _get_partitioned_variable(

~/anaconda2/envs/tensor/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py in _true_getter(name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, constraint)
    402           trainable=trainable, collections=collections,
    403           caching_device=caching_device, validate_shape=validate_shape,
--> 404           use_resource=use_resource, constraint=constraint)
    405 
    406     if custom_getter is not None:

~/anaconda2/envs/tensor/lib/python3.6/site-packages/tensorflow/python/ops/variable_scope.py in _get_single_variable(self, name, shape, dtype, initializer, regularizer, partition_info, reuse, trainable, collections, caching_device, validate_shape, use_resource, constraint)
    741                          "reuse=tf.AUTO_REUSE in VarScope? "
    742                          "Originally defined at:\n\n%s" % (
--> 743                              name, "".join(traceback.format_list(tb))))
    744       found_var = self._vars[name]
    745       if not shape.is_compatible_with(found_var.get_shape()):

ValueError: Variable my_int_variable already exists, disallowed. Did you mean to set reuse=True or reuse=tf.AUTO_REUSE in VarScope? Originally defined at:

  File "<ipython-input-5-5cbd14b8ee95>", line 2, in <module>
    initializer=tf.zeros_initializer)
  File "/home/dhankar/anaconda2/envs/tensor/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2910, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "/home/dhankar/anaconda2/envs/tensor/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2850, in run_ast_nodes
    if self.run_code(code, result):

In [8]:

my_int_variable1 = tf.get_variable("my_int_variable1", [1, 2, 3], dtype=tf.int32,initializer=tf.zeros_initializer)

print(type(my_int_variable1))

<class 'tensorflow.python.ops.variables.Variable'>

Variables vs. Tensors

Further notes to understand Variables and Tensors - also the Difference between the two

---

https://stackoverflow.com/questions/40866675/implementation-difference-between-tensorflow-variable-and-tensorflow-tensor

---

• Quoting Verbatim - the tf.tensor ==

In the Python API, a tf.Tensor object represents the symbolic result of a TensorFlow operation. For example, in the expression t = tf.matmul(x, y), t is a tf.Tensor object representing the result of multiplying x and y (which may themselves be symbolic results of other operations, concrete values such as NumPy arrays, or variables).

In this context, a "symbolic result" is more complicated than a pointer to the result of an operation. It is more analogous to a function object that, when called (i.e. passed to tf.Session.run()) will run the necessary computation to produce the result of that operation, and return it to you as a concrete value (e.g. a NumPy array).

Your assumption most closely matches the definition of a (C++) tensorflow::Tensor object. The (Python) tf.Tensor object is more complicated because it refers to a function for computing a value, rather than the value itself.

---

The (Python) tf.Tensor object is more complicated because it refers to a function for computing a value, rather than the value itself.

---

• Distinction of Variables ==

A similar distinction exists for variables. In the Python API, a tf.Variable is the symbolic representation of a variable, which has methods for creating operations that read the current value of the variable, and assign values to it. In the C++ implementation, a tensorflow::Var object is a wrapper around a shared, mutable tensorflow::Tensor object.

• In Memory Buffer ==

An in-memory buffer is simply a contiguous region of memory that has been allocated with a TensorFlow allocator. tensorflow::Tensor objects contain a pointer to an in-memory buffer, which holds the values of that tensor. The buffer could be in host memory (i.e. accessible from the CPU) or device memory (e.g. accessible only from a GPU), and TensorFlow has operations to move data between these memory spaces.

• Whats a "handle" ==

The TensorFlow codebase uses "handle" to refer to a name for a stateful object (like a tf.FIFOQueue or tf.TensorArray) that can be passed around without copying all of the values (i.e. call-by-reference).

• Internal implementation Tensor vs. Variable ==

In C++, a tensorflow::Tensor and tensorflow::Var are very similar; the only different is that tensorflow::Var also has a mutex that can be used to lock the variable when it is being updated.

In Python, the essential difference is that a tf.Tensor is implemented as a dataflow graph, and it is read-only (i.e. by calling tf.Session.run()). A tf.Variable can be both read (i.e. by evaluating its read operation) and written (e.g. by running an assign operation).

• Tensors and variables serve different purposes.
• Tensors (tf.Tensor objects) can represent complex compositions of mathematical expressions, like loss functions in a neural network, or symbolic gradients.
• Variables represent state that is updated over time, like weight matrices and convolutional filters during training.
• While in principle you could represent the evolving state of a model without variables, you would end up with a very large (and repetetive) mathematical expression, so variables provide a convenient way to materialize the state of the model, and—for example—share it with other machines for parallel training.

---

Also read --

https://stackoverflow.com/questions/38556078/in-tensorflow-what-is-the-difference-between-a-variable-and-a-tensorhttps://github.com/tensorflow/tensorflow/issues/10855

• TensorFlow provides many convenient initializers. Alternatively, you may initialize a tf.Variable to have the value of a tf.Tensor.
• Note that when the initializer is a tf.Tensor you should not specify the variable's shape, as the shape of the initializer tensor will be used.
• Seen below - the initializer tensor == tf.constant
• Seen below - initializer=tf.constant([23, 42]) , here the shape of the initializer tensor == 23,42

---

In [9]:

other_variable = tf.get_variable("other_variable", dtype=tf.int32,initializer=tf.constant([23, 42]))
print(type(other_variable))

#<class 'tensorflow.python.ops.variables.Variable'>

<class 'tensorflow.python.ops.variables.Variable'>

In [ ]:

In [ ]:

In [ ]:

In [ ]:

In [8]:

print("EVERY TIME we run this Chunk - weights_3:0 and biases_3:0 --- increments to weights_4:0 , weights_5:0 etc etc ... Why ? ")
print(" "*90)


## Create two variables.
weights = tf.Variable(tf.random_normal([784, 200], stddev=0.35),name="weights")
biases = tf.Variable(tf.zeros([200]), name="biases")

print(type(biases))

print(biases)
print(weights)

EVERY TIME we run this Chunk - weights_3:0 and biases_3:0 --- inceremnts to weights_4:0 , weights_5:0 etc etc ... Why ? 
                                                                                          
<class 'tensorflow.python.ops.variables.Variable'>
<tf.Variable 'biases_4:0' shape=(200,) dtype=float32_ref>
<tf.Variable 'weights_4:0' shape=(784, 200) dtype=float32_ref>

In [ ]:

In [ ]:

In [ ]:

# from above == https://github.com/tensorflow/tensorflow/issues/14788
# Need to understand the Difference between --- dtype=int32_ref and dtype=int32
# is the _ref === REFRENCE ? 

"""
In [19]: v = tf.Variable([1])
In [20]: v
Out[20]: <tf.Variable 'Variable_2:0' shape=(1,) dtype=int32_ref>

In [21]: v3 = tf.identity(v)
In [22]: v3
Out[22]: <tf.Tensor 'Identity_1:0' shape=(1,) dtype=int32>
"""

In [ ]:

In [32]:

# A VARIABLE is NOT A TENSOR --- Variables are in-memory buffers containing tensors. 
#

from tensorflow.python.framework import ops

## Defining Variables ---@alextp == https://github.com/tensorflow/tensorflow/issues/14788
#
#I can see how the docstring is confusing if you think a variable is a tensor (it's not). 
#A variable is a thing which can be converted to a tensor containing its current value. 
#It also can be assigned to, partitioned, etc, which tensors can't


# Create three variables with some default values.
print("EVERY TIME we run this Chunk - weights_3:0 inceremnts to weights_4:0 , weights_5:0 etc etc ... Why ? ")
print(" "*90)

weights = tf.Variable(tf.random_normal([2, 3], stddev=0.1), name="weights")
print(weights)
print(type(weights))
print(" "*90)

biases = tf.Variable(tf.zeros([3]), name="biases")
print(biases)
print(type(biases))
print(" "*90)

custom_variable = tf.Variable(tf.zeros([3]), name="custom")
print(custom_variable)
print(type(custom_variable))
print(" "*90)


# Get all the variables' tensors and store them in a list.
all_variables_list = ops.get_collection(ops.GraphKeys.GLOBAL_VARIABLES)

EVERY TIME we run this Chunk - weights_3:0 inceremnts to weights_4:0 , weights_5:0 etc etc ... Why ? 
                                                                                          
<tf.Variable 'weights_5:0' shape=(2, 3) dtype=float32_ref>
<class 'tensorflow.python.ops.variables.Variable'>
                                                                                          
<tf.Variable 'biases_5:0' shape=(3,) dtype=float32_ref>
<class 'tensorflow.python.ops.variables.Variable'>
                                                                                          
<tf.Variable 'custom_5:0' shape=(3,) dtype=float32_ref>
<class 'tensorflow.python.ops.variables.Variable'>
                                                                                          

In [26]:

# Graphs 

a = tf.constant(3.0, dtype=tf.float32)
print(type(a)) ## <class 'tensorflow.python.framework.ops.Tensor'>

#b = tf.constant(4.0,66,77) # also tf.float32 implicitly
# TypeError: Cannot convert 66 to a dtype. data type not understood


b = tf.constant(66) # 
print(type(b)) ## <class 'tensorflow.python.framework.ops.Tensor'>


total = a + b  ## Seems we cant add Floats and INT's ??
print(a)
print(b)
print(total)

<class 'tensorflow.python.framework.ops.Tensor'>
<class 'tensorflow.python.framework.ops.Tensor'>

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-26-0ad9c4461e85> in <module>()
     12 
     13 
---> 14 total = a + b  ## Seems we cant add Floats and INT's ??
     15 print(a)
     16 print(b)

~/anaconda2/envs/tensor/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py in binary_op_wrapper(x, y)
    896       if not isinstance(y, sparse_tensor.SparseTensor):
    897         try:
--> 898           y = ops.convert_to_tensor(y, dtype=x.dtype.base_dtype, name="y")
    899         except TypeError:
    900           # If the RHS is not a tensor, it might be a tensor aware object

~/anaconda2/envs/tensor/lib/python3.6/site-packages/tensorflow/python/framework/ops.py in convert_to_tensor(value, dtype, name, preferred_dtype)
    930       name=name,
    931       preferred_dtype=preferred_dtype,
--> 932       as_ref=False)
    933 
    934 

~/anaconda2/envs/tensor/lib/python3.6/site-packages/tensorflow/python/framework/ops.py in internal_convert_to_tensor(value, dtype, name, as_ref, preferred_dtype, ctx)
   1020 
   1021     if ret is None:
-> 1022       ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
   1023 
   1024     if ret is NotImplemented:

~/anaconda2/envs/tensor/lib/python3.6/site-packages/tensorflow/python/framework/ops.py in _TensorTensorConversionFunction(t, dtype, name, as_ref)
    864     raise ValueError(
    865         "Tensor conversion requested dtype %s for Tensor with dtype %s: %r" %
--> 866         (dtype.name, t.dtype.name, str(t)))
    867   return t
    868 

ValueError: Tensor conversion requested dtype float32 for Tensor with dtype int32: 'Tensor("Const_17:0", shape=(), dtype=int32)'

In [ ]:

In [ ]:

In [ ]:

In [ ]:

In [ ]:

In [ ]:

In [ ]:

In [ ]:

In [ ]: