tf.keras.optimizers.Adam and other optimizers with minimization

I want to use the tf.contrib.keras to play around with it. However, there is something I don't understand. The classes from tf.train have a function minimize that you use for the optimization of your function. However, this minimize function does not exists for the classes in tf.contrib.keras.optimizers. Let's say we have the following code:

# tensorflow tf.train.AdamOptimizer(learning_rate=0.001) updateModel = trainer.minimize(loss)  # keras wrapper trainer=tf.contrib.keras.optimizers.Adam() updateModel = trainer.minimize(loss) # ERROR because minimize function does not exists 

The keras wrapper wont work because there is no minimize function. I'm trying to look for an example or workaround using tf.keras with tensorflow, but I don't find anything that helps me with that.

P.S. Here I'm using tf.contrib.keras.optimizers.Adam as a dummy example, but I would like to use other optimizers from the same package.

2 Answers

Answers 1

This caused because of differences between keras and tensorflow APIs. In keras optimizer is a function provided during model compilation which is used for a gradient descent optimization:

model.compile(optimizer='adam', ...) model.fit(..) <- optimiziation performed using Adam algorithm 

In tensorflow - you may use it in a custom manner as you presented.

Answers 2

Your confusion is caused by the fact that tf.contrib.keras API is not exactly tensorflow and isn't meant to be used like core tensorflow¹.

If you look at the source code, the classes from tf.contrib.keras.optimizers are almost identical to ones from keras.optimizers. E.g., the first Optimizer and the second Optimizer, the first SGD and the second SGD, and so on. Keras is being gradually incorporated in tensorflow, but right now it's more like another project bundled together with tensorflow and can't be easily used with the arbitrary tensorflow graph. Instead, keras optimizers should be used with keras layers.

So you can either stick to the all-tensorflow API and use tf.train.AdamOptimizer (like you do right now) or to all-keras API and use Adam (see Marcin's answer). I don't see any value in mixing the two.

---

^{1 At least in TF 1.x. Keras might be integrated more with TF in the future versions.}

If You Enjoyed This, Take 5 Seconds To Share It

• Facebook
• Twitter
• Google+
• StumbleUpon
• Digg
• Delicious
• LinkedIn
• Reddit
• Technorati