I want to know how my model is doing while it's training (i.e live data manner) by sshing and checking the plots.
Using the animation.FuncAnimation I'm able to make it save (& overwrite) a frame every time it updates on my local machine like this:
import numpy as np import matplotlib.pyplot as plt import matplotlib.animation as animation def animate(i): fig.clf() plt.suptitle('Title here') # actually parse model outputs and plot values = np.random.random_integers(0, 100, (100,)) plt.plot(np.arange(len(values)), values, label='random') plt.ylabel('demo data') plt.grid() plt.legend() plt.xlabel('Epochs') fig.savefig('Figure.png') fig = plt.figure() ani = animation.FuncAnimation(fig, animate, interval=10*60*1000) plt.show() Using this on a local machine is fine since the plt.show calls the $DISPLAY. However, when running on a remote server (via ssh of course) since there's no display, I get the RuntimeError: Invalid DISPLAY variable. When using other backends something like svg via matplotlib.use('svg'). The script exits without actually saving any image.
Also, my decision to use plt.show() and fig.savefig('Figure.png') inside the animate function were because without the plt.show() function after the call to FuncAnimation, it does NOT run the animate after given interval. I tried doing plt.savefig instead.
And regarding the fig.savefig('Figure.png'), doing so outside the animate function leads to a blank image. I'm guessing since I clear the image at the start of the animate function.
So, my question is: Is there a way to save figures generated on live data using animation (or FuncAnimation) like this over ssh till some event occurs (or perhaps a timeout)?
4 Answers
Answers 1
As an alternative to the webserver suggestion, you can use watch to run scp every n seconds and copy the png to your local machine.
Something like:
watch -n 5 scp user@remote:/path/to.png ./ which would copy the file every 5 seconds.
Or use rsync to only copy when it has actually changed. (There are also utilities that will use scp or rsync to copy it to your local machine when the file changes.)
Answers 2
The idea behind "animation.FuncAnimation" is that an animating function is calling a function to write/update a fig.
You seems to want to generate the "png" file remotely and get it via SSH or some other way.
Sharing the "fig" between two machines doesn't work.
If you are running the "animate" function on a remote machine, move the fig = plt.figure() into the function instead of the "fig.clf()"
def animate(i): fig = plt.figure() .... While it's possible to write SSH code that connects to the remote machine, I'd probably run a simple webserver on the remote that serves each frame.
So that you could call http:///server_frame?index=i to load the frame.
If you're a beginner and there's no huge performance requirement, this can be done easily with cherrypy (https://cherrypy.org/). An example question on how to serve images is here: How to serve multiple matplotlib images from cherrypy?
On the local side, you can now write a new "get_frame" function, that downloads the frame and puts it onto a local fig.
ani = animation.FuncAnimation(fig, get_frame, interval=10*60*1000) Answers 3
I honestly doubt using the animation framework is the proper approach here. You could have it easier just saving the figure in your own loop and viewing it from ssh or exporting with scp.
Anyway, here's a workaround for your code.
First you need to use a backend that doesn't need a working DISPLAY (mpl.use('Agg').
Then, I doubt the animation framework works at all in this scenario, it probably uses the graphics engine main loop for the drawing. You can use a MovieWriter though. Here I'm using a bare minimum movie writer that just saves the current frame in a png file.
Note that interval doesn't work in this case, it will save every frame you generate so you need to take care of the delay yourself in the generator (see the ugly time.sleep(1) below).
import numpy as np # headless backend import matplotlib as mpl mpl.use('Agg') import matplotlib.pyplot as plt import matplotlib.animation as animation import time # dummy movie writer class NullMovieWriter(animation.AbstractMovieWriter): """ from matplotlib/lib/matplotlib/tests/test_animation.py """ frame_size_can_vary = True def setup(self, fig, outfile, dpi, *args): self.fig = fig self.outfile = outfile self.dpi = dpi self.args = args self._count = 0 def grab_frame(self, **savefig_kwargs): self.savefig_kwargs = savefig_kwargs self._count += 1 print("saving frame: {}".format(self._count)) self.fig.savefig(self.outfile) def finish(self): pass fig = plt.figure() def animate(i): fig.clf() plt.suptitle('Title here') # actually parse model outputs and plot values = np.random.random_integers(0, 100, (100,)) plt.plot(np.arange(len(values)), values, label='random') plt.ylabel('demo data') plt.grid() plt.legend() plt.xlabel('Epochs') time.sleep(1) ani = animation.FuncAnimation(fig, animate, interval=10*60*1000) writer = NullMovieWriter() ani.save("Figure.png", writer=writer) Answers 4
Maybe it would be easier to save logs in plain text (or pickle) instead of rendering and every time you want to check how it's doing, just run a program on local machine that will download the data and plot it for you.
Or if you really want to render it on server side, then check out MoviePy library which can easily integrate with matplotlib and render gifs for example.
0 comments:
Post a Comment