How can we build and distribute python scripts in a windows environment

My team is enjoying using python to solve problems for our business. We write many small independent scripty applications.

However, we have to have a central windows box that runs these along with legacy applications.

Our challenge is going through a build and deploy process.

We want to have Bamboo check the script out of git, install requirements and run tests, then if all is green, just deploy to our production box.

We'd like libraries to be isolated from script to script so we don't have dependency issues.

We've tried to get virtualenvs to be portable but that seems a no go.

Pex looked promising, but it doesn't work on windows.

Ideally you'd see a folder like so:

AppOne   /Script.py   /Libs     /bar.egg     /foo.egg AppTwo   /Script2.py   /Libs     /fnord.egg     /fleebly.py 

Are we thinking about this wrong? What's the pythonic way to distribute scripts within an enterprise?

5 Answers

Answers 1

TL:DR:
Use Docker

---

A short story long:

You can use docker to create an independent image for every script that you want to deploy.

You can install a python image (-slim is the lightest) as a base environment for each script or a group of scripts/applications and use it like a "virtualenv" in which you can install all your dependencies for that script.

There is also an integration for Bamboo and Docker which you may find useful.

Here is the Docker documentation for reference.

You can test each script individually in a separated container and if it passes then you can use the same container to deploy it in your main server.

It is not exactly what you are asking, but you can use this solution in every platform (Windows, Linux, etc.), you can deploy all your scripts to the enterprise server (or anywhere for that matter) and use them across your company.

Good luck :)

Disclaimer: This is not THE solution, it is a solution that I am aware of which applies to the time of this answer (2017)

Answers 2

You may be able to do that with a neat if relatively unknown feature that was sneaked into Python 2.6 without much ado: executing zip files as Python applications. It got a bit (just a bit) more of publicity after PEP 441 (which is the one PEX is inspired in), although I think most people is still unaware of it. The idea is that you create a zip file (the recommeded extension is .pyz or .pyzw for windowed applications, but that's obviously not important) with all the code and modules that you want and then you simply run it with Python. The interpreter will add the contents of the zip file to sys.path and look for a top level module named __main__ and run it. Python 3.5 even introduced the convenience module zipapp to create such packaged applications, but there is really no magic in it and you may as well create it by hand or script.

In your case, I guess Bamboo could do the check out, dependency install and tests in virtualenvs and then package the application along with the environment libraries. It's not a one-click solution but it may do the trick without additional tools.

Answers 3

Another possibility is pyinstaller. It creates an executable that can be deployed. Python is not even required to be installed on the deployed production box. It is harder to debug problems that occur only on the deployed box. You also can't modify the scripts on the deployed box which depending on your trust of the owners of the machine is either a positive or negative. See http://www.pyinstaller.org/

Answers 4

As I understand it, you want to create self-contained application directories on a build server, then copy them over to a production server and run scripts directly from them. In particular, you want all dependencies (your own and external packages) installed within a Libs subdirectory in each application directory. Here's a way you could do that:

1. Create the top-level application directory (AppOne) and the Libs subdirectory inside it.
2. Use pip install --ignore-installed --target=Libs package_name to install dependencies into the Libs subdirectory.
3. Copy your own packages and modules into the Libs subdirectory (or install them there with pip).
4. Copy Script.py into the top-level directory.

5. Include this code at the top of Script.py:

   import os, sys app_path = os.path.dirname(__file__) lib_path = os.path.abspath(os.path.join(app_path, 'Libs')) sys.path.insert(0, lib_path) 

This will make packages like Libs\x and modules like Libs\x.py available to your script via import x. Without code like this, there's no way for your script to find those packages and modules.

If you want to streamline this part of your script, there are a couple of options: (1) Put these lines in a separate fix_path.py module in the top-level directory and just call import fix_path at the start of your script. (2) Create a Libs\__init__.py file with the line sys.path.insert(0, os.path.dirname(__file__)), and then call import Libs from your script. After that, Libs\x can be imported via import x. This is neat, but it's a nonstandard use of the package and path mechanisms (it uses Libs as both a library directory and a package), so it could create some confusion on your team about how importing works.

Once these directories and files are in place, you can safely copy this whole structure over to another system and then run it using python AppOne\Script.py. If you name your top-level script __main__.py instead of Script.py, then you can run your app just by executing python AppOne.

Or as @jdehesa pointed out, if your script is named __main__.py, you can compress the contents of the AppOne directory (but not the AppOne directory itself) into a file called AppOne.zip, and then copy that to your production server and run it by calling python AppOne.zip. (On Python 3.5 or later, you can also create the zip file via python -m zipapp AppOne if your script is called __main__.py. You may also be able to use python -m zipapp AppOne -m Script if your script is called Script.py. See https://docs.python.org/3/library/zipapp.html.)

Answers 5

This kind of thing can be easily dealt with python setup.py

Sample setup.py

from setuptools import setup  setup(     name=name_for_distribution,     version=version_number,     py_modules=[pythonfiles],     install_requires=[         python packages that need to be installed     ] ) 

Create a virtual environment , activate it and run : python setup.py install

I feel this is the most pythonic way to distribute and package your project.

Reading links:

https://pythonhosted.org/an_example_pypi_project/setuptools.html https://docs.python.org/2/distutils/setupscript.html

If You Enjoyed This, Take 5 Seconds To Share It

• Facebook
• Twitter
• Google+
• StumbleUpon
• Digg
• Delicious
• LinkedIn
• Reddit
• Technorati