In the last few years I've written a nice little program in python and now I'd like to distribute it, but my first attempt didn't encounter much enthusiasm, since many potential users didn't like the idea of downloading and installing python and all the dependencies necessary to run my application.
I looked around for weeks searching for a solution to compile somehow my code into something easy to run for everybody, but I'm not really satisfied of what I found.
I have no problem with the idea of freezing the application instead of really compiling it. It just means that the end user will wait a few seconds more to download the application and will occupy more space on his/her hard disk.
My problem is to find a way to package all the application into an executable for every OS (let's say the big ones: windows, mac, linux - obviously a different executable for every OS) without the need to install virtual machines or wine or similar options. I strongly suspect it is possible, since I saw a bunch of ren'py games packaged exactly the way I'd like to package my application, but I don't know how to obtain the same result for my program.
Schematically, my application requires a console to run and includes a few home-made packages, numpy and matplotlib. It also has a system to support multiple languages that dynamically includes a language file, overwriting the previous settings:
exec('from lang.%s import lang' % language) I suspect this could create some problems. Finally, the program requires a few folders to save its settings, logs, etc in the first run, so these should be included too. My computer runs ubuntu mate 16.10 (x64) and the python version is 2.7, but in the future I plan to translate it to python 3. The executables created can be single heavy files that contain almost everything needed or lightweight files and everything else can be found in a near folder; I don't really have a preference for one solution or the other.
I am aware that this question pops up from time to time and was probably already answered, but the last occurrence of this kind of question I could find dates back to 2014, which sounds like eons ago if we consider how quickly things change in this field.
Edit: Since there seems to be no solution with the conditions I proposed, I can think of setting up some emulators/wrapper/compatibility layer/whatever to run a freezer for different OSs, but I'm not sure what to do. I read here and there that wine doesn't always work properly and can't find anything to compile for mac on linux. As far as I understand the program shouldn't require a different compilation for 32/64 bit systems, but if there is an easy way to compile for every possibility it'd be nice. Virtual machines are still a no go, right now I don't have the space on my disk to set up two or more virtual machines (that's the great downside of ssd disks...), let alone paying for licences for OSs that would be used a few times a year to compile a free software.
8 Answers
Answers 1
Couple things first.
- Python is already cross-platform
- Python code is interpreted, not a natively compiled language and does not need compiling
- There exist standard methods in Python
setuptoolsto distribute executable scripts with cross-platform compatibility - Third party python libraries can include native code (in C) that will either need compiling when installing or for binary packages to have been available by the package maintainers. NumPy and matplotlib are in this category and do have binary packages available for windows AFAIK.
Here is some example setup.py code to install an executable script:
from setuptools import setup, find_packages setup(<...>, entry_points={'console_scripts': [ '<console cmd name here> = my_package.module:main' ]}) Where my_package.module is, for example
my_package/module.py:
<..> def main(): <what to run when <console cmd name here> is executed> The package can then be distributed as a standard python module, installable via pip. If the code is open source, it can be hosted on PyPi (see instructions on PyPi site for how to setup).
Otherwise, a source or binary wheel package (python setup.py sdist and python setup.py bdist_wheel respectively) can be built and distributed to your users.
Binary wheels will need to built on the platform they are to be installed on, source distributions will work on any platform but will require code to be executed at installation as they contain purely source code.
Note that building/installing binary wheel packages requires pip version >= 6.0 and setuptools >= 26.0.
When this package is installed, a < console cmd name here > executable will be available to run on all platforms the package can be successfully installed.
For a simple example package that does this, see hello-world-python-package, specifically its setup.py.
See pythonwheels site for more information on binary wheel packages.
Answers 2
The only platform that expects programs to be a self-contained package is ms-windows. This is mostly for historical reasons pertaining to the native toolchain. For example up to version 2016 of the ms-toolchain (IIRC), Python extension modules like numpy must be compiled with exactly the same version of the ms-toolchain as Python itself. This makes building extension modules for ms-windows a huge pain.
Most UNIX-like operating systems have functional package management (although in Apple's OS-X this is supplied by a third-party). This makes installing Python relatively easy.
What could be helpful is to pack your application into a zipfile like youtube-dl does. I've written a script called build.py that does this for some of my programs.
Answers 3
Since I haven't seen it on here yet I thought I would bring up Docker. It appears to be a distributable VM that contains a package or application of some sort. There is a free community edition available on github https://github.com/docker/docker. I have never used it myself but it seems to fit all of your criteria - lightweight, free, and platform-agnostic. Just something to explore.
Answers 4
What about JAVA? Wrap you python code into java distribute as jar or .exe?
http://www.jython.org/jythonbook/en/1.0/JythonAndJavaIntegration.html
You may find this QA useful in your case. Distributing my python scripts as jars with jython?
Answers 5
I am don't know what do you really thing by writing "freezing" application, but if I understand it, you need to compile your code into a installer, that simply install all libraries and files needed to run your program, so it can handle any noob computer user.
Ok. Now you have two choices:
A) Make installer for each system separately - You can use tools such as Inno setup or PackageBuilder (on mac) to create installer, what check if is python already installed (if not install it) and copy your files + make shortcut on desktop (or something like that) + run external scripts (if you need) or if you are need binaries for windows you can use Py2exe or on mac Py2app. On Linux there is not anything like that, but if someone using Linux usually is it advanced user.
If I know there is not something like automatic builder for all platforms to build python installers, but if you have some money you can boost your work with Bitrock Install Builder.
B) Make your own script - If user already have python you can make script for downloading and installing files manually (by me best solution if is product made for advanced users, but if you make some GUI it not will be problem for noob)
Answers 6
Ren'Py
You mention Ren'Py a couple of times. Its documentation describes how to add Python code to a game. The Python Statements section even describes how to add third party packages (e.g. numpy and matplotlib). It can generate files for Windows, Mac, and Linux. If you can get it to run your application the way you want, and you are already familiar with it, then you should use it.
PyInstaller
Another option is PyInstaller. In the Supporting Multiple Operating Systems section, it says this:
If you need to distribute your application for more than one OS, for example both Windows and Mac OS X, you must install PyInstaller on each platform and bundle your app separately on each.
The second paragraph describes how to do this on a single machine:
You can do this from a single machine using virtualization. The free virtualBox or the paid VMWare and Parallels allow you to run another complete operating system as a “guest”. You set up a virtual machine for each “guest” OS. In it you install Python, the support packages your application needs, and PyInstaller.
PyInstaller advertises support for numpy and matplotlib.
Virtual Machines and Hosted Operating Systems
You said you don't have room on your SSD for virtual machines. You could buy an external USB hard drive and store the virtual machines on it. Or you could use an operating system hosting provider. You can use Google to find out more about those. If you have questions about how to use a specific hosting provider, you should create new questions.
Cross Platform Testing
Even if you use Ren'Py, you will almost certainly need to test the results on virtual machines or hosted operating systems. The only way to be certain that your program will run correctly on other platforms is to test it on those platforms. The hosting solution allows you to avoid "paying for licences for OSs that would be used a few times a year to compile a free software". However, you will still be spending money for the hosting.
The Price of Free
If your goal is to develop Python applications for Windows, Mac, and Linux without spending any money, then you are probably stuck with Ren'Py and relying on users to help you test your application on their operating systems.
Answers 7
Initial reaction
- For windows use p2exe
- For linux use python's freeze
- For macos use possibly same freeze should work (just a guess held up by one google search)
(changes might be needed to your program e.g. __file__ can not be used) Regarding
Possible alternative approach to initial problem
<...> didn't encounter much enthusiasm, since many potential users didn't like the idea of downloading and installing python
Maybe user objections could be cured, by including python installer itself next to your program. (either actual python installer or http://portablepython.com/)
i.e. add not just link to installer, but all installers and possibly bat/sh scripts to install everything.
Answers 8
At work, I do something like what you want, and I ended up writing my own code to do it. Everything is built on Linux, but I ship both Linux and Windows packages. The packages include the Python interpreter and library, and they work identically whether or not the customer already has Python installed.
Essentially, you need a small program that embeds the Python interpreter and sets it up to load only the Python code you ship, ignoring any system-installed Python. Written properly, this program can be cross-compiled for each architecture you want to support. You then ship the Python DLLs you need for the architecture, along with a zip file that has the Python code you'll use (you only need the .pyc or .pyo files). This zip file must also contain as much of the standard library as you need.
Caveats:
- You will still want virtual machines for testing.
- Setting up the C cross-compilers correctly for each architecture can be challenging.
- Neither Python's
site.pynor the one that virtualenv uses completely isolates you from a system Python install. I had to make my own version ofsite.py. - It's easy to link against the system Python by mistake. Be very careful about how you use
pkg-configand compiler arguments in your build scripts. - It's easy for Python to try to load the system Python library by mistake. Be very careful about setting the Python path and probably
Py_IgnoreEnvironmentFlag. - For Windows, you can get quite far with py2exe, and it will be easier than what I'm suggesting. I don't use py2exe mainly because I'm embedding Python code in a non-Python application.
Because my application is in Go with some Python, I wrote my wrapper as a Go package that calls Python WSGI code via cgo. You can see most of the Python embedding code here. That repo is not the whole story, though: the scripts for the custom Python build and the build scripts and initialization code for the app itself aren't public right now.
0 comments:
Post a Comment