Showing posts with label compilation. Show all posts
Showing posts with label compilation. Show all posts

Friday, April 6, 2018

Configuring Gradle to speed up cordova-cli builds

Leave a Comment

I use Cordova CLI to build my hybrid Android app. In my effort to speed up compile times I have moved the entire Android installation to an SSD and upgraded my machine with an extra 8Gb of memory (total of 16Gb). Nevertheless I see zero effect on compile times which are stuck at pretty much the same figure as they were prior to these upgrades. I am starting to suspect that it is not enough to merely provide more resources. Somehow Gradle needs to be told to use them. However, I am not sure how I do this within the context of a hybrid Cordova app.

Acting on the ideas I found in this Reddit thread I created a gradle.properties file in the app/platforms/android folder where I put

org.gradle.parallel=true org.gradle.daemon=true org.gradle.configureondemand=true org.gradle.jvmargs=-Xms2048m -Xmx8096m -XX:PermSize=1024                     -XX:MaxPermSize=1024m -XX:ReservedCodeCacheSize=1024m 

This did have an effect - it made the build almost 50% slower! The first build with a new Gradle daemon is usually slower so I recompiled. There was a small saving but the build was still slower than before. Clearly, there is more to configuring Gradle than telling it to use tons of memory. However, I have no experience in the domain. Could someone here tell me how I should go about the process in order to get some real performance benefits?

2 Answers

Answers 1

I was recently working on tuning Jenkins pipelines and migrating an app from Maven to Gradle so I hope my knowledge might be useful.

First of all, you can measure difference in execution time in the following way:

  • remove all .gradle folders. It is Gradle's cache folders and it might be accidentally used.
  • execute gradle --stop. It will stop Gradle daemons so that no cache in daemons itself can be used.
  • run gradle clean build

Secondly, the gain that you've got is mostly due to using org.gradle.parallel=true. You may try to remove other things and I bet there will be no significant change in performance. org.gradle.daemon=true is the default value for the option. Also, I would revise JVM flags as e.g. -XX:MaxPermSize is not used in Java 8 (although I understand you're working on Android app)

Thirdly, regarding speeding up the compilation itself, you can use this flags:

GRADLE_OPTS="-XX:+TieredCompilation -XX:TieredStopAtLevel=1" 

It should turn off JVM profiling so it might do the trick for you.

Thirdly, check compile vs implementation. Using of implementation configuration theoretically may speed up compilation and recompilation process because of not using transitive dependencies.

I guess I don't know other ways to speed up compilation of a Gradle project. However, there are things like parallel test execution if you're interested in speeding up the whole build.

The last note: check you plugins. Sometimes they do useless things you don't really need (mine bottleneck was wsdl2java) e.g. copying many unused generated sources.

Hope it helps!

Answers 2

Move your project's build folder to SSD: there are tons of write operations during build and write is always much slower than read.

And configure antivirus software (if any) to exclude all java binaries, project's build folder and ~/.gradle from scanning.

These two actions should be enough to see the difference.

Read More

Tuesday, April 11, 2017

ionic build number wrong in config.xml file

Leave a Comment

My problem is that if I compile my project, it adds a 2 to the build number.

My Ionic config.xml

<widget id="at.company.or.so" version="0.0.1" ios-CFBundleVersion="55" android-versionCode="72" xmlns="http://www.w3.org/ns/widgets" xmlns:cdv="http://cordova.apache.org/ns/1.0"> 

My AndroidManifest.xml

<manifest android:hardwareAccelerated="true" android:versionCode="72" android:versionName="0.0.1" package="at.tiddlr.app" xmlns:android="http://schemas.android.com/apk/res/android"> 

If i build a debug or release Version should be 0.0.1(72) but when i install the App it is 0.0.1(722)

Output of aapt aapt d badging apk.apk

package: name='at.company.or.so' versionCode='722'  versionName='0.0.1'platformBuild VersionName='5.1.1-1819727' sdkVersion:'14' targetSdkVersion:'22' 

I think it's a problem while compiling but i have no idea where to search.

1 Answers

Answers 1

Not sure if you still have an issue with this, but I ran into the same build issue with Android where mine would increment with an 8. My solution for now is to..... dramatic pause.... comment out some lines in the build.gradle file. Whenever I upgrade the CLI I have to go do it again....

This line:

versionCode cdvVersionCode ?: Integer.parseInt("" + privateHelpers.extractIntFromManifest("versionCode") + "0") 

Change to:

versionCode cdvVersionCode ?: Integer.parseInt("" + privateHelpers.extractIntFromManifest("versionCode")) 

Then I also comment out the whole if/else block about 13 lines down which starts with:

if (Boolean.valueOf(cdvBuildMultipleApks)) { 

Now I have to manually update my build number for android before I deploy to the store in the config file, but my increment issue has stopped. Hopefully this help somebody out if you have already passed the issue. I have included a link to the issue on Apache's issues board below. IF the link ever dies check out CB-8976 on Apache's issue site. I will also note that I have not experienced any issues with commenting this out... but I can not guarantee since you should not mess with this file technically as they state // GENERATED FILE! DO NOT EDIT!)

Apache Issue's / Version Code

Read More

Monday, April 3, 2017

Compilation errors in simple Java program with Scanner

Leave a Comment
import java.util.*; public class strings {     public static void main (String [] args) {         Scanner keyboard = new Scanner(System.in);         System.out.print("Type your first integer: ");         int first = keyboard.next();         System.out.print("Type your seconds integer : ");         int second = keyboard.next();         System.out.print("The sum of your two integers are:");     } } 

I dont know why im getting 2 errors on strings cannot converted to int.

13 Answers

Answers 1

keyboard.nextInt();  

instead of

keyboard.next(); 

If you want to use keyboard.next();, then change int first to String first.

Modify your code to this:

import java.util.*; public class Demo {     public static void main (String [] args) {         Scanner keyboard = new Scanner(System.in);         System.out.print("Type your first integer: ");         int first = keyboard.nextInt();          System.out.print("Type your seconds integer : ");         int second = keyboard.nextInt();          System.out.print("The sum of your two integers are:" +(first+second));     } } 

Change your class name to another instead of string.

Answers 2

There are multiple ways to do this, depending on how much validation you want to do.

In the solution below first number is obtained using .nextInt(). Note that if it is not an integer we need to clear the stream input, so in catching of InputMismatchException we do call .next(). Otherwise the same exception would keep occurring since the input has not been cleared

The second number is obtained using .next() which returns a string and we cast that string to the integer. Note that here we catch a different exception NumberFormatException to ensure that the input is still good. (we do not call .next())

 public static void main (String [] args) {     try (Scanner keyboard = new Scanner(System.in)) {         System.out.print("Type your first integer: ");         int first;         do {             try {                 first = keyboard.nextInt();                 break;             } catch (InputMismatchException e) {                 keyboard.next();                 System.out.print("Invalid first integer, try again: ");             }         } while (true);         System.out.print("Type your seconds integer : ");         int second;         do {             try {                 second = Integer.valueOf(keyboard.next());                 break;             } catch (NumberFormatException e) {                 System.out.print("Invalid seconds integer, try again: ");             }         } while (true);         System.out.print("The sum of your two integers are: " + (first + second));     } } 

One last thing I did was try on the resource, this way the input stream will be closed regardless if the program exits normally or throws an exception.

Answers 3

Scanner class next() method returns String value. You can do anyone of the following steps to solve the compilation error.

  1. You need to convert the String value to int.

    int first = Integer.parseInt(keyboard.next()); 
  2. You can use nextInt() method to get int value directly from the user.

Answers 4

This will read the whole line then convert it to an Integer

int first = Integer.paresInt(keyboard.nextLine());

or

If you are sure input is in digit

int first = keyboard.nextInt();

Answers 5

I have three solutions for this problem :

First

In your code you are using keyboard.next() which will return a string that you have to convert it into Integer. For that you can use keyboard.nextInt(), it will directly convert the string into integer.

import java.util.Scanner;  public class strings {     public static void main(String[] args) {         Scanner keyboard = new Scanner(System.in);         System.out.print("Type your first integer: ");         int first = keyboard.nextInt();         System.out.print("Type your seconds integer : ");         int second = keyboard.nextInt();         System.out.print("The sum of your two integers are:"+(first+second));     } } 

Second

or you can convert the string into Integer by using Integer.parseInt(**String to be converted to Integer**), this will convert the string into Integer.

import java.util.Scanner;  public class strings {     public static void main(String[] args) {         Scanner keyboard = new Scanner(System.in);         System.out.print("Type your first integer: ");         int first = Integer.parseInt(keyboard.next());         System.out.print("Type your seconds integer : ");         int second = Integer.parseInt(keyboard.next());         System.out.print("The sum of your two integers are:"+(first+second));     } } 

Third

or you can convert the string into Integer by using Integer.valueOf(**String to be converted to Integer**), this will convert the string into Integer.

import java.util.Scanner;  public class strings {     public static void main(String[] args) {         Scanner keyboard = new Scanner(System.in);         System.out.print("Type your first integer: ");         int first = Integer.valueOf(keyboard.next());         System.out.print("Type your seconds integer : ");         int second = Integer.valueOf(keyboard.next());         System.out.print("The sum of your two integers are:"+(first+second));     } } 

Answers 6

You can use scanner.nextLine() (reads the input till end of the line) which gives String output and convert it to int using Integer.parseInt as shown below:

          Scanner keyboard = new Scanner(System.in);           try {               System.out.print("Type your first integer: ");               int first = Integer.parseInt(keyboard.nextLine());               System.out.print("Type your seconds integer : ");               int second = Integer.parseInt(keyboard.nextLine());               int sum  =first+second;               System.out.print("The sum of your two integers are:"+sum);           } catch(Exception exe) {               System.out.println(" Error!!!! Please try again !!!! ");           } finally {               keyboard.close();           } 

Also, try to close the resources (inside finally block, shown above) as a practice.

Answers 7

You are using next() method from Scanner class. This method return String and you cannot store string at int type variable. So you can use nextInt() method for int type input or you can just Store the value at String variable and after that you can convert the String variable to int.

Please follow this code:

import java.util.*; public class strings {     public static void main (String [] args) {         Scanner keyboard = new Scanner(System.in);         System.out.print("Type your first integer: ");         int first = keyboard.nextInt();         System.out.print("Type your seconds integer : ");         int second = keyboard.nextInt();         System.out.print("The sum of your two integers are:" +(first+second));     } } 

or Follow this code

import java.util.*; public class strings {     public static void main (String [] args) {         Scanner keyboard = new Scanner(System.in);         System.out.print("Type your first integer: ");         //Get String input and stroe at String variable         String firstString = keyboard.next();         //Convert the String variable to int          int first=Integer.parseInt(firstString);         System.out.print("Type your seconds integer : ");         //Get String input and stroe at String variable         String secondString = keyboard.next();         //Convert the String variable to int          int second=Integer.parseInt(secondString);         System.out.print("The sum of your two integers are:" +(first+second));     } } 

Answers 8

As others have stated you can use

int first = keyboard.nextInt(); 

but you can also use the next() method along with a parseInt, which can have the handy benefit of showing an error message if you'd like.

import java.util.*; public class myClass { public static void main (String [] args) {     Scanner keyboard = new Scanner(System.in);     System.out.println("Type your first integer: ");     String firstStr = keyboard.next();     System.out.println("Type your seconds integer : ");     String secondStr = keyboard.next();     int first = -1; //assuming only positive integers     int second = -1;      do{       try{       first = Integer.parseInt(firstStr);       second = Integer.parseInt(secondStr);       int sum = first + second;       System.out.println("The sum of your two integers are: " + sum);      } catch (NumberFormatException exception) {       System.out.println("Integers only.");      }   } while (first=-1 && second=-1);  } } 

Answers 9

Change your first and second variables from

int first = keyboard.next();   int second = keyboard.next(); 

to

int first = keyboard.nextInt(); int second = keyboard.nextInt(); 

Answers 10

Problem is here:

int first = keyboard.next(); 

Scanner.next returns String, and you are expecting int, so you need to replace it with

int first = keyboard.nextInt(); 

As well

  int second = keyboard.nextInt(); 

Answers 11

you should replace next with nextInt, like this

int first = keyboard.nextInt(); 

From the API, you can see that the return type of next is String,while nextInt() is int.

Answers 12

Try using

keyboard.nextInt(); 

not

keyboard.next(); 

it displays that it cannot convert because .next is for strings, but for other data types is .next and then the data types with a capital. For example:

long l=keyboard.nextLong();  

and boolean b=keyboard.nextBoolean();

Answers 13

keyboard.next() will return the next complete token from the scanner ,i think of string type so it needs to be typecast to integer and better option is to use keyboard.nextInt() method to take the integer input .

Read More

Tuesday, March 7, 2017

Freeze a python script for every OS

Leave a Comment

In the last few years I've written a nice little program in python and now I'd like to distribute it, but my first attempt didn't encounter much enthusiasm, since many potential users didn't like the idea of downloading and installing python and all the dependencies necessary to run my application.

I looked around for weeks searching for a solution to compile somehow my code into something easy to run for everybody, but I'm not really satisfied of what I found.

I have no problem with the idea of freezing the application instead of really compiling it. It just means that the end user will wait a few seconds more to download the application and will occupy more space on his/her hard disk.

My problem is to find a way to package all the application into an executable for every OS (let's say the big ones: windows, mac, linux - obviously a different executable for every OS) without the need to install virtual machines or wine or similar options. I strongly suspect it is possible, since I saw a bunch of ren'py games packaged exactly the way I'd like to package my application, but I don't know how to obtain the same result for my program.

Schematically, my application requires a console to run and includes a few home-made packages, numpy and matplotlib. It also has a system to support multiple languages that dynamically includes a language file, overwriting the previous settings:

exec('from lang.%s import lang' % language) 

I suspect this could create some problems. Finally, the program requires a few folders to save its settings, logs, etc in the first run, so these should be included too. My computer runs ubuntu mate 16.10 (x64) and the python version is 2.7, but in the future I plan to translate it to python 3. The executables created can be single heavy files that contain almost everything needed or lightweight files and everything else can be found in a near folder; I don't really have a preference for one solution or the other.

I am aware that this question pops up from time to time and was probably already answered, but the last occurrence of this kind of question I could find dates back to 2014, which sounds like eons ago if we consider how quickly things change in this field.

Edit: Since there seems to be no solution with the conditions I proposed, I can think of setting up some emulators/wrapper/compatibility layer/whatever to run a freezer for different OSs, but I'm not sure what to do. I read here and there that wine doesn't always work properly and can't find anything to compile for mac on linux. As far as I understand the program shouldn't require a different compilation for 32/64 bit systems, but if there is an easy way to compile for every possibility it'd be nice. Virtual machines are still a no go, right now I don't have the space on my disk to set up two or more virtual machines (that's the great downside of ssd disks...), let alone paying for licences for OSs that would be used a few times a year to compile a free software.

8 Answers

Answers 1

Couple things first.

  • Python is already cross-platform
  • Python code is interpreted, not a natively compiled language and does not need compiling
  • There exist standard methods in Python setuptools to distribute executable scripts with cross-platform compatibility
  • Third party python libraries can include native code (in C) that will either need compiling when installing or for binary packages to have been available by the package maintainers. NumPy and matplotlib are in this category and do have binary packages available for windows AFAIK.

Here is some example setup.py code to install an executable script:

from setuptools import setup, find_packages setup(<...>,       entry_points={'console_scripts': [           '<console cmd name here> = my_package.module:main' ]}) 

Where my_package.module is, for example

my_package/module.py:

<..> def main():     <what to run when <console cmd name here> is executed> 

The package can then be distributed as a standard python module, installable via pip. If the code is open source, it can be hosted on PyPi (see instructions on PyPi site for how to setup).

Otherwise, a source or binary wheel package (python setup.py sdist and python setup.py bdist_wheel respectively) can be built and distributed to your users.

Binary wheels will need to built on the platform they are to be installed on, source distributions will work on any platform but will require code to be executed at installation as they contain purely source code.

Note that building/installing binary wheel packages requires pip version >= 6.0 and setuptools >= 26.0.

When this package is installed, a < console cmd name here > executable will be available to run on all platforms the package can be successfully installed.

For a simple example package that does this, see hello-world-python-package, specifically its setup.py.

See pythonwheels site for more information on binary wheel packages.

Answers 2

The only platform that expects programs to be a self-contained package is ms-windows. This is mostly for historical reasons pertaining to the native toolchain. For example up to version 2016 of the ms-toolchain (IIRC), Python extension modules like numpy must be compiled with exactly the same version of the ms-toolchain as Python itself. This makes building extension modules for ms-windows a huge pain.

Most UNIX-like operating systems have functional package management (although in Apple's OS-X this is supplied by a third-party). This makes installing Python relatively easy.

What could be helpful is to pack your application into a zipfile like youtube-dl does. I've written a script called build.py that does this for some of my programs.

Answers 3

Since I haven't seen it on here yet I thought I would bring up Docker. It appears to be a distributable VM that contains a package or application of some sort. There is a free community edition available on github https://github.com/docker/docker. I have never used it myself but it seems to fit all of your criteria - lightweight, free, and platform-agnostic. Just something to explore.

Answers 4

What about JAVA? Wrap you python code into java distribute as jar or .exe?

http://www.jython.org/jythonbook/en/1.0/JythonAndJavaIntegration.html

You may find this QA useful in your case. Distributing my python scripts as jars with jython?

Answers 5

I am don't know what do you really thing by writing "freezing" application, but if I understand it, you need to compile your code into a installer, that simply install all libraries and files needed to run your program, so it can handle any noob computer user.

Ok. Now you have two choices:

A) Make installer for each system separately - You can use tools such as Inno setup or PackageBuilder (on mac) to create installer, what check if is python already installed (if not install it) and copy your files + make shortcut on desktop (or something like that) + run external scripts (if you need) or if you are need binaries for windows you can use Py2exe or on mac Py2app. On Linux there is not anything like that, but if someone using Linux usually is it advanced user.

If I know there is not something like automatic builder for all platforms to build python installers, but if you have some money you can boost your work with Bitrock Install Builder.

B) Make your own script - If user already have python you can make script for downloading and installing files manually (by me best solution if is product made for advanced users, but if you make some GUI it not will be problem for noob)

Answers 6

Ren'Py

You mention Ren'Py a couple of times. Its documentation describes how to add Python code to a game. The Python Statements section even describes how to add third party packages (e.g. numpy and matplotlib). It can generate files for Windows, Mac, and Linux. If you can get it to run your application the way you want, and you are already familiar with it, then you should use it.

PyInstaller

Another option is PyInstaller. In the Supporting Multiple Operating Systems section, it says this:

If you need to distribute your application for more than one OS, for example both Windows and Mac OS X, you must install PyInstaller on each platform and bundle your app separately on each.

The second paragraph describes how to do this on a single machine:

You can do this from a single machine using virtualization. The free virtualBox or the paid VMWare and Parallels allow you to run another complete operating system as a “guest”. You set up a virtual machine for each “guest” OS. In it you install Python, the support packages your application needs, and PyInstaller.

PyInstaller advertises support for numpy and matplotlib.

Virtual Machines and Hosted Operating Systems

You said you don't have room on your SSD for virtual machines. You could buy an external USB hard drive and store the virtual machines on it. Or you could use an operating system hosting provider. You can use Google to find out more about those. If you have questions about how to use a specific hosting provider, you should create new questions.

Cross Platform Testing

Even if you use Ren'Py, you will almost certainly need to test the results on virtual machines or hosted operating systems. The only way to be certain that your program will run correctly on other platforms is to test it on those platforms. The hosting solution allows you to avoid "paying for licences for OSs that would be used a few times a year to compile a free software". However, you will still be spending money for the hosting.

The Price of Free

If your goal is to develop Python applications for Windows, Mac, and Linux without spending any money, then you are probably stuck with Ren'Py and relying on users to help you test your application on their operating systems.

Answers 7

Initial reaction

  • For windows use p2exe
  • For linux use python's freeze
  • For macos use possibly same freeze should work (just a guess held up by one google search)

(changes might be needed to your program e.g. __file__ can not be used) Regarding


Possible alternative approach to initial problem

<...> didn't encounter much enthusiasm, since many potential users didn't like the idea of downloading and installing python

Maybe user objections could be cured, by including python installer itself next to your program. (either actual python installer or http://portablepython.com/)
i.e. add not just link to installer, but all installers and possibly bat/sh scripts to install everything.

Answers 8

At work, I do something like what you want, and I ended up writing my own code to do it. Everything is built on Linux, but I ship both Linux and Windows packages. The packages include the Python interpreter and library, and they work identically whether or not the customer already has Python installed.

Essentially, you need a small program that embeds the Python interpreter and sets it up to load only the Python code you ship, ignoring any system-installed Python. Written properly, this program can be cross-compiled for each architecture you want to support. You then ship the Python DLLs you need for the architecture, along with a zip file that has the Python code you'll use (you only need the .pyc or .pyo files). This zip file must also contain as much of the standard library as you need.

Caveats:

  • You will still want virtual machines for testing.
  • Setting up the C cross-compilers correctly for each architecture can be challenging.
  • Neither Python's site.py nor the one that virtualenv uses completely isolates you from a system Python install. I had to make my own version of site.py.
  • It's easy to link against the system Python by mistake. Be very careful about how you use pkg-config and compiler arguments in your build scripts.
  • It's easy for Python to try to load the system Python library by mistake. Be very careful about setting the Python path and probably Py_IgnoreEnvironmentFlag.
  • For Windows, you can get quite far with py2exe, and it will be easier than what I'm suggesting. I don't use py2exe mainly because I'm embedding Python code in a non-Python application.

Because my application is in Go with some Python, I wrote my wrapper as a Go package that calls Python WSGI code via cgo. You can see most of the Python embedding code here. That repo is not the whole story, though: the scripts for the custom Python build and the build scripts and initialization code for the app itself aren't public right now.

Read More

Wednesday, February 15, 2017

Visual Studio : compile list of modules on each platform and configuration

Leave a Comment

I am working on a huge C++ project, targeting many platforms with several configurations for each platform.

Because of the long compilation time, build the entire project on every platform to test if a change compile successfully, isn't an option.

What I usually do, is compile the single cpp modules I modified on different combination of platform/configuration. I'd like to automate this process, either using a script, a VS extension, whatever, I am open to evaluate different options.

What I need exactly is taking a list of cpp files and compile each file, for each platform and each configuration (basically iterating through all combination of the configuration manager).

Is this possible? any good suggestion on how to approach the problem?


EDIT:

I am aware that this is way far to be a perfect solution, and will spot only a subset of errors. I will still have to face linking errors, compiler errors on other cpp units depended on a modified header, and so on..

I also, don't have any chance to modify the current build system, or project generation.

I am mostly interested in a local solution, to reduce the amount of possible issues and facing the huge building time process.

2 Answers

Answers 1

I would suggest that you "simply" write a script to do this (using Python for instance, which is very powerful for this kind of this)

You could:

  • Parse the .sln file to extract the list of configurations, platforms ( GlobalSection(SolutionConfigurationPlatforms) entry) and projects (Project entry)
  • If needed, you can parse every project to find the list of source files (that's easier than parsing the .sln, as vcxproj files are in xml). Look for ClCompile xml nodes to extract the list of .cpp files.
  • Then you can identify which projects needs some files to be recompiled (getting list of modified files as script input parameter or based on timestamp checking)

Finally, to rebuild, you have two options:

  • Call "msbuild " to recompile the whole project (vcxproj) (for instance msbuild project.vcxproj /p:Configuration=Debug;TargetFrameworkVersion=v3.5)
  • You could also recompile a single file (cl simple.cpp). To do so, you need to know what are the cl build options to be sure you compile the file exactly the same way as Visual Studio would. If you earlier did a full build of the solution (it could be a rquirement for your script to work), then you should be able to find that from Visual Studio logs (within the target folder). In my solutions, I can find for every project (vcxproj file) a build log per configuration (in %OUTPUT_DIR%\lib\%libname%\%libname%.dir\%configuration%\%libname%.tlog\CL.command.1.tlog), this file reports the exact cl arguments that were used to compile every file of the project. Then you can manually invoke cl command and this should end up recompiling the file the same way Visual Studio would do it.

Additionnaly, you could add a project in your Visual Studio solution that would fire this script as a custom command.

Such a script should be able to identify which projects has to be rebuilt and rebuild them.

Answers 2

This is a very common requirement, it is never solved this way. What you are proposing is not completely impossible, but it is certainly very painful to implement. You are overlooking what should happen when you modify a .h file, that can force a bunch of .cpp files to be recompiled. And you are not considering linker errors. While you'll have a shot at discovering .cpp files, discovering #include file dependencies is very gritty. You can't get them from the project or make file. Compiling with /showIncludes and parsing the build trace files is what it takes. Nothing off-the-shelf afaik.

Don't do this, you'll regret it. Use the solution that everybody uses: you need a build server. Preferably with a continuous integration feature so the server kicks-off the build for all target platforms as soon as you check-in a code change. Many to choose from, this Q+A talks about it.

Read More

Sunday, April 24, 2016

Get the commands distutils passes to the compiler

Leave a Comment

Lets say I have this Python code in a setup.py script to build a C extension:

from distutils.core import setup, Extension  module1 = Extension('demo', sources = ['demo.c'])  setup (name = 'PackageName',        version = '1.0',        description = 'This is a demo package',        ext_modules = [module1]) 

Easy enough. Now I call the setup.py script with this line:

C:/> python setup.py build_ext --compiler=mingw32 

Ok, but whats the question?

When distutils calls mingw32 and passes all the necessary and operating system independant flags and options to it, how does it figure those flags out?

Where does distutils keep the commands related to each platform, and how can I access them?

1 Answers

Answers 1

It's not as simple as a set of options but you can see how it works. In your python source directory look for this

distutils/ccompiler.py 

In that file each compiler has an entry like this

# Map compiler types to (module_name, class_name) pairs -- ie. where to # find the code that implements an interface to this compiler.  (The module # is assumed to be in the 'distutils' package.) compiler_class = { 'unix':    ('unixccompiler', 'UnixCCompiler',                                "standard UNIX-style compiler"),                    'msvc':    ('msvccompiler', 'MSVCCompiler',                                "Microsoft Visual C++"),                    'cygwin':  ('cygwinccompiler', 'CygwinCCompiler',                                "Cygwin port of GNU C Compiler for Win32"),                    'mingw32': ('cygwinccompiler', 'Mingw32CCompiler',                                "Mingw32 port of GNU C Compiler for Win32"),                    'bcpp':    ('bcppcompiler', 'BCPPCompiler',                                "Borland C++ Compiler"),                    'emx':     ('emxccompiler', 'EMXCCompiler',                                "EMX port of GNU C Compiler for OS/2"),                  }     

You can find the code you're looking for in

distutils/cygwinccompiler.py 

If you edit your setup.py script and add this

from distutils.core import setup,Extension from distutils.cygwinccompiler import Mingw32CCompiler from pprint import pprint  module1 = Extension('demo', sources = ['demo.c'])  m32 = Mingw32CCompiler() pprint (vars(m32))   setup (name = 'PackageName',    version = '1.0',    description = 'This is a demo package',    ext_modules = [module1]) 

You can see quite a few of the options available...

{'archiver': ['ar', '-cr'],  'compiler': ['gcc', '-O', '-Wall'],  'compiler_cxx': ['g++', '-O', '-Wall'],  'compiler_so': ['gcc', '-mdll', '-O', '-Wall'],  'dll_libraries': None,  'dllwrap_version': None,  'dry_run': 0,  'force': 0,  'gcc_version': LooseVersion ('4.2.1'),  'include_dirs': [],  'ld_version': None,  'libraries': [],  'library_dirs': [],  'linker_dll': 'dllwrap',  'linker_exe': ['gcc'],  'linker_so': ['dllwrap', '-mdll', '-static'],  'macros': [],  'objects': [],  'output_dir': None,  'preprocessor': None,  'ranlib': ['ranlib'],  'runtime_library_dirs': [],  'verbose': 0} 

To access individual options you can use them as follows...

print m32.compile ['gcc', '-O', '-Wall'] 

There's no simple set of flags. A lot of the flags are configured at runtime and the code above shows you were to look to see how they're generated etc.

Read More

Saturday, April 16, 2016

Weird error when trying to build

Leave a Comment

I have this code in a class called ctime:

#include <__config> #include <time.h>  #if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER) #pragma GCC system_header #endif  _LIBCPP_BEGIN_NAMEPSACE_STD  using ::clock_t; using ::size_t; using ::time_t; using ::tm; using ::clock; using ::difftime; using ::mktime; using ::time; using ::asctime; using ::ctime; using ::gmtime; using ::localtime; using ::strftime;  _LIBCPP_END_NAMESPACE_STD 

But I keep getting these errors:

errors

Anyone knows what the problem is?

2 Answers

Answers 1

Issue may be related to C++ standard library not found at the compile time

You can try the following ways:

  • Add an empty .mm file to you project! Ensure that C++ Standard Library is set to libstdc++ (GNU c++ standard library) in the Build Settings
    Ref: http://stackoverflow.com/a/19250215/4557505
    else
  • add -lstdc++ to Other Linker Flags! in the build settings
    else
  • add libstdc++.dylib/tbd and libstdc++.6.dylib/tbd in the build phases -> Link Binary with Library

Answers 2

So I found the answer, it was probably because I had a framework with a header inside called time.h which is saved name. I changed my file to mytime.h and it compiled perfectly.

Read More

Friday, April 15, 2016

Access restrictions in Eclipse Multiple Projects with Gradle

Leave a Comment

I have a Gradle Project in Eclipse consisting of multiple subprojects. I currently have subprojects A, B and C.

Project A should have access to Project B. Project B should have access to Project C. But project A should not have access to Project C.

A -> B, B -> C, but not A -> C 

I can easily test this by having a java example class in Project A which tries to use a class from Project C.

I've achieved this with Gradle using the following setup in the main build.gradle file and using the transitive property:

project(':projectA') {     dependencies {         compile project(':projectB'), {             transitive = false         }     } }  project(':projectB') {     dependencies {         compile project(':projectC'), {             transitive = false         }     } } 

Running Gradle's compileJava on Project A's example class gives the correct error message. I would like to have this error should up as a compile error in Eclipse. I was also able to manually configure the classpath in a way that the desired relationship holds, but a Gradle Refresh/Rebuild resets the classpath again.

Is it possible to use Gradle's Java Compiler instead of the Eclipse Compiler? Or should I influence the classpath files when doing a Gradle Refresh/Rebuild? Is there maybe a different solution?

I'd like to hear what is the preferred approach for this situation. Thanks!

2 Answers

Answers 1

You can use the gradle eclipse plugin to modify eclipse project settings from gradle, including changes to eclipse classpath. I would try something like this in build.gradle:

allprojects{     apply plugin:'eclipse'      eclipse.classpath.file {         beforeMerged { classpath -> classpath.entries.removeAll{it} }     } } 

and then run gradle eclipseClasspath to re-generate the .classpath file, and a general refresh / F5 in eclipse to pick up the modified .classpath file.


Demo:

I start with transitive=true in build.gradle. When a class in A instantiates a class in C, I do not get compile errors.

enter image description here

Now I change to transitive=false in build.gradle This causes compile failures from commandline but eclipse is happy with the old dependency information. Right-Click->Gradle->Refresh Project has no effect. To make gradle pick up the change, run gradle eclipseClasspath

gradle eclipseClasspath  :eclipseClasspath  :A:eclipseClasspath  :B:eclipseClasspath  :C:eclipseClasspath 

and have Eclipse pick up the changed .classpath files by doing a refresh.

enter image description here

Which makes eclipse recognize the missing transitives and throw compile errors:

enter image description here

My full root build.gradle at this point:

allprojects{     apply plugin:'java'     apply plugin:'eclipse'      eclipse.classpath.file {         beforeMerged { classpath -> classpath.entries.removeAll {it} }     } }  project(':A').dependencies {     compile project(':B'),{ transitive = false } }  project(':B').dependencies {     compile project(':C'),{ transitive = false } } 

Answers 2

The only way I was able to achieve this was to refer to the class files of the project, thus:

project(':projectA') {     dependencies {         compile files("../projectB/build/classes/main")     } } 

The path ../projectB/build/classes/main should point to where the class files are generated and stored in your project.

I also looked into replacing the Eclipse compiler with Gradle's compiler, but this is currently not supported by Eclipse.

Read More