I'd like to extract the actual address of java objects for some research purpose. Just to be clear, I actually want the 48bits virtual address of the object, not ID or hashcode or any unique identifier, and I understand that those addresses are moved around by the GC. I've been reading the other posts from stackoverflow like here or here.
For the following I use the @Peter Lawrey -> Is there a way to get a reference address? method. So it uses the Unsafe class with the arrayBaseOffset method. What I found strange about those methods is that they give the same result for every run (at least on my computer) which is very unlikely to happen. Memory allocation is supposed to be randomized for security reasons.
Moreover, I tried to verify those methods with Pintools which is the instrumentation tool from Intel that I used to extract memory traces of the run. My problem is that I am not able to correlate what I see in the memory trace of Pintools with the addresses given by the above methods to get the memory addresses. The given addresses are never accessed in my memory trace.
So I am wondering what is returned by those methods and how those results have been verified it against other tools.
Some infos: my OS is an Ubuntu x86_64, my JVM is the openJDK 64bits 1.8.0_131, pintools version is v3.2
=================== Big Edit: I realize that my question is not well put, so let me get a more atomic example, here is the java that I try to analyze:
`import sun.misc.Unsafe; import java.lang.reflect.Field; public class HelloWorld { public static void main(String[] args) throws Exception { Unsafe unsafe = getUnsafeInstance(); Integer i = new Integer(42); long addr_fromArray; long addr_fromObject; ///////////////////////////////////// Object[] objects = {i}; long baseOffset = unsafe.arrayBaseOffset(Object[].class); addr_fromArray = unsafe.getLong(objects, baseOffset); long factor1 = 8; long addr_withFactor = (unsafe.getInt(objects, baseOffset) & 0xFFFFFFFFL) * factor1; ///////////////////////////////////// class Pointer { Object pointer; } Pointer pointer = new Pointer(); pointer.pointer = i; long offset = unsafe.objectFieldOffset(Pointer.class.getDeclaredField("pointer")); addr_fromObject = unsafe.getLong(pointer, offset); System.out.println("Addr of i from Array : 0x" + Long.toHexString(addr_fromArray)); System.out.println("Addr of i from Object : 0x" + Long.toHexString(addr_fromObject)); System.out.println("Addr of i from factor1 : 0x" + Long.toHexString(addr_withFactor)); System.out.println("!=1");//Launch the pintools instrumentation for(int a= 0 ; a < 123 ;a++){ i = 10; } System.out.println("!=1");//Stop the pintools instrumentation } private static Unsafe getUnsafeInstance() throws SecurityException, NoSuchFieldException, IllegalArgumentException, IllegalAccessException { Field theUnsafeInstance = Unsafe.class.getDeclaredField("theUnsafe"); theUnsafeInstance.setAccessible(true); return (Unsafe) theUnsafeInstance.get(Unsafe.class); } }` I get the pointer to the i Integer from different methods that I have seen on stack overflow. Then I do a loop on i for an arbitrary number of time so I could recognize it in my memory trace (Note: I checked that no GC calls occurs within this code)
When pintools see the specific "!=1" written in the standard output, it starts /stop the instrumentation
On every access during the instrumentation phase, I execute this code:
VOID RecordAccess(VOID* ip, int id_thread , VOID * addr, int id) { PIN_GetLock(&lock, id_thread); if(startInstru) { log1 << "Data accessed: " << addr << "\tThread:" << id_thread << endl; nb_access++; uint64_t dummy = reinterpret_cast<uint64_t>(addr); if(accessPerAddr.count(dummy) == 0) accessPerAddr.insert(pair<uint64_t,uint64_t>(dummy, 0)); accessPerAddr[dummy]++; } } With this pintools, I generate a memory trace + a histogramm on how many times each memory addresses is accessed. Note: the pintool is launched with the "follow_execv" option in order to instrument every threads.
I see 2 Problems:
1) I see no accesses to any of the printed i addresses (or close to this address). I tend to trust Pintools because I have used quite a lot before but maybe Pintools is not able to retrieve the correct addresses here.
2) I see no addresses being accessed 123 times (or close to this). My thoughts for this is that maybe JVM performs optimization here because it sees that code executed has no effect so it doesn't execute it. However, I tried with a more complex instruction (that cannot be optimized like storing a random number) inside the loop than just a store to i without better results.
I don't care much about the GC effect here, maybe in the second step. I only want to be able to extract native addresses from my java app that I am pretty sure Pintools is giving me.
1 Answers
Answers 1
So when I am instrumenting this run with pintools, with a similar script as here . I don't see any access performed on the mentionned addresses or nearby addresses
I think you should give more information how you run and what you see.
To explore object layouts you could use http://openjdk.java.net/projects/code-tools/jol.
import org.openjdk.jol.info.GraphLayout; import java.io.PrintWriter; import java.util.Arrays; import java.util.Collections; import java.util.SortedSet; public class OrderOfObjectsAfterGCMain2 { public static void main(String... args) { Double[] ascending = new Double[16]; for (int i = 0; i < ascending.length; i++) ascending[i] = (double) i; Double[] descending = new Double[16]; for (int i = descending.length - 1; i >= 0; i--) descending[i] = (double) i; Double[] shuffled = new Double[16]; for (int i = 0; i < shuffled.length; i++) shuffled[i] = (double) i; Collections.shuffle(Arrays.asList(shuffled)); System.out.println("Before GC"); printAddresses("ascending", ascending); printAddresses("descending", descending); printAddresses("shuffled", shuffled); System.gc(); System.out.println("\nAfter GC"); printAddresses("ascending", ascending); printAddresses("descending", descending); printAddresses("shuffled", shuffled); System.gc(); System.out.println("\nAfter GC 2"); printAddresses("ascending", ascending); printAddresses("descending", descending); printAddresses("shuffled", shuffled); } public static void printAddresses(String label, Double[] array) { PrintWriter pw = new PrintWriter(System.out, true); pw.print(label + ": "); // GraphLayout.parseInstance((Object) array).toPrintable() has more info SortedSet<Long> addresses = GraphLayout.parseInstance((Object) array).addresses(); Long first = addresses.first(), previous = first; pw.print(Long.toHexString(first)); for (Long address : addresses) { if (address > first) { pw.print(Long.toHexString(address - previous) + ", "); previous = address; } } pw.println(); } With this tool I have approximately the same results:
Before GC # WARNING: Unable to attach Serviceability Agent. Unable to attach even with escalated privileges: null ascending: 76d430c7850, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, descending: 76d430e4850, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, shuffled: 76d43101850, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, After GC ascending: 6c782859856d88, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, descending: 6c78285e856eb8, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, shuffled: 6c782863856fe8, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, After GC 2 ascending: 6c7828570548a8, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, descending: 6c78285c0549d8, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, shuffled: 6c782861054b08, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, 18, Process finished with exit code 0 With this example http://hg.openjdk.java.net/code-tools/jol/file/018c0e12f70f/jol-samples/src/main/java/org/openjdk/jol/samples/JOLSample_21_Arrays.java you could test GC affects on arrays.
0 comments:
Post a Comment