Java is a safe programming language and prevents programmerfrom doing a lot of stupid mistakes, most of which based on memory management.But, there is a way to do such mistakes intentionally, using This article is a quick overview of Unsafe instantiationBefore usage, we need to create instance of public static Unsafe getUnsafe() { Class cc = sun.reflect.Reflection.getCallerClass(2); if (cc.getClassLoader() != null) throw new SecurityException('Unsafe'); return theUnsafe;} This is how java validates if code is trusted.It is just checking that our code was loaded withprimary classloader. We can make our code “trusted”. Use option java -Xbootclasspath:/usr/jdk1.7.0/jre/lib/rt.jar:. com.mishadoff.magic.UnsafeClient But it’s too hard.
Field f = Unsafe.class.getDeclaredField('theUnsafe');f.setAccessible(true);Unsafe unsafe = (Unsafe) f.get(null); Note: Ignore your IDE. For example, eclipse show error “Access restriction…”but if you run code, all works just fine. If the error is annoying, ignore errors on Preferences -> Java -> Compiler -> Errors/Warnings ->Deprecated and restricted API -> Forbidden reference -> Warning Unsafe APIClass sun.misc.Unsafeconsists of
Interesting use casesAvoid initialization
class A { private long a; // not initialized value public A() { this.a = 1; // initialization } public long a() { return this.a; }} Instantiating it using constructor, reflection and unsafe givesdifferent results. A o1 = new A(); // constructoro1.a(); // prints 1A o2 = A.class.newInstance(); // reflectiono2.a(); // prints 1A o3 = (A) unsafe.allocateInstance(A.class); // unsafeo3.a(); // prints 0 Just think what happens to all your Singletons. Memory corruptionThis one is usual for every C programmer.By the way, its common technique for security bypass. Consider some simple class that check access rules: class Guard { private int ACCESS_ALLOWED = 1; public boolean giveAccess() { return 42 == ACCESS_ALLOWED; }} The client code is very secure and calls In fact, it’s not true. Here is the code demostrates it: Guard guard = new Guard();guard.giveAccess(); // false, no access// bypassUnsafe unsafe = getUnsafe();Field f = guard.getClass().getDeclaredField('ACCESS_ALLOWED');unsafe.putInt(guard, unsafe.objectFieldOffset(f), 42); // memory corruptionguard.giveAccess(); // true, access granted Now all clients will get unlimited access. Actually, the same functionality can be achieved by reflection.But interesting, that we can modify any object, even onesthat we do not have references to. For example, there is another unsafe.putInt(guard, 16 + unsafe.objectFieldOffset(f), 42); // memory corruption Note, we didn’t use any reference to this object. sizeOfUsing public static long sizeOf(Object o) { Unsafe u = getUnsafe(); HashSet<Field> fields = new HashSet<Field>(); Class c = o.getClass(); while (c != Object.class) { for (Field f : c.getDeclaredFields()) { if ((f.getModifiers() & Modifier.STATIC) == 0) { fields.add(f); } } c = c.getSuperclass(); } // get offset long maxSize = 0; for (Field f : fields) { long offset = u.objectFieldOffset(f); if (offset > maxSize) { maxSize = offset; } } return ((maxSize/8) + 1) * 8; // padding} Algorithm is the following: go through all non-static fields including allsuperclases, get offset for each field, find maximum and add padding.Probably, I missed something, but idea is clear. Much simpler public static long sizeOf(Object object){ return getUnsafe().getAddress( normalize(getUnsafe().getInt(object, 4L)) + 12L);}
private static long normalize(int value) { if(value >= 0) return value; return (~0L >>> 32) & value;} Awesome, this method returns the same result as our previous In fact, for good, safe and accurate Shallow copyHaving implementation of calculating shallow object size, we can simplyadd function that copy objects. Standard solution need modify your code with Shallow copy: static Object shallowCopy(Object obj) { long size = sizeOf(obj); long start = toAddress(obj); long address = getUnsafe().allocateMemory(size); getUnsafe().copyMemory(start, address, size); return fromAddress(address);}
static long toAddress(Object obj) { Object[] array = new Object[] {obj}; long baseOffset = getUnsafe().arrayBaseOffset(Object[].class); return normalize(getUnsafe().getInt(array, baseOffset));}static Object fromAddress(long address) { Object[] array = new Object[] {null}; long baseOffset = getUnsafe().arrayBaseOffset(Object[].class); getUnsafe().putLong(array, baseOffset, address); return array[0];} This copy function can be used to copy object of any type, its size will be calculateddynamically. Note that after copying you need to cast object to specific type. Hide PasswordOne more interesting usage of direct memory access in Most of the APIs for retrieving user’s password, have signatureas It is completely for security reason, because we can nullify array elements after we don’t need them.If we retrieve password as This trick creates fake String password = new String('l00k@myHor$e');String fake = new String(password.replaceAll('.', '?'));System.out.println(password); // l00k@myHor$eSystem.out.println(fake); // ????????????getUnsafe().copyMemory( fake, 0L, null, toAddress(password), sizeOf(password));System.out.println(password); // ????????????System.out.println(fake); // ???????????? Feel safe. UPDATE: That way is not really safe. For real safety we need to nullifybacked char array via reflection: Field stringValue = String.class.getDeclaredField('value');stringValue.setAccessible(true);char[] mem = (char[]) stringValue.get(password);for (int i=0; i < mem.length; i++) { mem[i] = '?';} Thanks to Peter Verhas for pointing out that. Multiple InheritanceThere is no multiple inheritance in java. Correct, except we can cast every type to every another one, if we want. long intClassAddress = normalize(getUnsafe().getInt(new Integer(0), 4L));long strClassAddress = normalize(getUnsafe().getInt('', 4L));getUnsafe().putAddress(intClassAddress + 36, strClassAddress); This snippet adds (String) (Object) (new Integer(666)) One problem that we must do it with pre-casting to object. To cheat compiler. Dynamic classesWe can create classes in runtime, for example fromcompiled byte[] classContents = getClassContent();Class c = getUnsafe().defineClass( null, classContents, 0, classContents.length); c.getMethod('a').invoke(c.newInstance(), null); // 1 And reading from file defined as: private static byte[] getClassContent() throws Exception { File f = new File('/home/mishadoff/tmp/A.class'); FileInputStream input = new FileInputStream(f); byte[] content = new byte[(int)f.length()]; input.read(content); input.close(); return content;} This can be useful, when you must create classes dynamically, some proxiesor aspects for existing code. Throw an ExceptionDon’t like checked exceptions? Not a problem. getUnsafe().throwException(new IOException()); This method throws checked exception, but your code not forced to catch or rethrow it.Just like runtime exception. Fast SerializationThis one is more practical. Everyone knows that standard java
Popular high-performance libraries, like kryohave dependencies, which can be unacceptable with low-memory requirements. But full serialization cycle can be easily achieved with unsafe class. Serialization:
You can also add compression to save space. Deserialization:
Actually, there are much more details in correct inplementation, but intuition is clear. This serialization will be really fast. By the way, there are some attempts in Big ArraysAs you know Here is class SuperArray { private final static int BYTE = 1; private long size; private long address; public SuperArray(long size) { this.size = size; address = getUnsafe().allocateMemory(size * BYTE); } public void set(long i, byte value) { getUnsafe().putByte(address + i * BYTE, value); } public int get(long idx) { return getUnsafe().getByte(address + idx * BYTE); } public long size() { return size; }} And sample usage: long SUPER_SIZE = (long)Integer.MAX_VALUE * 2;SuperArray array = new SuperArray(SUPER_SIZE);System.out.println('Array size:' + array.size()); // 4294967294for (int i = 0; i < 100; i++) { array.set((long)Integer.MAX_VALUE + i, (byte)3); sum += array.get((long)Integer.MAX_VALUE + i);}System.out.println('Sum of 100 elements:' + sum); // 300 In fact, this technique uses Memory allocated this way not located in the heap and not under GC management, so take care of itusing It can be useful for math computations, where code can operate with large arrays of data.Also, it can be interesting for realtime programmers, where GC delays on large arrays canbreak the limits. ConcurrencyAnd few words about concurrency with For example, consider the problem to increment value in the shared objectusing lot of threads. First we define simple interface interface Counter { void increment(); long getCounter();} Then we define worker thread class CounterClient implements Runnable { private Counter c; private int num; public CounterClient(Counter c, int num) { this.c = c; this.num = num; } @Override public void run() { for (int i = 0; i < num; i++) { c.increment(); } }} And this is testing code: int NUM_OF_THREADS = 1000;int NUM_OF_INCREMENTS = 100000;ExecutorService service = Executors.newFixedThreadPool(NUM_OF_THREADS);Counter counter = ... // creating instance of specific counterlong before = System.currentTimeMillis();for (int i = 0; i < NUM_OF_THREADS; i++) { service.submit(new CounterClient(counter, NUM_OF_INCREMENTS));}service.shutdown();service.awaitTermination(1, TimeUnit.MINUTES);long after = System.currentTimeMillis();System.out.println('Counter result: ' + c.getCounter());System.out.println('Time passed in ms:' + (after - before)); First implementation is not-synchronized counter: class StupidCounter implements Counter { private long counter = 0; @Override public void increment() { counter++; } @Override public long getCounter() { return counter; }} Output: Counter result: 99542945Time passed in ms: 679 Working fast, but no threads management at all, so result is inaccurate.Second attempt, add easiest java-way synchronization: class SyncCounter implements Counter { private long counter = 0; @Override public synchronized void increment() { counter++; } @Override public long getCounter() { return counter; }} Output: Counter result: 100000000Time passed in ms: 10136 Radical synchronization always work. But timings is awful.Let’s try class LockCounter implements Counter { private long counter = 0; private WriteLock lock = new ReentrantReadWriteLock().writeLock(); @Override public void increment() { lock.lock(); counter++; lock.unlock(); } @Override public long getCounter() { return counter; }} Output: Counter result: 100000000Time passed in ms: 8065 Still correct, and timings are better. What about atomics? class AtomicCounter implements Counter { AtomicLong counter = new AtomicLong(0); @Override public void increment() { counter.incrementAndGet(); } @Override public long getCounter() { return counter.get(); }} Output: Counter result: 100000000Time passed in ms: 6552
class CASCounter implements Counter { private volatile long counter = 0; private Unsafe unsafe; private long offset; public CASCounter() throws Exception { unsafe = getUnsafe(); offset = unsafe.objectFieldOffset(CASCounter.class.getDeclaredField('counter')); } @Override public void increment() { long before = counter; while (!unsafe.compareAndSwapLong(this, offset, before, before + 1)) { before = counter; } } @Override public long getCounter() { return counter; } Output: Counter result: 100000000Time passed in ms: 6454 Hmm, seems equal to atomics. Maybe atomics use In fact this example is easy enough, but it shows some power of As I said,
Actually, in real it is more hard than you can imagine. There are a lot of problems likeABA Problem, instructions reordering, etc. If you really interested, you can refer to the awesome presentation about lock-free HashMap UPDATE: Added Kudos to Nitsan Wakart BonusDocumentation for
ConclusionAlthough, |
|