C++ is an important part of Snap’s client development strategy due to its portability along with its ability to utilize the full potential of client hardware. It powers Snap’s Lens technology which is used in the Snapchat mobile app, Lens Studio, and Snap Camera. It also powers our voice and video calling, messaging, mobile UI frameworks, and a growing set of client infrastructure components.
Djinni is a project originally created by Dropbox that generates bridging code between C++ and other programming languages. It consists of an interface definition language (IDL), a code generator, and a small support library. Using interfaces generated by Djinni allows developers to seamlessly integrate components across Java and Objective-C while retaining the original feel of each language.
We chose to use Djinni as our cross-platform middleware because of its focus on mobile development and its ease of use. Today, the majority of Snapchat’s C++ code is exposed as Djinni interfaces. As our usage of Djinni grew, we saw opportunities to improve its performance, stability, and features to better suit our needs. In this post we will take a closer look at some of the improvements we made to Djinni.
Dropbox stopped maintaining the original Djinni repository in 2018. But even before that, we at Snap had already developed an internal fork of it. This fork is now public on Github and Snap is committed to ongoing development and maintenance of this repository.
String marshalling performance
A common concern amongst Android developers is the performance of calling library functions over Djinni’s generated Java Native Interface (JNI) code. We built a performance test app to measure the actual cost of these calls. What immediately caught our attention was that it was slow to pass large strings across the JNI boundary. For example, passing a 16KB string took more than 2 milliseconds on a fairly fast device. That is quite expensive for just one argument in one single call.
After some research, we discovered that the bottleneck was in Djinni’s string encoding and decoding routines. These functions convert between C++ std::string (assumed to be standard UTF-8 encoding) and Java’s modified UTF-8, which JNI functions expect. Because modified UTF-8 is a Java-specific encoding, it does not have a ready to use implementation in the C++ standard library. Djinni used a handwritten implementation which was not tuned for performance.
We experimented with a few methods to optimize this code and eventually settled on encoding C++ strings to standard UTF-16, which JNI functions can also accept. Switching to a standard encoding means we could take advantage of the high quality string encoding implementation in the C++ standard library. The string conversion functions went from 250 lines to just 37 lines while running much faster than before.
Our tests showed an improvement of 3-10x depending on string size (except for tiny strings of size < 16, where the overhead of the JNI call dominates):
Zero-copy buffers
Snapchat passes large chunks of binary data back and forth across the Djinni boundary. For example, media files, uncompressed video frames, or large serialized objects.
Djinni’s only option for passing binary data was the “binary” type, which copies data every time it crosses the boundary. This approach is straightforward and works well with small data objects. But when the data being passed gets bigger, the cost of copying can become significant.
To address this limitation, we developed a pair of new types to support passing data without copying: DataView and DataRef:
DataView is a non-owning viewport into a memory buffer. This is the fastest way to share data across the boundary as the only thing passed is a (pointer, size) pair. Being just a pointer to the memory, it is the programmer’s responsibility to keep the memory around while it’s being accessed on the other side of the Djinni boundary.
DataRef, on the other hand, is a native wrapper for a platform data object (ByteBuffer on Android and NSData on iOS). For example, on Android, when you create a DataRef object in C++, you essentially create a ByteBuffer object in the JVM heap, managed by the new DataRef object. This is why it’s called DataRef, as it works as a reference to the actual platform data object.
Unlike DataView, the ByteBuffer and NSData objects created by DataRef own their buffers. It is therefore safe to keep these ByteBuffer and NSData objects beyond the scope they are called with.
In short, if you need the ByteBuffer/NSData objects to keep the memory buffer alive then use DataRef. If you do not need them to manage the lifetime of the buffer and simply want a viewport into the memory managed elsewhere, then use DataView as it’s cheaper to create and pass.
One cool trick the DataRef class has is that in C++ it can take over the memory of a std::vector<uint8_t> or std::string without copying it, if you construct it with an R-value reference like this:
After the DataRef constructor call, the original vector becomes empty. Its data is moved into the DataRef object, which will continue to live on the other side of the Djinni boundary. This is an efficient way to pass large C++ vectors and have the Java/ObjC side taking ownership of the data.
Eliminating Java Finalizers
Since Djinni sits at the foundation of the Snapchat app’s C++ components, stability is of the utmost importance to us. One of the first problems we noticed after adopting Djinni was these finalizer crashes in Android:
The crash stacks were different each time, but they all happened in the Djinni generated CppProxy class’s finalizers. The CppProxy classes are what Java code uses to access C++ objects. They manage lifetime of a C++ object by calling its nativeDestroy() method in a finalizer:
This code is not incorrect by itself. The culprit is when this is combined with the way Android’s garbage collection loop operates. A simplified version of it looks like this:
If a GC cycle happens while the app is entering background mode, the Android system can suspend the process at the context switch to native code, in the finalizer. When the process resumes, it will see a very long execution time and raise a TimeoutException, even though the finalizer wasn’t running at all while suspended.
The chance of the OS suspending our process in the middle of a GC cycle is very low, so these crashes do not happen all that often. You probably will never see it if you only test your apps locally. But at Snapchat's scale, we have to make sure even the slightest chance of failure is addressed.
To fix this issue, we changed the code generator to make CppProxy objects register themselves with a NativeObjectManager object in their constructors and removed the finalizer code completely. This means we take over the task of finalizing native objects from the JVM and do it ourselves in a dedicated low priority cleanup thread outside of the Java GC.
As this change rolled out to users, we stopped seeing TimeoutExceptions in CppProxy objects.
Summary
Snap’s Djinni fork provides significant stability and performance improvements over the original version. These are just a few examples of them.
Djinni continues to be an important part of the Snap strategy for cross-platform development. We at Snap are committed to developing and maintaining Djinni in its new home.