Sunday, 24 May 2015

Modifying OppoCamera's Plug ins

While fixing live effects in face beauty plug in, i needed to hook into Facebeauty plug in's APK (though i did it in past while debugging another plug ins, but i couldn't get time to write tutorial for it). Oppo has well written framework for loading plug ins. It uses DexClassLoader to load dex files at run time which interacts with camera app through predefined interfaces (defined in com.oppo.camera.capmode.CapModeBase). Each plug in's main class extends CapModeBase class and overrides needed methods. All plug ins have jar file included within them which are loaded at runtime

Now there are two ways to hook into Plug in:

  1. Using Xposed Framework (Fairly simple)
  2. Directly injecting code into smali

Using Xposed Framework:

Using Xposed to hook into plug in is very simple. All plug ins are loaded/managed by class com.oppo.camera.Plugin.PluginManager. Every time a plug in in installed, this class sets up environment to run that plug in by extracting libs and jar files. Have a look at following code

public static Map<String, ClassLoader> mClassLoaderIsLoaded;

private void updatePluginClassLoader(Plugin plugin) {
     DexClassLoader dexClassLoader = new DexClassLoader(modeJar.toString(), this.mContext.getFilesDir().getAbsolutePath(), libraryPath, this.mContext.getClassLoader());
     try {
          mClassLoaderIsLoaded.put(plugin.getCameraMode(), dexClassLoader);
          Constructor myConstructor = dexClassLoader.loadClass(plugin.getClassPackageName() + "." +      plugin.getCameraModeClassName()).getDeclaredConstructor(new Class[]{Activity.class, CameraInterface.class, CameraUIInterface.class});
          myConstructor.setAccessible(true);
          this.mPluginManagerListener.updateCameraMode((CapModeBase) myConstructor.newInstance(new Object[]{(Activity)      this.mContext, this.mCameraInterface, this.mCameraUIInterface}));
     } catch (Exception e3) {
          e2 = e3;
          DexClassLoader dexClassLoader2 = dexClassLoader;
     }
}
On startup, updatePluginClassLoader() method puts all classloaders for plug ins into HashSet mClassLoaderIsLoaded. So you first need to hook into PluginManager class to retrieve mClassLoaderIsLoaded. This can be done easily by
Map<String, ClassLoader> loaders = (Map) XposedHelpers.getObjectField(param.thisObject, "mClassLoaderIsLoaded");
Now you can retrieve plugin specific class loader by get method of Map like following snippet shows retrieval of classloader of Professional Camera Mode plug in
ClassLoader dexLoader = (DexClassLoader) loaders.get("professional");
Once we have classloader we can use it to hook into many class of target plug in like
Class manualFocusController = XposedHelpers.findClass("com.oppo.camera.professional.ManualFocusController", dexLoader);
You can find whole example code on pastebin

Directly Injecting smali code:

One problem is CameraApp is not loading plug in apk directly, instead it loads its jar file from assets directory. Code contained in jar file and classes.dex file are same. As usual we decompile classes.dex file and inject code into smali but things are a bit different here because classes.dex is never loaded, it is just placebo (may be because Android does not allow APKs without classes.dex?). So modifying smali code after normal decompilation will have no effect at run time. For that we need to decompile jar file included in assets directory. After modifying it, we also need to sign that jar file using same key used to sign APK file.


Sunday, 26 April 2015

Enabling 50MP shots on CameraNext (COS12 stock camera app)

From the day i ported ColorOS camera on OPO, i was curious about technique behind 50MP shot function. Today while trying to fix preview bug on COS12, i came to following line in log
 E/QCameraParameters( 246): setSuperZoomMode:str_val:0,m_bSuperZoomEnabled:0  
Recall that Oppo named 50MP shot function as SuperZoom feature. Besides, these lines are printed by HAL so obviously it took my attention. I started further investigation and found that HD Picture plug in was utilizing these two variables. By investigating a little bit i came to know that there are two parameters which are used to manipulate SuperZoom function. It's easy to enable by setting following two parameters high-resolution and superzoom. superzoom parameter accepts binary value (i.e 0 or 1) and high-resolution param has something great to do (can't tell for sure because this technique is not revealed publicly by Oppo. more information is given below). So now that is confirm that 50MP function is provided by HAL, that's why it doesn't work on OxygenOS but works on COS12 (if i'm not wrong both CM11s and COS12 HAL are same. More surprisingly, those are written by Oppo)

So we just need to set two parameters to enable 50MP shots. I thought about testing it on CameraNext. I decompiled it and wasted much time on understanding f***ing cyngn's obfuscated code. But there was a surprise waiting for me. While finding proper place to inject my code i came to class com.android.camera.ap which basically checks preferences and sets camera parameters accordingly.I was surprised when  i saw these parameters were already defined in some function
public static void c(Parameters parameters, boolean z) {
    if (!ap.j(parameters)) {
        return;
    }
    if (z) {
        parameters.setFlashMode("off");
        ap.a(parameters, false);
        parameters.setPictureSize(4160, 3120);
        parameters.set("high-resolution", "1300");
        parameters.set("superzoom", "0");
        return;
    }
    parameters.set("high-resolution", "0");
    parameters.set("superzoom", "0");
}
So i started searching what was it using this function. After a bit messing with code i found that it was Clear Image which was using it. So Clear Image is actually SuperZoom technique but the difference was picture size remains same in case of Clear Image.

Now let me talk about how resolution is controlled in SuperZoom technique. It's too easy, it is controlled by high-resolution parameter provided by client. 1300 means 13MP and 5000 means 50MP (though i tried about 10000 value but it failed. Obviously there would be some guards in code :D)

So theoretical part ends here and now start injection of code into smali. As i told cyngn has obfuscated code of CameraNext so it is very hard to understand and implement. But another villain was ART. Yes ART was pain in a** while injecting code into smali because google decided not show much debugging messages when application is failed in compile time verification. Dalvik was showing dump of all registers in case of crash along with data types but ART just shows crash message. So it took longer than i expected.

Adding toggle was easy. I edited xml files under res/xml to add specific toggle for 50 MP. I decided to add 50MP functionality under Clear Image mode, This saved lots of my time. Now i just need to pass addition parameter with method c like
public static void c(Parameters parameters, boolean z, boolean z2) {
    if (!ap.j(parameters)) {
        return;
    }
    if (z) {
        parameters.setFlashMode("off");
        ap.a(parameters, false);
        parameters.setPictureSize(4160, 3120);
        parameters.set("high-resolution", z2 ? "5000" : "1300");
        parameters.set("superzoom", "0");
        return;
    }
    parameters.set("high-resolution", "0");
    parameters.set("superzoom", "0");
}
boolean z2 acts as a toggle for 50MP feature.

The most frustrating part was neither CyanogenMod nor OnePlus ever told us that Clear Image was actually SuperZoom technique by Oppo. I don't know what kind of deal they have but at least they should have informed us.

Thursday, 23 April 2015

Fixing 4k recording bug in Color OS Camera on OnePlus One - COS12

So when lollipop was released, it broke 4k recording capabilities in Color OS camera. First i doubt it was something related with libraries (camera client etc.) or some changes in parameters but when i looked into bacon's (OnePlus One) device tree, my doubts were cleared. Now it was clear that there is no difference between them (KK and LP) in terms of libraries and HAL.Same is the case for Cyanogen OS 12. There is no major difference between KK and LP's camera module (a quick overview + log tells they both are almost same)

I was confused, what was going wrong then? Whenever we try to record 4k recording, it always records it in FWVGA resolution. I started debugging with Xposed to make things much easier but everything was fine there. Parameters were set correctly, log was not showing error etc. There were no errors on log which was making it difficult to find errors.

Now i decompiled CameraNext apk from COS 12 to see if there are any changes with calls to native libs. But damn Cyngn obfuscated it and so it was nearly impossible to trace down. I left it there.

After a week's drop (damn studies :( ) i started messing with it again. i was iterating over same and same functions again and again. Then suddenly i got attention over package android.media and class CamcorderProfile. These both are declared inside OppoCamera's package. What happens here is that when we call any member of class CamcorderProfile, they's are used from local package instead of global one (i.e from android framework). I started checking it  found that some integers were defined inside it. Those integers represents the video quality (i.e/ resolution, fps etc). These integers must match with same declared in native layer (see CamcorderProfile.java). I matched them with OppoCamera's variables and found the reason why 4k was not working on CM12, COS12. The reason was that integers representing quality were changed in LP. In KitKat, 4k (QUALITY_2160P) was represented by value 12 and 8 was used for FWVGA. In LP these both were swapped and so in LP, when Oppocamera was requesting 4k, actually it was requesting FWVGA.

So it is just a minor change :) but actually it is not that easy. Why? the reason is optimizations provided by compiler. Compiler inlines constant values to improve performance. In lining means it hardcodes value directly into the code instead of fetching it statically from class. Actually this is causing more problems because i need to find everyplace where 12 is used to represent 4k quality.

Now adding 4k dci was also easier as it was represented by value 13, i just included it into list and it started working. I also added compatibility code for KK with hack by using API level numbering so that i don't need to release two versions.

Now we have 4k recording working on COS12 but preview is bugged. I don't have any solution for it yet. I'm much busy at the moment so it'll take some time.

Hope you enjoyed....
This is my first blog post so deal with it...:D