Fixing Manual Focus - A Cracker's way

Background:
Soon after providing find7 HAL for OnePlus One, people started complaining of broken RAW image capture functionality (i'm neither RAW image user nor any expert in camera sensor architecture). An obvious reason is find 7 HAL uses different format for RAW images, which seems broken (again i'm no expert here). Lots of formats are specified for RAW images. This can be easily fixed by tweaking camera to use desired (in our case, which is used by CM HAL) format. This is simple one line code using set(String, String) method of camera parameters. But this will require modification in all applications. I believe in providing full compatibility so i thought about modifying CM HAL to fix Manual Focus functionality.

I was aware that our HAL is same as find7's and thus it also supports Manual Focus because running dumpsys media.camera shows that min-focus-pos-index and max-focus-pos-index parameters are set by HAL. Professional mode plug in utilizes these two params (there is another one also). Also if you have noticed, when you slide MF slider from near to infinity and vice versa, you'll see focus moving, a little bit but it responds.

So, i again dumped camera parameters and checked values for min-focus-pos-index and max-focus-pos-index. It was 40 and 60 respectively. If you look at the code for MF Slider, it builds up slider in such a way that its value will jump between min-focus-pos-index and max-focus-pos-index. This made everything clear. Since slider was causing value to jump between 40 and 60 it had only a small effect on focus. Besides i loaded find7 HAL and checked values for these two params and it was 0 and 300 respectively.

So, conclusion is we need to change value of min-focus-pos-index from 40 to 0 and value of max-focus-pos-index from 60 to 300.

I haven't tested, but we can simply change values of these params by set method of camera parameters, but again it requires to be done from all applications so i left that way. All parameters are explicitly defined by HAL which can be overridden by Application Layer as per requirement. I started looking into QCOM Camera HAL source for understanding how they are utilized. All parameters related operations are defined under QCameraParameters.cpp (this one is from Nexus 6's HAL). If you look further, default parameters and values are set by initDefaultParameters() function. So now we know what we need to do. We need to hook into this function and set proper values for target parameters.

Note: This is not as simple as modifying smali code because of various reasons. It requires deep understanding of ARM architecture and assembly code. So be prepared with a cup of coffee :)

The real challange:
So let's start with patching HAL's blob. We'll use IDA PRO for analyzing/understanding code and HxD for modifying blob (hex editing). Here i assume you know how to use IDA (No worries if you don't. It's easy. You can comment here if you need any help) So, open up IDA PRO and load camera.vendor.bacon.so (our stock camera HAL, you can find it on /system/lib/hw).

As we already know we need to look for initDefaultParameters() function, search it from Function Window. Now double click and open up function.

Now let's find our target parameter keys. Press ALT+T to bring up search window and search max-focus-pos-index. Look for the match found at 0x0006169E and 0x0006168A for min-focus-pos-index. Notice that instruction to be executed after both these line is

BLX             _ZN7android16CameraParameters3setEPKci

This relates to call to function set(char const*,int). Obviously this is the place from where values for min-focus-pos-index and max-focus-pos-index are set. Now we need to find the address of the values which are being passed as parameter to this function. As we already know the value (40 & 60), we'll just search it (else you need to trace down parameter registers & function calls)

See the highlighted lines in screenshot just some lines above, it loads 0x28 (40) in R0 and 0x3c (60) in R1.
Conclusion: We need to replace 0x28 with 0x0 and 0x3c with 0x12c.
Let's start with replacing 0x28 with 0x0. Open up HxD and load camera.vendor.bacon.so. Now in IDA notice the line

.text:0006166A MOVS R0, #0x28

From this, we can see that offset is 0x6166A. Click on this line and open up Hex view-1 tab in IDA. You'll see two blocks high lighted. We are going to replace these two. Open up HxD and jump to given offset. We know that we want to replace this line with

.text:0006166A MOVS R0, #0x0

Here 20 is instruction code (Little endian format) for MOVS R0 and 28 is the Hex value to be stored. So replace 28 with 00 and its all done.

Now its turn for replacing 0x3c with 0x12c. This is not as simple as replacing 0x28 with 0x0 because each register can store number in range of 0-255 only. To store larger numbers we need to use wider instructions like MOV.W. MOV.W which are 4-byte instructions so 2-byte instructions like MOV can't be replaced by them. See STR.W is at 0x6166E. Such cases require use of Code Caves. Finding and utilizing code cave is hard and very complex job and requires huge amount of efforts. So for this tutorial i won't be including this topic. In next tutorial i'll write about how i achieved it.

If you want to try this by yourself, you can replace 0x3c with 0xFF (255) the same way we did with 0x28 to see practical effects. There is nothing wrong with it but focus will move between 0-255 in this case.

Hope i was able to make you understand everything.
You can share your thoughts, doubts, questions in comments :)

Have a nice day...

Modifying OppoCamera's Plug ins

While fixing live effects in face beauty plug in, i needed to hook into Facebeauty plug in's APK (though i did it in past while debugging another plug ins, but i couldn't get time to write tutorial for it). Oppo has well written framework for loading plug ins. It uses DexClassLoader to load dex files at run time which interacts with camera app through predefined interfaces (defined in com.oppo.camera.capmode.CapModeBase). Each plug in's main class extends CapModeBase class and overrides needed methods. All plug ins have jar file included within them which are loaded at runtime

Now there are two ways to hook into Plug in:

1. Using Xposed Framework (Fairly simple)
2. Directly injecting code into smali

Using Xposed Framework:

Using Xposed to hook into plug in is very simple. All plug ins are loaded/managed by class com.oppo.camera.Plugin.PluginManager. Every time a plug in in installed, this class sets up environment to run that plug in by extracting libs and jar files. Have a look at following code

public static Map<String, ClassLoader> mClassLoaderIsLoaded;

private void updatePluginClassLoader(Plugin plugin) {
     DexClassLoader dexClassLoader = new DexClassLoader(modeJar.toString(), this.mContext.getFilesDir().getAbsolutePath(), libraryPath, this.mContext.getClassLoader());
     try {
          mClassLoaderIsLoaded.put(plugin.getCameraMode(), dexClassLoader);
          Constructor myConstructor = dexClassLoader.loadClass(plugin.getClassPackageName() + "." +      plugin.getCameraModeClassName()).getDeclaredConstructor(new Class[]{Activity.class, CameraInterface.class, CameraUIInterface.class});
          myConstructor.setAccessible(true);
          this.mPluginManagerListener.updateCameraMode((CapModeBase) myConstructor.newInstance(new Object[]{(Activity)      this.mContext, this.mCameraInterface, this.mCameraUIInterface}));
     } catch (Exception e3) {
          e2 = e3;
          DexClassLoader dexClassLoader2 = dexClassLoader;
     }
}

On startup, updatePluginClassLoader() method puts all classloaders for plug ins into HashSet mClassLoaderIsLoaded. So you first need to hook into PluginManager class to retrieve mClassLoaderIsLoaded. This can be done easily by

Map<String, ClassLoader> loaders = (Map) XposedHelpers.getObjectField(param.thisObject, "mClassLoaderIsLoaded");

Now you can retrieve plugin specific class loader by get method of Map like following snippet shows retrieval of classloader of Professional Camera Mode plug in

ClassLoader dexLoader = (DexClassLoader) loaders.get("professional");

Once we have classloader we can use it to hook into many class of target plug in like

Class manualFocusController = XposedHelpers.findClass("com.oppo.camera.professional.ManualFocusController", dexLoader);

You can find whole example code on pastebin

Directly Injecting smali code:

One problem is CameraApp is not loading plug in apk directly, instead it loads its jar file from assets directory. Code contained in jar file and classes.dex file are same. As usual we decompile classes.dex file and inject code into smali but things are a bit different here because classes.dex is never loaded, it is just placebo (may be because Android does not allow APKs without classes.dex?). So modifying smali code after normal decompilation will have no effect at run time. For that we need to decompile jar file included in assets directory. After modifying it, we also need to sign that jar file using same key used to sign APK file.

Enabling 50MP shots on CameraNext (COS12 stock camera app)

From the day i ported ColorOS camera on OPO, i was curious about technique behind 50MP shot function. Today while trying to fix preview bug on COS12, i came to following line in log

 E/QCameraParameters( 246): setSuperZoomMode:str_val:0,m_bSuperZoomEnabled:0  

Recall that Oppo named 50MP shot function as SuperZoom feature. Besides, these lines are printed by HAL so obviously it took my attention. I started further investigation and found that HD Picture plug in was utilizing these two variables. By investigating a little bit i came to know that there are two parameters which are used to manipulate SuperZoom function. It's easy to enable by setting following two parameters high-resolution and superzoom. superzoom parameter accepts binary value (i.e 0 or 1) and high-resolution param has something great to do (can't tell for sure because this technique is not revealed publicly by Oppo. more information is given below). So now that is confirm that 50MP function is provided by HAL, that's why it doesn't work on OxygenOS but works on COS12 (if i'm not wrong both CM11s and COS12 HAL are same. More surprisingly, those are written by Oppo)

So we just need to set two parameters to enable 50MP shots. I thought about testing it on CameraNext. I decompiled it and wasted much time on understanding f***ing cyngn's obfuscated code. But there was a surprise waiting for me. While finding proper place to inject my code i came to class com.android.camera.ap which basically checks preferences and sets camera parameters accordingly.I was surprised when  i saw these parameters were already defined in some function

public static void c(Parameters parameters, boolean z) {
    if (!ap.j(parameters)) {
        return;
    }
    if (z) {
        parameters.setFlashMode("off");
        ap.a(parameters, false);
        parameters.setPictureSize(4160, 3120);
        parameters.set("high-resolution", "1300");
        parameters.set("superzoom", "0");
        return;
    }
    parameters.set("high-resolution", "0");
    parameters.set("superzoom", "0");
}

So i started searching what was it using this function. After a bit messing with code i found that it was Clear Image which was using it. So Clear Image is actually SuperZoom technique but the difference was picture size remains same in case of Clear Image.

Now let me talk about how resolution is controlled in SuperZoom technique. It's too easy, it is controlled by high-resolution parameter provided by client. 1300 means 13MP and 5000 means 50MP (though i tried about 10000 value but it failed. Obviously there would be some guards in code :D)

So theoretical part ends here and now start injection of code into smali. As i told cyngn has obfuscated code of CameraNext so it is very hard to understand and implement. But another villain was ART. Yes ART was pain in a** while injecting code into smali because google decided not show much debugging messages when application is failed in compile time verification. Dalvik was showing dump of all registers in case of crash along with data types but ART just shows crash message. So it took longer than i expected.

Adding toggle was easy. I edited xml files under res/xml to add specific toggle for 50 MP. I decided to add 50MP functionality under Clear Image mode, This saved lots of my time. Now i just need to pass addition parameter with method c like

public static void c(Parameters parameters, boolean z, boolean z2) {
    if (!ap.j(parameters)) {
        return;
    }
    if (z) {
        parameters.setFlashMode("off");
        ap.a(parameters, false);
        parameters.setPictureSize(4160, 3120);
        parameters.set("high-resolution", z2 ? "5000" : "1300");
        parameters.set("superzoom", "0");
        return;
    }
    parameters.set("high-resolution", "0");
    parameters.set("superzoom", "0");
}

boolean z2 acts as a toggle for 50MP feature.

The most frustrating part was neither CyanogenMod nor OnePlus ever told us that Clear Image was actually SuperZoom technique by Oppo. I don't know what kind of deal they have but at least they should have informed us.

Fixing 4k recording bug in Color OS Camera on OnePlus One - COS12

So when lollipop was released, it broke 4k recording capabilities in Color OS camera. First i doubt it was something related with libraries (camera client etc.) or some changes in parameters but when i looked into bacon's (OnePlus One) device tree, my doubts were cleared. Now it was clear that there is no difference between them (KK and LP) in terms of libraries and HAL.Same is the case for Cyanogen OS 12. There is no major difference between KK and LP's camera module (a quick overview + log tells they both are almost same)

I was confused, what was going wrong then? Whenever we try to record 4k recording, it always records it in FWVGA resolution. I started debugging with Xposed to make things much easier but everything was fine there. Parameters were set correctly, log was not showing error etc. There were no errors on log which was making it difficult to find errors.

Now i decompiled CameraNext apk from COS 12 to see if there are any changes with calls to native libs. But damn Cyngn obfuscated it and so it was nearly impossible to trace down. I left it there.

After a week's drop (damn studies :( ) i started messing with it again. i was iterating over same and same functions again and again. Then suddenly i got attention over package android.media and class CamcorderProfile. These both are declared inside OppoCamera's package. What happens here is that when we call any member of class CamcorderProfile, they's are used from local package instead of global one (i.e from android framework). I started checking it  found that some integers were defined inside it. Those integers represents the video quality (i.e/ resolution, fps etc). These integers must match with same declared in native layer (see CamcorderProfile.java). I matched them with OppoCamera's variables and found the reason why 4k was not working on CM12, COS12. The reason was that integers representing quality were changed in LP. In KitKat, 4k (QUALITY_2160P) was represented by value 12 and 8 was used for FWVGA. In LP these both were swapped and so in LP, when Oppocamera was requesting 4k, actually it was requesting FWVGA.

So it is just a minor change :) but actually it is not that easy. Why? the reason is optimizations provided by compiler. Compiler inlines constant values to improve performance. In lining means it hardcodes value directly into the code instead of fetching it statically from class. Actually this is causing more problems because i need to find everyplace where 12 is used to represent 4k quality.

Now adding 4k dci was also easier as it was represented by value 13, i just included it into list and it started working. I also added compatibility code for KK with hack by using API level numbering so that i don't need to release two versions.

Now we have 4k recording working on COS12 but preview is bugged. I don't have any solution for it yet. I'm much busy at the moment so it'll take some time.

Hope you enjoyed....
This is my first blog post so deal with it...:D