Android开发笔记： 3.11 System info, FileSystem, Security and Encryption

3.11 System info, FileSystem, Security and Encryption

3.11.1 System info

3.11.1.1 Retrieve an ID

Refer to so1, so2 and so3. According to this article, android.os.Build.SERIAL should be unique if it is available. From the article: Devices without telephony are required to report a unique device ID here; some phones may do so also.

The 1^st link has summary of phone:

All devices tested returned a value for TelephonyManager.getDeviceId()

1. All GSM devices (all tested with a SIM) returned a value for TelephonyManager.getSimSerialNumber()
2. All CDMA devices returned null for getSimSerialNumber() (as expected)
3. All devices with a Google account added returned a value for ANDROID_ID
4. All CDMA devices returned the same value (or derivation of the same value) for both ANDROID_ID and TelephonyManager.getDeviceId() -- as long as a Google account has been added during setup.
5. I did not yet have a chance to test GSM devices with no SIM, a GSM device with no Google account added, or any of the devices in airplane mode.

So if you want something unique to the device itself, TM.getDeviceId() should be sufficient. Obviously some users are more paranoid than others, so it might be useful to hash 1 or more of these identifiers, so that the string is still virtually unique to the device, but does not explicitly identify the user's actual device. For example, using String.hashCode(), combined with a UUID:

final TelephonyManager tm = (TelephonyManager) getBaseContext().getSystemService(Context.TELEPHONY_SERVICE);
final String tmDevice, tmSerial, androidId;
tmDevice = "" + tm.getDeviceId();
tmSerial = "" + tm.getSimSerialNumber();
androidId=""+android.provider.Settings.Secure.getString(getContentResolver(),android.provider.Settings.Secure.ANDROID_ID);
UUID deviceUuid = new UUID(androidId.hashCode(), ((long)tmDevice.hashCode() << 32) | tmSerial.hashCode());
String deviceId = deviceUuid.toString();

might result in something like: 00000000-54b3-e7c7-0000-000046bffd97

3.11.2 Filesystem

/data/app: apk files of all installed apps.

/data/app-lib: package subfolder which contains libs

/data/data: package subfolder which contains app files, include shared_prefs

3.11.3 Protect App source from decompiling: proguard

Refer to hackernews and this post for security apps. For security programing, refer to android crypto and this tutor. My thought and learnt from other app: use JNI which cannot be easily decompiled; rename class name to meaning-less; put in a lot of redundant code to confuse decompiler; Use filesize info as usually decompile and recompile will change the size significantly. To protect Java code against decompiler, here is several links, SO, Eclipse, xda.

1. You can use an obfuscator, like proguard or Ygard, but it is not too complex to decrypt strings and rename classes, fields and methods. DexGuard is related to proguard and is specially for Android, but is not open source, though a free quote available.
2. You can encrypt your classes with a private key, and use a custom classloader to decrypt your classes with a public key before loading into memory, but it is not too complex to modify the classloader to save onto a disc all the classes loaded.
3. You can try crash decompilers. JAD is one of the best decompilers but if you add corrupted entries in the constant pools, all products powered by JAD crash. However, some decompilers are still working.

The only way to protect your software, is to deploy it in a SaaS/PaaS. Refer this for IaaS, PaaS and Saas: the importance of applications in a virtualized environment has brought about three major trends in cloud computing:

• Infrastructure as a Service (IaaS): Used to manage low-level resources like VMs and disks. The end user is responsible for what is running within the VM, starting with the OS. IaaS is most closely related to a regular automated virtualized system. Amazon Web Services is maybe the best-known provider for an IaaS-style cloud service, but there are numerous others in the market.
• Platform as a Service (PaaS): Provides faster development and deployment platforms by abstracting the user from the OS while adding well-defined APIs to many essential services (such as the Web, databases, mail, queues, and storage) that the developer must use. Both sides benefit; development should be faster and the end product should be more reliable more quickly. Typically PaaS also provides monitoring and administrative consoles tightly designed around the platform, making operations easier. At the same time, by maintaining tight control over what is executed when, PaaS can make better decisions about machine load, at least in theory. Another side effect is a vendor lock-in, desirable to the vendor but maybe not to the developer. Windows Azure is a good example of a PaaS-style cloud.
• Software as a Service (SaaS): Instead of writing and maintaining every application, one uses online services. Examples include Google Mail and Salesforce. It is important to note that the SaaS software provided by a vendor might not be running in a cloud, but SaaS offerings often make sense when developing an application for the cloud. For instance, if I develop an e-commerce solution in the cloud, I might not want to reinvent my own load balancers, databases, or payment-processing system. I might not even want to run or operate them; perhaps all I want is to use these services. Using SaaS vendors allows me to concentrate on my own application.

Refer this for setting proguard with Android Studio. Refer this SO for enable proguard in Eclipse/ADT project. So for Android Studio, change build.gradle minifyEnabled to true. For Eclipse, uncomment this line in project.properties file:

proguard.config=${sdk.dir}/tools/proguard/proguard-android.txt:proguard-project.txt

this include one txt from SDK root(which has common setting), and one in the root of project folder(user defined). Then right mouse click the project=>Android Tools=>Export Signed/Unsigned Application Package. After, might need run zipalign, as mentioned in 3.9.1, like:

~/android-sdks/build-tools/23.0.3/zipalign -v 4 iFunTV.apk iFunTV_18.apk . After that, use adb install -r iFunTV_18.akp to update.

If got problem with ProGuard, follow instruction here to get around it. Usually ignore warning for external jar: -dontwarn org.apache.**

To keep a particular package/class: -keep class javax.** { *; }

Keep	From being removed or renamed	From being renamed
Classes and class members	-keep	-keepnames
Class members only	-keepclassmembers	-keepclassmembernames
Classes and class members, if class members present	-keepclasseswithmembers	-keepclasseswithmembernames

Note, for example, if want to keep hdp.http.MyApp =>public static String getTumd5(), should put inside -keepclasseswithmembers class hdp.http.MyApp either '*** getTumd5()' or 'public static ** getTumd5()', no 'String'.

Be careful of code like UpdateInfo apkInfo = (UpdateInfo) gson.fromJson(jsontxt, UpdateInfo.class), have to keep the class. Refer to Gson guide and SO. Another problem is Gson with ArrayList. Refer to this SO and this SO, and Gson Proguard rule.

The popular algerithm is RSA and AES. Refer to this SO for AES implementation for NDK: google openssl, and just AES implement.

Android开发笔记： 3.10.2 Capture touch event and simulate

3.10.2 Capture touch event and simulate

Refer to Android input. Android provide three utility for this: input, getevent and sendevent. Interestingly, cannot find official document about sendevent and input. Run ‘input’ and ‘sendevent’ alone will get usage info, such as usage: sendevent device type code value => Note: getevent output hex but sendevent only take decimal parameters, and no error report if pass hex value, just won’t work. Some info can be found from superuser, so and this blog: capture, convert with awk then playback: adb shell getevent | grep --line-buffered ^/ | tee /tmp/touch-events.log; Perform some touching, then ctrl-c and run: awk '{printf "%s %d %d %d\n", substr($1, 1, length($1) -1), strtonum("0x"$2), strtonum("0x"$3), strtonum("0x"$4)}' /tmp/touch-events.log | xargs -l adb shell sendevent. Note there might be timing with sendevent to simulate touch.

For touch events only 2 event types are used: EV_ABS (3) and EV_SYN (0).

Touching the display (in case of Type A protocol) will result in an input report (sequence of input events) containing the following event codes:

• ABS_MT_TRACKING_ID (57 = 0x39) - ID of the touch (important for multi-touch reports)
• ABS_MT_POSITION_X (53 = 0x35) - x coordinate of the touch
• ABS_MT_POSITION_Y (54 = 0x36) - y coordinate of the touch
• ABS_MT_TOUCH_MAJOR (48 = 0x30) - basically width of your finger tip in pixels
• ABS_MT_PRESSURE (58 = 0x3A) - pressure of the touch
• SYN_MT_REPORT (2) - end of separate touch data
• SYN_REPORT (0) - end of report

Try getevent with: adb shell -- getevent -p, adb shell -- getevent -lp /dev/input/event2, adb shell -- getevent -lt /dev/input/event2

‘input’ implementation might be different per device, with a terminal app running, adb shell input text "hello” will get “hello” show after the shell prompt.

Android开发笔记: 3.9 De-compile APK

3.9 De-compile APK

'dexdump' is from Android SDK can decompile .dex files.

3.9.1 apktool and ApkStudio

ApkTool can decode resources to nearly original form and rebuild them after making some modifications; it makes possible to debug smali code step by step. Also it makes working with an app easier because of project-like file structure and automation of some repetitive tasks like building apk, etc. A tutor of ApkTool is here. To build it, git clone git://github.com/iBotPeaches/Apktool.git; CD to ApkTool, and run ./gradlew build fatJar which will create ./brut.apktool/apktoo-cli/build/libs/apktool-xxxxx.jar; rename it to apktool.jar, and put it with Linux wrapper script.

apktool if xyz_framework.apk =>install apk to ~/apktool, not sure why need to install the frameworks

apktool d xyz.apk => decompile the apk to current folder. The Smali files are responsible for the functionality of the app

apktool b <decompiled-folder> => recompile to <decompiled-folder>/dist.

zipalign -v 4 <recompiled-apk.apk> <final-apk.apk> =>zipalign available from <sdk>/build-tools/<sdk-ver>

When I try 'adb install' to install the app, I got “INSTALL_PARSE_FAILED_NO_CERTIFICATES”. According to app-signing and SO, apk needs to be signed, even not for release to market, even with debug build; And the sign needs to be done before zipalign. So like this:

keytool -genkey -v -keystore release-key.keystore -alias myky -keyalg RSA -keysize 2048 -validity 10000

jarsigner -verbose -sigalg SHA1withRSA -digestalg SHA1 -keystore release-key.keystore my.apk myky

zipalign -v 4 my.apk my_align.apk

To use the debug key, refer to this SO: jarsigner -verbose -sigalg SHA1withRSA -digestalg SHA1 -keystore /the/path/debug.keystore -storepass android -keypass android my_application.apk androiddebugkey

On Windows, the keystore is in %HOMEPATH%\.android\debug.keystore, on linux it is ~/.android/debug.keystore,

ApkStudio is a IDE of ApkTool, available at github: git clone http://github.com/vaibhavpandeyvpz/apkstudio.git (don't use https as it requires login). Building ApkStudio needs qt, which is a little annoying to install. Especially it needs Qt5. Build steps:

cd apkstudio
lrelease res/lang/en.ts
qmake apkstudio.pro CONFIG+=release

If complains: QRegularExpression: No such file. The author said should use qt5 instead of qt4. However I installed both qtbase5-dev and qttools5-dev, but still cannot get it resolved. Find the author's wiki: need to use Qt Creator to load and build the project. At the end, it just like the apktool, no java source would be created.

3.9.2 Jadx

Open-source APK and DEX decompiler jadx decompiles .class and .jar files, but also it produces Java source code from Android Dex and Apk files. The problem with Jadx is bugs creating wrong code, like make <activity android:theme="@style/Theme.AndroidDm" wrongly to <activity android:theme="@style/Theme_AndroidDm". Also, Jadx kind of remove all binaries except Java and xml files. So for the iFunTV case, all JNI lib, png/gif/txt files are not there. Some unreachable code which unlikely be the original code. It also put res folder under asset. The resource xml files might also have issue which will impact Eclipse to properly creat the R.java. Also, pay attention that Jadx may fail to decompile certain code, and it will post a exception at runtime with code like this:

throw new UnsupportedOperationException("Method not decompiled: hdp.http.MyApp.Getcodeing(java.io.File):java.lang.String");

Another big problem with Jadx output is somehow for some switch/case, it won't generate 'break' instruction.

Version 0.6.1 fixed some issue as the generated code can be imported to Eclipse as Android project, w/o needs of update xml files.

3.9.3 Dex2jar, JD-GUI, and AndroidDecompiler

Dex2jar is a tools to work with android .dex and java .class files, refer how-to-use-dextojar. JD-GUI is a Java Decompiler. I did git clone https://github.com/pxb1988/dex2jar, then go to the folder and export JAVA_HOME to jdk root and run gradlew. This can build, but I cannot find shell script for running it. So maybe the easy way is download from SF. Run d2j-dex2jar.sh iFunTV.apk will create iFunTV-dex2jar.jar. After this, still need JD-GUI(source available at github) or JAD to decompile class file to java source. JD-GUI may also has bug, like incorrectly translate for loop to label. Also in Java an object ref can be reused, if see a decompiled has an Object type variable which may be casted to different type later, and there is no range overlap for the different type, better change it to two ref with the corresponding type. So may try the out-date JAD which is not open source and not maintained since 2011.

-s option for change output file extension: jad -sjava example1.class

-p for output redirection: jad -p example1.class > myexm1.java

For my case, extract the jar of dex2jar's output will create a tree: my-dex2jar, then run jad -o -r -sjava -dsrc my-dex2jar/**/*.class

Too bad seeing this: The class file version is 50.0 (only 45.3, 46.0 and 47.0 are supported)

AndroidDecompiler is a reference of using ApkTool, dex2jar and jd-core. It has not been update since Oct 2 2014. The last one has Dex2Jar :Version 0.0.9.15, apktool : Version 1.5.2, JD-Core-Java : Version 1.2 and Artistic Style (astyle) : Version 2.04

Except the Astyle, all other in the git are just jar and scripts. So no need to build. This tools set is also very buggy, run it with bash like this: bash decompileAPK.sh -o out_folder target.apk, and it generates a lot of java exception during run. Any way, just take the output as reference only.

3.9.4 Tips and Conclusion

As mentioned above, no tools is perfect. My experience is using Jadx for decompile the Java source. Find out all external library and remove those Java code for those libs, use those libs' Jar instead. Remove the res folder from Jadx's output, take the res from apktool's output. Also copy all asset raw files and JNI libs over. Fix compiling errors. Use this online converter for viewing unicode.

Android开发笔记: 3.8 Commercialize

3.8 Commercialize

3.8.1 App and Google Play setting

Google Play sign up link;

To be able to upload bundle instead of APK, need to opt in Goole App Sign, with your keystore file my_keystore.jks and your key as $key, run:

java -jar pepk.jar --keystore=my_keystore.jks --alias=my_android --output=my_key –encryptionkey=$key => will create my_key file, which needs to be uploaded to GooglePlay console.

With Android Studio to create the keystore and key and build, it will generate an apk as: app/build/outputs/apk/app-release-unaligned.apk. This apk still cannot be uploaded to app store as it needs to be zip-aligned. app/app-release.apk is the one can be uploaded. Refer this, ProGuard mapping file app/build/outputs/mapping/release/mapping.txt can be uploaded for deobfuscate crash stack traces.

If following the instruction either 'Open Module Setting'/F4 or click menu=>Build=>Edit Build Types, click 'signing' tab, fill all info, which will be automatically copied to app build.gradle file. To make it get password either from ENV, modify the gradle file:

storePassword System.getenv("KSTOREPWD")
keyPassword System.getenv("KEYPWD")
Or if build from command line(which I haven't figured out how):
storePassword System.console().readLine("\nKeystore password: ")
keyPassword System.console().readLine("\nKey password: ")

Each time update the app, make sure modify build.gradle(Module: app) to update the versionCode,

 optionally update versionName. Refer this for more about version code. In a project, it

 is set in AndroidManifest.xml:
<manifest xmlns:android="http://schemas.android.com/apk/res/android" android:versionCode="29"

 android:versionName="1.2.9" package="some_corp.com">

Refer this SO for detail of the CERT.RSA and CERT.SF file under META-INF folder. 

3.8.2 Webserve

For some app, may need to provide internet service such as public shared files/resource,

dbase support for user autherize. Google Drive, Box.net, Dropbox may all provide cloud storage,

but mostly can not be accessed directly by apps. For example, if the app needs to pull in a 

configuration file, with Google Drive even shared through a link, the link is not directly to

the file, but a view of the file provide by Google. I found OpenShift from this post, which is

backed by Redhat with Node.js host, and is free. Code there is maintained with git, which needs a pub key: ssh-keygen -t rsa. Upto 3 free

Apps can be maintained through web console or command line client tools rhc. For install rhc,

just apt-get install ruby-full, no need of rubygems, then sudo gem install rhc. If see Error

installing rhc: net-ssh requires Ruby version >= 2.0, follow this SO to upgrade ruby:

 ppa:brightbox/ruby-ng and installed ruby2.4. And for me it is installed as:

 /var/lib/gems/1.9.1/gems/rhc-1.38.4/bin/rhc. Openshift was re-designed one years ago, and use

 oc command line utility. It needs Openssl 1.0.1(Ubuntu16.04/xenial has 1.0.2g), following au 

 won’t fix the lib problem, tried to download openssl 1.0.1/ and build it. Since I need the shared

 lib, have to do ./config shared before make it. And make install will copy the so files to

 /usr/local/ssl/lib. But that doesn’t work either. Refer to issues/21061. To build oc:

 git clone https://github.com/openshift/origin Start from openshift debian-ubuntu.html.

I installed http://www.nodeclipse.org/updates/ for developing node.js within Eclipse.

<app_name> to get tar of all packages and logs. Seems Node.js' http won't accept txt as

Content-type. I updated app.js to support downloading the txt file. For dbase, refer 5.3.

PostgreSQL is great, but for some project, MonGoDB(refer this) is more simple to use.

RDBMS	MongoDB
Database	Database
Table	Collection
Tuple/Row	Document
column	Field
Table Join	Embedded Documents
Primary Key	Primary Key (Default key _id provided by mongodb itself)
Database Server and Client
Mysqld/Oracle	mongod
mysql/sqlplus	mongo

When creating collection, refer to this SO, try to avoid name of database object method such as 'stats', 'group'.
This github is a working sample with node+express+mongodb
URL $OPENSHIFT_MONGODB_DB_URL: mongodb://user:pw@$OPENSHIFT_MONGODB_DB_HOST:$OPENSHIFT_MONGODB_DB_PORT/

I'm using node.js code from this SO for query user's IP:

var ip = req.headers['x-forwarded-for'] || req.connection.remoteAddress || req.socket.remoteAddress || req.connection.socket.remoteAddress;
clustrmaps provides a real-time map of visitors from around the world. 

 In order to tracking user IP, may need node.js proxy code to forward the request to another

 html page with in openshift domain, such as the index.htm. Refer to proxy SO and redirect SO.

 It won’t work as the js is expected to be run by the visitor device’s web browser. Similar

 widgets are mentioned here, which are target to portal website or blog.
App may need to access cloud storage for update info, resource. Need directly access with a

 link, which is not supported by most cloud service such as Google Drive, Dropbox, Box.net. 

But some mentioned tricks to achieve that, in short in this trick we are simply replacing 

“www.dropbox.com” with “dl.dropboxusercontent.com”. Change the query “?dl=1” can download the

file, but it still is not direct link.

Refer this(a little old) and this for host video stream service cost.

Android开发笔记: 3.7 Resource and Drawable

3.7 Resource and Drawable

Refer to resources. Whereas XML resource files in other res/ subdirectories define a single resource based on the XML filename, files in the values/ directory describe multiple resources. For a file in this directory, each child of the <resources> element defines a single resource. So the filename under values/ doesn't matter, resource in all files there will be picked up into the package for pre-v21. Refer compatibility, for Android 5.0/API level 21 and newer, it takes resource from alternative place such as res/values-v21 and res/layout-v21.

For drawable, refer to Working with Drawable, w3 SVG path, google design of icons, drawable resource, and downloadable google icons. When using Android Studio to create a new project, it may create multiple set of drawables, such as from drawable-v21 for target with API v21 and newer, and res/values/drawables.xml for target before Android 5.0/API level 21. The drawables.xml is referring to system icon in SDK android.jar (refer here) like: <item name="ic_menu_mpl" type="drawable">@android:drawable/ic_menu_camera</item>

Same res can be found under SDK data subfolder. Not all icons there could be used directly, such as compass_base, when compile, it complains “Resource is not public”. To use non public icon, will have to copy the icon over to local res folder of your project.

Target before Android 5.0 does not support Vector Drawables, which might lead to InflateException with binary XML file.

If use png or other image files under drawable, use the filename w/o ext as the id of the drawable. Interesting when I use an icon for navigation drawer menu, no matter I pick the white color or black color of the google icon in xml, it displays same. I guess it will change the color bases on the theme. Here is a opensource Android Asset Studio, which can create all size of icons base on user provided png file.

There is strange problem with Android 5.0/API21, android:pathData="M19.43 12.98c.04-.32.07-.64.07-.98s-.03-.66-.07

This will cause runtime exception: java.lang.NumberFormatException: Invalid float: "-.32.07". To get around it, add space between data.

Note the difference between @+id and @id, refer to this SO. The '+' is telling the parser to create a new ID to R.java.

A free resource site: https://commons.wikimedia.org/wiki/Main_Page, include free image, sound and video.

Note of data URI, refer to wikipedia, is frequently used for embedded small png image like: <img src="data:image/x-png;base64,iVBO..."> is a barcode. To display the image in Android, follow this SO: need to convert the Base64 encoded string to a Bitmap like this:

byte[] decodedString = Base64.decode("Your Base64 String", Base64.DEFAULT);
Bitmap bitMap = BitmapFactory.decodeByteArray(decodedString, 0, decodedString.length);
For multi-language support, refer to languages and localization.

Android开发笔记: 3.6 Android NDK

3.6 Android NDK

Refer to ndk. Run <sdk_root>/tools/android to update app project file. Then under JNI subfolder run <ndk_root>/ndk-build to build native code. If build directly from Eclipse, use mouse right click Project=>Android Tools=>Add native Support, might see many not resolved error as Eclipse might not use the right include path. Also, Eclipse will auto delete all .so files under libs(jar files is ok) when building NDK code. So if have other prebuilt .so files, don't put them under libs. Either put them somewhere else, or follow this SO to treat as prebuilt. Refer Ndk guide for using prebuilt librarries. Refer to this SO and JNI tip for calling Java from native code. Eclipse ADK might have problem with NDK: it can build, but report Semantic Error with Method/Symbol could not be resolved. Refer to Android NDK build, Method could not be resolved and Eclipse compiles successfully but still gives semantic errors as it might be a bug with Eclipse plugin. For me, restart Eclipse may be ok.

Regarding armeabi/armeabi-v7a, refer to ABIs and application_mk guide. APP_ABI := armeabi armeabi-v7a x86 mips can only be set in Application.mk. By default, add NDK support from Eclipse ADT doesn't create this Application.mk. So if needs to generate other than the default armeabi binary, will need to add a Application.mk with the APP_ABI line. Use TARGET_ARCH_ABI in Android.mk won't do it.

Get Android NDK: WARNING: APP_PLATFORM android-16 is larger than android:minSdkVersion 14 in ./AndroidManifest.xml: Refer to this SO; need to add APP_PLATFORM := android-14 in jni/Application.mk, otherwise the setting is taken from project.properties file, which is the setting of compileSdkVersion.

Below is Android.mk and Application.mk for NagaSoft vjplayer prebuilt libs(armeabi-v7a only):

#Android.mk

LOCAL_PATH := $(call my-dir)

include $(CLEAR_VARS)

LOCAL_MODULE := vjplayer

LOCAL_SRC_FILES := libvjplayer_jni.so

include $(PREBUILT_SHARED_LIBRARY)

include $(CLEAR_VARS)

LOCAL_MODULE := p2pcore

LOCAL_SRC_FILES := libp2pcore.so

include $(PREBUILT_SHARED_LIBRARY)

#Application.mk

APP_ABI := armeabi-v7a

APP_PLATFORM := android-14

Problem with NDK r13: this NDK toolchain mostly are 4.9, but when I use this NDK to build OpenSSL-Vitamio, it always looks for 4.8 toolchain. The NDK CHANGELOG.MD mentioned `NDK_TOOLCHAIN_VERSION` now defaults to Clang. So I have to do build like:

ndk-build NDK_TOOLCHAIN_VERSION=4.9. Get problem fox x86_64: BN_ULONG redefined. As mentioned in check-all-builds.sh: The $HOST_OS-x86_64 is currently broken because the single <openssl/opensslconf.h> header is tailored for 32-bits. So just build for arm v7a: ndk-build NDK_TOOLCHAIN_VERSION=4.9 APP_ABI=armeabi-v7a