滤掉烦人的广告

 Google,Facebook等大佬也要靠网页上的广告挣钱，就像本blog也有广告。但君子爱财，取之有道，那种硬弹，骗点最为深恶痛绝。好在浏览器提供插件block这些下三烂的东西，弹出广告可用NoScript，骗链可用Adblock Plus。下图是block前和block后的比较。

 

Highlighted广告在第二个图都被滤掉了，不论标题是多么的迷惑人。关于正确设置ABP filter, 参见：https://adblockplus.org/filter-cheatsheet

 

Remote Windows PC Uart Access

Last August I posted using Python for Remote Serial Port Access. In fact, there is many other way to do the same, without the need to run a script. The widely used is opening pseudo-ttys via SSH connection, as mentioned https://github.com/npat-efault/picocom:

$ ssh -t user@termbox picocom -b 115200 /dev/ttyS0 

Picocom is a terminal emulator, similar like minicom, PuTTy and Teraterm, with very small footprint. One problem is that it is tty based, and Windows does not expose Serial Port/UART as TTY devices. Cygwin does not provide picocom package. Even we can compile picocom from source for Cygwin, it is worthless as it cannot open Windows native serial port device.

The solution is WSL. With WSL, it is possible to access native device from the Linux environment running as Windows Sub-System. Nowadays, model PC/Laptop does not have RS-232 port, mostly, external device with UART output will be connected to PC with a Uart-over-USB (FTDI chip). Unfortuenately, as of today, WSL2 still does not support USB device. As I mentioned here, may need to stick with WSL1 for this feature at this moment.

Two things need to be done on WSL: install picocom and setup SSH server.

Install picocom

Most Linux distribution provide picocom install package, such as Ubuntu/Debian, user can run sudo apt install picocom to have it installed in seconds. However there is one issue: may run into error as 'FATAL: failed to add port: Cannot get the device attributes: Inappropriate ioctl for device'. This is a known issue with custom baud-rate support, and as mentioned in the ticket the solution is either build the binary with CPPFLAGS=-DNO_CUSTOM_BAUD make or run picocom with env setting: NO_CUSTOM_BAUD=1. If taking the env setting way, need to make sure the setting is taking effect for SSH session. So better use the build option. 

To build it, download the picocom source from https://github.com/npat-efault/picocom, install gcc and make for WSL if you haven't done so:

sudo apt install gcc make

If runs into issue, likely you will need to do: sudo apt update --fix-missing

Then build the binary as mentioned above: CPPFLAGS=-DNO_CUSTOM_BAUD make

Once build is done, copy the executable to a executable path such as /usr/local/bin.

Setup SSH server

May refer this for how to setup SSH on WSL. Before try the SSH access, make sure on SSH server side, has port 22 opened from Windows Firewall.

Install and setup SSH server just like on a Linux setup: sudo apt install openssh-server

By default, it will have PasswordAuthentication off and PubkeyAuthentication on. And for pubkey, it may be using ECDSA but not RSA. So if running into SSH permission denied (publickey) issue, may need to play with sshd config setting: sudo nano /etc/ssh/sshd_config

May try ssh -vvv username@the_wsl_host to triage the error. The '-vvv' option should give out lot of useful information of the error. Once sshd_config is updated, run sudo service ssh --full-restart to restart the sshd (systemctl restart sshd for old version Linux).

 

Open UART remotely

ssh -t user@wsl_host picocom -b 115200 /dev/ttyS10

Above will try to open the COM10 on the wsl_host pc remotely. Note picocom is a terminal emulator, just like minicom, there is no GUI interface. To quit, press C-a, then C-x. Looking for help? Press C-a, then C-h.


The journey for updating Realtek NIC MAC

Realtek Network Interface Controller (NIC, also known as a Network Interface Card) is widely used in PC. Usually an EEPROM would be connected to the RTL NIC chip to provide programmable storage for saving some network related setting, such as MAC address. IP address is bound with the MAC. So if multiple NIC has same MAC, it would be a disaster for switch/bridge/router to correctly forward IP packet.

On Linux and Windows, it is possible to change the MAC address temporarily by using some command such as ifconfig, but the setting won't persist during reboot. It is also possible to override the MAC address with a saved file or value stored in Registry table and keep the change permanently. However, this change can also get lost if the card is moved to a different setup, or the OS gets reinstalled.

So to permanently change the MAC, will need to reprogram the EEPROM. There is an Open Source tools, rtl8168-eeprom, supposed can be used to update EEPROM. However, several issues to get it working:

1) open('/dev/mem') may fail, due to a new kernel feature 'KERNEL_LOCKDOWN'.

2) Rebuild kernel from source. After build and install, it cannot boot the new kernel with error

/boot/vmlinuz-x.y.z-generic has invalid signature

This is due to for secure boot, customer kernel has to be signed. Refer to this.

3) Tried to build kernel with CONFIG_SECURITY_LOCKDOWN_LSM disabled, will run into problem during kernel_config, as KERNEL_LOCKDOWN is forced on due to config policy setting

4) Build old kernel 4.4 from source for Ubuntu 20.04 (Focal Fossa) doesn't work very well, as the kernel version is too low which cause build problem.

5) So the way to run the tools is install kernel 4.4 deb package from LaunchPad. However, the tools cannot properly access RTL8168 EEPROM as it failed to read back the ID from EEPROM.

Ethtool has option '-m' for dump module EEPROM, '-e' for dump EEPROM, and '-E' for update MAC. However, running ethtool with these option may get  'Operation Not Permitted' error. Run 'ethtool -i devname' may get 'support-eeprom-access: no'. So the NIC driver does not support EEPROM access.

As a side note, Realtek might have released some tools for this purpose, which can run on Windows, Linux, and UEFI shell. You can search for 'RTNicPG' from Internet. I'm able to find several links but none of them from Realtek official website. Maybe it got leaked out as an internal used tools.

Update build.gradle for build native code with NDK

Because of Android Studio/Gradle keeps evolution, and the special need of Android NDK's configuration, sometimes it is a little difficult to get things working smoothly.

There is some references like:

https://riptutorial.com/android/example/13985/enable-experimental-ndk-plugin-support-for-gradle-and-androidstudio

https://stackoverflow.com/questions/16667903/android-studio-gradle-and-ndk

https://android-developers.googleblog.com/2020/02/native-dependencies-in-android-studio-40.html

Will see some mismatch between above posts, or things not working with Android Studio, such as with Android Studio 4.1.2, Gradle model 6.5, the ndk setting in above first link will lead to weird error. The 2nd link from stackoverflow has the correct setting, i.e. ndk setting has to be put under defaultConfig, specially the abiFilters setting. As explained in the 3rd link, when deal with native code, frequently may need to link with prebuilt lib, which may only support one special ABI such as armeabi-v7a, but not all others such as 64 bit or x86. Here is an example app build.gradle file for android-eye:

apply plugin: 'com.android.application'

android {
    compileSdkVersion 14
    buildToolsVersion "30.0.3"
    ndkVersion "17.2.4988734"

    defaultConfig {
        applicationId "teaonly.droideye"
        targetSdkVersion 14
        minSdkVersion="14"
        ndk {
            abiFilters 'armeabi-v7a'
            // 64-bit support requires an Android API level higher than 19; Namely 21 and higher
            //abiFilters 'armeabi', 'armeabi-v7a', 'arm64-v8a', 'x86', 'x86_64'
            //cppFlags.add("-O2 -Werror -Wall")
            //ldLibs.addAll(['log', 'z', 'ld'])
            stl = "c++_static"
        }

    }
    buildTypes {
        release {
            minifyEnabled false
            proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.txt'
        }
    }
    externalNativeBuild {
        ndkBuild {
            path file('src/main/jni/Android.mk')
        }
    }
}

dependencies {
    implementation files('libs/java_websocket.jar')
}

Building under Android Studio may hide the detail of the build, which makes it difficult to root cause when something goes wrong.

Beside gradle related setting, sometimes developer might get sysroot setting issue. Such as compiling x264 with NDK (specially old version) sometimes need some tweak, such as sysroot setting. Refer to SO1, SO2, and BuildSystemMaintainers for latest new NDK. For old NDK, at compile time sysroot is $NDK/sysroot, and at link time sysroot points to $NDK/platforms/android-$API/arch-$ARCH. One way is refer to standalone_toolchain, run $NDK/build/tools/make_standalone_toolchain.py --arch arm --api 14 --install-dir /tmp/my_toolchain to create a standalone toolchain. May need to specify CFLAGS=-D__ANDROID_API__=$API as mentioned here. The other possible way is refer to this: Using clang instead of gcc also works, e.g. for autotools-based projects CC=arm-linux-androideabi-clang CXX=arm-linux-androideabi-clang++ ./configure --cross-prefix=arm-linux-androideabi- ... works without disabling unified headers and without specifying __ANDROID_API__. 

With new version NDK,  'Application.mk' has to be provided. Also, NDK will assume the 'Application.mk' and 'Android.mk' are all located under a 'jni' sub-folder. If that is not the case, then user will have to specify the path of 'Application.mk' while invoking 'ndk-build' script, by setting NDK_APPLICATION_MK, and in 'Application.mk', user has to specify the path of 'Android.mk' by set 'APP_BUILD_SCRIPT'. Refer to https://developer.android.com/ndk/guides/ndk-build for more detail.

学习TensorFlow - 设置Python环境

 TensorFlow的网址：https://www.tensorflow.org，Wikipedia介绍，是由Google开发，后来捐献成开源项目，提供machine-learning的API和库支持。虽然TensorFlow的core是用C++开发的，TensorFlow同时提供多种语言如C++, Python, Java等的外包(wrapper)，而Python目前还是ML最流行的开发语言。TensorFlow适用于底层建模，上层可用Keras等module来简化。

TensorFlow的教程很多，我也还在学习。设置Python环境是第一步，很多教程提供了不同的设置却没有明确的解释为什么。Python有很多不同的版本，其中2.x和3.x差别很大，另外Python有很多软件包，这些包也有不同版本。一个复杂的项目可能用到特定版本软件包，如果把多个项目安装在同一环境下，可能会遇到软件包版本冲突，所以简单的解决办法就是隔离每个项目的运行环境。为了达到这个目的，可以用Python的env包来设置虚拟环境，或安装Docker container，或安装Anaconda/MiniConda并建立虚拟环境。Python虚拟环境的实质就是把所有的软件包和工具安装在单独的目录下来避免冲突。所有每增加一个环境都要占用更多的硬盘空间。实际上我之前在“Get fun with google assistant”提到的Google Assistant虚拟环境可以与TensorFlow共用。

有的教程要安装PyCharm， 其实完全没必要。PyCharm是Jetbrains开发的Python集成开发环境(IDE)，是个很不错的软件(Google的Android Studio是基于Jetbrains开发的IntelliJ IDEA, JetBrains的TeamCity是仅次于Jenkins的Continous Integration Tool)，用Visual Studio Code开发也很适手。

注意TensorFlow 2.0有很多改变不同于1.0，所以遇到AttributeError: module 'tensorflow' has no attribute 'placeholder'，请参照https://www.tensorflow.org/guide/migrate，简单的办法是把

import tensorflow as tf

改成：

import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()

@ Mar 11, 2023

Windows上设置miniconda有时有点very frustrate，主要是安装过程可能没有设置系统路径(System Environment

 Variable: PATH),所以当用户在自己打开的cmd console下运行conda command时，只会得到找不到命令的错误。

所以最好把conda的path加到Environment Variable Path setting, 注意conda.exe通常是在Scripts子目录下。

VSCode通常无法detect到conda/env setting,如果conda executable不在系统路径里。关于在VS Code中设置

Python Environment,可参见： https://code.visualstudio.com/docs/python/environments