Friday, October 18, 2019

计算机是上大学的首选专业

    2000年我和很多人一样移民到加拿大,有很多专对华人的培训中心,提供Advanced Java,J2EE之类的速成,把本来的化工博士,医生,建筑工程师都包装成电脑工程师,只因为计算机的工作最好找。快20年过去了,哪怕再过10年,我认为计算机还是最容易找工作的行业,只因为计算机应用只会越来越多。律师,医生,老师这些被普遍认为最好的工作都会面临被计算机顶替的风险。
    听起来有点太夸张。我上学那会,听说第三次浪潮,产业革命,都是很时髦的概念,经常挂嘴上夸夸其谈,虽然不知其所以然。但现在回头看,社会不没有沿着那些大家推崇的方向发展。单从计算机的发展来说,人工智能(AI),神经元网络,网络计算机在20多年前就被当作计算机的发展前沿,互联网,电子商务,网络交易,虚拟空间,新近的大数据,云存储,云计算,比特币,量子计算机,IoT, AR, VR, 电动汽车自动驾驶等,也都是一会热,一会凉。但总体来说,计算机正在被应用到越来越多的角落,同时创造来更多计算机技术应用的机会和需求,同时也推动了计算机的教学和研究的普及深入。软件技术的发展伴随着电子技术工艺的提高,更高集成化,更快速度,更小功耗,一体化,多功能,新材料,新领域,新着眼点,整体发展多样化伴随着技术进步。
    哪些技术会继续成为发展的热点,哪些会成为新的业界宠儿?互联网正把世界的各个角落拉的更近,电子商务交易会更普及并取代更多的实体商店;电子货币会因此取代纸币甚至信用卡。比特币没有银行和国家做后台保障,同时有安全风险,很难成为流通交易的媒介。电子交易极大地推动商业的发展,给更多的人提供更多的机会,更多的竞争,意味着更低价,更高性能,更好更多样的服务;网络交易推动了云服务的发展,而云的发展给小企业商户提供低价优质多样稳定的服务,就连大企业也逐渐从自建网络服务转移到更多依赖专业云供应商的服务,一方面减少开发维护的费用,一方面可以得到更专业更稳定性价比更高的服务,并可以很容易的扩展或缩减服务需求;分工合作就像当年汽车制造采用流水线作业一样,极大提高了效率。从云服务收集到的大数据给云供应商,政府,企业提供导向,使企业政府的决策更精准。伴随语音图像识别和人工智能的发展,人类的生活更方便,像Amazon, Google, Apple等提供的智能家居助手,安保,把我们的生活变得更方便舒适和现代化。未来,你的马桶可能会时刻记录你的体重,血糖,消化代谢系统的状态;你的冰箱会提醒你那些食物要过期了,你还有什么食物储备;你的电子锅会自动为你做出可口的饭菜;你可以随时随地与亲朋好友视频通话;你的垃圾桶每周三早上会自动跑到马路边等着被收集。人脸识别会提供很多便利,乘车,购物,出入,对室外公众场合的安全提供极大的保障。但基于隐私的担心会成为这一技术普及的注意障碍。5G6G的普及,万物互联,给日常生活带来更多便利,提供更多的可能,你的汽车可能会跟临近的汽车协调路线,交通事故将成为历史,你可以随时随地处理公务和私事,办公,求学,就医,讨论,救助。
    美国用世界上最好的很多大学,有世界上最先进的技术,有最好的高科技公司,有很多很多聪明能干的工程师企业家。但美国上到总统,下到平民,也有普遍的自高自大,傲慢偏见,唯我独尊。世事无常,所谓的民主会滞肘决策协调,所谓的精英政治只不过少数集团的一己私利。没有长远的历史可借鉴,没有深厚的文化的积垫,没有广泛的外界民众的拥戴,缺少与别国互惠互利的利益共享,繁盛终会走向衰落。
    计算机技术相关的最好的大学基本都在美国,像麻省理工,斯坦福,加州伯克利等等。加拿大最好的计算机大学是滑铁卢大学。要进这些大学,需要好好准备,不只是英语成绩,还需要很好的平时成绩单,推荐信,并通过面试。这些好学校竞争激烈,淘汰率也很高。
本科基本是打基础,如想深入,特别是跟导师做研究,就要申请研究生和博士生。如果你有两年左右的工作经验,特别是做过一些项目或课题,再争取较好的TOEFL/GRE成绩,可能会有更多机会。
    国内给我的感觉是过于急功近利,很多公司急于赚钱,而不搞开发投入。只有华为给我印象不错。中国的软件业不重视深入研究,不重视知识产权保护,没有良好的保护,就不会有强劲的开发动力,大多抄来抄去,什么热门就做什么,昨天做AR/VR,今天做大数据,云计算,比特币,明天又去搞自动驾驶人工智能。现在的高科技要厚重积发,要有很多的技术储备。特别体现在微电子领域,中国一直就没有一款像样的CPU。不提丢人现眼的汉芯基于MIPS指令的龙芯的最新一代3A4000/4B3000转到28nm,也主要用于办公为代表的事务处理应用;华为总算有个麒麟芯片,但其核心是ARM的IP,而且芯片要由台积电来造,因为国内的FAB只能量产28nm的晶片,14nm还在研发,而台积电,三星,Intel都已经移到10纳米,7纳米,甚至5纳米。芯片集成的工艺非常重要,不只要高集成度,还要保证成品率,同时要低漏电,不然就会造成高功耗和过热问题。芯片集成不只是一个很烧钱的行业,技术垄断(马太效应)也很严重,不知道你有没有听说过芯片集成必须的光刻机制造商荷兰的ASML,一台光刻机要一亿多美元,一年只造几台,买家要排队等,可能还限制出口中国。一个芯片厂可能更要上百亿美元的投入。再说芯片的IP, Intel/AMD,ARM,Apple, Qualcomm,Motorola等等都做了很大投入,中国也就华为买了ARM开发权。这个链接是清华大学微电子所所长魏少军谈中国的芯片产业:https://mil.news.sina.com.cn/dgby/2019-05-20/doc-ihvhiqay0078144.shtml
所以说这方面,中国要有很长的路要走,特别是现在美国在搞对中国的技术封锁打压,希望国内的企业和有识之士能蹚出一条路来。
   中国在电子产业方面有自己的优势,巨大的市场,开放的环境,众多的软硬件工程师。在新兴的人工智能等方面,中国很容易与全世界的同行同步前进,不像通信行业,AI领域还没有什么行业标准和技术垄断,所以这方面中国有很多机会,比如很多城市使用的人脸识别,在西方所谓的民主社会,会因为一些人的对隐私的担心而无法应用。中国的信用卡体系不是很完善,反倒促进电子支付的普及。在西方人们只用信用卡,从商店购物是没问题,但无法用于个人间的交易。另外欧美的收入较高,同时会造成人工成本高启。中国的手机电视无人机和其它小家电销售到世界各地。所以在IoT领域我认为中国会有很多机会。我当年在北京时,曾经做过不少小项目,经常软硬件都自己一个人做,中关村跑两趟,从电路板设计,元件选用采购,到电路板定制焊接,软硬件调试,经常两三周就搞定。而这边没有电子市场,一切都要从网上订购,费用很贵,而且很费时。
   取决于个人的长远计划,是在小的startup做起,还是到大的高科技公司从普通工程师做起。小公司有风险也有机遇,而大公司只能一级一级从低到高慢慢来,好处是可以从其他资深工程师学习,同时通常大公司会提供很好的培训资源。

Wednesday, October 2, 2019

用微軟的family setting來控制小孩玩手機和電腦的時間

很久以前我写过一个C#程序限制小孩子看计算机的时间的小程序。现在有更好更简单易行的办法了,不过需要用到微软的服务:在Windows 10的PC上,不需要安装任何东西,只需要设置好Windows parental control;在Android tablet 和手机上需要安装Microsoft Launcher; (我没有试过如何在Apple的device上设置,另外Linux上恐怕也不起作用)。
截屏如下,可以设置和监控Activity, Screen time, App and game limits等。具体参见微软网站:https://account.microsoft.com/family
一旦设置好后,你就可以远程控制小孩使用手机和电脑的时段和总的电脑使用时间。需要时,小孩可以要求更多时间,你可以远程授予或拒绝。你可以login到微软的family account page进行设置和监控:https://account.microsoft.com/family
同时也可以监控Xbox。虽然不是尽善尽美,但是个良好的开始,而且完全free, Google已经开始跟进了。

Tuesday, October 1, 2019

Setup VNC server on Ubuntu

Update @ Jan 2021:
New version of Ubuntu 20.04 and 20.10 has been out for a while. I have been seeing many post about setting up VNC server on Ubuntu, most talking about specifically with TightVNC server and Xfce, without detail explanation of the choice. Xfce is one of Desktop environment for Ubuntu, among other popular environment such as Gnome (default for Ubuntu 20.x), KDE, etc. Basically, Xfce is light weight, so more suitable for VNC remote connection. And for the server, TightVncServer is not the only choice. 
-----------------------------------
I have used several different VNC solution (most are free, and even Open Source), here is my feeling:
  • vino, available on Ubuntu by default, but performance is not good
  • Vnc4Server is very stable, but might be out-of-date, depends on the desktop environment, such as Gnome or Kde, may need some tweak with x server and authentication configuration; 
  • TightVnc, UltraVnc, noMachine RealVnc and several other open-source VNC all have some cons and pros, and neither of them is integrated to Ubuntu as good as x11vnc. Some found TightVnc is easy to setup and use, some found noMachine is very powerful. But like TightVnc, it won't allow local user and remote user have the view at same time, so it cannot be used for remote assistant.
  • x11vnc is the only one which can be up with the user login screen, so you can type in account credential in the Ubuntu login GUI. One shortage is x11vnc relies on the hardware display. But for headless (no monitor/display adapter), user can set up Xvfb, Xvnc or dummy display. So x11vnc is my favorite one.
As of Ubuntu 18.04, it switched from LightDM to GDM3. Connecting to the login screen with VNC while using GDM3 is currently not possible. The easiest way to get this VNC functionality back is to simply switch back from GDM3 to LightDM as mentioned here: https://askubuntu.com/questions/1033274/ubuntu-18-04-connect-to-login-screen-over-vnc/
Install LightDM is very simple as: apt install lightdm

This post shows how to setup VNC with TightVNC server + Xfce desktop environment:
https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-vnc-on-ubuntu-18-04
And this combination might be easier to be setup for headless (no monitor connected Linux station), with xserver-xorg-video-dummy package installed. For Xfce, if the Terminal launcher doesn't work, check Settings Manager > Preferred Applications setting, and switch to a different Terminal app.

There might be issue to run Microsoft's Visual Studio Code with xrdf. Refer to vscode ticket 3451. The workaround provide by 'groone' in 2016 still works as of Jun 2020:
  • make a backup first
  • sudo sed -i 's/BIG-REQUESTS/_IG-REQUESTS/' /usr/lib/x86_64-linux-gnu/libxcb.so.1
libxcb.so.1 is a symbolic link (such as libxcb.so.1.1.0). So just remove the symbolic link, and copy the '.so' file like this: 'cp libxcb.so.1.1.0 libxcb.so.1', then run 'sed' to replace the string. This way it won't break other application which may rely on libxcb (such as VLC player).

Below is some other note which might be old.
vnc server =>refer help.ubuntu.com/community/VNC/Servers. x11vnc works pretty well. Start x11vnc server(apt-get install x11vnc):
x11vnc -storepasswd => provide password
x11vnc -auth guess -forever -loop -noxdamage -repeat -rfbauth /home/lab/.vnc/passwd -rfbport 5900 -shared
Have x11vnc start automatically via systemd in any environment (Vivid+, i.e. Ubuntu15 and up):
sudo nano /lib/systemd/system/x11vnc.service => add following:
[Unit]
Description=Start x11vnc at startup.
After=multi-user.target
[Service]
Type=simple
ExecStart=/usr/bin/x11vnc -auth guess -forever -loop -noxdamage -repeat -rfbauth /home/lab/.vnc/passwd -rfbport 5900 -shared
[Install]
WantedBy=multi-user.target
Then do: sudo systemctl daemon-reload && sudo systemctl enable x11vnc.service
For Ubuntu14, sudo nano /etc/init/x11vnc.conf
# description "Start x11vnc at boot"
description "x11vnc"
start on runlevel [2345]
stop on runlevel [^2345]
console log
respawn
respawn limit 20 5
exec /usr/bin/x11vnc -auth guess -forever -loop -noxdamage -repeat -rfbauth /home/lab/.vnc/passwd -rfbport 5900 -shared

No desktop but a terminal only with ubuntu14/vnc4server=> log shows x-window-manager: not found
This link works for me:
# apt-get install gnome-panel gnome-settings-daemon metacity nautilus gnome-terminal
and use this xstartup file
#!/bin/sh
export XKL_XMODMAP_DISABLE=1
unset SESSION_MANAGER
unset DBUS_SESSION_BUS_ADDRESS
[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
xsetroot -solid grey
vncconfig -iconic &
gnome-panel &
gnome-settings-daemon &
metacity &
nautilus &
gnome-terminal &
Use gnome classic: refer to this au, or install gnome-session-fallback
Many search points to ubuntu14.04 gnome-fallback broken, use startxfce4, which is not gnome.
For log show xinitrc denied, change exec /etc/X11/xinit/xinitrc to . exec /etc/X11/xinit/xinitrc or exec sh /etc/…
Some of our PC has vino installed, which is slow sometimes

Refer this for vnc through SSH. Basically run this from local PC: ssh -L 5900:localhost:5900 lab@pv-host39, then VncViewer open localhost:5900.
For some PC may get “channel 3: open failed: connect failed: Connection refused”. Solution here doesn’t work for me. For me, x11 failed to start.

Display, remote and dummy

If seeing monitor shows signal out of range, refer this AU. If mode not available, try cvt 1920 1080 60. The output looks like below. Copy the modeline after Modeline to use it in the next command:
# 1360x768 59.80 Hz (CVT) hsync: 47.72 kHz; pclk: 84.75 MHz
Modeline "1360x768_60.00" 84.75 1360 1432 1568 1776 768 771 781 798 -hsync +vsync
Create a new mode with the copied modeline and xrandr.:
xrandr --newmode "1360x768_60.00" 84.75 1360 1432 1568 1776 768 771 781 798 -hsync +vsync
With the following command, you get the connected port: xrandr --query | grep connected
The output looks like this. As you can see, in my case the connected port is DVI-0.:
HDMI-0 disconnected (normal left inverted right x axis y axis)
DVI-0 connected 1920x1080+0+0 (normal left inverted right x axis y axis) 521mm x 293mm
VGA-0 disconnected (normal left inverted right x axis y axis)
Add the new mode using your connected port.: xrandr --addmode DVI-0 "1360x768_60.00"
Change the monitor resolution.: xrandr --output DVI-0 --mode "1360x768_60.00"
So for our case, as monitor does not support 1920x1200, mostly need to run: xrandr --output HDMI1 --mode "1920x1080"
Fore remote display, refer to au. Not like VNC, it only opens a display remotely, but not allow any remote input.
For PC without monitor(headless), refer to this to add a dummy display: sudo apt-get install xserver-xorg-video-dummy, then update/create /usr/share/X11/xorg.conf.d/xorg.conf:
Section "Device"
Identifier "Configured Video Device"
Driver "dummy"
EndSection
Section "Monitor"
Identifier "Configured Monitor"
HorizSync 31.5-48.5
VertRefresh 50-70
EndSection
Section "Screen"
Identifier "Default Screen"
Monitor "Configured Monitor"
Device "Configured Video Device"
DefaultDepth 24
SubSection "Display"
Depth 24
Modes "1024x800"
EndSubSection
EndSection
With above, won’t see display after connect the PC with a monitor. Seeing error: /usr/bin/xinit: XFree86_VT property unexpectedly has 0 items instead of 1. Boot to recovery mode and try dpkg or failsafeX won’t resolve the problem as Xorg still take the modified xorg.conf. Delete xorg.conf.

Saturday, March 16, 2019

Android开发笔记: 3.11 System info, FileSystem, Security and Encryption


3.11 System info, FileSystem, Security and Encryption

3.11.1 System info

3.11.1.1 Retrieve an ID

Refer to so1, so2 and so3. According to this article, android.os.Build.SERIAL should be unique if it is available. From the article: Devices without telephony are required to report a unique device ID here; some phones may do so also.
The 1st link has summary of phone:
All devices tested returned a value for TelephonyManager.getDeviceId()
  1. All GSM devices (all tested with a SIM) returned a value for TelephonyManager.getSimSerialNumber()
  2. All CDMA devices returned null for getSimSerialNumber() (as expected)
  3. All devices with a Google account added returned a value for ANDROID_ID
  4. All CDMA devices returned the same value (or derivation of the same value) for both ANDROID_ID and TelephonyManager.getDeviceId() -- as long as a Google account has been added during setup.
  5. I did not yet have a chance to test GSM devices with no SIM, a GSM device with no Google account added, or any of the devices in airplane mode.
So if you want something unique to the device itself, TM.getDeviceId() should be sufficient. Obviously some users are more paranoid than others, so it might be useful to hash 1 or more of these identifiers, so that the string is still virtually unique to the device, but does not explicitly identify the user's actual device. For example, using String.hashCode(), combined with a UUID:
final TelephonyManager tm = (TelephonyManager) getBaseContext().getSystemService(Context.TELEPHONY_SERVICE);
final String tmDevice, tmSerial, androidId;
tmDevice = "" + tm.getDeviceId();
tmSerial = "" + tm.getSimSerialNumber();
androidId=""+android.provider.Settings.Secure.getString(getContentResolver(),android.provider.Settings.Secure.ANDROID_ID);
UUID deviceUuid = new UUID(androidId.hashCode(), ((long)tmDevice.hashCode() << 32) | tmSerial.hashCode());
String deviceId = deviceUuid.toString();
might result in something like: 00000000-54b3-e7c7-0000-000046bffd97

3.11.2 Filesystem

/data/app: apk files of all installed apps.
/data/app-lib: package subfolder which contains libs
/data/data: package subfolder which contains app files, include shared_prefs

3.11.3 Protect App source from decompiling: proguard

Refer to hackernews and this post for security apps. For security programing, refer to android crypto and this tutor. My thought and learnt from other app: use JNI which cannot be easily decompiled; rename class name to meaning-less; put in a lot of redundant code to confuse decompiler; Use filesize info as usually decompile and recompile will change the size significantly. To protect Java code against decompiler, here is several links, SO, Eclipse, xda.
  1. You can use an obfuscator, like proguard or Ygard, but it is not too complex to decrypt strings and rename classes, fields and methods. DexGuard is related to proguard and is specially for Android, but is not open source, though a free quote available.
  2. You can encrypt your classes with a private key, and use a custom classloader to decrypt your classes with a public key before loading into memory, but it is not too complex to modify the classloader to save onto a disc all the classes loaded.
  3. You can try crash decompilers. JAD is one of the best decompilers but if you add corrupted entries in the constant pools, all products powered by JAD crash. However, some decompilers are still working.

The only way to protect your software, is to deploy it in a SaaS/PaaS. Refer this for IaaS, PaaS and Saas: the importance of applications in a virtualized environment has brought about three major trends in cloud computing:
  • Infrastructure as a Service (IaaS): Used to manage low-level resources like VMs and disks. The end user is responsible for what is running within the VM, starting with the OS. IaaS is most closely related to a regular automated virtualized system. Amazon Web Services is maybe the best-known provider for an IaaS-style cloud service, but there are numerous others in the market.
  • Platform as a Service (PaaS): Provides faster development and deployment platforms by abstracting the user from the OS while adding well-defined APIs to many essential services (such as the Web, databases, mail, queues, and storage) that the developer must use. Both sides benefit; development should be faster and the end product should be more reliable more quickly. Typically PaaS also provides monitoring and administrative consoles tightly designed around the platform, making operations easier. At the same time, by maintaining tight control over what is executed when, PaaS can make better decisions about machine load, at least in theory. Another side effect is a vendor lock-in, desirable to the vendor but maybe not to the developer. Windows Azure is a good example of a PaaS-style cloud.
  • Software as a Service (SaaS): Instead of writing and maintaining every application, one uses online services. Examples include Google Mail and Salesforce. It is important to note that the SaaS software provided by a vendor might not be running in a cloud, but SaaS offerings often make sense when developing an application for the cloud. For instance, if I develop an e-commerce solution in the cloud, I might not want to reinvent my own load balancers, databases, or payment-processing system. I might not even want to run or operate them; perhaps all I want is to use these services. Using SaaS vendors allows me to concentrate on my own application.
Refer this for setting proguard with Android Studio. Refer this SO for enable proguard in Eclipse/ADT project. So for Android Studio, change build.gradle minifyEnabled to true. For Eclipse, uncomment this line in project.properties file:
proguard.config=${sdk.dir}/tools/proguard/proguard-android.txt:proguard-project.txt
this include one txt from SDK root(which has common setting), and one in the root of project folder(user defined). Then right mouse click the project=>Android Tools=>Export Signed/Unsigned Application Package. After, might need run zipalign, as mentioned in 3.9.1, like:
~/android-sdks/build-tools/23.0.3/zipalign -v 4 iFunTV.apk iFunTV_18.apk . After that, use adb install -r iFunTV_18.akp to update.
If got problem with ProGuard, follow instruction here to get around it. Usually ignore warning for external jar: -dontwarn org.apache.**
To keep a particular package/class: -keep class javax.** { *; }
Keep
From being removed or renamed
From being renamed
Classes and class members
-keep
-keepnames
Class members only
-keepclassmembers
-keepclassmembernames
Classes and class members, if class members present
-keepclasseswithmembers
-keepclasseswithmembernames
Note, for example, if want to keep hdp.http.MyApp =>public static String getTumd5(), should put inside -keepclasseswithmembers class hdp.http.MyApp either '*** getTumd5()' or 'public static ** getTumd5()', no 'String'.
Be careful of code like UpdateInfo apkInfo = (UpdateInfo) gson.fromJson(jsontxt, UpdateInfo.class), have to keep the class. Refer to Gson guide and SO. Another problem is Gson with ArrayList. Refer to this SO and this SO, and Gson Proguard rule.

The popular algerithm is RSA and AES. Refer to this SO for AES implementation for NDK: google openssl, and just AES implement.

Android开发笔记: 3.10.2 Capture touch event and simulate


3.10.2 Capture touch event and simulate

Refer to Android input. Android provide three utility for this: input, getevent and sendevent. Interestingly, cannot find official document about sendevent and input. Run ‘input’ and ‘sendevent’ alone will get usage info, such as usage: sendevent device type code value => Note: getevent output hex but sendevent only take decimal parameters, and no error report if pass hex value, just won’t work. Some info can be found from superuser, so and this blog: capture, convert with awk then playback: adb shell getevent | grep --line-buffered ^/ | tee /tmp/touch-events.log; Perform some touching, then ctrl-c and run: awk '{printf "%s %d %d %d\n", substr($1, 1, length($1) -1), strtonum("0x"$2), strtonum("0x"$3), strtonum("0x"$4)}' /tmp/touch-events.log | xargs -l adb shell sendevent. Note there might be timing with sendevent to simulate touch.
For touch events only 2 event types are used: EV_ABS (3) and EV_SYN (0).
Touching the display (in case of Type A protocol) will result in an input report (sequence of input events) containing the following event codes:
  • ABS_MT_TRACKING_ID (57 = 0x39) - ID of the touch (important for multi-touch reports)
  • ABS_MT_POSITION_X (53 = 0x35) - x coordinate of the touch
  • ABS_MT_POSITION_Y (54 = 0x36) - y coordinate of the touch
  • ABS_MT_TOUCH_MAJOR (48 = 0x30) - basically width of your finger tip in pixels
  • ABS_MT_PRESSURE (58 = 0x3A) - pressure of the touch
  • SYN_MT_REPORT (2) - end of separate touch data
  • SYN_REPORT (0) - end of report
Try getevent with: adb shell -- getevent -p, adb shell -- getevent -lp /dev/input/event2, adb shell -- getevent -lt /dev/input/event2
input’ implementation might be different per device, with a terminal app running, adb shell input text "hello” will get “hello” show after the shell prompt.