Monday, February 11, 2019

Running Linux on Windows

讲了不少虚拟机,container,模拟器等,大部分都是运行在Linux上。我想微软认识到了Linux支持的重要性,特别是在开发应用中,比如Android开发,Linux的开发者用户群很大,所以一年多前微软就通过Windows 10的Insider/Creators program (WSL, Windows Subsystem for Linux )引人Ubuntu的支持,可以在Windows console下运行Ubuntu命令:如 bash, ssh, git, apt等等。现在这一feature已经正式出现在Windows App Store: https://www.microsoft.com/en-us/p/ubuntu/9nblggh4msv6,甚至可以在旧版Windows 10 (如1709)上运行。从微软网站摘录的:
Note that Windows 10 S does not support running this app. Before installing Ubuntu on Windows or before the first run please open the Control Panel, visit Programs and Features' submenu Turn Windows features on or off and select Windows Subsystem for Linux. Click OK, reboot, and then your system is ready to run this app. Instead of the above steps You can set up your system by executing the following command in the Administrator PowerShell prompt: Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux When your system is ready simply start "ubuntu" in cmd.exe or launch Ubuntu from the start menu.

Linux上的VM



Linux有很好的VM support,如Kernel Virtual Machine (KVM), 由kernel module提供支持。QEMU (https://www.qemu.org, 一个开源的machine emulator and virtualizer)包含KVM的userspace component。如果你做过Android开发,用过Android的虚拟机AVD,那就是用QEMU实现的。KVM和QEMU是基于Linux内核的,所有有很小的footprint,效率却很高。When used as a virtualizer, QEMU achieves near native performance by executing the guest code directly on the host CPU. QEMU supports virtualization when executing under the Xen hypervisor or using the KVM kernel module in Linux. When using KVM, QEMU can virtualize x86, server and embedded PowerPC, 64-bit POWER, S390, 32-bit and 64-bit ARM, and MIPS guests. 

KVM vs VirtualBox

Both are opensource. According to this: KVM is better integrated with Linux, it's smaller and faster, and while you can use it with other guests besides Linux. Vbox supports suspends, and may be better for other guest OS such as BSD and Solaris.

Running Windows program on Linux and MacOS: Wine

因为很多原因我工作之外更喜欢使用Linux/Ubuntu操作系统。我在家的常用计算机是双操作系统,Windows10 + Ubuntu。Boot来boot去的到也很快,但还是不很痛快。比如,Linux可以access Windows的硬盘分区,可是Windows不支持Linux的ext4分区。又如Google Drive App只有Windows和Mac版,没有Linux版(很奇怪)。有时我会玩老游戏如微软的帝国时代和红警,感谢开源Wine (Wine Is Not Emulator) enables Linux, Mac, FreeBSD, and Solaris users to run Windows applications without a copy of Microsoft Windows. Wine is free software under constant development. Other platforms may benefit as well. 
https://wiki.winehq.org/Ubuntu 提供了源程序和binary,很容易安装。wine FAQ 提及At present there are some significant bugs that prevent many 32 bit applications from working in a 64 bit wineprefix。很多Windows程序都是32位的,所以需要32位Linux support:

If your system is 64 bit, enable 32 bit architecture (if you haven't already):
sudo dpkg --add-architecture i386
为了用apt安装WINE,需要把WINE加到Ubuntu的软件list里,download and add the repository key:

sudo apt-key add winehq.key

Ubuntu 18.04: sudo apt-add-repository 'deb https://dl.winehq.org/wine-builds/ubuntu/ bionic main'

sudo apt update && sudo apt install --install-recommends winehq-stable


安装完后,在home folder的.wine目录下,可以找到drive_c,就是mapped的C drive。运行的程序并不必须copy到这个目录下。在这个.wine目录下运行winecfg会自动安装.net等package。之后可以运行wine notepad来确认wine设置成功。

因为我的游戏是中文汉化版,为了中文支持,首先需要在Host OS上加中午支持。
User guide: For Chinese font support, refer to test language: need to install language package for linux first, such as:
sudo apt-get install language-pack-zh-hans
then run like: LANG=zh_CN.UTF-8 wine empires2.exe to start Empire 2.
注意怎样在Ubuntu上添加拼音输入支持,先要安装如上的pack,然后在setting/Language and Region input source里加入拼音输入,注意,一定要加pinyin 而不是Chinese,不然,就没法用拼音。

帝国时代游戏里有选项可以改变游戏分辨率。取决于winecfg的Graphics设置,可能会有不同的效果和问题。如右图所示, Graphics有4个checkbox,最重要的是第3项和第4项:
Allow the window manager to control the windows
Emulate a virtual desktop 和 Desktop size
当这两项都没有选取时,游戏显示是正常的,在游戏中改变分辨率,游戏的显示还是正常,但是游戏的快捷键可能不起作用。如果只选第3项,键盘工作正常,但切换游戏分辨率后,一部分屏幕可能不能正常刷新。我相信应该有办法tweak wine解决这个问题,不过最简单的办法就是把3和4都选上,显示是virtual的,而不是全屏的,看起来画面更细致。
 




WINE也可以安装到Mac OS上。

On MacOS, refer to https://wiki.winehq.org/MacOS
如上述链接所说的,需要先确保Xquartz已安装。Xquartz也是一个开源软件: https://www.xquartz.org/, an open-source effort to develop a version of the X.Org X Window System that runs on OS X. 在MacOS上32bit wine是基本安装, 64bit 是可选项。设置运行和Linux /Ubuntu基本一样。
总的来说,WINE的solution与VM和container完全不同。 VM是在指令一层simulate,container则必须运行在同种OS,而WINE是把Windows API calls 转换成POSIX API calls on-the-fly,各有千秋。 

Update in 2020: 参见winetricks: A useful tool for using common workarounds to current deficiencies in Wine,很方便用来安装runtime support。


Wednesday, February 6, 2019

学习舵轮(kubernetes)和埠(docker) - 创建NodeJS+MongoDB docker image


讲了很多概念,下面是一些docker的源代码,用来创建前面讲的NodeJS+MongoDB的应用。DockerHub提供很多常用的container image,包括NodeJS image。如下的Dockerfile就会载入最新的NodeJS container,并运行npm(Node Package Manager)根据用户提供的package.json文件来安装需要的Node package。
FROM node:latest
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY package.json /usr/src/app/ #package.json like this contains dependencies and entry point
RUN npm install
COPY . /usr/src/app
EXPOSE 3000
CMD [ “npm”, “start” ]
用这个Dockerfile可以build一个NodeJS的container,要加MongoDB支持,需要创建如下YAML compose file:
version: "1"
services:
app:
  container_name: app #optional
  build: . # Dockerfile location for building nodejs App
  ports:
  - "8080:3000" # port forwarding
  depends_on: # same as links?
  - mongo
mongo:
  container_name: mongo #optional
  image: mongo
  ports:
  - "27017:27017"
  volumes:
  - data-volume:/data/db
volumes:
data-volume:
把这个compose file 存为my-compose.yml,然后运行: 
docker-compose -f my-compose.yml up
就会创建和运行一个提供NodeJS+MongoDB的container。打开localhost:8080就可以测试。
 
对我来说,Docker的NodeJS image的size有点大。我更prefer小巧的alpine Linux,所以就创建了如下的Docker file:
FROM alpine
COPY . /app
WORKDIR /app
RUN apk update && apk add nodejs
EXPOSE 3000
CMD ["node","app.js"]
这是基于alpine image,安装NodeJS,并运行app.js script。app.js的内容如下:
var http = require('http');
http.createServer(function (req, res) {
 res.write('Hello World!'); //write a response to the client
 res.end(); //end the response
}).listen(3000);
这是一个简单的‘HelloWorld’ web server code。用类似如下命令build和运行:
docker image build -t my_project:v0.1 .
docker run -d -p 8080:3000 my_project:v0.1 #detach mode, remap port to 8080,or use docker-machine ip to find container ip.
运行结果是web server 不work,查看log就会发现,虽然NodeJS和MongoDB都安装了,但Node的Mongo package没有安装,JS code 就没办法talk to MongoDB。所以需要更新Dockerfile的‘RUN’这一行:
RUN apk add --update nodejs nodejs-npm mongodb; RUN npm install mongodb 
重新build会看到message:
 Step 5/7 : RUN npm install mongodb
---> Running in 216d30b5ce3b
npm WARN saveError ENOENT: no such file or directory, open '/app/package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN enoent ENOENT: no such file or directory, open '/app/package.json' …
再运行还是有问题,这次是JS知道怎样access mongoDB,但MongoDB没有起来,需要运行MongoDB daemon:
Refer this for starting mongod: mongod --port 27017 --dbpath /app/ --fork --logpath /var/log/mongodb.log
#need specify --dbpath before we add the volume mount point; use --fork to run as daemon. Will get:
WARNING: Access control is not enabled for the database. Read and write access to data and configuration is unrestricted.
WARNING: You are running this process as the root user, which is not recommended.
WARNING: This server is bound to localhost. Remote systems will be unable to connect to this server. Start the server with --bind_ip
to specify which IP addresses it should serve responses from, or with --bind_ip_all to bind to all interfaces. If this behavior is desired, start the server with --bind_ip 127.0.0.1 to disable this warning.
Refer to alpine_Init_Scripts, after mongo is installed, mongodb and mongos are created under /etc/init.d. But openrc isn’t installed by default. So need add ‘openrc’ to RUN apk add command in dockerfile. 
不要在Dockerfile里加上RUN上面的启动mongod daemon 的 command line, 那样做会造成生成的significantly size increase of docker container image, from 207M to 508M. It’s a little complex to get the service setup properly (rc-update add mongodb default, may need to setup environment for mongo), 所以我把 mongod cmdline and nodejs cmdline 放在一个 shell script, 然后把run这个shell script 的 CMD line 加到 dockerfile。这样创建的docker image只有208M左右。

常用的MongoDB命令:
mongo cmds: db.stats(), show dbs, use my_db, show collections.
MongoDB v3.0+有些改动,如遇到TypeError: db.collection is not a function,refer to this SO, for mongodb>=3.0, code like var col_history = db.collection('history') will lead to TypeError: db.collection is not a function. Change the code to: var col_history = db.db().collection('history')

把database 数据文件存到另外mount的volume上: use a separate volume for db, run: docker volume create mongo_db, update Dockerfile with VOLUME /data and set --dbpath /data/ in mongod cmdline. Then start container like this: docker container run -d -p8080:3000 --mount type=volume,source=mongo_db,destination=/data my_project:v0.2