Developing code with WSL: 1 - Common problem

<WIP>

The two things Microsoft did recent years I would give a thumb up are WSL and VisualStudio Code. Both are gaining a lot of attraction along all software developers, I believe. Now, I'd like to make some note with WSL first. I had talked about WSL before, as:
Feb 2019: Running Linux on Windows

Jun 2020: Windows10 运行Ubuntu - WSL2

I'm going to create multiple post for what I learned while using WSL. This first one would be some common tips. 

Accessing files cross host Windows and WSL filesystem 

share folder: /mnt/c

running windows application

share git: cache issue

If user frequently or occasionally would switch between Windows and WSL, and run git command, may see that git will refresh index. As explained from this stackoverflow post:

• The thing you're using here, which Git variously calls the index, the staging area, or the cache, does in fact contain cache data.

• The cache data that it contains is the result of system calls.

• The system call data returned by a Linux system is different from the system call data returned by a Windows system.

Especially if have VS-Code installed on Windows, and w/o using WSL extension to access the repo, VS-Code will run the windows native git.exe with the GitLens extension.

Another thing user may need to consider is: under WSL, the host Windows' git.exe is faster than the WSL linux git. So a simple and efficient solution would be set an alias by adding one line in ~/.bashrc file with:

alias git=git.exe

With this, under WSL it will always invoke the Windows git.exe for fetching the index. It's faster, no refresh when switch between Windows and WSL Linux. A side note is this won't work for tcl/expect which is used by the autobld script. This probably is due to the way how WSL is handling running navtive Windows app under WSL. For that, we can take the other approach mentioned in the same stackoverflow post: set environment variable of GIT_INDEX_FILE

 Makefile : Clock skew detected

Makefile : Clock skew detected - Stack Overflow, this is not the case for WSL build. In fact: WSL2 Clock is out of sync with Windows - Stack Overflow. Another easy way to get around this is: close all WSL session, then from a console, issue: wsl --shutdown. Then open a new WSL session, it will automatically sync the time with Windows host.

GUI: Windows 11, or MobaXterm and 32 bit/64bit executable

Refer to Run Linux on Windows - WSL

RISC-v on ZC706 EVB - Part III: build w/ Win11/WSL2/Vivado 2016.2

 Continue my journey with of RISC-v on ZC706 Evaluation Board - Part II. Finally get some spare time can back on this. Since it may need some effort to get PetaLinux setup and debug the executable issue, I changed my focus to setup Vivado 2016.2 on another Windows 11 + WSL2 PC. Luckily Xilinx_Vivado_SDK_2016.2_0605_1.tar.gz is just around 11.1GB, even much smaller than 2016.4 package. And I chose WSL2 as on Windows 11 it support GUI, and would be less hassle do the build under Linux. However, I do run into several problem with fresh installed WSL2:

• Got 'libawt_xawt.so: libXrender.so.1: cannot open shared object file: No such file or directory' when running Vivado xsetup to install Vivado. Search from internet shows I need to do "sudo apt-get install libxrender1" , and that did work. Then hit next issue as below:
• Got 'libawt_xawt.so: libXtst.so.6: cannot open shared object file: No such file or directory', turns out that I need do 'sudo apt install libxtst6'. Then next issue:
• Got 'libawt_xawt.so: libXi.so.6: cannot open shared object file: No such file or directory', need to do "sudo apt install libxi6"

With that, xsetup is up and running. For installing option, I selected Vivado HL Design Edition as the WebPACK one might not have all features I need, and I don't need DSP support (for choices, may refer to Xilinx 2016.4 WebPACK Vivado and SDK Install on Windows 7 SP1 (css-techhelp.com) for more detail). Then the installation stuck with the flash screen for quite long time, which made me doubt whether it hanged or crashed. Luckily, it is not. After a while, the install option diag shows up, and I can select and continue the installation. At beginning I checked the option for creating Desktop shortcut, but not sure where it will go as I don't have Ubuntu Desktop installed. The installation not proceeding after a while, with log file name showed in console. By checking the log, likely there is issue with the shortcut. So re-run the installer, this time, not select the shortcut and create group box, however, installer still hangs, and log file shows it probably is running tcl script for generating the device list:

Executing script Generating installed device list: /home/Suser/Vivado/2016.2/bin/vivado [-nolog, -nojournal, -mode, batch, -source, /home/$user/.xinstall/Vivado_2016.2/scripts/xlpartinfo.tcl, -tclargs, /home/$user/Vivado/2016.2/data/parts/installed_devices.txt]
Tried 'sudo apt install tcl' but doesn't help. If manually run about  line, will get:
application-specific initialization failed: couldn't load file "librdi_commontasks.so": libncurses.so.5: cannot open shared object file: No such file or directory

Per this link, need to do: sudo apt install libncurses5

With that, I successfully installed Vivado 2016.2 under WSL. Then I start to pull fpga-zynq.git and build with it as described in fpga-zynq: 2) Pushing your rocket modifications to the FPGA, steps as:

• Start WSL Ubuntu terminal
  
• git clone https://github.com/ucb-bar/fpga-zynq.git to /mnt/c/workspace which can be easily accessed from Windows
  
• Run sudo apt install openjdk-8-jdk-headless; sudo update-alternatives --install "/usr/bin/java" "java" "/usr/lib/jvm/java-8-openjdk-amd64/bin/java" 1
• There is a strange thing happened. To install JDK8 under WSL, sometime would need to do sudo apt update, but somehow WSL give me an error as: "Release file for http://archive.ubuntu.com/ubuntu/dists/jammy-updates/InRelease is not valid yet (invalid for another 1d 3h 20min 52s). Updates for this repository will not be applied.". https://itsfoss.com/fix-repository-not-valid-yet-error-ubuntu/ gives me a clue on this: my WSL clock is skewed for some reason and it is lagged more than 1 day. So have to issue 'wsl --shutdown' then restart WSL to fix the problem
  
• For fresh WSL, would need do: sudo apt install make
  
• cd to git repo zc706 subfolder, run:
  

  make init-submodules

• Run below command, may get a lot of warn and error related to sbt, probably just ignore them:
  

  make rocket

• If see Cannot run program "dtc", do: sudo apt install device-tree-compiler
• Once 'make rocket' successfully completed, run Vivado setting script in the installation path such as: source ~/Vivado/2016.2/settings64.sh
• run  make project, and soon will see something like 'Wrote  : </mnt/c/code/fpga-zynq/zc706/zc706_rocketchip_ZynqFPGAConfig/zc706_rocketchip_ZynqFPGAConfig.srcs/sources_1/bd/system/system.bd>' at the end. 'system.bd' is folder contains system folder.
  
• Now run

  make fpga-images-zc706/boot.bin

  will see output like:

Successfully read diagram <system> from BD file </mnt/c/code/fpga-zynq/zc706/zc706_rocketchip_ZynqFPGAConfig/zc706_rocketchip_ZynqFPGAConfig.srcs/sources_1/bd/system/system.bd>

Verilog Output written to : /mnt/c/code/fpga-zynq/zc706/zc706_rocketchip_ZynqFPGAConfig/zc706_rocketchip_ZynqFPGAConfig.srcs/sources_1/bd/system/hdl/system.v
Verilog Output written to : /mnt/c/code/fpga-zynq/zc706/zc706_rocketchip_ZynqFPGAConfig/zc706_rocketchip_ZynqFPGAConfig.srcs/sources_1/bd/system/hdl/system_wrapper.v
Wrote  : </mnt/c/code/fpga-zynq/zc706/zc706_rocketchip_ZynqFPGAConfig/zc706_rocketchip_ZynqFPGAConfig.srcs/sources_1/bd/system/system.bd>
INFO: [BD 41-1029] Generation completed for the IP Integrator block proc_sys_reset_0 .
WARNING: [xilinx.com:ip:processing_system7:5.5-1] system_processing_system7_0_0: The Zynq BFM requires an AXI BFM license to run. Please ensure that you have purchased and setup the AXI BFM license prior to running simulation with this block. Please contact your Xilinx sales office for more information on purchasing this license
INFO: [BD 41-1029] Generation completed for the IP Integrator block processing_system7_0 

...

Attempting to get a license for feature 'Synthesis' and/or device 'xc7z045'
WARNING: [Common 17-348] Failed to get the license for feature 'Synthesis' and/or device 'xc7z045'
3 Infos, 1 Warnings, 0 Critical Warnings and 1 Errors encountered.
synth_design failed
ERROR: [Common 17-345] A valid license was not found for feature 'Synthesis' and/or device 'xc7z045'. Please run the Vivado License Manager for assistance in determining
which features and devices are licensed for your system.
Resolution: Check the status of your licenses in the Vivado License Manager. For debug help search Xilinx Support for "Licensing FAQ".
INFO: [Common 17-206] Exiting Vivado at Fri Apr  7 19:48:34 2023...
[Fri Apr  7 19:48:34 2023] synth_1 finished
wait_on_run: Time (s): cpu = 00:00:00.15 ; elapsed = 00:00:28 . Memory (MB): peak = 1186.469 ; gain = 0.000 ; free physical = 241 ; free virtual = 1777
# launch_runs impl_1 -to_step write_bitstream
ERROR: [Common 17-70] Application Exception: Failed to launch run 'impl_1' due to failures in the following run(s):
synth_1
These failed run(s) need to be reset prior to launching 'impl_1' again.
...

Likely would need a license to run synth and impl. As mentioned at https://dzone.com/articles/create-a-bootbin-program-an-sd-card-and-boot-a-zc7, The ZC706 has an XC7Z045-2FFG900 C SoC. This chip is not supported in WebPACK (see [link]) like the chip on the ZC702. You can get a 30-day license at [link].

Next, might see 'bootgen: command not found'. This could happen if user didn't run the Xilinx Vivado settings64.sh shell script, or XiLinx SDK wasn't installed. If the SDK is installed, bootgen should be available under Vivado/2016.2/bin folder. Without SDK, make boot.bin would fail. So need to select SDK during installation as below (I only select Zynq-7000 to save space and installing time. Also, not select Acquire or Manage a License Key as I will deal with that separately):

If already have Vivado installed, then it won't allow you to re-install at same location. Would need to launch Vivado, then goto help->Add Design Tools or Devices...->Check SDK and click install.

After all this, finally can successfully build the boot.bin. With this boot.bin copied to SD card, it can successfully boot the PetaLinux (connect the Uart USB, and use root/root to login). But running "./fesvr-zynq pk hello" still get "ERROR: No cores found".

Also tried following the prompted message for the problem 5 mentioned in Risc-v on zc706 evaluation board - Part I: ERROR: [BD_TCL-109] This script was generated using Vivado <2016.2> and is being run in <2022.1> of Vivado. Please run the script in Vivado <2016.2> then open the design in Vivado <2022.1>. Upgrade the design by running "Tools => Report => Report IP Status...", then run write_bd_tcl to create an updated script.

I opened the project file zc706_rocketchip_ZynqFPGAConfig.xpr created by 2016.2 Vivado with 2022.1 Vivado, and keep the project as Read-only while the new version Vivado prompt for choice. It would still treat the board type as error:

• [Board 49-67] The board_part definition was not found for xilinx.com:zc706:part0:1.0. This can happen sometimes when you use custom board part. You can resolve this issue by setting 'board.repoPaths' parameter, pointing to the location of custom board files. Valid board_part values can be retrieved with the 'get_board_parts' Tcl command.

For this, no need to update update zc706/src/tcl/zc706_rocketchip_ZynqFPGAConfig.tcl, but update the project xpr file directly as it is XML format, just update <Option Name="BoardPart" Val="xilinx.com:zc706:part0:1.0"/> to xilinx.com:zc706:part0:1.4. For above update instruction, I cannot find 'write_bd_tcl' script, but did find there is a man doc as Vivado/2022.1/doc/eng/man/write_bd_tcl. And 'Report IP' now is under menu/Report but not Tools. Open the Tcl Console (menu/Window->Tcl Console), issue command write_bd_tcl with output filename. It may prompt:  ERROR: [BD 5-229] Please open or create a block design first. So need to select from left Flow Navigator->IP INTEGRATOR-> Open Block Design before running the write_bd_tcl command.

CodeFormer: AI Face Enhancement, 图片人脸智能增强

最近看到这个由周尚长博士领队的几位南洋理工大学S-Lab成员开发的开源程序，用来恢复和增强照片中的人脸，效果惊人。可以点进以下3分半的you-tube看一下就知我所言不虚(包括1927 Fifth Solvay International Conference群英会的照片)。

 

Face Restoration

Face Color Enhancement and Restoration

Face Inpainting

 

https://shangchenzhou.com/projects/CodeFormer/ 是项目主页，有些简单介绍。code的github网址是：https://github.com/sczhou/CodeFormer，附有简单的安装使用说明，除了要git clone代码，和下载样本数据外，主要是要设置conda。关于conda, 可参见：学习TensorFlow - 设置Python环境。Miniconda就足够了（参见https://stackoverflow.com/questions/45421163/anaconda-vs-miniconda）

注意，CUDA不是必须的，像我的电脑是AMD APU，不支持nVida的CUDA，也可以完美的运行CodeFormer。

当你用CodeFormer转化自己的照片时，最初可能并不会注意到转化后的照片有什么不一样，记住，一定要zoom in，放大再放大，看细节,一定有惊喜。


除了增强照片，Codeformer还支持video的人脸增强(需要引入FFMpeg)。有空要研究研究怎样加入AMD ROMc的支持，加快在我的AMD计算机上的处理速度。

 

Setup ssh tunnel/port forward on Windows

 Reference link:

https://docs.oracle.com/cd/E24628_01/install.121/e22624/preinstall_req_cygwin_ssh.htm

https://linuxize.com/post/how-to-setup-ssh-tunneling/

Make sure for Cygwin, has openssh installed. Then run

ssh-host-config

*** Query: Enter the value of CYGWIN for the daemon: [] binmode ntsec

Then run this to start the service: cygrunsrv -S sshd

May need to change Windows firewall setting to open the port 22.

 

SSH tunneling or SSH port forwarding is a method of creating an encrypted SSH connection between a client and a server machine through which services ports can be relayed.

There are three types of SSH port forwarding:

• Local Port Forwarding. - Forwards a connection from the client host to the SSH server host and then to the destination host port.
• Remote Port Forwarding. - Forwards a port from the server host to the client host and then to the destination host port.
• Dynamic Port Forwarding. - Creates a SOCKS proxy server that allows communication across a range of ports.

RISC-v on ZC706 Evaluation Board - Part II

With RISC-v on ZC706 Evaluation Board - Part I, I was successfully building the boot.bin. I thought I got the work done, but realized it was not when I tried the binary I made: after replaced the 'boot.bin' on the SD card with the one I built, the Init LED turns to green for about 1 second, then turns to red, no output from UART port, so something is wrong. Actually there is a warning while making the boot.bin:

[WARNING]: Partition FSBL.elf.0 range is overlapped with partition rocketchip_wrapper.bit.0 memory range
[INFO]   : Bootimage generated successfully

Would need to check the partition layout.

Beside this problem, I'm seeing another issue when playing more, with the fetched SD binaries: running "./fesvr-zynq pk hello" got "ERROR: No cores found", same error for any command with fesvr-zynq. Search the symptom returned same issue:

https://github.com/chipsalliance/rocket-chip/issues/1390

https://github.com/ucb-bar/fpga-zynq/issues/87

Likely there is an update for zedboard but not for zc706. So likely will have to build this fesvr-zynq by myself. Run 'make fesvr-zynq' lead me realized that all sub-modules of rocket-chip/riscv-tools didn't get cloned, especially 'riscv-fesvr' which is needed for making fesvr-zynq. With that repo cloned, the build did make some progress, but eventually would complain cannot find 'arm-xilinx-linux-gnueabi-g++'. Though I did find the executable under Xilinx Vitis, running environment setup script, or even adding the path of the executable won't help, as it needs the PetaLinux build environment, which seems hasn't been fully integrated to Xilinx Vitis. Have to download PetaLinux Tools from Xilinx site.

Will update this once get some progress on this.

Below is quoted from https://github.com/ucb-bar/fpga-zynq

Overview of System Stack

Our system will allow you to run a RISC-V binary on a rocket core instantiated on a supported Zynq FPGA. This section will outline the stack of all of the parts involved and by proxy, outline the rest of the documentation. Going top-down from the RISC-V binary to the development system:

Target Application (RISC-V binary) will run on top of whatever kernel the rocket chip is running. Compiled by riscv-gcc or riscv-llvm.

RISC-V Kernel (proxy kernel or RISC-V Linux) runs on top of the rocket chip. The proxy kernel is extremely lightweight and designed to be used with a single binary linked against Newlib while RISC-V Linux is appropriate for everything else.

Rocket Chip (rocket core with L1 instruction and data caches) is instantiated on the FPGA. Many of its structures will typically map to various hard blocks including BRAMs and DSP slices. It communicates to the host ARM core on the Zynq via AXI.

Front-end Server (riscv-fesvr) runs on the host ARM core and provides an interface to the rocket chip running on the FPGA (connected via AXI).

Zynq ARM Core (actually dual Cortex A9) runs Linux and simplifies interfacing with the FPGA.

FPGA Board (Zybo, Zedboard, or ZC706) contains the Zynq FPGA and several I/O devices. At power on, the contents of the SD card are used to configure the FPGA and boot Linux on the ARM core.

External Communication (TTY over serial on USB or telnet/ssh over ethernet) allows the development system to communicate with the FPGA board.

Development System (PC with SD card reader) generates the images to configure the FPGA.