Remote Serial Port Access

Not just because of COVID-19 people may need to work from home, there is a lot of case that people need to access a serial port on a remote setup, such as access a Raspberry-Pie UART on a host PC. VirtualHere provides a commercial solution to create a Virtual USB on local host to virtually represent the USB port on the remote host. This works pretty well with FTDI USB-to-Serial chips. There are many other solutions. Specially if you like me want to have a simple manageable solution, such as using script to do the job. A good example I found from here: http://fabacademy.org/archives/2015/doc/WebSocketConsole.html, which create a web view of UART output with purely Python script.

Too lazy to check what the license is for posting the code. So I won't try to copy and paste the article here. It was published in 2015, a little old, but I assume it should still work well. When I tried it out on Windows recently, I do see some issue, mainly due to known issue for python package on Windows.

Python 2.x is too old. So the first thing is updating for Python 3.x, such as convert all "print '...' " to "print('...')". There is also some tab/space mixing issue need to be fixed.

For using pyserial to open serial port on windows such as COM6, no need to use the awkward device name "//./COM6", just open "COM6" should be fine.

The big issue is the multiprocessing pickling error on Windows, will get error:

self = reduction.pickle.load(from_parent)
EOFError: Ran out of input

Google it will get a bunch of similar topic, such as this pytorch one: use if __name__ == '__main__': to protect your main function (more details here). Can also try to run the script in command prompt. If the error persists, then you can try to set num_worker of DataLoader to zero.

And with Stackoverflow. As explained here: Windows doesn't have fork, so there's no way to make a new process just like the existing one. So the child process has to run your code again, but now you need a way to distinguish between the parent process and the child process, and __main__ is it.  

'if __name__ == '__main__'' is already in the original code, the num_worker seems is pytorch specific. So for this case, looks like need to fix empty input problem, maybe need to force the serial port read as a blocking read? Tried changing serialworker.py as:

self.sp = serial.Serial(SERIAL_PORT, SERIAL_BAUDRATE, timeout=0)

but it doesn't work out. So maybe need to consider using pyserial in thread instead.

--- to be continue ---

Though I wasn't able to figure out the reason, but I'm able to get it working by removing the serialworker class completely, put all serial I/O code to the worker of multiprocessing.Process.

Now new issue surfaced: 

1) appdata\local\programs\python\python38\lib\site-packages\tornado\platform\asyncio.py", line 79, in add_handler

    self.asyncio_loop.add_reader(
  File "c:\users\kokat\appdata\local\programs\python\python38\lib\asyncio\events.py", line 498, in add_reader
    raise NotImplementedError

This is a known issue as tornado issue 2068, and also mentioned in Tornado site. The fix/workaround is adding below in \Lib\site-packages\tornado\platform\asyncio.py

import sys

if sys.platform == 'win32':
     asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())

Or add above line in main, right after the if __name__ == '__main__' check (add import asyncio)

2) error TypeError: __init__() got an unexpected keyword argument 'io_loop'

One solution is as User a local Jupyter Notebook, downgrade tornado with:

 pip install tornado==4.5.3

This may also get rid of the first issue. But I more prefer to use the new version of Tornado. So the other solution is remote the obsoleted keyword 'io_loop'.

3) checkQueue() is invoked by a scheduler, with 100 as interval, the 'if' check in the code only can pick one line serial port output from the queue, which will lead to bad throughput. Should change the 'if' check as 'while', like this:

    while not output_queue.empty():

        message = output_queue.get()

4) WebSocketHandler is not working as expected, one reason is that main.js failed to be loaded. Other reason could be with the Javascript and the html script.

--- to be continue ---

After some research, debugging and testing, I got a solution to have a html file with inline JavaScript. It's not perfect, still has some minor issue, such as the 'clear' button doesn't work as expected. The code is like below (modified based on tutorialspoint code):

<!DOCTYPE html>

<html>

   <head>

      <meta charset = utf-8>

      <title>WebSocketConsole</title>

      <body>

         <section id = "wrapper">

            <header><h1>WebSocketConsole</h1></header>

            <style>

               body { margin: 0px; padding: 20px; }

               #status { padding: 5px; color: #fff; background: #ccc;}

               #status.fail { background: #c00; }

               #status.success { background: #0c0; }

               #status.offline { background: #c00; }

               #status.online { background: #0c0; }

               #received { width: 500px; height: 400px; border: 1px solid #dedede; font-family: "Lucida Console", Courier, monospace; font-size: 10px;

                           overflow-y:scroll; display:flex; flex-direction:column-reverse}

               #cmd_str { width: 97%; }

               .message { font-weight: bold; }

               .message:before { content: ' '; color: #bbb; font-size: 14px; }

            </style>

            <article>

               <p id = "status">Not connected</p>

               <p>Users connected: <span id = "connected">0</span></p>

               <p>Data received from serial port</p>

               <div id="received"></div>

               <button id="clear">Clear</button>

               <p>Send data to serial port</p>

               <form onsubmit = "send_cmd(); return false;">

                    <input type = "text" id = "cmd_str" placeholder = "type cmd and press enter" />

               </form>

            </article>

            <script>

               var state = document.getElementById("status");

               var connected = document.getElementById("connected");

               var received = document.getElementById("received");

               var clear = document.getElementById("clear");

               clear.onclick = function() {

                   received.empty();  //TODO: this does not clear the log. Need fix

               }

               var cmd_str = document.getElementById("cmd_str");

               if (window.WebSocket === undefined) {

                  state.innerHTML = "sockets not supported";

                  state.className = "fail";

               }else {

                  if (typeof String.prototype.startsWith != "function") {

                     String.prototype.startsWith = function (str) {

                        return this.indexOf(str) == 0;

                     };

                  }

                  window.addEventListener("load", onLoad, false);

               }

               function onLoad() {

                  websocket = new WebSocket("ws://"+location.host+"/ws");

                  websocket.onopen = function(evt) { onOpen(evt) };

                  websocket.onclose = function(evt) { onClose(evt) };

                  websocket.onmessage = function(evt) { onMessage(evt) };

                  websocket.onerror = function(evt) { onError(evt) };

               }

               function onOpen(evt) {

                  state.className = "success";

                  state.innerHTML = "Connected to server";

               }

               function onClose(evt) {

                  state.className = "fail";

                  state.innerHTML = "Not connected";

                  connected.innerHTML = "0";

               }

               function onMessage(evt) {

                  var message = evt.data;

                  if (message.startsWith("connected:")) {

                     message = message.slice("connected:".length);

                     connected.innerHTML = message;

                  } else {

                    //console.log("receiving: " + message);

                    received.append(message);

                    var linebreak = document.createElement("br"); //refer to link

                    received.appendChild(linebreak);

                  }

               }

               function onError(evt) {

                  state.className = "fail";

                  state.innerHTML = "Communication error";

               }

               function send_cmd() {

                  var message = cmd_str.value;

                  cmd_str.value = "";

                  websocket.send(message);

               }

            </script>

         </section>

      </body>

   </head>  

</html>