Sunday, July 14, 2024

Learning SDL Part II

 In Learning SDL Part I I talked about the 2DLight example from https://glusoft.com/sdl2-tutorials. It might be a bit too advanced for beginner. The examples have external dependencies, and might not easy to be understood without basic concept of SDL. Code might have not been maintained after years. And may due to build tools update or package update, code cannot be built out-of-box. So I decide to start from simple one: https://lazyfoo.net/tutorials/SDL

The first example is "01_hello_SDL". Since it is a simple one, I'm going to try to build it with NMake. A simple Makefile for x64 would be like this:

# Compiler
CC=cl
# Include directory for SDL headers
SDL_INCLUDE=C:\path\to\SDL\include\SDL2
# Library directory for SDL library files
SDL_LIB=C:\path\to\SDL\lib\x64
# Compiler flags, appending /Zi if want to debug
CFLAGS=/I$(SDL_INCLUDE) /Dmain=SDL_main
# Linker flags
LFLAGS=/link /LIBPATH:$(SDL_LIB) SDL2.lib SDL2main.lib SDL2_image.lib Shell32.lib /SUBSYSTEM:CONSOLE

# Target executable name
TARGET=01_hello_SDL.exe
# Source files
SOURCES=01_hello_SDL.cpp

# Rule to make the target
$(TARGET): $(SOURCES)
    $(CC) $(CFLAGS) /Fe:$(TARGET) $(SOURCES) $(LFLAGS)

# Clean target
clean:
    del $(TARGET) *.obj

Note: 1) have to specify /SUBSYSTEM:CONSOLE, otherwise may get entry point not defined error as Win32 app will look for WinMain as entry point.

2)  Have to link to SDL2main.lib and Shell32.lib. SDL2main.lib provides SDL entry function. The Shell32.lib is needed, otherwise, will get error 'LNK2019: unresolved external symbol __imp_CommandLineToArgvW referenced in function main_getcmdline'

3) For some projects need more image format support such as png, would need link to more lib, such as 06_extension_libraries_and_loading_other_image_formats, would need libpng16-16.dll and zlib1.dll which is a dependency of the png dll. If this zlib1 is missing, will only see error complaining "Failed loading libpng16-16.dll" which might be misleading.

4) For projects which needs font, would need link to SDL2_ttf.lib, and needs libfreetype-6.dll at runtime

5) For projects which needs audio, would need link to SDL2_mixer.lib

Run this to setup x64 env: "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\Build\vcvars64.bat", then run 'NMake' to build. Can copy the x64 SDL dll to C:\Windows\System32 folder, then won't need to copy them to working folder every time.

6) For projects using OpenGL, OpenGL very likely is already installed on the system. May check whether opengl.dll and glu.dll under \windows\system32. To link the lib, add these two libs:

opengl32.lib glu32.lib

7) For OpenGL Extension Wrangler library, can be downloaded from https://glew.sourceforge.net/

Learning SDL Part I

 

While trying to add more feature to the Tanks game, I realized that I need dive deeper into SDL programming. For this reason, I went to https://glusoft.com/sdl2-tutorials/ and downloaded project files. As I didn't have the Visual Studio installed, but only the build tools, so need some tweak to build the examples. The downloaded package comes with solution (.sln) and project (.vcxproj) files. So the project can be built with command like: msbuild 2DLight.sln /p:Configuration=Release /p:Platform="x86"

Also need to update the vcxproj file with v141=>v143, set WindowsTargetPlatformVersion to 10.0.22621.0 which matches my environment, and set AdditionalIncludeDirectories and AdditionalLibraryDirectories to have my SDL2 header file and lib paths, and need to make sure the setting is in 'x64' session if using x64 as platform in above command line. And for the first 2DLight example, would need download https://github.com/trylock/visibility and SDL2_gfx as well. Note SDL_gfx is not from libsdl.org as other SDL lib. And note, there is SDL2_gfx-1.0.4.tar.gz (.zip) and SDL_gfx-2.0.27.tar.gz, though the 2.0.27 is a newer release (Ver 1.0.4 – 11 Feb 2018 vs Ver 2.0.27 – Sun Dec 10 2023), the later is not compatible with SDL2, and though it comes with a SDL2 patch (also needs some tweak such as needs to link with SDL2.lib instead of SDL.lib, rename the project file as SDL2_gfx.vcxproj), the resulted lib may not work with the example code from sdl2-tutorials.

Need to make similar update for SDL2_gfx.vcxproj. And it does not have x86, but Win32, so build command is:
msbuild SDL2_gfx.sln /p:Configuration=Release /p:Platform="Win32"

May see this error: SDL2_gfx-1.0.4\SDL2_gfxPrimitives_font.h(1559,1): warning C4819: The file contains a character that cannot be represented in the current code page (936). Save the file in Unicode format to prevent data loss

Just remove these characters (all are inline comment) should be OK.

For error  SDL2_gfx-1.0.4\SDL2_gfxPrimitives.c(1771,2): error C2169: 'lrint': intrinsic function, cannot be defined, refer to https://www.ferzkopp.net/wordpress/2016/01/02/sdl_gfx-sdl2_gfx. Just comment out the define/implementation as this is already defined as intrinsic function.

There would be a bunch error for building test if those vcxproj files have not been updated, and can be ignored for now. If all test project files got updated, 3 of the 4 tests should work out-of-box, but for TestImageFilter, would not see any thing displayed or printed to terminal. By checking the code, this Image filter test is using 'printf' to log information, and the test has no graphics display. The 'printf' won't spite out anything either, probably due to SDL hijacked the console I/O. The C code also includes 'windows.h' when 'WIN32' is defined. And in fact, w/ or w/o this header file makes no difference. The way to get some print out is replacing 'printf' in the code with 'SDL_Log' call. With SDL_Log, I got 23 of 27 passed OK.

Now, back to the 2DLight example, all build are OK, but will get runtime error as "The application was unable to start correctly. 0xc00007b". Turns out this is due to mixed using x64 build tools to build x86 dll and executable. And the behavior is very weird. So, just avoid doing that.

Now with x86 build environment, build SDL2_gfx as Win32 target, and use other x86 SDL2 lib/dll, also make sure the image/PNG files are copied to the run folder, no more 0xc00007b, however, seeing an assert for vector subscript out of range. Per Copilot:

The vx.reserve(result.size()) call in the code snippet reserves memory for result.size() elements in the vector vx. However, it does not change the size of the vector. The reserve() function is used to allocate memory in advance to prevent frequent reallocations when adding elements to the vector.

To actually change the size of the vector to match the reserved capacity, you can use the resize() function instead of reserve(). The resize() function not only reserves memory but also sets the size of the vector to the specified value.

After I replaced the two reserve() with resize(), the code mostly works as expected, may still see 'Expression: vector subscript out of range' sometimes, likely usually happened when moving the mouse cursor out of the frame, probably due to missing boundary checking. Updated source code was pushed to forked https://github.com/quyq/2DLight-SDL.

If you are using VSCode as editor like me, you may create task.json like this to allow building the project with MSBuild using shortcut 'ctrl+shift+b' (the top half set build env for x86, the bottom half run MSBuild with the solution file):

{
    "version": "2.0.0",
    "windows": {
      "options": {
        "shell": {
          "executable": "cmd.exe",
          "args": [
            "/C",
            // The path of batch file and platform parameter for setting the build env
            "\"C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/VC/Auxiliary/Build/vcvarsall.bat\"",
            "x86",
            "&&"
          ]
        }
      }
    },
    "tasks": [
      {
        "type": "shell",
        "label": "MSBuild.exe build active file",
        "command": "MSBuild.exe",
        "args": [
            "2DLight.sln",
            "-p:Configuration=Release"
        ],
        "problemMatcher": ["$msCompile"],
        "group": {
          "kind": "build",
          "isDefault": true
        }
      }
    ]
}

 If want to use VSCode built-in debugger, may create launch.json as:

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "2DLight",
            "type": "cppvsdbg",
            "request": "launch",
            "program": "2DLight.exe",
            "args": [],
            "stopAtEntry": false,
            "cwd": "${workspaceFolder}/Release",
            "environment": [],
            "console": "externalTerminal"
        }
    ]
}

Wednesday, July 10, 2024

Update Tanks (坦克大战) with network support

With Revisit 坦克大战编程, can easily build Tanks on Windows or under WSL. However, with one player the game would be a bit boring and a bit frustrate; and with two player2 on same PC, it would be a bit crowd for four hands on one keyboard. If would be more attractive if two players can play together remotely, or even play against each other, would be much fun. So, we need to bring in network support. Would need to choose a  suitable networking library, (e.g., SDL_net, which complements SDL, or another networking library like Boost.Asio for C++). Need to initialize the networking in the application, setting up one instance as the server and the other as the client.

I haven't been using either network library. So will pick SDL_net since we already have used other library from SDL. SDL_net was moved to github as: https://github.com/libsdl-org/SDL_net. Similar as other SDL libraries, I downloaded SDL2_net-devel-2.2.0-VC.zip.

Now it's time to deep dive to the code to understand the design. The main loop is App::run with state of m_app_state check/switch, eventProces, m_app_state->update, m_app_state->draw, and FPS control. m_app_state initially is pointing to an instance of Menu, then it is pointing to an instance of Game, Scores and Menu sequentially after invoke of nextState, which updates the state machine. Menu is the state to allow user to select player and quit, Scores is the state to show the score. So we most are interested of the Game state.

It would be complex to maintain two state machine on two hosts, even one is a mirror of the other. Need to clone all object on fly, with dynamic create and destroy, and need to merge input from both side, and one side needs to be the master for random number generation, state control by user. Also consider network latency, might be a bit messy to maintain the sync between server and client. The easier approach would be let the master to maintain the state machine, for client side, just relay the input to server, and mirror the render result from server.

(to be continue)

Sunday, June 30, 2024

Program D-Link DCS-5010L - Part 3

 As mentioned in Program D-Link DCS-5010L - Part 2, I wasn't able to successfully open DCS-5010L stream with OpenCV+GStreamer, so I'm going to explore OpenCV+ffmpeg. Ffmpeg is famous OpenSource Multimedia framework which contains a set of utility with almost all codec supported. Here is a Ffmpeg vs. GStreamer comparison.

To going this approach, there is FfmpegCV, which might be served as OpenCV replacement with ffmpeg support, has compatible API as OpenCV. Here I'm going to give it a try. Similar as using GStreamer, would need to install ffmpeg executable separately. Then do: pip install ffmpegcv

This will install the stable version, without cuda acceleration as I don't have nVidia GPU. After trying around, I realized that I need to use this API to open the stream:

cap = ffmpegcv.VideoCaptureStream(stream_url)

However, I don't see how it can help to play the audio.

Then I turned to PyAV. With that, I figured out that the 'video.cgi' would only provide video stream. Need to open 'audio.cgi' for the audio stream which is in wave format.

I added my code to forked dlink-dcs-python-lib github repository, with arrow key support for tile/pan the camera. There is no need to put the OpenCV video into a Tcl/tk gadget, but may create a TK GUI for setting configuration/options such as video file saving path, resolution, motion detecting and so on.

Saturday, June 22, 2024

Program D-Link DCS-5010L - Part 2

OpenCV is mainly focusing on video. It does not support audio directly. To playback audio, would rely on ffmpeg or GStreamer. ffmpeg is mostly a standalone offline multimedia converting tools. OpenCV has GStreamer built-in support, but it is optional. I used pip installed opencv-python on Windows for Python 3.12, and GStreamer isn't enabled. May refer to github opencv-python and https://discuss.bluerobotics.com/t/opencv-python-with-gstreamer-backend/8842. Even install the full package with pip install opencv-contrib-python won't get GStream enabled CV2. Might need to build with source. Either way, would need to install GStreamer first, which is available here: https://gstreamer.freedesktop.org/download. To build GStreamer enabled opencv-python:

git clone --recursive https://github.com/opencv/opencv-python.git
cd .\opencv-python
set CMAKE_ARGS="-DWITH_GSTREAMER=ON"
or $env:CMAKE_ARGS="-DWITH_GSTREAMER=ON" for PowerShell
pip wheel . --verbose

The build gets setuptools 59.2.0, I'm using miniConda and I'm seeing "conda 24.1.2 requires setuptools>=60.0.0, but you have setuptools 59.2.0 which is incompatible". This is not my environment setuptools out-of-date issue, have tried: pip install setuptools --upgrade, python -m pip install pip --upgrade, conda update -n base setuptools, nothing works. And after this, seeing:

  Running command Getting requirements to build wheel
  Traceback (most recent call last):
    File "C:\ProgramData\miniconda3\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
      main()
...
    File "C:\Users\squ\AppData\Local\Temp\pip-build-env-lekizdpu\overlay\Lib\site-packages\pkg_resources\__init__.py", line 2172, in <module>
      register_finder(pkgutil.ImpImporter, find_on_path)
                      ^^^^^^^^^^^^^^^^^^^
  AttributeError: module 'pkgutil' has no attribute 'ImpImporter'. Did you mean: 'zipimporter'?
  error: subprocess-exited-with-error

Sounds same as opencv-python issues 988, one comment there mentioned:

I encountered the same problem while running pip wheel . --verbose |& tee install_opencv.log. It worked after removing the ==59.2.0" from the "setuptools==59.2.0" https://github.com/opencv/opencv-python/blob/4.x/pyproject.toml#L16.

The pyproject.toml file is under the root, with the setuptools line removed, the original problem is resolved. Build under Windows would need Visual Studio Build Tools (refer to Revisit 坦克大战编程)., need to do the build under the Native build environment. The build will try Ninja, Visual Studio and NMake generator, and will try from v144 to v141. I have v143, and have "Developer Command Prompt for VS 2022" environment launched, all Ninja and NMake will fail as no support of platform, VS v143 progress a bit far,  but still get error as:

 -- Trying 'Visual Studio 17 2022 x64 v143' generator

  -- The C compiler identification is unknown
  CMake Error at CMakeLists.txt:3 (ENABLE_LANGUAGE):
    No CMAKE_C_COMPILER could be found
.

After some research, turns out Windows SDK is needed (I deselected it when installing Visual Studio Build Tools to save some space). It takes a while to build, depends on your computer. But it works. The opencv_python-*.whl file will be generated in the root folder. Just run pip to install it.

At beginning of the build, it will show the configuration, make sure Gstreamer is ON. It will stay OFF if GStreamer develop isn't installed. And with Gstreamer is enabled, I'm getting runtime error as:

ImportError: DLL load failed while importing cv2: The specified module could not be found

which is similar as opencv-python issues 856. Installed Microsoft Visual C++ Redistributable for Visual Studio 2022, but that didn't help. My Windows is not the 'N' one, so not a problem of Windows Media Feature Pack. I have tried the build whl before Gstreamer option enabled and not seeing problem, so I believe the problem is with GStreamer dll, but adding "C:\gstreamer\1.0\msvc_x86_64\lib\gstreamer-1.0" to path didn't help. Have to hook up Sysinternals' Process Monitor, and it shows the run is looking for a bunch of gstreamer's runtime dll which were not in site-packages\cv2 folder. Added "C:\gstreamer\1.0\msvc_x86_64\bin" to environment variable 'PATH' didn't help. Copy dlls over works, but not a clean way. Per https://stackoverflow.com/questions/214852/python-module-dlls, since version python 3.8 they added a mechanism to do this more securely. Read documentation on os.add_dll_directory https://docs.python.org/3/library/os.html#os.add_dll_directory. So doing this in code would work:

import os
gst_root = os.getenv('GSTREAMER_1_0_ROOT_MSVC_X86_64', 'C:/gstreamer/1.0/msvc_x86_64/')
os.add_dll_directory(gst_root+'bin')
import cv2

With Gstreamer enabled, I'm still not able to open the stream with log as:

[ WARN:0@2.620] global cap_gstreamer.cpp:2840 cv::handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module souphttpsrc0 reported: Could not establish connection to server.
[ WARN:0@2.623] global cap_gstreamer.cpp:1698 cv::GStreamerCapture::open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0@2.623] global cap_gstreamer.cpp:1173 cv::GStreamerCapture::isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
[ WARN:0@2.623] global cap.cpp:206 cv::VideoCapture::open VIDEOIO(GSTREAMER): backend is generally available but can't be used to capture by name

A bit frustrate on this. The http url is all right as I can open it without using GStreamer. Most example I can find are using rtsp protocol but not http. Will leave this aside until I can figure out how to use GStreamer alone to open the stream. I'm going to try OpenCV+ffmpeg next