Started the long-pending rewrite of Easy Diffusion’s server code. v4 intends to replace the Python (and PyTorch) based server with a simple C++ version. The reason for rewriting the server in C++ is to achieve sub-second startup time for the UI, and to reduce the download size (won’t need to distribute Python along with Easy Diffusion) or mess with conda/venv etc. And it’s also something that I want to do for personal taste, i.e. de-bloating what doesn’t need to be bloated.
Submitted a code fix in Blender to fix the order of calling modal operators for XR Actions - https://projects.blender.org/blender/blender/pulls/152575
For Z-Image, the performance of the stock version of chromaForge is poorer than sd.cpp’s. Mainly because chromaForge isn’t able to run the smaller gguf quantized models that sd.cpp is able to run (chromaForge fails with the errors that I was fixing yesterday). If I really want to push through with this, it would be good to fix the remaining issues with gguf models in chromaForge. Only then can the performance be truly compared (in order to decide whether to release this into ED 3.5). I want to compare the performance of the smaller gguf models, because that’s what ED’s users will run typically.
Worked on fixing Z-Image support in ED’s fork of chromaForge (a fork of Forge WebUI). Fixed a number of integration issues. It’s now crashing on a matrix multiplication error, which looks like an incorrectly transposed matrix (mostly due to reading the weights in the wrong order). I’ll try to install a stock version of chromaForge to see its raw performance with Z-Image (and whether it’s worth pursuing the integration), and also use it to help investigate the matrix multiplication error (and any future errors).
Combining the worklogs for a few days. Worked on testing Freebird and Puppetry with the new XR API changes coming in Blender 5.1 (related to making navigation_scale read-only). Continuing to discuss and give feedback to Blender devs on their #xr chat channel. Investigated why grease pencil strokes render incorrectly in VR, by digging into Blender’s source for that. Haven’t found the reason/fix yet. It seems related to draw_grease_pencil_lib.glsl, but winmat and viewport_res don’t change when we change xr_session_state.navigation_scale or xr_session_settings.base_scale. Added the ability to set the location of the camera preview in Puppetry. Submitted a fix for the regression introduced in Blender 5.0.1 (causes Blender to crash when Freebird or Puppetry are started) - https://projects.blender.org/blender/blender/pulls/152237
Collecting the worklog over the past few weeks. Enabled Flash-Attention and CPU offloading by default in sdkit3 (i.e. Easy Diffusion v4). Added optional VAE tiling (and VAE tile size configuration) via config.yaml in Easy Diffusion v4. Created Easy Diffusion’s fork of Forge WebUI, in order to apply the patches required to run with ED. And also to try adding new features like Z-Image (which are missing in the seemingly-abandoned main Forge repo). Improved the heuristics used for killing and restarting the backend child process, since /ping requests are unreliable if the backend is under heavy load. Merged a few PRs (1 2) for torchruntime that improve support for pinning pre-cu128 torch versions and fix the order of detection of DirectML and CUDA (prefers CUDA). Added progress bars when downloading v4 backend artifacts.
Collecting the worklog over the past 2 weeks. Added the ability to add and edit Text objects in Freebird while inside VR. This is useful for adding notes and labels while working in VR - https://x.com/freebirdxr/status/2004091164946059451 Added a “Camera Preview” feature in Puppetry, to allow the user to see the live view from the scene Camera, while recording. This is helpful in avoiding surprises after recording, for e.g. realizing that the movements weren’t captured correctly by the scene Camera.
Wrote a simple script to convert ONNX to GGML. It auto-generates C++ code that calls the corresponding ggml functions (for each ONNX operator). This file can then be compiled and run like a normal C++ ggml program, and will produce the same results as the original model in PyTorch. The generated file can work on multiple backends: CPU, CUDA, ROCm, Vulkan, Metal etc, by providing the correct compiler flags during cmake -B, e.g. -D GGML_CUDA=1 for CUDA.
Migrated findstarlink.com back to S3 (from Cloudflare Pages), and started rewriting the website to improve loading speed.
Built a simple screen recorder for myself using Python and Tkinter, and a few Windows-specific calls (via ctypes). I wanted something just like the Windows Snipping Tool, but with a few customizations for my workflow.