~ / cmdr2

• #freebird
• #dom

The next major version of Freebird (i.e. v3) will use a new internal architecture that’s much easier to program with. In some ways, it’s an evolution of the architecture used in Freebird v2, but taken to its logical conclusion. The current version of Freebird (v2) uses a DOM-like model, and borrows a lot of programming patterns from browser-based programming. An underlying runtime abstracts away input events (like trigger_press, drag, enter, leave etc). It follows an event dispatch model (using add_event_listener and dispatch_event). Visual elements like menus, transform handles etc are DOM Nodes, which respond to events like drag and click. It also uses CSS-like styling to provide an easy way to style groups of related elements (like menu buttons).

• #gpu
• #ai
• #sdkit

A possible intuition for understanding GPU memory hierarchy (and the performance penalty for data transfer between various layers) is to think of it like a manufacturing logistics problem: CPU (host) to GPU (device) is like travelling overnight between two cities. The CPU city is like the “headquarters”, and contains a mega-sized warehouse of parts (think football field sizes), also known as ‘Host memory’. Each GPU is like a different city, containing its own warehouse outside the city, also known as ‘Global Memory’. This warehouse stockpiles whatever it needs from the headquarters city (CPU). Each SM/Core/Tile is a factory located in different areas of the city. Each factory contains a small warehouse for stockpiling whatever inventory it needs, also known as ‘Shared Memory’. Each warp is a bulk stamping machine inside the factory, producing 32 items in one shot. There’s a tray next to each machine, also known as ‘Registers’. This tray is used for keeping stuff temporarily for each stamping process. This analogy can help understand the scale and performance penalty for data transfers.

• #mlir
• #easydiffusion
• #sdkit

Good post on using MLIR for compiling ML models to GPUs. It gives a good broad overview of a GPU architecture, and how MLIR fits into that. The overall series looks pretty interesting too! Making a note here for future reference - https://www.stephendiehl.com/posts/mlir_gpu/

• #easydiffusion
• #samplers
• #c++

Wrote a fresh implementation of most of the popular samplers and schedulers used for image generation (Stable Diffusion and Flux) at https://github.com/cmdr2/samplers.cpp. A few other schedulers (like Align Your Steps) have been left out for now, but are pretty easy to implement. It’s still work-in-progress, and is not ready for public use. The algorithmic port has been completed, and the next step is to test the output values against reference values (from another implementation, e.g. Forge WebUI). After that, I’ll translate it to C++.

• #easydiffusion
• #sdkit
• #compilers

Some notes on machine-learning compilers, gathered while researching tech for Easy Diffusion’s next engine (i.e. sdkit v3). For context, see the design constraints of the new engine. tl;dr summary The current state is: Vendor-specific compilers are the only performant options on consumer GPUs right now. For e.g. TensorRT-RTX for NVIDIA, MiGraphX for AMD, OpenVINO for Intel. Cross-vendor compilers are just not performant enough right now for Stable Diffusion-class workloads on consumer GPUs. For e.g. like TVM, IREE, XLA. The focus of cross-vendor compilers seems to be either on datacenter hardware, or embedded devices. The performance on desktops and laptops is pretty poor. Mojo doesn’t target this category (and doesn’t support Windows). Probably because datacenters and embedded devices are currently where the attention (and money) is.

• #easydiffusion
• #sdkit
• #engine

The design constraints for Easy Diffusion’s next engine (i.e. sdkit v3) are: Lean: Install size of < 200 MB uncompressed (excluding models). Fast: Performance within 10% of the best-possible speed on that GPU for that model. Capable: Supports Stable Diffusion 1.x, 2.x, 3.x, XL, Flux, Chroma, ControlNet, LORA, Embedding, VAE. Supports loading custom model weights (from civitai etc), and memory offloading (for smaller GPUs). Targets: Desktops and Laptops, Windows/Linux/Mac, NVIDIA/AMD/Intel/Apple. I think it’s possible, using ML compilers like TensorRT-RTX (and similar compilers for other platforms). See: Some notes on ML compilers.

• #tailscale
• #networking

Tailscale is genuinely super well-made. It’s crazy how well it works.

• #freebird
• #vr
• #api

Freebird v2.2.2 released. It now exposes the states/values of the VR buttons (as custom properties) in FB-Controller-Right and FB-Controller-Left (see: XR Tracking Objects). These values will be updated every frame, when VR is running. You can use these properties to drive shapekeys, or use them in other scripts: To drive a shapekey, please right-click a property, e.g. ’trigger’, and click Copy as New Driver. Then right-click on your shapekey value, and select Paste Driver. To use in a script, use the custom property directly. E.g. bpy.data.objects["FB-Controller-Right"]["trigger"]

• #freebird

Freebird v2.2.0 released - Freebird now exposes the VR headset and controller positions via three empty objects in the scene: FB-Headset, FB-Controller-Right, and FB-Controller-Left. These three empties live-track the position of the headset and the VR controllers. For e.g. you can attach objects to these empties to animate objects or bones.

• #tkinter
• #ui

Spent some time playing with Tkinter, and building a real desktop app with it. It’s pretty specific to my needs, but is open to customization by others. Building UIs with Tkinter was interesting (not frustrating), and it feels almost-there-but-not-quite-there. I still think that HTML/CSS/JS is the best API out there for UI (the good parts), but Tkinter’s mental model and API is quite nice too. Fairly intuitive.