Bokeh

Exploring Uncut - March 3rd, 2026

Tue, 03 Mar 2026 00:00:00 +0200

Update 2026-03-07: Added example of generated output. Clarified usage of Codex instead of ChatGPT. Some styling improvements.

Introduction

This time, the focus is solely on providing an update on my exploration of getting a Mandelbrot set renderer running on an STM32F746G-DISCO board.

I had hoped to also cover “The Egg Project” and some Home Assistant improvements I’ve been working on. However, there is already plenty to discuss here, and the other projects have been moving forward more slowly due to a collection of minor but frustrating issues, from ordering the wrong components, to chasing down unexpected sensor behavior, to tracking a Wi-Fi authentication failure caused by a simple case mismatch.

Mandelbrot update

Removing Swift Numerics

Last time, I was faced with the issue that as soon as I introduced an additional external loop around my working code, it crashed.

At this stage I was still using Swift Numerics. I didn’t think this was the culprit, but to rule it out definitively, I removed the package dependency and reimplemented to core computation using only Floats.

    let maxIterations: UInt8 = 100    let cReal: Float = -1.0    let cImag: Float = 1.0    var zReal: Float = 0.0    var zImag: Float = 0.0    var iterations: UInt8 = 0    for _ in 0..<1 {      while iterations < maxIterations {        let magnitudeSquared = zReal * zReal + zImag * zImag        if magnitudeSquared >= 4.0 {  // 2.0 squared = 4.0          break        }                // z = z * z + c        // (a + bi) * (a + bi) = (a² - b²) + (2ab)i        let zRealTemp = zReal * zReal - zImag * zImag + cReal        zImag = 2.0 * zReal * zImag + cImag        zReal = zRealTemp                iterations += 1      }      print("Iterations: \(iterations)\r")    }

The behavior was exactly the same. If I change the range of that for loop to have more than one iteration, the code crashes.

No progress so far.

Moving to C

To further simplify things, I implemented the main algorithm in C and called it from Swift.

uint8_t oneRound() {  uint8_t maxIterations = 100;  float cReal = -1.0;  float cImag = 1.0;  float zReal = 0.0;  float zImag = 0.0;  uint8_t iterations = 0;  while (iterations < maxIterations) {    float magnitudeSquared = zReal * zReal + zImag * zImag;    if (magnitudeSquared >= 4.0) {  // 2.0 squared = 4.0      break;    }        // z = z * z + c    // (a + bi) * (a + bi) = (a² - b²) + (2ab)i    float zRealTemp = zReal * zReal - zImag * zImag + cReal;    zImag = 2.0 * zReal * zImag + cImag;    zReal = zRealTemp;    iterations++;  }  return iterations;}

    for _ in 0..<2 {      let iterations: UInt8 = oneRound()      print("Iterations: \(iterations)\r")    }

This worked correctly for one, two, twenty, or any number of iterations through the loop.

Passing coordinates

However, this isn’t very interesting, since the calculation is always performed for the same coordinate.
What we would need is to perform it for different coordinates.
So we need to pass cReal and cImag as parameters.

uint8_t oneRound(float cReal, float cImag) {  uint8_t maxIterations = 100;  float zReal = 0.0;  float zImag = 0.0;  uint8_t iterations = 0;  while (iterations < maxIterations) {    float magnitudeSquared = zReal * zReal + zImag * zImag;    if (magnitudeSquared >= 4.0) {  // 2.0 squared = 4.0      break;    }        // z = z * z + c    // (a + bi) * (a + bi) = (a² - b²) + (2ab)i    float zRealTemp = zReal * zReal - zImag * zImag + cReal;    zImag = 2.0 * zReal * zImag + cImag;    zReal = zRealTemp;    iterations++;  }  return iterations;}

But for our first test, let’s still pass the same coordinates for each call.

    for _ in 0..<20 {      let iterations: UInt8 = oneRound(-1.0, 1.0)      print("Iterations: \(iterations)\r")    }

And with this, we’re back to our previous behavior: one pass through the loop is fine; more than one, and execution crashes.

Experimenting some more in this codebase, I updated the loop to what it typically is for rendering the typical Mandelbrot set.

    let xMin: Float = -2.5    let xMax: Float = 1.0    let yMin: Float = -1.25    let yMax: Float = 1.25    for y: Int32 in 0..<10 {      for x: Int32 in 0..<10 {        let cReal: Float = xMin + (Float(x) / 10.0) * (xMax - xMin)        let cImag: Float = yMin + (Float(y) / 10.0) * (yMax - yMin)        let iterations: UInt8 = oneRound(cReal, cImag)        print("Iterations: \(iterations)\r")      }    }

As anticipated, this also crashes.

Computing the coordinates in C

Trying one more variation, I decided to pass x and y to the C function and let it calculate the cReal and cImag values.

uint8_t oneRound(uint32_t x, uint32_t y) {  uint8_t maxIterations = 100;  float xMin = -2.5;  float xMax = 1.0;  float yMin = -1.25;  float yMax = 1.25;  float zReal = 0.0;  float zImag = 0.0;  uint8_t iterations = 0;  float cReal = xMin + (x / 10.0) * (xMax - xMin);  float cImag = yMin + (y / 10.0) * (yMax - yMin);  while (iterations < maxIterations) {    float magnitudeSquared = zReal * zReal + zImag * zImag;    if (magnitudeSquared >= 4.0) {  // 2.0 squared = 4.0      break;    }        // z = z * z + c    // (a + bi) * (a + bi) = (a² - b²) + (2ab)i    float zRealTemp = zReal * zReal - zImag * zImag + cReal;    zImag = 2.0 * zReal * zImag + cImag;    zReal = zRealTemp;    iterations++;  }  return iterations;}

    for y: UInt32 in 0..<10 {      for x: UInt32 in 0..<10 {        let iterations: UInt8 = oneRound(x, y)        print("Iterations: \(iterations)\r")      }    }

And this worked fine.

Setting some flags

Slightly running out of ideas on variations to try out, I explained all those tests to ChatGPT and asked for advice.The fact that passing int parameters worked but not float ones led it to believe there might be an ABI mismatch issue with e.g. one language using soft-float ABI and the other using hard-float ABI.
The suggestion was to add explicit build flags to ensure everything was aligned.
I ended up adding

      "-Xcc", "-mcpu=cortex-m7",      "-Xcc", "-mthumb",      "-Xcc", "-mfpu=fpv5-d16",      "-Xcc", "-mfloat-abi=hard"

to the Swift compiler options in my toolset.json file.
However, that was not enough. This only ensures that the C code compiled as part of the Swift build gets those flags applied.
I still needed to ensure that the same flags were used when compiling the standalone C source files (i.e. Support.c). This required adding a completely new section to toolset.json

  "cCompiler": {    "extraCLIOptions": [      "-mcpu=cortex-m7",      "-mthumb",      "-mfpu=fpv5-d16",      "-mfloat-abi=hard"    ]  },

Looking at the commands executed during the build, I could see those flags properly passed everywhere they were required (and confirmed they were not specified before).

But this did not change anything.

Single vs double precision

The astute reader might notice the use of fpv5-d16 for the FPU specification.
This indicates a double-precision-capable FPU, which I thought was the case, as glancing through the 250+ page STM32F7 Series and STM32H7 Series Cortex®-M7 processor programming manual, I had read

Floating-point unit
The FPU provides IEEE754-compliant operations on 32-bit single-precision and 64-bit double-precision floating-point values.

Unfortunately, this refers to the specifications of the Cortex-M7 processor options as used in general in different STM32 products.

Since I’m using an STM32F746G-DISCO board, I glanced at the 1700+ page of theSTM32F75xxx and STM32F74xxx advanced Arm®-based 32-bit MCUs - Reference manual, without finding any additional information.

Finally, in the Datasheet - STM32F745xx STM32F746xx - ARM®-based Cortex®-M7 32b MCU+FPU, 462DMIPS, up to 1MB Flash/320+16+ 4KB, another 250+ page document, it states

The Cortex®-M7 core features a single floating point unit (SFPU) precision which supports all Arm® single-precision data-processing instructions and data types.

So I changed the fpu flag definition to fpv5-sp-d16 instead of fpv5-d16.But it crashed just the same.

Info

Full disclaimer, ChatGPT had proposed to test using single precision and I also used it afterwards to find the exact documents where the information could be found.
I only had a vague recollection of seeing that information.

Revising the FPU enable code

The last suggestion of ChatGPT was that I was enabling the FPU too late. This seemed strange to me, as it was the first instruction called in my Swift code.Still, it proposed to move it earlier, directly in the reset handler.

So I removed the enableFPU() call that started the main() method in Application.swift and added some bits of assembly directly in the reset routine in startup.S

  // ---- enable FPU here ----  ldr   r1, =0xE000ED88  ldr   r0, [r1]  orr   r0, r0, #(0xF << 20)  str   r0, [r1]  dsb  isb  // -------------------------

And this worked just fine. The loops are going over all coordinates and printing the expected iteration numbers, as shown in the last few lines of the console output from the board.

…Iterations: 100Iterations: 5Iterations: 3Iterations: 1Iterations: 1Iterations: 2Iterations: 3Iterations: 3Iterations: 3Iterations: 4Iterations: 11Iterations: 4Iterations: 2Starting echo

I do not understand the reason why this works, which is frustrating me, but it does.

Now that I had something working, I reverted some of the other changes I made, such as adding the compiler flags, or implementing the logic in C instead of Swift, verifying at each step of the way that it was still properly working.
And eventually, I got back to using SwiftNumerics and the Complex<Float> type in my calculations and all was well.

Getting a grasp on what’s happening

As mentioned above, I don’t like the black magic feeling of changing things around and having something working without a clue as to why.
I’m strongly in the camp that there’s always a good reason for why something is working or not. It might be very complex and intricate, you might not understand it, but there is logic underneath.
At least in the context of software engineering, when it comes to philosophy, psychology or quantum physics, things are less clearly defined. This annoys and fascinates me at the same time and although this is primarily a technical blog, I may write about that topic at some point.

But I digress. To avoid that lingering frustration, and since ChatGPT had been helpful so far, I used AI again, this time using the Codex CLI, to ask for an explanation.
When building the project, quite a few files are generated, including a .build/armv7em-apple-none-macho/release/Application.disassembly that contains the “readable” assembly listing of the whole program.

So I built the project twice, once where I enabled the FPU at the top of the main() function of my Swift code and once where I enabled it from the reset routine. For each, I saved the Application.disassembly file and then I asked ChatGPT to compare the two, explain the differences in the code and the crash I was observing.

It pointed out some of the obvious differences (like the one in the reset routine) but also similarities.
For both projects, the way the main() function from Application.swift was called was identical, including the fact there’s a generated function prologue that includes a vpush {d8-d13} instruction.

So I asked follow-up questions to understand exactly what was going on, like what does vpush do and what is this function prologue?

Given those answers, let me try to explain what’s happening here.

Processor or FPU registers being used in the code of a function fall in two categories:

call-clobbered (or caller-saved) registers hold temporary information and there’s no requirement on their values after the call. If the caller wants to keep the values in such registers around, it needs to save them itself.
callee-saved registers, on the other hand, are used to hold long-lived values that should be preserved across calls, and it is the responsibility of the callee to save those registers.

The function prologue is compiler-generated code that performs some preliminary setup before the function is called.

In our case, as FPU callee-saved registers were used in the code of our main() function, the compiler generated code to save those as part of the prologue. This is what the vpush instruction does.

Schematically, you can represent the calling sequence as
Reset → main prologue (vpush) → enableFPU() → code using FPU

Since the prologue executes before the function body, the vpush is being executed before executing the first instruction of our main() function, which is enabling the FPU. Executing an FPU instruction before the FPU is enabled led to a crash.

By enabling the FPU in the reset routine, the calling sequence looked like
Reset (enableFPU) → main prologue (vpush) → code using FPU
making sure the FPU is enabled before the vpush is executed.

But why did this crash occur only when I added an outer loop?
The way the compiler generates and optimizes the code and decides on which register to use for which operations is obviously quite complex (at least to me). But by saving the disassembly files for an outer loop with one iteration or a loop with more than one iteration, I could confirm that different registers were used and that only in the latter case were callee-saved registers involved. This meant the prologue only contained the vpush instruction for that latter case. In the first case, our calling sequence looked like Reset → enableFPU() → code using FPUwhich is perfectly fine.

Using the same mechanism, I confirmed that the behavior was exactly the same between the code that was calling my C function using int parameters vs the call using float parameters.

Conclusion

In this post, I showed you how I debugged the implementation of the Mandelbrot set computation in Embedded Swift on the STM32F746G-DISCO board and how ChatGPT has helped both find and fix the issues but also understand what was really happening.

The source code for this experiment, including all the intermediate steps, is available on GitHub.

But so far, I have code that prints the number of iterations at each coordinate. The end goal is of course to draw a picture, and to do that, that iteration count needs to be mapped to a color value.

If the journey so far is any indication as what lies ahead, I’m sure I’ll be facing quite a few challenges and will have more stories to share. Stay tuned for a follow-up on this topic and on the other ongoing projects.

Exploring Uncut - December 6th, 2025

Sat, 06 Dec 2025 00:00:00 +0200

Introduction

I’ve had drafts of Exploring Uncut posts on my computer for months now. They contain bullet points of things I wanted to talk about, sometimes with additional notes, sometimes a draft of a few paragraphs. But as time went by, the potential post only became bigger and the time it would take me to finish it would only grow.

So here’s a new post focusing on what I’m working on right now. I’ll tackle those other topics when the time is right.

Mandelbrot

The goal

While preparing the talk I did at SwiftLeeds on Open-Source packages Apple offers, I wanted to find a nice use case for Swift Numerics. As it includes a module to handle complex numbers, I thought drawing a Mandelbrot set would fit nicely.

And as I also wanted to show the package could work on Embedded, I opened a PR to the Swift Numerics repository to ensure it would compile in that mode. Nothing major, disabling Codable and CustomDebugStringConvertible conformance under Embedded Swift.

Once this was done, I created a quick test to generate a Mandelbrot set based on the stm32-lcd-logo example from the swift-embedded-examples GitHub repository.It compiled just fine but when executing on my STM32F746G-DISCO board, it just crashed.

Investigating the issue

To investigate the issue, I worked on a minimal test case, starting from the stm32-uart-echo example.
In that test, I’m evaluating the sequence of the typical f(z) = z² + c function for only one given complex number and with a very limited number of iterations.

  let maxIterations: UInt8 = 5  let c = Complex<Float>(-1.0, 1.0)  var z = Complex<Float>.zero  var iterations: UInt8 = 0  while iterations < maxIterations && z.magnitude < 2.0 {z = z * z + citerations += 1  }  print(iterations)

Before going into more details, let’s do the math manually. The sequence of values for z go like

Calculation	z	Magnitude	Iteration
z₀ = (0.0, 0.0)	(0.0, 0.0)	0.0	1
z₁ = (0.0, 0.0)² + (-1.0, 1.0) = (0.0, 0.0) + (-1.0, 1.0)	(-1.0, 1.0)	1.414	2
z₂ = (-1.0, 1.0)² + (-1.0, 1.0) = (0.0 - 1.0, -2.0 + 1.0) + (-1.0, 1.0) = (-1.0, -1.0) + (-1.0, 1.0)	(-2.0, 0.0)	2.0	3

As you can see, we get out of the loop as soon as iterations equals 3 because of the magnitude test.
Building and running the above code on the board worked fine and printed the value of 3.
But changing maxIterations to a bigger value, starting at 9, would make the program crash.

That left me scratching my head, in both cases, as we’ve seen above, the program should only go through the loop 3 times, as the condition on the magnitude would make it exit, not the condition on the number of iterations.
What I also observed was that the final application binary had different sizes based on that value. So the generated code was different.
I did take a quick look at the assembly but I need to take a refresher and learn ARM assembly. The last time I worked with assembly was in the previous century on 68k and x86 processors.
I tried several flags but could only make it crash more often, not make it work.

I posted about this on the “The Swift Programming Language” Discord server, in the embedded channel and after some back and forth, got an answer from CMD that, based on the assembly output, indicated I probably needed to initialize the FPU coprocessor.
This was confirmed soon after by Rauhul, pointing me to sample code for performing that.
And this fixed my problem. A huge thank you to both of them for the help.

Although I understand the reason for the crash and the fix, I’m at loss as to how to get there when faced with the initial problem.
I suppose learning ARM assembly and the tools to inspect a binary build and reading the ARM / STMicro technical documentation would help but that seems like a huge learning curve to get a simple example working.

On to the next one

Now that I had this fixed, I could proceed with my initial project, but it still crashed.
So back to the test snippet and I embedded the above calculation in a loop.

let maxIterations: UInt8 = 100for _ in 0..<2 {  let c = Complex<Float>(-1.0, 1.0)  var z = Complex<Float>.zero  var iterations: UInt8 = 0  while iterations < maxIterations && z.magnitude < 2.0 {z = z * z + citerations += 1  }  print(iterations)}

And this crashes.

As said above, I’m again not sure where to start to investigate that problem.
I can change some bits of the code, maybe not use Swift Numerics and Complex types but do the math using only Float types.
And I’ll definitely read more on ARM assembly and technical literature on the board, as it anyway interests me to know more about that, but I think trying to get an actual debugger working with Embedded Swift and the different boards I’m using would be the bigger win.

The Egg Project

Background

In May 2025, Jony Ive and Sam Altman announced that io Products joined OpenAI.
The team is working on a new kind of product, incorporating OpenAI technology in a new form factor.
Not much is really known about it, just a lot of rumors.

In Episode 640, the ATP podcast team discussed the announcement at length and speculated on what the device could be.
Marco Arment was going for a watch while John Siracusa proposed a potato (or more specifically that if the product delivers on its features, it could be shaped like a potato and still be great).

In Episode 641 they reviewed information that was now available, mainly from the rumor mill.
That includes the fact that it’s not a phone, it’s not glasses, and Jony Ive was not keen on a wearable device. That device would probably have cameras and microphones for environmental detection.
And this is where the egg form factor idea came from.

After that first mention of the egg, I thought about creating such a device.
But the idea of creating a 3D printed egg with electronics fitted inside made me think it wasn’t an appropriate shape.
I then considered an “all-seeing eye pyramid” as both an easier shape to model and an appropriate symbol for a device that would capture your environment. And then I moved on.

But as I was listening to Episode 667, where they discussed the latest news (not much) and reiterated the egg idea, I thought “Why not?”.
And so that’s one more project on the list.

The project

I’ll disclose as much information on the project as io Products does 😉.
It’s an egg,
it uses Swift as much as possible,
and it’s fully open source.

You can find the repository on GitHub at The Egg Project.

Egg Modeler

The first step is to model an egg for 3D printing.

After a quick search, I discovered there was ample literature on the subject and many equations available to model an egg.
So I created a SwiftUI-based app to play around with different equations, their parameters, and come up with a shape I liked.
It can generate a 3D model using Cadova, a Swift library made for that purpose. I discovered it only a few weeks ago and used it for several small prints and I’m loving it. I’ll definitely dedicate a post to it.

As for the egg modelling equations, I based the math in that tool on The Universal Egg Equation and A “universal” formula for egg shape.

The Egg Modeler allows you to test different equations and parameters to model an egg shape

What’s next

The UI can certainly use a little more polish and the code a bit of cleaning.
But the main focus at this stage will be on the 3D model improvements, as it needs to be updated to support integrating the different electronic components.

Exploring Uncut - July 14th, 2025

Mon, 14 Jul 2025 00:00:00 +0200

Game follow-up

As we had another GeekClub meeting at the beginning of the month, I wanted to make some progress on my embedded game development so I would have something cool to show my friends.

3D Modeling

Back in March, I described how I modeled and 3D-printed a screen support, partly using ChatGPT to help me get there.

On the one hand, there were small issues with the print, like a small gap at the top of the screen, a bit of a loose fit on the LEGO® base and the screws I had in stock were too long so the screen would easily detach from its support. On the other hand, from a modeling point of view, I used hacks to reach the final result, I used extra volumes and steps and the final shape was not exactly what I had in mind. So I wanted to clean-up the model.

AI had helped get there, but it didn’t really teach me anything. The next time I would need to model something, I would still struggle and hack my way around it (probably again with some help from AI).

As I want to learn and get better at modeling, I searched for some video tutorials (this works best for me, I read reference material but prefer watching tutorials). And I found a great playlist on YouTube: Fusion 360 Tutorials For Absolute Beginners.

I watch a few episodes at a time, take some notes, then apply the learnings in some of my modeling. This gives me foundational knowledge and a deeper understanding of what I do, why I do it and some additional context. I might still use AI for specific questions but at least I’m equipped with the tools to understand the answer and better adjust my prompts.

The same channel also proposes Fusion 360 Tutorials for Intermediate Users but I’m not at that level yet.

The new support removes the gap that was present around the screen

The resulting model is freely available for download: Seeed Studio Round Display Support

Game progress

It seems I didn’t really post follow-ups on the game’s development since that March post ( Exploring Uncut - March 24th, 2025), in which I only explained how I got the screen output and joystick input working.

Over the last few months, I had the ship rotating around the screen, controlled by the joystick and I could fire some projectiles that would disappear towards the center of the screen.

But so far, I had no enemies. I wasn’t sure how they should appear, their movement patterns, if they should animate, and I couldn’t find appropriate graphical assets to use. So as a first try, I added meteorites that would come from the center towards the ship.

The animation worked OK, but nothing happened when the meteorite hit the spaceship or was hit by a missile.

Collisions

So I needed to detect collisions between my objects. I first checked if LVGL offered collision checking between images out of the box, but I didn’t find anything. I then searched for information on how to implement collision detection and found this YouTube playlist: 2D Game Physics.

This not only provided me with the basic information I needed but the intro video hinted at a lot of interesting topics to be covered by the series. Unfortunately, the playlist only contains four episodes. It’s a pity, but I can’t blame the author. I understand the amount of work that goes into producing such content.

Armed with that information, I did a first implementation of collision detection on my LVGL images struct.

struct LVGLImage: LVGLObject {  var pointer: UnsafeMutablePointer<lv_obj_t>?⋮    func collideWith(_ other: LVGLImage) -> Bool {    var myCoord = lv_area_t()    var otherCoord = lv_area_t()      lv_obj_get_coords(self.pointer, &myCoord)    lv_obj_get_coords(other.pointer, &otherCoord)      return otherCoord.x1 < myCoord.x2 && myCoord.x1 < otherCoord.x2        && otherCoord.y1 < myCoord.y2 && myCoord.y1 < otherCoord.y2  }}

And used it to handle collisions between missiles and meteorites. This worked fine and was visually OK. So I applied the same logic for the spaceship collision but there was a catch.

The way I implemented the spaceship rotation around the center of the screen was by moving the spaceship away from the center at start-up and using the center of the screen for the rotation anchor point.

And the LVGL manual indicates

Note that the real coordinates of image objects won’t change during transformation. That is lv_obj_get_width/height/x/y() will return the original, non-zoomed coordinates.

This means that in effect, the coordinates of the ship image never change and my collision detection formula above is bogus.

Rotating the ship image around its center and moving that center by computing its coordinates is geometrically equivalent. With that change, the ship now has proper coordinates to work with.

But computing a bounding box around a rotated shape is tricky. If you rotate the bounding box, it’s not axis-aligned anymore and the basic collision formulas stop working. If you don’t rotate the box and still want it to enclose the whole shape, it’ll vary in size based on the rotation angle and leads to false positives in the detection.

The easiest option when rotation is involved, is to use bounding circles instead of bounding boxes. So I created the types to support that.

struct Point {    let x: Int16    let y: Int16}struct Circle {    var center: Point    var radius: Int16    func collideWith(_ other: Circle) -> Bool {        // Using (r1+r2)^2>(x2-x1)^2+(y2-y1)^2 for efficient detection, no square roots involved        return (radius + other.radius) * (radius + other.radius)            > (other.center.x - center.x) * (other.center.x - center.x)            + (other.center.y - center.y) * (other.center.y - center.y)    }}

And used it in my image struct.

struct LVGLImage: LVGLObject {  var pointer: UnsafeMutablePointer<lv_obj_t>?⋮    var boundingCircle: Circle {    var coords = lv_area_t()    lv_obj_get_coords(pointer, &coords)    let width = lv_area_get_width(&coords)    let height = lv_area_get_height(&coords)    return Circle(      center: Point(x: (coords.x1 + coords.x2) / 2, y: (coords.y1 + coords.y2) / 2),      radius: min(width, height) / 2)  }  func collideWith(_ other: LVGLImage) -> Bool {    return boundingCircle.collideWith(other.boundingCircle)  }}

Note that I don’t compute the exact radius. This would require using Pythagoras formula, which requires calculating a square root, which is expensive. There are several approximation methods. But in the end, given the use case and the shapes of my different graphical elements, using the minimum between the width and the height got me the best gameplay feeling.
I’m leaving out a few details here that I’ll be sure to address when I write a series of posts on the complete implementation of the game.

The spaceship avoiding a meteorite and firing some missiles

Swiftly release

Back in April, Apple released swiftly, that makes installing the Swift toolchain a breeze. There was however a small glitch when used in combination with SourceKit-LSP , which I mentioned in Swiftly and Swift 6.1 released. That was not problematic as a workaround was available (see Swiftly and SourceKit-LSP issue follow-up).

Swiftly 1.0.1 is now available and fixes the issue. No workaround is necessary anymore and selecting the desired toolchain with swiftly is all that is required.

Conferences season

The September to November period in Europe is packed with Swift related conferences. This means a lot of them have their Call for Papers deadline in June or July.
So I spent a bit of time writing proposals for several talk ideas I had and submitted to two conferences. There are two more submissions I plan to make in July, but I’m waiting for feedback on the current ones to potentially tweak them a bit.

WWDC

WWDC week was quite intense. As many, I watched the keynote and the State of the Union “live”. I then downloaded all the session videos as they became available (all on the first day for the first time). I installed macOS 26 and iOS 26 on test devices. I participated in five group labs. I watched several of the sessions that interested me most. And I tried to compile some of my code with the latest SDKs.

I’ve made some tweaks to my apps, fixing some issues that appeared with the new design, but I haven’t taken the time yet to push forward with some ideas I have to make good use of the new features that are now available. I’m particularly excited about SpeechAnalyzer and the Foundation Models framework and think it’ll be a great fit to make progress on my PAL project.

I’m also happy seeing Apple propose some additional in person events to help developers work with the newly announced technologies. Unfortunately, I didn’t get accepted in the first one to which I applied. Hopefully I’ll have better luck next time.

Retro-computing

Last week I was in Eindhoven, visiting the OpenRemote offices and meeting with the whole team for a few days. And I finally managed to go visit the Home Computer Museum. It’s less than 20 minutes away by car but although I’ve been around a few times this year, I never managed to fit that into my schedule.

I first heard about the museum when watching HomeComputerMuseum: The interactive computer museum as a business – Bart van den Akker. I had no idea there was such a museum so close to Belgium.

By the way, the Vintage Computer Federation – VCF has a ton of other videos (and other information) available; it’s a great resource for retro-computing enthusiasts.

Back to the Home Computer Museum. It has a very large collection of home computers. You’ll find the usual suspects from Apple, Atari, Commodore, IBM, Sinclair and many others. But you’ll also encounter computers that you’ve probably never heard of, let alone see.

A small section of the computers on display

The Holborn computer, a machine I never heard of

That’s already a great treat but the best is that you can use a lot of them. Play a game, sit in front of the keyboard and try to remember some of those BASIC instructions you typed several decades ago.

The early days: Altair 8800 and IMSAI 8080

The DAI, a home computer made in Belgium

There’s also a huge collection of games, some magazines and books, the main game consoles powered-up ready for you to remember those killer moves.

Finally, the staff is super friendly and ready to help you and answer your questions. They also offer a service to have their technical staff repair your old machines. And have some items for sale if you want to add something to your collection (not taking that PowerMac G5 home with me seriously tested my willpower).

If retro-computing is a topic of interest for you, follow me on Mastodon. Every Thursday I post some fond memories I have about old computers, software or gadget I’ve encountered through the years.

Miscellaneous

As June came, so did the nice weather. Meaning spending more time in the garden, relaxing but also taking care of the growing grass and vegetation. I could use that as an excuse to explain I didn’t write any post last month, but who would I be fooling?

That did give me an excuse to buy a new toy though. After watching many reviews on YouTube but also getting a better understanding of the RTK technology, I finally took the plunge and bought a Mammotion LUBA 2 AWD 3000X.

The Luba handling one of the tricky slopes

I’ve only had it for about a month now but I’m quite happy with it. The garden looks definitely better than it did the previous years and it’s freed me a bit of time, mostly to take care of more manual tasks in the garden. The only caveat so far is that I was hoping for a slightly higher positioning accuracy to mow closer to fences without getting stuck.

Sadly, June also brought us the passing of Bill Atkison. As many, I admired what he accomplished over his years at Apple. There are several videos available on-line with interviews of him. I remember enjoying his discussions on Triangulation:

Finally, the Embedded Swift Community Hour held its fourth occurence last Friday. This was a quite successful session with about 25 participants, a big increase compared to the previous sessions.
We had interesting discussions about ESP32, bare metal vs SDK usage and build systems. You can find a more detailed summary of the session on the Swift Forums - Embedded Swift Community Hour (July 11, 2025).

Exploring Uncut - May 26th, 2025

Mon, 26 May 2025 00:00:00 +0200

HWS+ livestream

Saturday, May 17th marked another Hacking with Swift+ subscriber livestream. This time, Paul walked us through building a macOS application that uses the AppStoreConnect API to retrieve information about our published apps.

It was the longest session I ever attended but was super interesting, both from a coding perspective and in terms of the resulting app.

In fact, it was so compelling that Paul decided to open source the app, called ShipShape.To help those who want to contribute, he also opened up the livestream recording to everyone.

You might already be using Helm for this purpose. I tested it in the past, it works great and I have a lot of respect for the work done by Pol and Hidde. But being able to participate in the creation of the application and somehow drive where it’s going is really interesting. Plus, enabling the application to be available to the widest audience possible feels really rewarding.

If you have some ideas, design or coding skills or a bit of time, join me in contributing to this great initiative.

SwiftCraft

SwiftCraft held its second edition last week, and as for the first one, I was there.

The conference takes place in Folkstone, UK at a great venue call the Leas Cliff Hall.

A great view on the beach from the venue

They made some changes compared to the previous edition, the major one being that it was now a single track, which I appreciated very much.
The speaker lineup was diverse, featuring returning speakers, high-profile keynotes presenters and new speakers who where taking their first stab at presenting.
Overall the quality was excellent and I connected with most sessions.

They also hosted a one-hour slot for lightning talks, and I had the privilege of giving a 5-minute presentation on cryptography basics. I was thrilled to receive positive feedback afterward and will probably submit a longer version to upcoming conferences.
If you want to check it out, the slides are available on GitHub.

I was surprised attendance wasn’t higher for this second edition, but honestly, I didn’t mind. The cozy vibe was a great opportunity to talk with the speakers and the participants. I thoroughly enjoyed talking to familiar faces and meeting new ones.

Creating a badge

Of course, when going to a conference, I always have an idea for a specific project, and of course it’s always a last minute thing.

When presenting at #pragma 2024 last year, I had created the Arc Reactor to show case an Embedded Swift project but also as a high-tech conference badge. It turned out rather un-practical, with the finished contraption being quite heavy.

This time, I tried another spin on the badge concept, using a lightweight and power efficient eInk screen.

Badge based on eInk screen, clipped on shirt pocket

It is still powered by Embedded Swift, running on a nRF52840 microcontroller. It displays typical badge info, a logo of the conference or a QR code linking to this blog. All of this is controlled from an iPhone app over BLE.

Badge using nRF52840 microcontroller, in 3D printed enclosure

While I reused some code from the Arc Reactor project, it was still done in a rush, with some code debugging on-site during session breaks.
As with my other projects, I intend to fully open source it, including the 3D models that I created for the enclosure. It might take a while though, as the code is really not in a “share-worthy” state.

Embedded Swift documentation effort

A few weeks ago, Rauhul posted on the Swift forums about Evolving Embedded Swift Documentation and invited community contributions.

Given my involvement with Embedded Swift and my interest in writing about it, I was immediately on board.

Last Thursday, just after returning from SwiftCraft, I joined a call with some community members to discuss the next steps. Over the week-end, I reviewed the existing documentation and submitted some early fixes.

I believe the current state of this documentation is already a major step forward compared to what was available last year when Embedded Swift was announced. But we must, as a community, continue the effort, in order to foster a wider adoption of Swift in this space.

It doesn’t have to be big, even just indicating what you feel is missing from the doc or which part you had trouble understanding, are incredibly valuable. Take a look at the aforementioned post and welcome on-board.

Tech news

A lot has happened in the tech world over the past few days.

Anthropic announced Claude 4 models, and they look promising. While I still mostly use ChatGPT, I’ve started trying Claude for programming related tasks.

Google hosted its Google IO conference, during which they made a ton of announcements related to AI but also in the AR/XR space. I’ve only watched a quick recap but I get the feeling Android is not the star of the show it once was.

And of course, it was difficult to miss the Sam and Jony show, OpenAI acqui-hire of Jony Ive’s io company.
That last one reignited my interest in AI wearables and I finally took a bit of time to do a quick, super long overdue, update on the list of such products I maintain (more updates to come).

Conclusion

That’s it for this time, and honestly, it’s probably too much. I should limit the number of projects I’m working on. But then again, who’s counting when you’re having fun?

If you’d like to share which one is your favorite, ping me on Mastodon.

Swiftly and SourceKit-LSP issue follow-up

Sun, 04 May 2025 00:00:00 +0200

Update 2025-07-12: Swiftly 1.0.1 has now been released and it fixes the problem described in this post. No workaround is necessary anymore.

Introduction

A few weeks ago, I wrote a post about Swiftly and mentioned that I experienced issues with code-completion and the likes when doing embedded development in VSCode. I pointed to a SourceKit-LSP error I was seeing but did not take the time to investigate much further.

Last week, Kuba Mracek kindly tagged me in a GitHub issue he opened in the SourceKit-LSP project.
The issue describes a similar problem and Ben Barham provided a workaround for it.

compile_commands.json

Based on the explanation in the aforementioned issue, I double-checked the content of the generated compile_commands.json file.

{  "directory": "…",  "command": "/Users/…/.swiftly/bin/swiftc -target armv7em-none-none-eabi …",  "file": "/Users/…/EmbeddedSwift-nRF52-BLEExamples/L3_E2/Main.swift",  "output": "CMakeFiles/app_swift.dir/Main.swift.obj"},

Here we can see that the command refers to swiftc within the .swiftly folder in my home directory.
Looking at that file (swiftc -> /Users/…/.swiftly/bin/swiftly), we can see it’s in fact a symbolic link to the swiftly executable itself.

Pointing to the proper compiler

Based on the comment in the issue, I tried to set the SWIFT_EXEC environment variable appropriately. Working in VSCode and with the nRF Connect SDK, I used a similar technique to what I did before when defining environment variables and added the following snippet to the .vscode/settings.json file within my project

    "nrf-connect.west.env": {                "$base": "terminal",        "SWIFT_EXEC": "$(swiftly use -p)/usr/bin/swiftc"    },

But this did not change anything.

Looking at Swift - Frequently Asked Questions, I can see SWIFT_EXEC being mentioned, but in the context of SwiftPM or Xcode, not CMake.

Searching the CMake documentation, I instead found that SWIFTC was the environment variable that indicated the Swift compiler to use. And so tried this instead

    "nrf-connect.west.env": {                "$base": "terminal",        "SWIFTC": "$(swiftly use -p)/usr/bin/swiftc"    },

but this gave an error

  Could not find compiler set in environment variable SWIFTC  $(swiftly use -p)/usr/bin/swiftc.

It looks like the environment variable does not get evaluated but is used verbatim instead.

I searched the VSCode documentation on User and workspace settings and Variables reference but could not really find a way to make this work.

So I tried hardcoding the path

    "nrf-connect.west.env": {                "$base": "terminal",        "SWIFTC": "/Users/…/Library/Developer/Toolchains/swift-latest.xctoolchain/usr/bin/swiftc"    },

and this worked fine.

In compile_commands.json the command references the actual swiftc compiler and code-completion now works as expected (if you have a Swift file opened in an editor, close and re-open it for things to work properly)

{  "directory": "…",  "command": "/Users/…/Library/Developer/Toolchains/swift-latest.xctoolchain/usr/bin/swiftc -target armv7em-none-none-eabi …",  "file": "/Users/…/EmbeddedSwift-nRF52-BLEExamples/L3_E2/Main.swift",  "output": "CMakeFiles/app_swift.dir/Main.swift.obj"},

This works but we’ve now hardcoded the path to our swiftc compiler. Using the swift-latest.xctoolchain symbolic link makes it slightly better though, as this we won’t need to change the configuration every time we install a new snapshot release and it’s probably the version we want to use anyway.

But can we do slightly better?

Set the compiler from within CMake

Taking another look at what the CMake documentation says about SWIFTC, we read that it’s used to set the value of the CMAKE_Swift_COMPILER variable.

Looking at our existing CMakeLists.txt file, we see that at the top, certain similar variables are already set

# Use the armv7em-none-none-eabi target triple for Swiftset(CMAKE_Swift_COMPILER_TARGET armv7em-none-none-eabi)# Enable "wmo" as needed by Embedded Swiftset(CMAKE_Swift_COMPILATION_MODE wholemodule)

Let’s try something similar

execute_process(COMMAND bash -c "swiftly use -p"OUTPUT_VARIABLE toolchainPathOUTPUT_STRIP_TRAILING_WHITESPACE)set (CMAKE_Swift_COMPILER "${toolchainPath}/usr/bin/swiftc")

This works fine and we can indeed see in compile_commands.json that the actual path to the currently selected toolchain is used

{  "directory": "…",  "command": "/Users/…/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2025-04-03-a.xctoolchain/usr/bin/swiftc -target armv7em-none-none-eabi -j 10 …",  "file": "/Users/…/EmbeddedSwift/Public/EmbeddedSwift-nRF52-BLEExamples/L3_E2/Main.swift",  "output": "CMakeFiles/app_swift.dir/Main.swift.obj"},

Info

I first had an issue properly building the path, caused by an extra newline in the output of the command.
Looking at the execute_process — CMake 4.0.1 Documentation pointed me to the OUTPUT_STRIP_TRAILING_WHITESPACE option that solved the issue.
It’s always a good idea to check the doc!

Being generic

The above change in the CMakeLists.txt works great when Swiftly is installed but what happens if it’s not.

My first reaction was to be cautious and test for the presence of Swiftly before executing the code

find_program(SWIFTLY "swiftly")IF(SWIFTLY)    execute_process(        COMMAND bash -c "swiftly use -p"        OUTPUT_VARIABLE toolchainPath        OUTPUT_STRIP_TRAILING_WHITESPACE    )    set (CMAKE_Swift_COMPILER "${toolchainPath}/usr/bin/swiftc")ENDIF()

So in this case, we’re falling back to the old behavior and defining the TOOLCHAIN environment variable to indicate to compiler to use.

While writing this post, as I wanted to explain why I made the above test, I tried the original code without Swiftly installed. And to my surprise, it also worked.

The explanation is that, if Swiftly is not installed, the toolchainPath variable is empty. And CMAKE_Swift_COMPILER resolves to /usr/bin/swiftc, which is executing the proper compiler.

Conclusion

In this post, I explained how to get code-completion working within VSCode when installing the Swift toolchain using Swiftly.

A big thank you to Kuba Mracek and Ben Barham for providing the initial information on the workaround. I’m grateful for the spirit of the Swift community, with so many people helping each other and striving to help grow the platform.

I’m hoping the blog posts I write and the code I open source are a small contribution to this. If you have any topic you’d like me to cover, don’t hesitate to reach out on Mastodon.

Exploring Uncut - April 28th, 2025

Mon, 28 Apr 2025 00:00:00 +0200

It’s been a while since the last “Exploring Uncut” post, so without further ado, here’s a recap of some of the fun and important things from the past few weeks.

Home Assistant

I’ve been interested in home automation for a very (very) long time, nearly 40 years.
As a teenager, I had seen an article on how to control your home using some custom electronics on your Commodore 64. That piqued my curiosity and I started researching the available products and technologies.
Looking at X10, CEBus, Phast and drooling in front of Crestron touch panels (considered state of the art back then they were ridiculously tiny monochrome panels costing thousands of dollars).

As I started working and earning more money, I bought smart remote controls from Nevo, then Phillips Pronto. Eventually, this became my part time job and I went after all kinds of certifications: AMX VIP, RTi certified, Vantage, Crestron, Lutron, …

In the early 2010s, I came across OpenRemote, an open source project that, at the time, was focused on home automation. They were doing cool things, so I used it at home and joined the team.
But home automation is a complicated market and unless your community reaches some critical mass or you’re a major player that can somehow impose a standard (and even then, HomeKit is OK but it did not became a defacto standard), this will be a constant struggle. There will always be that one other device or one other protocol that you absolutely need to support.
The project moved away from the home market and is now an IoT platform focusing on use cases such as Energy Management, Fleet Management or Smart Cities. And part of my professional time is still spent as a member of this team.

I kept using it for my home for several years after the switch, but eventually I had to move on and find some other solution.
At the time, the main project I was working on (in the digital identity domain, nothing to do with home automation), was moving to kubernetes and micro-services. So as a research project, I developed a custom home automation system. The idea was to use an MQTT bus as the backbone and have each pod encapsulate a protocol adapter or some business logic. I developed each service using Java and SpringBoot and had a web UI developed with Angular and using MQTT over WebSocket. Eventually I implemented a native Swift client, mainly to have a UI accessible on my AppleTVs.

It was a great learning experience and fun to get it running but eventually the maintenance burden became too significant. Every new device I would have liked to add required custom development, and keeping up with the dependencies updates or fixing the occasional glitch was just too much.

So I decided to take the plunge and try Home Assistant. I’ve known about it for several years. It’s become the market leader, most of my geek friends are using it, but I never tried it.

For testing purposes, I installed it inside a VM on my Mac mini. I then went through the configuration steps and was amazed at how many devices were immediately discovered or could be added with a few clicks. In no time, I had a simple dashboard with control over some basic elements like lights, media players or security cameras.

Going further will still take a bit of reading or watching some videos. And we’ll see how stable the system will be and how much effort I’ll need to put into keeping it running, but so far I’m impressed with what I’ve seen.

I’ll still do some custom developments and experiments though, as I have ideas for small devices that I could interface using Embedded Swift (although I could change my mind when looking at ESPHome).

Computer History Museum videos

Some of you following me on Mastodon might have seen my #ThrowbackThursday toots, featuring all kinds of older software or hardware from several decades ago. Looking at that brings back memories of those times when, as a child, I was discovering computers and programming; experimenting with side projects as I ran into some cool new tech or previous jobs I worked at.
But unrelated to my personal experience, I feel strongly about the fact that knowing the history of our field is important. If you’re studying painting, literature, architecture or many other topics, you’ll take history classes. You’ll understand how we got here, how the techniques evolved over time and why, who were the key contributors.

I’ve got a big collection of books about the history of computers and software programming but I also sometimes re-read old technical books, as they convey the ideas underpinning the technology at the time. And I also watch several YouTube channels about retro computing and computing history.

Lately I’ve started watching several of the Oral History videos published by the Computer History Museum, where they have long-format interviews with key players in our field, giving them the opportunity to tell their story as they recall.

Being slightly biased towards Apple and NeXT history, I’ve started with interviews from Rich Page, Avie Tevanian and Bertrand Serlet.
Having seen the people speaking about macOS change over the years in the Apple keynotes, it’s funny to see the link with NeXT’s history. Avie was Bertrand’s boss and mentor, Bertrand worked with Scott Forestall on AppKit and later hired Craig Federighi to work on EOF.

Miscellaneous

I continued working on porting the examples from the Bluetooth Low Energy Fundamentals course from Nordic Developer Academy to Embedded Swift. I posted all examples from lesson 2 (see Embedded Swift version of the Nordic Dev Academy BLE examples) and have all examples from lesson 3 running (but a bit of cleaning is required before posting).

I’m now using Swiftly to manage my Swift toolchains. Overall it makes things easier but there are a couple of glitches still. I wrote about it in Swiftly and Swift 6.1 released.

And I just submitted two talk proposals for SwiftLeeds. Let’s see how that goes.

I really enjoy presenting at conferences, so if you have any topics you’d like to see me talk about, let me know on Mastodon and I’ll see if I can bring some interesting thoughts on the subject.

Swiftly and Swift 6.1 released

Mon, 14 Apr 2025 00:00:00 +0200

Update 2025-07-12: Swiftly 1.0.1 has now been released and it fixes the problem described in this post. No workaround is necessary anymore.
Update 2025-05-04: There is now a workaround for the SourceKit-LSP issue mentioned at the end of this post, see Swiftly and SourceKit-LSP issue follow-up for more information.

Introduction

End of March was a busy time for Apple, with the release of swiftly 1.0 and a few days later Swift 6.1.

Let’s explore what this means for toolchain selection, mostly when doing Embedded Swift development on macOS.

Xcode toolchain

Xcode 16.3 comes bundled with Swift 6.1.If you don’t install anything else, open a terminal and check the current version of swift

swift -print-target-info{  "compilerVersion": "Apple Swift version 6.1 (swiftlang-6.1.0.110.21 clang-1700.0.13.3)",  "target": {    "triple": "arm64-apple-macosx15.0",    "unversionedTriple": "arm64-apple-macosx",    "moduleTriple": "arm64-apple-macos",    "compatibilityLibraries": [ ],    "librariesRequireRPath": false  },  "paths": {    "runtimeLibraryPaths": [      "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx",      "/usr/lib/swift"    ],    "runtimeLibraryImportPaths": [      "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/macosx"    ],    "runtimeResourcePath": "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift"  }}

You see a final 6.1 release

Info

Running swift --version or swift -v --version gives us some information but the above command also gives information about the runtime path.

Creating a main.swift file with this one-liner

print("Hello World!")

we can compile it using

swift main.swift

this generates an executable main file.

Let’s try running

swiftc -enable-experimental-feature Embedded main.swift

that gives an error

error: Whole module optimization (wmo) must be enabled with embedded Swift.

This indicates that the compiler recognizes Embedded mode.

But running

swiftc -enable-experimental-feature Embedded -wmo main.swift

gives

<unknown>:0: error: module 'Swift' cannot be imported in embedded Swift mode

We could maybe try to specify a target ? Let’s check what targets are available.

clang -print-targets  Registered Targets:    aarch64    - AArch64 (little endian)    aarch64_32 - AArch64 (little endian ILP32)    aarch64_be - AArch64 (big endian)    arm        - ARM    arm64      - ARM64 (little endian)    arm64_32   - ARM64 (little endian ILP32)    armeb      - ARM (big endian)    thumb      - Thumb    thumbeb    - Thumb (big endian)    x86        - 32-bit X86: Pentium-Pro and above    x86-64     - 64-bit X86: EM64T and AMD64

Clearly not all that’s required for embedded, but let’s try one

swiftc -enable-experimental-feature Embedded -wmo -target thumb-none-none-eabi main.swift error: unableToFind(tool: "swift-autolink-extract")

And what about just creating an object file, no linking

swiftc -enable-experimental-feature Embedded -wmo -c -target thumb-none-none-eabi main.swift<unknown>:0: error: unable to load standard library for target 'thumb-none-none-eabi'

Enough for the default Swift installation, let’s check with another one.

Manually downloading a toolchain

Since starting with Embedded Swift, what I’ve done so far is go to https://www.swift.org/install/macos/ and download the latest development snapshot. This would give me a .pkg file. Installing it would add a toolchain under /Library/Developer/Toolchains.

I then extract the toolchain identifier using

plutil -extract CFBundleIdentifier raw /Library/Developer/Toolchains/swift-latest.xctoolchain/Info.plistorg.swift.62202504031a

And references the toolchain I want to use with

export TOOLCHAINS='org.swift.62202504031a'

Info

This only works if you have Xcode installed, see https://forums.swift.org/t/embedded-swift-on-esp32-with-idf-error-on-build/72230/8 for more details.

Info

When using nRF Connect SDK from within VSCode, I added this variable to the West environment, as explained in nrfx-blink Step by Step.

With this in place, let’s check the Swift version we now have

swift -print-target-info{  "compilerVersion": "Apple Swift version 6.2-dev (LLVM 3f3fde0d5f85709, Swift 0c5fd6a3017961d)",  "target": {    "triple": "arm64-apple-macosx15.0",    "unversionedTriple": "arm64-apple-macosx",    "moduleTriple": "arm64-apple-macos",    "platform": "macosx",    "arch": "arm64",    "pointerWidthInBits": 64,    "pointerWidthInBytes": 8,    "swiftRuntimeCompatibilityVersion": "6.0",    "compatibilityLibraries": [ ],    "librariesRequireRPath": false  },  "paths": {    "runtimeLibraryPaths": [      "/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2025-04-03-a.xctoolchain/usr/lib/swift/macosx",      "/usr/lib/swift"    ],    "runtimeLibraryImportPaths": [      "/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2025-04-03-a.xctoolchain/usr/lib/swift/macosx"    ],    "runtimeResourcePath": "/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2025-04-03-a.xctoolchain/usr/lib/swift"  }}

Running

swiftc -enable-experimental-feature Embedded -wmo main.swift

now works and generates an executable mainfile.

We also have many more targets

clang -print-targets  Registered Targets:    aarch64    - AArch64 (little endian)    aarch64_32 - AArch64 (little endian ILP32)    aarch64_be - AArch64 (big endian)    arm        - ARM    arm64      - ARM64 (little endian)    arm64_32   - ARM64 (little endian ILP32)    armeb      - ARM (big endian)    avr        - Atmel AVR Microcontroller    mips       - MIPS (32-bit big endian)    mips64     - MIPS (64-bit big endian)    mips64el   - MIPS (64-bit little endian)    mipsel     - MIPS (32-bit little endian)    ppc32      - PowerPC 32    ppc32le    - PowerPC 32 LE    ppc64      - PowerPC 64    ppc64le    - PowerPC 64 LE    riscv32    - 32-bit RISC-V    riscv64    - 64-bit RISC-V    systemz    - SystemZ    thumb      - Thumb    thumbeb    - Thumb (big endian)    wasm32     - WebAssembly 32-bit    wasm64     - WebAssembly 64-bit    x86        - 32-bit X86: Pentium-Pro and above    x86-64     - 64-bit X86: EM64T and AMD64

So let’s try building for one of them.

swiftc -enable-experimental-feature Embedded -wmo -c -target riscv32-none-none-eabi main.swift

properly generates a main.o object file.

Enter swiftly

Installing swiftly is straightforward, go to Swift.org - Getting Started with Swiftly on macOS and follow the instructions.

As noted in the documentation, running swiftly init automatically downloads the latest swift toolchain.

Let’s check the installed version.

swift -print-target-info{  "compilerVersion": "Apple Swift version 6.1 (swift-6.1-RELEASE)",  "target": {    "triple": "arm64-apple-macosx15.0",    "unversionedTriple": "arm64-apple-macosx",    "moduleTriple": "arm64-apple-macos",    "compatibilityLibraries": [ ],    "librariesRequireRPath": false  },  "paths": {    "runtimeLibraryPaths": [      "/Users/…/Library/Developer/Toolchains/swift-6.1-RELEASE.xctoolchain/usr/lib/swift/macosx",      "/usr/lib/swift"    ],    "runtimeLibraryImportPaths": [      "/Users/…/Library/Developer/Toolchains/swift-6.1-RELEASE.xctoolchain/usr/lib/swift/macosx"    ],    "runtimeResourcePath": "/Users/…/Library/Developer/Toolchains/swift-6.1-RELEASE.xctoolchain/usr/lib/swift"  }}

The version reported is very similar to the one displayed for the default Xcode version, except it doesn’t show a specific swiftlang and clang version.

Let’s try to compile in Embedded mode

swiftc -enable-experimental-feature Embedded -wmo main.swifterror: link command failed with exit code 1 (use -v to see invocation)ld: library 'System' not foundclang: error: linker command failed with exit code 1 (use -v to see invocation)

This also results in an error, but only during the linking phase.

What if we just generate an object file?

swiftc -enable-experimental-feature Embedded -wmo -c main.swift

properly generates a main.o object file.

And what about other targets?

clang -print-targets  Registered Targets:    aarch64    - AArch64 (little endian)    aarch64_32 - AArch64 (little endian ILP32)    aarch64_be - AArch64 (big endian)    arm        - ARM    arm64      - ARM64 (little endian)    arm64_32   - ARM64 (little endian ILP32)    armeb      - ARM (big endian)    avr        - Atmel AVR Microcontroller    mips       - MIPS (32-bit big endian)    mips64     - MIPS (64-bit big endian)    mips64el   - MIPS (64-bit little endian)    mipsel     - MIPS (32-bit little endian)    ppc32      - PowerPC 32    ppc32le    - PowerPC 32 LE    ppc64      - PowerPC 64    ppc64le    - PowerPC 64 LE    riscv32    - 32-bit RISC-V    riscv64    - 64-bit RISC-V    systemz    - SystemZ    thumb      - Thumb    thumbeb    - Thumb (big endian)    wasm32     - WebAssembly 32-bit    wasm64     - WebAssembly 64-bit    x86        - 32-bit X86: Pentium-Pro and above    x86-64     - 64-bit X86: EM64T and AMD64

swiftc -enable-experimental-feature Embedded -wmo -c -target riscv32-none-none-eabi main.swift

also properly generates a main.o object file.

By the way, the clang version is quite similar to the one reported by the default Xcode toolchain

clang --versionApple clang version 17.0.0 (https://github.com/swiftlang/llvm-project.git 901f89886dcd5d1eaf07c8504d58c90f37b0cfdf)Target: arm64-apple-darwin24.4.0Thread model: posixInstalledDir: /Users/…/Library/Developer/Toolchains/swift-6.1-RELEASE.xctoolchain/usr/binBuild config: +assertions

Using snapshot versions

Swiftly allows us to download additional toolchains, including snapshot ones.Let’s see what’s available

swiftly list-available main-snapshotAvailable main development snapshot toolchains----------------------------------------------main-snapshot-2025-04-03main-snapshot-2025-04-02main-snapshot-2025-03-28main-snapshot-2025-03-25…

This lists a lot of versions, going way back to 2021.

Let’s install and use that latest one available at the time of writing (which is the same version as the one I manually downloaded above).

swiftly install main-snapshot-2025-04-03swiftly use main-snapshot-2025-04-03

And check the version

swift -print-target-info{  "compilerVersion": "Apple Swift version 6.2-dev (LLVM 3f3fde0d5f85709, Swift 0c5fd6a3017961d)",  "target": {    "triple": "arm64-apple-macosx15.0",    "unversionedTriple": "arm64-apple-macosx",    "moduleTriple": "arm64-apple-macos",    "platform": "macosx",    "arch": "arm64",    "pointerWidthInBits": 64,    "pointerWidthInBytes": 8,    "swiftRuntimeCompatibilityVersion": "6.0",    "compatibilityLibraries": [ ],    "librariesRequireRPath": false  },  "paths": {    "runtimeLibraryPaths": [      "/Users/…/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2025-04-03-a.xctoolchain/usr/lib/swift/macosx",      "/usr/lib/swift"    ],    "runtimeLibraryImportPaths": [      "/Users/…/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2025-04-03-a.xctoolchain/usr/lib/swift/macosx"    ],    "runtimeResourcePath": "/Users/…/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2025-04-03-a.xctoolchain/usr/lib/swift"  }}

Just like with the main version, building in Embedded mode fails

swiftc -enable-experimental-feature Embedded -wmo main.swifterror: link command failed with exit code 1 (use -v to see invocation)ld: library 'System' not foundclang: error: linker command failed with exit code 1 (use -v to see invocation)

What? This is really strange, it worked when we installed this toolchain manually and referenced it with the TOOLCHAIN variable. So why does it fail here?

Are the toolchains different? Let’s check with

diff -r /Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2025-04-03-a.xctoolchain ~/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2025-04-03-a.xctoolchain

No difference, they’re exactly the same.
I have so far no explanation for the different behaviors.

Let’s move on, building just an object file

swiftc -enable-experimental-feature Embedded -wmo -c main.swift

including for another target

swiftc -enable-experimental-feature Embedded -wmo -c -target riscv32-none-none-eabi main.swift

works fine, similar to the main toolchain.

By the way, if you try to build for a non-existent target, it provides you with a list of the supported ones.

swiftc -enable-experimental-feature Embedded -wmo -target m68k-none-none-eabi main.swift<unknown>:0: error: could not find module 'Swift' for target 'm68k-none-none-eabi'; found: riscv32-none-none-eabi, arm64e-apple-none-macho, armv4t-none-none-eabi, riscv64-none-none-eabi, i686-unknown-none-elf, armv7-apple-none-macho, armv6m-none-none-eabi, aarch64-none-none-elf, wasm64-unknown-none-wasm, arm64-apple-macos, x86_64-unknown-none-elf, arm64-apple-none-macho, armv7-none-none-eabi, armv7m-apple-none-macho, arm64e-apple-ios, armv7em-none-none-eabi, arm64e-apple-macos, x86_64-apple-macos, avr-none-none-elf, armv6-apple-none-macho, armv6m-apple-none-macho, wasm32-unknown-none-wasm, arm64-apple-ios, armv7em-apple-none-macho, armv6-none-none-eabi, at: /Users/…/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2025-04-03-a.xctoolchain/usr/lib/swift/embedded/Swift.swiftmodule

Toolchain priorities

With the snapshot version installed via swiftly, I tried to perform other tests with the toolchain I manually downloaded and so set the TOOLCHAIN variable.

export TOOLCHAINS='org.swift.62202504031a'

However, printing the version information shows I’m still using the toolchain installed via swiftly.

swift -print-target-info{  "compilerVersion": "Apple Swift version 6.2-dev (LLVM 3f3fde0d5f85709, Swift 0c5fd6a3017961d)",  "target": {    "triple": "arm64-apple-macosx15.0",    "unversionedTriple": "arm64-apple-macosx",    "moduleTriple": "arm64-apple-macos",    "platform": "macosx",    "arch": "arm64",    "pointerWidthInBits": 64,    "pointerWidthInBytes": 8,    "swiftRuntimeCompatibilityVersion": "6.0",    "compatibilityLibraries": [ ],    "librariesRequireRPath": false  },  "paths": {    "runtimeLibraryPaths": [      "/Users/…/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2025-04-03-a.xctoolchain/usr/lib/swift/macosx",      "/usr/lib/swift"    ],    "runtimeLibraryImportPaths": [      "/Users/…/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2025-04-03-a.xctoolchain/usr/lib/swift/macosx"    ],    "runtimeResourcePath": "/Users/…/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2025-04-03-a.xctoolchain/usr/lib/swift"  }}

As we’ve seen above, the toolchain installed manually and via swiftly are identical. But they reside at different locations on the filesystem.
Since the one installed by swiftly resides in my user account, and the manually installed one is in a system-wide location, the swiftly version takes precedence.
Logging in as another user that does not use swiftly, setting the TOOLCHAINS variable and checking the version correctly indicates that I’m using the manually installed version.

Impact for Embedded Swift

As shown in nrfx-blink Step by Step, I used to have the following snippet in my settings.json file for Visual Studio Code.

    "nrf-connect.west.env": {                "$base": "terminal",        "TOOLCHAINS": "org.swift.61202408261a"    }

This ensured the proper toolchain would be used when building Embedded Swift projects based on nRF Connect SDK.

This is not required anymore, the toolchain selected by swiftly is properly used by West.

But code-completion and jump-to-definition features, although working for C code (e.g. from BridgingHeader.h), are broken for Swift files.

Looking at the SourceKit-LSP logs (under Output tab, SourceKit Language Server), we see this

[Error - 17:27:48] Request textDocument/codeAction failed.  Message: No language service for 'file:///Users/…/Development/Nelcea/EmbeddedSwift/Public/EmbeddedSwift-nRF52-Examples/Timer/Main.swift' found  Code: -32001

I have not found a way to make it work so far, except mostly going back to the old way of working.
I say mostly because I still used swiftly to install the snapshot toolchain, but I commented out the call to ~/.swiftly/env.sh from my .zprofile file and re-added the TOOLCHAINS definition in settings.json.

Update 2025-05-04: There is now a workaround for this issue, see Swiftly and SourceKit-LSP issue follow-up for more information.

Conclusion

In this post, I’ve shown how I used to manually select a toolchain for my builds and how swiftly changes that.
I’ve also shown how different toolchains behave when working in Embedded mode.

However, some behaviors are not always what I expected, and I don’t know exactly what some error messages imply. If you have more information on what’s happening here, don’t hesitate to reach out on Mastodon.

I did not really talk about Swift 6.1 in this post. I will simple mention that it introduces package traits in packages definitions. This is indicated as a useful option for Embedded Swift but given all environments I use are CMake based, I haven’t looked into that yet.

Exploring Uncut - April 7th, 2025

Mon, 07 Apr 2025 00:00:00 +0200

Battery low

As you might have noticed, there was no post last week. I had a couple of bullet points in my note but come Sunday evening, I had not started writing any draft. I was also contemplating writing a more in-depth article on a particular topic, as I did before, but I could not decide on which one and commit to it.

I just needed to disconnect and relax and did not have the energy for it.

So instead of pushing through it, most probably missing the point of what I had to say, or shifting by one day, I decided to just skip a week.

So here is an update on what happened during the last two weeks.

Embedded Swift Community Hour

On Friday, March 28th, the second edition took place.

Kuba started by doing a show and tell of configuring SourceKit-LSP with Embedded Swift on different editors so code completion would work correctly.
The main attention points are:

you need a working build in CMake first for SourceKit-LSP to work
the CMAKE_EXPORT_COMPILE_COMMANDS is required (a compile_commands.json file must be present)
you must make sure (e.g. check in the logs) that SourceKit-LSP picks up the correct toolchain.
If Xcode is not installed, exporting a TOOLCHAINS variable will not work. You then need to add the /usr/bin folder of your toolchain to the PATH variable.
Now that swiftly 1.0 has been released, it’s the easiest method to install and reference Swift toolchains.

The open discussion then moved on to debugging and other pain point areas of Embedded Swift development. I mentioned one, where the linker would fail with no error message whatsoever.
I went on to implement a test project showing such a case. You can find it inEmbeddedSwift-nRF52-ClassVsStruct.
Kuba replied a few days later with a solution for this particular case. It turns out the optimization level was causing the issue. Using -O instead of -Osize fixes the issue.
I would like to further understand how the optimization level is affecting the generated code and causing the link issue. This aligns with the research I started doing several weeks ago on getting a better understanding of SIL and LLVM IR. Still a lot of work on that front.

iOS Dev Happy Hour

On Saturday, I had some availability in the afternoon and so attended the iOS Dev Happy Hour virtual meet-up for the first time.

Vincent Pradeilles 👨🏻‍💻 started with a presentation of some interesting SwiftUI features introduced in previous versions of iOS (from iOS 15 onwards). I always find this kind of presentation useful as it’s nearly impossible to remember all the SwiftUI types and modifiers unless you use them frequently. Moreover, when you’re faced with a particular problem, finding the appropriate way to tackle it is also not easy. Having regular reminders of how people use SwiftUI to implement particular UI/UX behaviors helps me deal with that.

After the presentation, the participants split into different discussion rooms, where smaller groups (of around 10 people maximum) can chat about a particular topic or simply get to know each other.
I had the opportunity to discuss upcoming conferences, AVFoundation, and visionOS as well as get to know some community members (and say hello to some I already know).

I enjoyed the format a lot, which remembers me of the vibe and energy you get from networking at physical conferences. I’ll definitely try to be part of future occurrences.

BLE in Embedded Swift

Last week, in The Swift Programming Language Discord, somebody posted a question about working with BLE in Embedded Swift. As I had done quite a bit of work on that for my Arc Reactor project, I happily answered.

But I never published the BLE code as it required cleanup. It was rushed together just in time for it to be ready for a conference talk. Twice!
The first time, for Pragma in November 2024, I had quite a bit of C glue code to relay between the Zephyr BLE API and my Swift code. That bothered me.
Later, for FOSDEM in February 2025, I managed to get rid of the C code altogether. But it was still prototype code with known issues (e.g. memory leaks).
At that time, I also started putting together smaller examples showing specific features of using BLE from Embedded Swift. The goal was to publish that code and write a series of articles to accompany it. But I put that on hold and never progressed on it.
Now that I know these examples would actually benefit someone, I’ll take the required time to publish something.
I’ll publish the examples on GitHub at Embedded Swift version of the Nordic Dev Academy BLE examples. The first example is already online, others will follow as I find the time to properly implement them.

That’s it for this week, as always, feel free to get in touch on Mastodon, I’d love to hear your thoughts.

Exploring Uncut - March 24th, 2025

Mon, 24 Mar 2025 00:00:00 +0200

Structs, classes and memory

I continued my reflection on how value and reference semantic impacts the memory allocation in the generated code.

Given there’s now an active Embedded Swift channel on The Swift Programming Language Discord, I asked the question there. Some comments led me to revisite the test examples I made earlier and update them. It started to make slightly more sense but not yet totally.

But then AliMark71 posted a link to the Explore Swift performance video by John McCall from WWDC 2024. And this video answers most of the questions I had, explaining in detail the different cases of how Swift would decide where to allocate memory from. I highly recommend watching this video and in fact will watch it again as it contains quite a bit of information to ingest.

And as is often the case with WWDC videos, I went down the dependency tree and started watching referenced videos:

Understanding Swift Performance by Kyle Macomber and Arnold Schwaighofer from WWDC 2016
Building Better Apps with Value Types in Swift by Doug Gregor and Bill Dudney from WWDC 2015
Protocol-Oriented Programming in Swift by Dave Abrahams from WWDC 2015

It’s a pity but it seems videos of the later are not available on-line anymore. However you can check the WWDC Notes for all those videos:

Those presentations contain a lot of useful information, sometimes covering things I’m using in my Swift code but did not grasp the fundamentals of or the ideas behind them to their full extent.

Getting the round display working

Debugging the screen

The Logic Analyzer I ordered arrived last week. This is a cheap Chinese version I ordered on Ali Express. Quite incredible that this thing costs less than 6 €. For the software, there are multiple choices available, but to start I decided to try out Logic 2 from Saleae. The freely available software is designed for their hardware but also works with many other ones. I’m not sure how they feel about people using their software this way but I can’t justify spending 500 € or more on their hardware for my usage. I might move to the open source PulseView - sigrok software in the future tough.

In order to connect the signals to capture to the analyzer, I connected the display to a bread board first and from there to the boards. First to the XIAO board, that works as expected, then to the nRF 52840 DK.

Cabling the display through a bread board to allow connecting the logic analyzer.

I was hoping to clearly see some missing or wrong signal, but at first glance, I could not see much difference between the two captures. Also this is my first time working with a logic analyzer and I only have a very high level knowledge of how SPI works.

Capture of the working display on a Seeed Studio XIAO nRF52840.

Capture of the blank display on a nRF52840 DK. I don’t know what to make out of this.

The logical next step I would take would be to study in more detail what I don’t know. Use a simple sensor connected via SPI where I could send a few bytes at a time and check the capture. Read about what the capture software can do and experiment with it. Understand SPI in much more details. Look at the source code of the display driver and how it uses SPI under the hood.

But I wasn’t thrilled about how much time this would take and was eager to get the display working.

So I gave ChatGPT a try. I explained what I wanted to do, indicated the connections I had made, the PINs I used and the DTS configuration I wrote. It pointed out that my cabling was wrong and proposed another one. It turned out that the proposed cabling was even less correct but it had correctly discovered an issue with mine. After triple checking I found the error and fixed it.

But the display stayed blank.

I continued my discussion with ChatGPT and as I had enabled logging at the SPI level using the following snippet

CONFIG_LOG=yCONFIG_DISPLAY_LOG_LEVEL_DBG=yCONFIG_SPI_LOG_LEVEL_DBG=y

in my prj.conf file, I also provided debug logs (those seemed to indicate that the SPI transactions were successful tough).
ChatGPT proposed other configuration changes (mostly incorrect) but also listed some additional checks and changes I could try. One of those was talking about the frequency used for SPI and indeed, that turned out to be the issue.
When I set the frequency to a lower value, I finally had the display working as expected.

Integrating the joystick control

Instead of having the spaceship go round the screen, I now wanted to control it with the joystick. So I hooked up my adapter to some available PINs, configured the GPIOs in the project, wrote the code to read the joystick position and move the spaceship accordingly. And nothing happened—the spaceship was motion less!

This time I had a good hunch for what the issue could be (I knew I was just being lazy but took my chance anyway). Looking into the board DTS files (nrf52840dk_nrf52840-pinctrl.dtsi and nrf52840dk_nrf52840.dts) confirmed that the PINs I was connected to were used by SPI1, enabled by default. As it is not in used, I disabled it in my overlay and all was fine.

I had quickly slapped together some test code and now wanted to organise it a bit better.
I thought “why re-invent the wheel ?”. Could I use some standard API, one that Swift developers might already know?
Apple has a whole Game Controller API available, could I make an implementation of that ? Turns out, this API is quite generic and has a large surface, too much for my first prototype here.

I only took a bit of inspiration from it and ended up with this simple code for now.

struct DirectionPadState {  var upPressed: Bool  var downPressed: Bool  var leftPressed: Bool  var rightPressed: Bool}struct Joystick {    private(set) var dpadState: DirectionPadState  private(set) var fireButtonPressed: Bool  private var upGPIO: UnsafeMutablePointer<gpio_dt_spec>  private var downGPIO: UnsafeMutablePointer<gpio_dt_spec>  private var leftGPIO: UnsafeMutablePointer<gpio_dt_spec>  private var rightGPIO: UnsafeMutablePointer<gpio_dt_spec>  private var buttonGPIO: UnsafeMutablePointer<gpio_dt_spec>    init(upGPIO: UnsafeMutablePointer<gpio_dt_spec>,        downGPIO: UnsafeMutablePointer<gpio_dt_spec>,        leftGPIO: UnsafeMutablePointer<gpio_dt_spec>,        rightGPIO: UnsafeMutablePointer<gpio_dt_spec>,        buttonGPIO: UnsafeMutablePointer<gpio_dt_spec>) {    self.upGPIO = upGPIO    self.downGPIO = downGPIO    self.leftGPIO = leftGPIO    self.rightGPIO = rightGPIO    self.buttonGPIO = buttonGPIO    gpio_pin_configure_dt(upGPIO, GPIO_INPUT)    gpio_pin_configure_dt(downGPIO, GPIO_INPUT)    gpio_pin_configure_dt(leftGPIO, GPIO_INPUT)    gpio_pin_configure_dt(rightGPIO, GPIO_INPUT)    gpio_pin_configure_dt(buttonGPIO, GPIO_INPUT)    dpadState = DirectionPadState(upPressed: false, downPressed: false, leftPressed: false, rightPressed: false)    fireButtonPressed = false  }  mutating func readState() {    self.dpadState = DirectionPadState(upPressed: gpio_pin_get_dt(upGPIO) == 1,                                downPressed: gpio_pin_get_dt(downGPIO) == 1,                                 leftPressed: gpio_pin_get_dt(leftGPIO) == 1,                                rightPressed: gpio_pin_get_dt(rightGPIO) == 1)    self.fireButtonPressed = gpio_pin_get_dt(buttonGPIO) == 1  }}

Modelling a screen support

Now that I started developing some basic interactions in the game, I needed to look at the screen more often. But with this contraption of cables connecting it to the board, it was difficult to get a clear view on it as it had a tendency to mostly lay flat on my desk.

I thus decided to 3D print a support for it and as I had already used some Lego© blocks to host my board in another project, I designed the support to attach to a Lego© plate.

I fired up Fusion 360, extracted some basic enclosure I already did in the past and extended it. The Seeed Studio round display has three threads at its back that you can use to screw it to a support. I remembered that in my previous project, although I made some measurement and tests, I was still slightly off for my support holes and I had used a drill to make them loose for the screws to fit.

Three threads at the back of Seeed Studio Round Display allow to attach it to a support.

There’s an STL file on the Seeed Studio website with some enclosure for that display, with the holes properly positioned. As I did not know how to import the STL in Fusion AND keep the proper dimensions, I opened the STL file in Bambu Studio and use the measurement tool in there. This gave me some measurements but not the ones I was looking for (the angles between the holes as measured from the center of the display).

I could compute those angles based on some trigonometry formulae. But I never remember if I need to use sine or cosine, for the adjacent or opposite side… So I was lazy (and curious). Having drawn a schematic representation of the back of the display, with the measurements I had and the values I was looking for, I asked ChatGPT.

The doodle I gave ChatGPT to work with.

I eventually got the values I was after but it took several tries and I had to give additional information to ChatGPT to get there. In particular, the values it proposed did not pass the most basic test of common sense. For instance, it first gave me a value of alpha of about 30 degrees while it’s clear on the graph that it’s an obtuse angle. After I pointed that out, it answered something like 220 degrees. Again, obtuse angle. I also had to point out that beta and gamma where equals and that the sum of all three angles had to be 360 degrees.

What ChatGPT can accomplish is quite extraordinary, and it amazed me that it could make any sense out of the doodle I gave it. But I interpret this example as a proof that the generative AI extracts features of a text corpus but does not learn a model of the world based on it.
It is interesting to note that having that common sense is sometimes also missing in humans (and I’m certainly guilty as charged in certain situations to behave as such). For instance children learning maths or physics will trust the result of their calculation even when the answer doesn’t make any sense.

I kept using ChatGPT throughout the design process. For instance, I would capture my current Fusion 360 sketch, annotate it with some scribble of what I was after and ask for step by step instructions on how to make that happen. And for this task, the results were really good. It got me to the desired result much faster than using a search engine to lookup software features.

And at the end of the afternoon, I had my screen nicely vertically mounted. I was thrilled with the result so far and spend a good minute making the spaceship go left and right.
Now, all that’s left is writing more code.

The final setup, with the display attached in its 3D printed support.

Hacking with Swift+ livestream

Past Saturday, Paul Hudson hosted another of his monthly livestream during which we build an application from scratch. This time, it was a iOS word game.

Although I did not learn much in terms of code or API, I still get a lot of value out of those sessions.Practicing using the APIs for real applications, seeing when and why choosing one over the other makes sense and what the final result is gets them deeper under your skin (or in your head). So when it comes time to develop your own application, you’re that much more productive. As the saying goes “practice makes perfect”.

Then there’s the community of course. Working together towards that goal feels great. Seeing the questions other people have is really interesting. You get to help them if you can but it also makes you think about things in a different way. And as someone would likes to write and present talks, it’s interesting to understand what concepts people have more difficulties to grasp and what explanation makes more sense to them.

And finally, seeing what can be accomplished in less than two hours is a great reminder that you don’t need that much time to explore some idea you have and build something that people are interested in. This is an important source of motivation.

Scramblet, a word game Paul had us made in a bit more than an hour.

Embedded Swift Community Hour

The next meet-up happens this Friday March 28th at 6PM CET. Join us for a nice conversation between people passionate about Embedded Swift.

In the meantime, feel free to get in touch on Mastodon.

Exploring Uncut - March 17th, 2025

Mon, 17 Mar 2025 00:00:00 +0200

Embedded Swift Community Hour

Last Friday, Rauhul organized an Embedded Swift Community Hour call on Discord, where members of the community interested in Embedded Swift gathered, presented their projects and talked about the future of this topic.

It was super interesting to virtually meet such a nice group of people, motivated to explore, share their findings and push the boundaries of Embedded Swift forward. I discovered super nice projects such as thelittlegame by Morten.

One of the core ideas of Swift is progressive disclosure — the ability to start simple but then progressively move to more complex projects without starting all over again.

We discussed how this is not currently true when targeting embedded platforms with Swift, the current hurdles that newcomers face, and the fact that the diversity of available platforms makes this even worse.
Starting up requires dealing with quite a few low-level concepts and working with C APIs. The true power of Swift only comes when you’ve created abstractions on top of those.
A few people, myself included, have created such abstractions and we discussed the challenge of sharing this code. We then extrapolated to how to potentially create a Swift based HAL.

Update 2025-03-18: For a more factual / in-depth recap of the meeting, check out Rauhul’s post: Embedded Swift Community Hour – March 14th, 2025 – Recap

Game follow-up

As I mentioned last week, I haven’t succeeded in connecting the Seeed Studio Round Display to the nRF 52840 DK. I did try two different SPIs, read some posts online, but quickly ran out of ideas on how to easily debug this with the information at hand. I thought maybe the easiest would be to check the SPI signals using a logic analyzer. As I don’t own one, I ordered a cheap one online and parked that problem for a bit.

Another aspect I wanted to explore for the game is using a joystick as input. I dug an old joystick from my Amiga times out of the attic, now all I needed was a way to connect it to my microcontroller board.

So I soldered a breakout cable to connect any old “DB-9 connector” joystick (such as used on the Commodore 64 and Amiga) to the nRF 52840 DK.

An old joystick connected to a nRF52840 DK board via a custom made breakout cable.

I then wrote a quick test program to detect the movements of the joystick and fire button presses and print information on the console. I’ll write a full article on this topic soon.

Down the rabbit hole

As indicated two weeks ago, I did not finish writing all I had been exploring in time for publishing the post, so I postponed writing about it until this week.

After publishing Addressing previous oversights, I noticed some other weird behavior in my code during further testing. Re-reading the post, I feel like I introduced more confusion than I clarified things and realized I do not grasp the subject as well as I wanted to.

Part of the issue, I think, is understanding how the reference vs value type dichotomy translates into heap vs stack allocations.

One might think the rule is simple: reference types are allocated on the heap and value types on the stack. But this is not true. I’ve tried conducting experiments with different code samples, just observing the output of print statements but I could not deduce any conclusive rules.

I’ve thought about several ways to get a better understanding, from using a custom build toolchain that I could run in a debugger (or simply adding some log statements in it at appropriate places) to looking at the final assembly code.

I’m wondering whether reading the generated SIL output would get me the information I want and would be the simplest option.

Anyway, these are all well beyond my comfort zone and I did not know exactly where to start to fill in the numerous gaps in my current knowledge.

So I searched for conference talks about SIL, as this is my preferred learning method.I have already watched

and I started watching 2015 LLVM Developers’ Meeting: J. seph Groff & Chris Lattner “Swift’s High-Level IR: A Case Study…" but that last one is a bit tough for a compiler newbie

I now need to put that into practice and look at some of the SIL code from my test projects. I’ll be sure to report further as I do that.

That’s it for this week, as always, feel free to get in touch on Mastodon, I’d love to hear your thoughts.

Exploring Uncut - March 10th, 2025

Mon, 10 Mar 2025 00:00:00 +0200

Not much happened this week in terms of exploration.
Being an introvert, I try to space out social events so I have time to recharge in between. This week was a bit of a failure on that front, with one concert, two family dinners, lunch with a friend, and a GeekClub meeting.
You might be wondering—what exactly is GeekClub?

I’ve been a CoderDojo coach for several years now. We meet about once a month to help kids between 7 and 17 years discover programming.
Watching kids tackle exciting programming projects at CoderDojo is always inspiring. But as coaches, we started to feel like spectators and were longing for more.
Some of the coachs and I invited a few friends and started meeting once a month to discuss tech (and more) over drinks. This month’s topics included full stack development in Swift or Python, Software Defined Radios and antenna shapes, Home Assistant and Modus, 3D printing filaments and more.

Round display follow-up

Last week, I built a quick prototype of a spaceship orbiting the edge of a round display. The next step is to control the ship using buttons or an old joystick.

The Seeed Studio Round Display integrates seamlessly with the XIAO chips, such as the nRF 52840. Solder header pins on the XIAO module, plug it into the display and voilà!

A XIAO nRF52840 board on the Seeed Studio Round Display.

But there’s a catch: the XIAO plugs directly into the display, using up all its GPIO pins—leaving no room for extra controls, which I need for game input.

I’ve identified two options to overcome this:

use the recently released Plus version of the XIAO chip, that exposes 9 more pins
or make custom connections to a different board

Pinout for the XIAO nRF52840 Sense Plus.

I explored the second option with a nRF52840 DK and ran some preliminary tests, but without much success. The backlight turns on, but nothing gets drawn on the screen.

Unwrap live

This past Saturday, Paul Hudson hosted his Unwrap Live event. This year, the theme was Apple Intelligence, with a big focus on using App Intents in your apps.
They allow an app to expose actions and content, making it part of the Apple Intelligence mechanism. At least, that’s the promise. For now, it’s mostly exposed to shortcuts and Siri. But that’s already a big deal and it revived some ideas I had for existing and future apps.

As usual, it was a very interesting (and intense) session offered by Paul where I learned not only about the main topic but picked up a few additional tidbits along the way.

For instance, I always forget how to initialize @Statevariables using an initializer and the _ notation to initiliaze the wrapped value. The following example demonstrates using that technique for safely initializing a SwiftData container.

@mainstruct MyApp: App {    @State private var modelContainer: ModelContainer…    init() {        let modelContainer: ModelContainer        do {            modelContainer = try ModelContainer(for: MyModel.self)        } catch {            print("Error loading ModelContainer, switching to in-memory storage.")            let config = ModelConfiguration(isStoredInMemoryOnly: true)            modelContainer = try! ModelContainer(for: MyModel.self, configurations: config)        }        self._modelContainer = .init(initialValue: modelContainer)    }}

Or that I can use the shorter, key path based, notation to access a property with a function that takes in a closure, such as in a map operation, using

items.map(\.name)

instead of

items.map { $0.name }

That’s it for this week, as always, feel free to get in touch on Mastodon, I’d love to hear your thoughts.

Exploring Uncut - March 3rd, 2025

Mon, 03 Mar 2025 00:00:00 +0200

Introduction

I would like to publish more often on this blog. I really do.

In fact, I have a long list of topics to write about. I even have several draft posts that I should just finalize and publish.

One main issue is that, as I reread a post to put the finishing touches on it, I ask myself more questions. Is it totally exact ? Does it need more background information ? Why can I write this and can I prove it somehow ?

I don’t think that this is wrong per se, but it’s in conflict with the desire to publish more often. And I believe a lot of the information that will end up in those posts would already be helpful to some people right now. Even if incomplete or potentially wrong, as long as I make that clear in my writing.
I also think that my journey to get to that final post and the process I use to overcome the challenges along the way have value in themselves.

I thus decided to start this new “Exploring Uncut” series, through which I’ll make public the explorations that have been keeping me busy during the week.
I might certainly abandon some of those or put them aside for a while. There will probably be more questions than answers, failures than success, ideas than achievements. But I don’t care, at least I’ll get something out there that one person might find interesting or useful.
And if not, I will have cleared my head for a fresh week.

Shall we play a game

Getting an idea

Several weeks ago, Frank Lefebvre posted a video of a small game he was developing in Embedded Swift, in preparation of his workshop at ARCtic Conference. I find this really cool and was inspired to think about games I could develop using Embedded Swift. And why not even build a portable gaming console (we’ll see for that one).

As I have a couple of Seeed Studio Round Displays lying around, that I’ve already used in a project, I was wondering what would be a good use for it.

The format of the screen reminded me of a video game I loved to play as a kid on my Commodore 64: Gyruss. I think this can be a great fit, as the game graphics are arranged in a circle.

I did some quick research and found this YouTube video: The History of Gyruss - Arcade console documentary, from which I learned that this kind of layout / animation in games has a name: Tubeshooters.

Finding this video had the adverse side effect of loosing me quite a bit of time. Being a fan (and collector) of old computers, I did spend the next few hours looking at retro gaming / retro computer videos.

A first test

After that interlude, I started the first test. Can I make a spaceship go round the border of the display?
So first step was to get an icon for my ship. I remembered Paul Hudson mentioning a great site with free artwork that could be used for game development during an HWS+ session: Kenney.
And sure enough, I found my space ship (Space Shooter Extension · Kenney).

I already had used LVGL (Light and Versatile Graphics Library), a free and open-source graphics library widely used in the embedded space for a previous project. And I already had written some Swift types to wrap the core features I needed, like managing Screens and displaying Images.
Starting from there, I added the ability to move and rotate images and was nearly ready for some testing.

The final step was to convert my spaceship image to the appropriate format. First, since the screen has a diameter of 240 pixels, I needed to resize the icon I downloaded. But more importantly, the most efficient way to use an image from LVGL is to have it as a byte array compiled in the code, not to load it from a file (certainly not in a compressed format).

There’s an on-line Image Converter that you can use for that, but there’s also an offline version that I had installed before. I went to a terminal and used the following command

./lv_img_conv.js SpaceShip.png  -f -c CF_INDEXED_2_BIT

This generates C code. Although I could use it as is and Swift C Interop would perfectly handle that, the C code is so simple that I preferred to convert it to Swift.This now looks something like

enum SpaceShip {  static private var data: [UInt8] = [    /*Pixel format: Alpha 8 bit, Red: 5 bit, Green: 6 bit, Blue: 5 bit  BUT the 2  color bytes are swapped*/    0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, …  ]   static let description = {    Self.data.withUnsafeBufferPointer { ptr in      lv_img_dsc_t(        header: lv_img_header_t(cf: UInt32(LV_IMG_CF_TRUE_COLOR_ALPHA), always_zero: 0, reserved: 0, w: 40, h: 30),        data_size: 1200 * UInt32(LV_IMG_PX_SIZE_ALPHA_BYTE),        data: ptr.baseAddress)    }  }()

I already thought about making a fork so the tool can directly generate the Swift code, but this is not my priority right now.

The first test was not very conclusive, as the ship was not rotating around an axis perpendicular to the screen but kind of flipping around a vertical axis in a way not really correlated to the provided angle.

The LVGL documentation is quite good, so reading it provided valuable information.

The transformations require the whole image to be available. Therefore indexed images, alpha only images or images from files can not be transformed.

Indeed as you might have noticed, when using the converter, we need to indicate a color format. And I initially used CF_INDEXED_2_BIT, the image being pretty simple, that would have limited the size.
Changing the color format to CF_TRUE_COLOR_ALPHA made the rotation work as expected.

The result

Here’s the result of the first test, a spaceship going round the screen.

The spaceship going round the screen.

It’s not super fast and there are some artefacts when the screen is redrawn, but we’ll see how this goes as we progress with the game development

I haven’t pushed any code yet at this stage but here is most of what I’ve got so far.Note that nRF Connect SDK v2.7 is using LVGL version 8.4. Most APIs have been renamed in the newer LVGL 9 versions.

@mainstruct Main {  static func main() {    let lvgl = LVGL(device: display_dev)    let screen = LVGLScreen()    screen.setActive()    var spaceShip = LVGLImage(parent: screen, imageDescription: SpaceShip.description)    spaceShip.setPosition(0, 100)    spaceShip.setPivot(20, -85)    lvgl.taskHandler()    lvgl.displayBlankingOff()    var value: Int16 = 0    while true {      k_msleep(16)      value += 3      if value > 360 {        value = 0      }      spaceShip.setAngle(value * 10)      lvgl.taskHandler()    }  }}// MARK: LVGLclass LVGLScreen: LVGLObject, Equatable {  static private var screens = [LVGLScreen]()  init(pointer: UnsafeMutablePointer<lv_obj_t>? = lv_obj_create(nil)) {    self.pointer = pointer    Self.screens.append(self)  }  var pointer: UnsafeMutablePointer<lv_obj_t>?  static func getActive() -> LVGLScreen? {    let activeScreen = lv_scr_act()    return Self.screens.first { $0.pointer == activeScreen }  }  static func == (lhs: LVGLScreen, rhs: LVGLScreen) -> Bool {    lhs.pointer == rhs.pointer  }  var isActive: Bool {    Self.getActive() == self  }  func setActive() {    if let pointer {      lv_disp_load_scr(pointer)    }  }  deinit {    Self.screens.removeAll { $0 == self }  }}struct LVGL {  var device: UnsafePointer<device>  init(device: UnsafePointer<device>) {    if !device_is_ready(device) {      fatalError("Device not ready, aborting test")    }    self.device = device  }  func taskHandler() {    lv_task_handler()  }  func displayBlankingOff() {    display_blanking_off(device)  }}protocol LVGLObject {  var pointer: UnsafeMutablePointer<lv_obj_t>? { get }}extension LVGLObject {  func setPosition(_ x: Int16, _ y: Int16) {    lv_obj_set_pos(self.pointer, x, y)  }}public enum LVGLAlignment: UInt8 {  case `default` = 0  case topLeft  case topMid  case topRight  case bottomLeft  case bottomMid  case bottomRight  case leftMid  case rightMid  case center  case outTopLeft  case outTopMid  case outTopRight  case outBottomLeft  case outBottomMid  case outBottomRight  case outLeftTop  case outLeftMid  case outLeftBottom  case outRightTop  case outRightMid  case outRightBottom}struct LVGLImage: LVGLObject {  var imgDesc: UnsafeMutablePointer<lv_img_dsc_t>  var pointer: UnsafeMutablePointer<lv_obj_t>?  public init(    parent: LVGLScreen? = nil,    imageDescription: lv_img_dsc_t,    alignment: LVGLAlignment = .center  ) {    guard let imgObj = lv_img_create(parent?.pointer) else {      fatalError("Failed to create image")    }    self.pointer = imgObj    imgDesc = UnsafeMutablePointer<lv_img_dsc_t>.allocate(capacity: 1)    imgDesc.initialize(to: imageDescription)    lv_img_set_src(imgObj, imgDesc)    lv_obj_align(imgObj, alignment.rawValue, 0, 0)  }  func setPivot(_ x: Int16, _ y: Int16) {      lv_img_set_pivot(self.pointer, x, y)  }  func setAngle(_ angle: Int16) {    lv_img_set_angle(self.pointer, angle)  }}

Reviving an old project

Nearly a year ago, I posted about Portable AI devices and started exploring open source propositions in the field. I found an interesting project that back then was called Friend (it’s now called Omi) and started playing around with it.Although I generally liked what they were doing, and even made some small contributions, a few things were bothering me:

all data was sent to a server in the cloud (they later released the source code for the server side too)
I was not very familiar with the development stack used
the way development efforts on the project were organized was a bit too chaotic for me
I was more interested in learning than using an existing product and wanted to freedom to explore in whichever direction I felt interesting or intriguing

So I did assemble a device as per their instructions and wrote an iOS app that would record and store the audio on the phone. My first use case was to record work meetings and I’ll figure out later how I can make use of the data I collected.

Back then I did not have a 3D printer and quickly created a case with what I had lying around.

The original enclosure used for the device, a recycled clear plastic box with holes in it.

BTW around the same time, StuffMc pointed me to the Embedded Swift vision document and that’s how I got interested in this technology, ended presenting about it at several conferences and writing a lot about it on this blog.

Although I wanted to explore processing the data on the phone (transcribing and then being able to query using an LLM), I thought exploring that on a server first would be easier and offer more possibilities.
I thus started writing server code that could receive the audio data from the phone. As I already had iOS code in Swift, and this was an exploration project, I decided to write the server code in Swift also (and at some point rewrite the existing firmware in Swift).

Then several things happened at the same time: I took a sabbatical and did not have any meeting to record anymore, I had acquired a 3D printer and taken the device apart so I could model a new case, I hit a bug in Vapor that halted my backend development and I spent most of my time exploring Embedded Swift. The project came to an halt and the device (rather its components) slept in a drawer.

In the meantime, the bug in Vapor has been fixed, my sabbatical is over, I again attend more work meetings and I got a bit of experience modelling and printing 3D objects. As I always wanted to continue the project, now seemed like a good time to do so.

To start recording again, I wanted to put the device back together. So last week, I finished the 3D model and printed it.

The new enclosure, looks a bit like a pack of gum.

Device inside the new enclosure, pretty tightly packed.

I can now look back at the code and continue The PAL Project.

I also want to address the on-line meetings aspect. For now, I’m using Audio Hijack from Rogue Amoeba to gather the audio. We might talk about open source alternatives to Rewind.ai some other time.

Oh yes, I know there are lots of questions about these types of devices, their usefulness and certainly their commercial viability (good bye Humane) but I still want to explore the field and maybe others want to follow along.

Posting

That’s it for this week! There’s a bit more I wanted to talk about but it’ll have to wait for next week, as otherwise I will just postpone posting this, exactly the opposite of what I’m going for with this series.

Until next time, and I’d love to hear any thoughts or suggestions. You can reach me on Mastodon, my preferred communication channel.

Addressing previous oversights

Sat, 08 Feb 2025 00:00:00 +0200

Introduction

Back in November 2024, as I closed my Blink them to death using Embedded Swift presentation at Pragma Conf, I promised to release the source code of the 3 projects I presented before the end of the year. I thought this would be an easy goal to achieve and that I’d finish way sooner.

Of course the projects had been prototypes so far, so I wanted to cleanup the code and recheck everything was working as expected. This ended up taking much longer than expected (plus, daily life got in the way—but that’s another story).

In the end, I pushed code for the third project on the last day of 2024 (and I cheated a bit as I removed some features compared to what I had in the prototypes). So all 3 projects are available:

And while doing the cleaning, I discovered that I had some loose ends in how I did things so far, and that included some of the code in my previously published posts.In this post, I’ll review those oversights and provide an improved version of what I initially presented.

Memory safety is nice

Swift is a language that provides memory safety and this is really nice, it catches a whole category of bugs at compile time.But with Embedded Swift, you sometimes need to perform some manual memory management and work without that safety net.

I was bitten by this issue as I experienced a very strange bug when working on Swatak, where the LEDs seemed to lit up in a completely random fashion. The root cause turned out to be a memory corruption.

Let me walk you through a simplified reproduction of the problem.

Based on the Button code that I published in Creating a Swift type for button input on nRF52 - Part 2, here is a bit of code that uses a Context object to store some message and print it out on button press.

@mainstruct Main {  static func main() {    let context: Context = Context(message: "Hello")        let _ = Button<Context>(gpio: &button, context: context) { _, callback, _ in      let context = Button<Context>.getContext(callback)      print("Btn pressed >\(context.message)<")    }    while true {      k_msleep(5000)    }  }}class Context {  let message: String  init(message: String) {    self.message = message  }}

And here is the output of running this on an nRF52840DK board

*** Booting nRF Connect SDK v2.7.0-5cb85570ca43 ****** Using Zephyr OS v3.6.99-100befc70c74 ***Btn pressed >Hello<Btn pressed ><

On the first button press, the message is properly printed out, but on the second one, nothing is printed out.

Something strange is going on with Context here. Let’s implement a deinitializer and print a message when it gets deallocated.

  deinit {    print("Context deinit")  }

Flashing this version to the board, we see that the message is printed out immediately. Even before the button is pressed.

*** Booting nRF Connect SDK v2.7.0-5cb85570ca43 ****** Using Zephyr OS v3.6.99-100befc70c74 ***Context deinit

Let’s look at the Button code. Below is an extract, keeping only the code related to the context storage.

struct Button<T: AnyObject> {  var context: T  init(gpio: UnsafePointer<gpio_dt_spec>, context: T, handle: GpioCallbackHandler?) {    self.context = context        self.pin_cb_data.pointee.context = Unmanaged.passUnretained(context).toOpaque()

As we have a property in Button that references the context, the latter will be kept alive as long is the button exists.
In addition, we store a pointer to the context in the pin_cb_datastructure, and there we use passUnretained as to not keep an extra reference to the context.
Could it then be that the Button instance itself is getting deallocated?
Let’s add a deinitialized with a print statement to Button (changing it to a class for this test).

  deinit {    print("Button deinit")  }

Our output is now

*** Booting nRF Connect SDK v2.7.0-5cb85570ca43 ****** Using Zephyr OS v3.6.99-100befc70c74 ***Button deinitContext deinit

And indeed, we can see that the Button instance is deallocated immediately, which in turns triggers the deallocation of the context.

Changing our main code to keep a reference to the button when it is created fixes the issue.

let btn1 = Button<Context>(gpio: &button, context: context) { _, callback, _ in  let context = Button<Context>.getContext(callback)  print("Btn pressed >\(context.message)<")}

*** Booting nRF Connect SDK v2.7.0-5cb85570ca43 ****** Using Zephyr OS v3.6.99-100befc70c74 ***Btn pressed >Hello<Btn pressed >Hello<

But this now generates a warning during compilation because the value is not used.

.../ButtonDeinitBug/Main.swift:6:9: warning: initialization of immutable value 'btn1' was never used; consider replacing with assignment to '_' or removing it 4 |     let context: Context = Context(message: "Hello") 5 |      6 |     let btn1 = Button<Context>(gpio: &button, context: context) { _, callback, _ in   |         `- warning: initialization of immutable value 'btn1' was never used; consider replacing with assignment to '_' or removing it 7 |       let context = Button<Context>.getContext(callback) 8 |       print("Btn pressed >\(context.message)<")

AFAIK there is still no way to silence this single warning in Swift.However, it is possible to rewrite the code in such a way that we retain the button and avoid the warning.

    _ = Unmanaged.passRetained(Button<Context>(gpio: &button, context: context) { _, callback, _ in      let context = Button<Context>.getContext(callback)      print("Btn pressed >\(context.message)<")    })

But this makes the code slightly less readable.
I leave it up to you to decide which is the lesser of the two evils.

Making it safe by default

We now understand why we saw an undefined behaviour and how to prevent it. But the Button code is still unsafe and we could easily fall into the same trap again later.We should instead make it so that users of this code would be Falling Into The Pit of Success, a concept I heard about in Alex Ozun’s talk at Swift Craft 2024 on Type-Driven Design with Swift.

One safe thing to do is to ensure that, if we would not retain the button, and it gets deallocated, we don’t access the context data i.e. making sure its callback closure is not called anymore.And just as nRF Connect SDK provides a gpio_add_callback function, there’s the corresponding gpio_remove_callbackone that we can use here.

By the way, also notice that in our Button initializer, we allocated a pin_cb_data structure

self.pin_cb_data = UnsafeMutablePointer<extended_callback>.allocate(capacity: 1)

which we never deallocated.

Taking care of both of the above issues, we now add a deinitializer to our Button class.

  deinit {    print("Button deinit")    gpio_remove_callback(gpio.pointee.port, &pin_cb_data.pointee.callback)    pin_cb_data.deallocate()  }

Having reverted our main code to not retain the button and flashing the board, we observe the following output

*** Booting nRF Connect SDK v2.7.0-5cb85570ca43 ****** Using Zephyr OS v3.6.99-100befc70c74 ***Button deinitContext deinit

So the Button and the Context instances are deallocated at startup. But now, we can press the button as much as we want and nothing will happen. I might not do what we want, but the code is safe from accessing random data or crashing the board.

Class or struct

Our Buttonstarted as a struct and we now made it a class, only because we needed a deinitmethod. An alternative would have been to make the struct non copyable, this also supports deinit. Which one makes the most sense ?

If you consider the Button type as a representation of the physical button, I would argue that there should only be one user of this button at a time. This aligns nicely with the ownership semantics of non-copyable structs.

But if you consider the Button type as representing the handler associated with the button press event, it does not really hurt to have multiple references to it and a class would work fine.

At this point, I’m only raising the point so you can consider this aspect in your own design. I’ll address this topic in more details in a future post.

Better understanding the underlying SDK

In a previous post, Controlling an LED using Embedded Swift on nRF52, I created a Led struct with simple on, off and toggle methods, controlling one of the on-board LED on the nRF52840 dk board. As it is one of the default LED for the board, its low-level definition is part of the SDK (in a device tree configuration file).

For the Swatak project mentioned above, I slightly changed the code, so that I could have a state property to proxy for the LED status.

  var state: Bool {    didSet {      gpio_pin_set_dt(gpio, state ? 1 : 0)    }  }

I mostly kept the same initializer code as in the original example, except I added the initialization of the state variable.

init(gpio: UnsafePointer<gpio_dt_spec>, state: Bool = false) {  self.gpio = gpio  self.state = state  gpio_pin_configure_dt(gpio, GPIO_OUTPUT | GPIO_OUTPUT_INIT_HIGH | GPIO_OUTPUT_INIT_LOGICAL)}

And with the blinking LED example, I did not notice anything wrong.

Invalid initial state of the physical LED

However, when I started using the code in Swatak, the LEDs were all on, even though the default state value is false.

In reusing old example code, I did not pay attention to the parameters of the gpio_pin_configure_dt() function. But of course, they matter.
Looking at the definition of the GPIO_OUTPUT_INIT_HIGH constant and its companion GPIO_OUTPUT_INIT_LOW provides some useful information.

/* Initializes output to a low state. */#define GPIO_OUTPUT_INIT_LOW    (1U << 18)/* Initializes output to a high state. */#define GPIO_OUTPUT_INIT_HIGH   (1U << 19)

So, by always passing GPIO_OUTPUT_INIT_HIGH to the gpio_pin_configure_dt() call, the LED always started on, irrelevant of the initial state.
The problem can be fixed by adapting the initializer and passing the proper constant based on the initial state.

init(gpio: UnsafePointer<gpio_dt_spec>, state: Bool = false) {  self.gpio = gpio  self.state = state  gpio_pin_configure_dt(gpio, GPIO_OUTPUT                       | (state ? GPIO_OUTPUT_INIT_HIGH : GPIO_OUTPUT_INIT_LOW)                       | GPIO_OUTPUT_INIT_LOGICAL)}

When the physical and virtual world don’t agree

For both the TrafficLight and Swatak projects, I needed to use external GPIO PINs on which I had attached my own LEDs. This meant using a custom device tree configuration (dts) file.

So I opened the original dts file for the board (boards/nordic/nrf52840dk/nrf52840dk_nrf52840.dts in the Zephyr SDK source) and looked up how it defined its LED.

led0: led_0 {gpios = <&gpio0 13 GPIO_ACTIVE_LOW>;label = "Green LED 0";};

I copied that snippet to my own project and adapted it for the GPIO pin my LED is connected to.

greenled: green_led {gpios = <&gpio1 10 GPIO_ACTIVE_LOW>;label = "LED P1.10";};

And the LED behavior I was seeing did not make any sense. After a moment, I understood that the LED visual state was the opposite of the actual statevariable value. Why is that ?

Let’s again turn to the constants definitions.

/** GPIO pin is active (has logical value '1') in low state. */#define GPIO_ACTIVE_LOW         (1 << 0)/** GPIO pin is active (has logical value '1') in high state. */#define GPIO_ACTIVE_HIGH        (0 << 0)

The above definition, using the GPIO_ACTIVE_LOW constant, means that there’s voltage on the PIN when the value is 0 and there’s no voltage when the value is 1.

Looking at the nRF52840 DK Hardware user guide (always a good idea) and more specifically the Buttons and LEDs section, one can read

The LEDs are active low, meaning that writing a logical zero (0) to the output pin turns on the LED.

So the above configuration is appropriate for the how the on-board LEDs are wired on the development board. But not for how my external LEDs are wired.Once I understood that, I could adapt the configuration.

greenled: green_led {gpios = <&gpio1 10 GPIO_ACTIVE_HIGH>;label = "LED P1.10";};

The LEDs now behaved as expected.

Info

Device tree configuration is a complex topic that goes far beyond the scope of this post.
I find the Practical Zephyr posts series on the Memfault Interrupt blog a very detailed, yet approchable and very well written source of information on the topic.

Conclusion

Although the goal of Embedded Swift is to allow us to use a familiar language and easily transpose the work we’ve been going on iOS, macOS or other platforms to the embedded world, it is not yet that straightforward.

At this stage, we still need to understand that the underlying system has its own rules, that we should be aware of and understand.

I’m hopeful that as the platform matures, the community will develop higher-level abstractions, allowing developers to distance themselves from some of the lower-level details. I’d like to believe that some of the articles I write and the code I publish help contribute to that evolution.

Rolling back a brew formula

Tue, 21 Jan 2025 00:00:00 +0200

Introduction

I find Homebrew super convenient to install and keep up to date the large collection of tools I rely on. But sometimes, things do not work out the way you plan.

Warning

At the time of this writing, a malware site masquerading as the Homebrew site is being spread through Google Ads, see this post on X by Ryan Chenkie for more details.

Some weeks ago, I was cleaning up the code for an Embedded Swift project before publishing it on GitHub, and as a last check, I wanted to verify everything still worked fine with the latest versions of the various components: latest Swift snapshot toolchain, nRF Connect SDK 2.9.0 and latest cmake version.
Since I had installed cmake using brew, I did a brew upgrade and this updated cmake from 3.31.1 to 3.31.2.

Suddenly, things began to fall apart. I could not complete a successful build after those updates.
I first rolled back nRF Connect SDK as this was probably where the biggest changes did occur. This did not help.
Next I tried with a previous Swift toolchain, which did not help either.
This pointed to cmake as the most probable culprit, so I wanted to roll it back.
And that proved to be more difficult than I thought.

TL;DR

In the rest of this post, I will outline my research process, the challenges I encountered, and the steps I followed to resolve them.

If you are not interested in the whole story, the following commands should work to install a specific version of a helm formula

brew tap-new $USER/local-cmakebrew tap homebrew/core --forcebrew extract --version=3.31.1 cmake $USER/local-cmakebrew install $USER/local-cmake/cmake@3.31.1

Trying to find an easy way

I opened a terminal and typed brew --help, hoping to find a revert/rollback command but I did not find any such option, only a full uninstall command.

So I turned to Google; and found several posts indicating how to install a specific version of a package, but most of those posts were outdated and referenced commands and ways to proceed that do not work anymore.For instance:

brew versions does not work, as the versions command is not available anymore.
same for brew switch
brew list --versions <package name> does work but always returns the latest, currently installed version for me
using the <package name>@<version> as done when using npm does not work (it happens that some package names use this format e.g. postgresql@14, but this is their naming convention, not a generic mechanism used by brew)

And indeed, brew --help install does not indicate any way to install a specific version.

Going the complex route

I found several posts indicating the possibility to install a specific version (using the install command) by forging a specific URL, referencing the formula using a git commit hash, so it would point to the desired version.
Something like brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/4cfc96448e261e9b16d9b51dc6d563c717003bfd/Formula/c/cmake.rb.

However, this method is no longer supported and returns the error message

Error: Non-checksummed download of cmake formula file from an arbitrary URL is unsupported! brew extract or brew create and brew tap-new to create a formula file in a tap on GitHub instead.

So I searched online for the suggested commands and tried

 brew tap-new $USER/local-cmake brew extract --version=3.31.1 cmake $USER/local-cmake

The first line worked OK and created a local tap, essentially a local git repository, located under /opt/homebrew/Library/Taps, to store formulas. See How to Create and Maintain a Tap — Homebrew Documentation for some background on this command.
The second failed though, with the message

Error: No available formula with the name "homebrew/core/cmake".Please tap it and then try again: brew tap homebrew/core

Executing the command that was suggested gave another error

brew tap homebrew/coreError: Tapping homebrew/core is no longer typically necessary.Add --force if you are sure you need it for contributing to Homebrew.

That error message leaves a door open. Although it does not make it super clear when you would really need to do it, the use of “typically” suggests this is still sometimes required.

I considered this was the case here

brew tap homebrew/core --force==> Tapping homebrew/coreCloning into '/opt/homebrew/Library/Taps/homebrew/homebrew-core'...

did indeed work, cloning the (pretty large) homebrew core repo locally.

I could now get back to the extract from above

brew extract --version=3.31.1 cmake $USER/local-cmake==> Searching repository history==> Writing formula for cmake at 3.31.1 from revision 4cfc964 to:/opt/homebrew/Library/Taps/ebariaux/homebrew-local-cmake/Formula/cmake@3.31.1.rb

which indeed, extracted a specific version of the cmake formula in the local git repo that I initially created.

And with that, I was finally able to install the desired version of the formula

brew install $USER/local-cmake/cmake@3.31.1

Conclusion

In this post, I explained how to install a specific version of a brew formula. This comes in handy when performing a general brew upgrade and discovering that certain commands no longer function as expected.

Timers in Swift on nRF52

Thu, 05 Dec 2024 00:00:00 +0200

Introduction

In this post, we’ll continue exploring Embedded Swift on an nRF 52840 dk, using the nRF Connect SDK. Our goal this time is to execute code after a certain delay, with optional periodic repetition.
In Swift, one would typically use a Timer class to implement such a feature. Can we use that with Embedded Swift?

Foundation

Embedded Swift is really about the Swift language, and the language only includes the Swift runtime and the standard library, both having some limitations in Embedded Swift.
Timer is not part of those, it is included in Foundation. In the Apple ecosystem, we take Foundation for granted, but really it is a separate library, that’s not part of the language.

Foundation is a complex system, originating over 30 years ago, it evolved from NeXT’s early days. And it’s still going through quite a bit of change, on its way to being fully open sourced and unified across all platforms where Swift is available.

The code is available on GitHub, mainly in 2 repositories:

swift-corelibs-foundation: The Foundation Project, providing core utilities, internationalization, and OS independence. It is providing the implementation of these functionalities on non Darwin platforms.
swift-foundation: The Foundation project, a cross platform implementation of several fundamental types and functions.

Info

If you’re interested in knowing more about this split and the structure of Foundation, check out the posts The Future of Foundation and Foundation Package Preview Now Available on the Swift.org Blog as well as the Swift & Interoperability video from this year’s ServerSide.swift Conference.

Let’s look at the code for Timer.swift in swift-corelibs-foundation.
A quick glance already shows that it’s a class inheriting from NSObject and that it integrates with the CFRunLoop mechanism.

Taking that code as is and building it with Embedded Swift is clearly not gonna fly, but maybe we can keep the same API and implement it for our platform?

Timers in nRF Connect SDK

Before we do that, we need to understand how timers work in nRF Connect SDK and what API is available.
So let’s check the nRF Connect SDK documentation for the Timers services.

Here are a few key takeaways from reading the documentation:

Timers trigger after a certain delay (timeout) and optionally repeat with a given period
A timer is defined using a variable of type k_timer
Timers can be defined at compile-time using macros or at run-time
In the later case, the timer is initialized with the k_timer_init function and started with k_timer_start
Timers can be stopped and restarted, potentially changing their delay and period
The expiry function code is executed in an interrupt context, its execution time should be kept to a minimum and it should not block; submit a work item to the system work queue if required

Equipped with that knowledge, we can now create our own implementation of the Foundation Timer API.

Re-implementing the Timer API

In addition to the Timer source code, we can also use the Timer Documentation as a guide to know how to implement the API.

Let’s go over the functions and properties defined by the API:

class func scheduledTimer(withTimeInterval: TimeInterval, repeats: Bool, block: (Timer) -> Void) -> Timer

Creates a timer and schedules it on the current run loop in the default mode.

✅ Can implement: We can implement that, dynamically creating a timer with k_timer_init and starting it with k_timer_start.

class func scheduledTimer(timeInterval: TimeInterval, target: Any, selector: Selector, userInfo: Any?, repeats: Bool) -> Timer

Creates a timer and schedules it on the current run loop in the default mode.

❌ Skip: Uses target and selector, an heritage from Objective-C, requiring dynamic runtime lookup of the implementation to call.

class func scheduledTimer(timeInterval: TimeInterval, invocation: NSInvocation, repeats: Bool) -> Timer

Creates a new timer and schedules it on the current run loop in the default mode.

❌ Skip: Uses an NSInvocation, similar issue to the previous function.

init(timeInterval: TimeInterval, repeats: Bool, block: (Timer) -> Void)

Initializes a timer object with the specified time interval and block.

❌ Skip: Only creates the timers but requires the timer to be added to a CFRunLoop, which is an API we don’t have / want to implement at this stage.

For the same reasons, we’ll skip the other initializers.

init(timeInterval: TimeInterval, target: Any, selector: Selector, userInfo: Any?, repeats: Bool)init(timeInterval: TimeInterval, invocation: NSInvocation, repeats: Bool)init(fire: Date, interval: TimeInterval, repeats: Bool, block: (Timer) -> Void)init(fireAt: Date, interval: TimeInterval, target: Any, selector: Selector, userInfo: Any?, repeats: Bool)

func fire()

Causes the timer’s message to be sent to its target.

✅ Can implement: We can store a closure and call it anytime.

func invalidate()

Stops the timer from ever firing again and requests its removal from its run loop.

✅ Can implement: The k_timer_stop function will prevent the timer from firing again.

var isValid: Bool

A Boolean value that indicates whether the timer is currently valid.

✅ Can implement: We can manage a state, setting it to invalid when invalidate is called.

var fireDate: Date

The date at which the timer will fire.

❌ Skip: Date is also part of Foundation, so we don’t have it by default in Embedded Swift and would need to implement something ourself. That’s something we’ll look at in a later post but let’s skip for now.

var timeInterval: TimeInterval

The timer’s time interval, in seconds.

✅ Can implement: That’s an information we have.

var userInfo: Any?

The receiver’s userInfo object.

❌ Skip: Embedded Swift does not support existential types, so Any is a no go.

var tolerance: TimeInterval

The amount of time after the scheduled fire date that the timer may fire.

❌ Skip: Although a Zephyr timer could fire later than expected if the system can’t honour the exact requested timing, we have no information or control over the potential shift.

static func publish(every: TimeInterval, tolerance: TimeInterval?, on: RunLoop, in: RunLoop.Mode, options: RunLoop.SchedulerOptions?) -> Timer.TimerPublisher

Returns a publisher that repeatedly emits the current date on the given interval.

class Timer.TimerPublisher

A publisher that repeatedly emits the current date on a given interval.

❌ Skip: Those 2 are about Combine, which we don’t have under Embedded Swift.

As we can see, we need to skip over a large chunk of the API, not implementing it or providing some stub implementation that does not really behave as its original counterpart.
In those conditions, implementing the same API doesn’t make a lot of sense and could actually prove dangerous as code using our implementation would behave differently than expected.

Our own API

So, instead, let’s implement a Timer type that provides similar functionalities, but with its own API (but still inspired by the existing Timer API).

We’ll keep the idea of having a static function create and start a timer and the initializer create the timer without starting it. We’ll thus need a method to start the timer.
When creating a timer, we want to pass a delay, an optional period and the code to execute on trigger.

In Zephyr, the expiry function type is defined as

typedef void (*k_timer_expiry_t)(struct k_timer *timer)

so we’ll create the equivalent in Swift

typealias TimerExpiry = @convention(c) (  _ timer: UnsafeMutablePointer<k_timer>?) -> Void

We can now define our code skeleton

struct Timer {  static func scheduledTimer(delay: UInt32, period: UInt32? = nil, handler: TimerExpiry?) -> Timer {  }  init(delay: UInt32, period: UInt32? = nil, handler: TimerExpiry?) {  }  func start() {      }}

The initializer is straightforward. We just add properties to remember the delay and period, and a property to store the k_timer structure; we initialize it with a reference to the code to execute.

struct Timer {  var timer = UnsafeMutablePointer<k_timer>.allocate(capacity: 1)  var delay: UInt32  var period: UInt32?  init(delay: UInt32, period: UInt32? = nil, handler: TimerExpiry?) {    self.delay = delay    self.period = period    k_timer_init(timer, handler, nil)  }}

Let’s now look at starting the timer, the Zephyr call to do that is

void k_timer_start(struct k_timer *timer, k_timeout_t duration, k_timeout_t period)

The function takes the duration and period parameters as k_timeout_t structures. Those are typically constructed in C using the macros K_MSEC() or K_NO_WAIT and K_FOREVER.

Trying to use that in our code

func start() {  k_timer_start(timer, K_MSEC(self.delay), K_MSEC(self.period))}

leads to compilation errors

macro 'K_MSEC' unavailable: function like macros not supported

We’ve already seen this error in previous posts, as the Swift C Interop only deals with simply macros.
Let’s look at the definition of the C macro

#define K_MSEC(ms)     Z_TIMEOUT_MS(ms)

which further expands to

# define Z_TIMEOUT_MS(t) Z_TIMEOUT_TICKS((k_ticks_t)k_ms_to_ticks_ceil64(MAX(t, 0)))

and this goes on several levels.

Going through the different definitions, we find out that the macro converts milliseconds to a k_timeout_t struct with its ticks element initialized. The conversion is based on the CONFIG_SYS_CLOCK_TICKS_PER_SEC constant and the results is a 64bit Int, rounded up.

With this understanding, we can write the following helper functions in Swift.

private func msToTimeout(_ delay: UInt32) -> k_timeout_t {  k_timeout_t(ticks: msToTick(delay))}private func msToTick(_ delay: UInt32) -> Int64 {  return Int64((Double(UInt64(CONFIG_SYS_CLOCK_TICKS_PER_SEC) * UInt64(delay)) / 1000.0).rounded(.up))}

and implement our start() function

func start() {  k_timer_start(timer, msToTimeout(self.delay), msToTimeout(self.period == nil ? 0 : self.period!))}

And finally, we can implement the static function

static func scheduledTimer(delay: UInt32, period: UInt32? = nil, handler: TimerExpiry?) -> Timer {  let t = Timer(delay: delay, period: period, handler: handler)  t.start()  return t}

Testing

Let’s write a small test program using what we have so far and observe its behaviour.

@mainstruct Main {  static func main() {    let _ = Timer.scheduledTimer(delay: 10000) { timer in      print("10 seconds have elapsed")        }    let _ = Timer.scheduledTimer(delay: 5000, period: 2000) { timer in      print("Ticking every 2 seconds")    }    while true {      k_msleep(5000)    }  }}

Building and flashing this to our board shows the following output

*** Booting nRF Connect SDK v2.7.0-5cb85570ca43 ****** Using Zephyr OS v3.6.99-100befc70c74 ***Ticking every 2 secondsTicking every 2 secondsTicking every 2 seconds10 seconds have elapsedTicking every 2 secondsTicking every 2 seconds...

Clean-up

The current implementation works but there’s one issue. We initialize the timer property by allocating a k_timer structure, which is never deallocated.

var timer = UnsafeMutablePointer<k_timer>.allocate(capacity: 1)

That’s a memory leak. To prevent it, we must ensure we deallocate the structure when it’s no longer required.
A good place to do it is in deinit. For this, we need to make our struct non-copyable (or switch to using a class).

Let’s do that and implement clean-up in the deinit

struct Timer: ~Copyable {...  deinit {    k_timer_stop(timer)    timer.deallocate()  }}

in which we stop the underlying Zephyr timer and release the previously allocated memory.

With this change in place, if we run our above example, no message will be printed anymore.
This is because we don’t keep a reference to the created timers and so they get de-initialized.

To make it work again, let’s change the code to

let timer1 = Timer.scheduledTimer(delay: 10000) { timer in  print("10 seconds have elapsed")    }let timer2 = Timer.scheduledTimer(delay: 5000, period: 2000) { timer in  print("Ticking every 2 seconds")}

User data

As with our button example from a previous post (Creating a Swift type for button input on nRF52 - Part 2), the closure can not capture its context and we need some work around to access data from it.
And as with the button, it’s possible to associate some context data with the timer.
The k_timer structure contains a user_data member; a void pointer to some data we want to pass around with the timer.

We’ll make the Timer generic for the user data, which can be any reference type

struct Timer<T: AnyObject>: ~Copyable {

add a property to store the user data

var userData: T

and update the initializer to take the user data as parameter and store it in the timer structure

init(userData: T, delay: UInt32, period: UInt32? = nil, handler: TimerExpiry?) {  self.delay = delay  self.period = period  k_timer_init(timer, handler, nil)  self.userData = userData  timer.pointee.user_data = Unmanaged.passUnretained(userData).toOpaque()}

As we keep a reference to userData in a property, we use passUnretained when storing the pointer in the timer structure’s user_data.

We’ll also provide a static function to extract the user data from a timer

static func getUserData(_ timer: UnsafeMutablePointer<k_timer>?) -> T {  return Unmanaged.fromOpaque(timer!.pointee.user_data).takeUnretainedValue()}

And update the scheduledTimer convenience method to pass the user data to the initializer

static func scheduledTimer(userData: T, delay: UInt32, period: UInt32? = nil, handler: TimerExpiry?) -> Timer {  let t = Timer(userData: userData, delay: delay, period: period, handler: handler)  t.start()  return t}

With that in place, we can for instance toggle an LED using a timer, such as in the sample code below (Led is a class in this case).

let led = Led(gpio: &led0)let timer = Timer<Led>.scheduledTimer(userData: led, delay: 5000, period: 2000) { timer in  Timer<Led>.getUserData(timer).toggle()}

Work queue

As was mentioned in the beginning of this post and in the original post on button (see Creating a Swift type for button input on nRF52 - Part 1), the closure is executed in an ISR context. The code of that closure should execute fast and never block.
If this is not the case, the typical way to address this is to use work queues and schedule the code as a work item so it gets executed in another thread.

This is a topic we’ll look at in a future post.

Conclusion

In this post, we’ve built a Timer struct, with an API inspired by (but not compatible with) the native Timer class from Foundation. It allows us to easily schedule a closure for later execution and optional repetition.

Foundation offers many more classes revolving around times and dates, an area to explore in a future post.

As always, the code from this post is available on GitHub.

If you have any questions, or specific topic you’d like me to cover, feel free to contact me on Mastodon, X or LinkedIn.

Randomness on nRF52 using Embedded Swift

Sat, 09 Nov 2024 00:00:00 +0200

Update 2025-07-12: Added import statement for getentropy() implementation in stubs.c

Introduction

Randomness comes in handy in many different places. It’s useful in the logic of many games to make enemies behave in an unpredictable fashion, it can distribute the timing of events when used within a retry mechanism and random data is sometimes integral to AI algorithms.
In this post, we’ll see how to generate random numbers in Embedded Swift on an nRF52840 DK using the nRF Connect SDK.

Reading documentation

The External dependencies section of the Embedded Swift — User Manual lists some external dependencies that need to be present at link-time, as the Swift standard library or Embedded Swift runtime might need to call them.
Of particular interest to us is:

using Hashable, Set, Dictionary, or random-number generating APIs.
dependency: void arc4random_buf(void *, size_t);

Testing the code

Let’s write some basic code to print a random digit every 5 seconds and try to compile that.

@mainstruct Main {  static func main() {    while true {      let randomDigit = Int.random(in: 0...9)      print("Digit: \(randomDigit)")      k_msleep(5000)    }  }}

From the documentation above, the linker should report an undefined symbol: arc4random_buf. However, the error message we receive is slightly different.

/opt/nordic/ncs/toolchains/f8037e9b83/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: /opt/nordic/ncs/toolchains/f8037e9b83/opt/zephyr-sdk/arm-zephyr-eabi/arm-zephyr-eabi/lib/thumb/v7e-m/nofp/libc_nano.a(lib_a-arc4random.o): in function `_rs_stir_if_needed':arc4random.c:(.text._rs_stir_if_needed+0x16): undefined reference to `getentropy'

indicating that the getentropysymbol is undefined.

C standard library

nRF Connect SDK (Zephyr) provides, by default, an implementation of the C standard library. In fact, it provides multiple.
We’re using newlib in our project. This is configured in prj.conf with the parameter

CONFIG_NEWLIB_LIBC=y

This is the configuration used in the nrfx-blink-sdk sample from Apple.

Looking at the source code of newlib, we find in file newlib/libc/stdlib/arc4random.c, the arc4random_buf()function that the Embedded Swift documentation instructs us to implement.
Following the code logic, we eventually find a call to getentropy(), the undefined symbol that our linker error mentioned.

As newlib does not provide an implementation of this function, it is up to us to do so.

Bridging the gap

Let’s start by researching the intended purpose of this function. The getentropy(3) - Linux manual page gives us the answer

fill a buffer with random bytes

Does Zephyr provide functionality to support this implementation ?
Looking through the documentation, we find a section on Random Number Generation.
This section indicates that indeed, Zephyr provides multiple possibilities for generating random numbers. Clicking through to the Random Function APIs documentation, we find the sys_rand_get() function that provides the functionality we’re looking for.

With this, we can glue everything together by implementing the required getentropy()function in the Stubs.c file.

#include <zephyr/random/random.h>⋮int getentropy(void *buffer, size_t length) {  sys_rand_get(buffer, length);  return 0;}

We also make sure we’re telling Zephyr to use one of the available random number generators by setting the proper configuration option in prj.conf

CONFIG_TEST_RANDOM_GENERATOR=y

Let’s build and flash what we have so far. Connecting to the board, we can see random digits being generated

*** Booting nRF Connect SDK v2.7.0-5cb85570ca43 ****** Using Zephyr OS v3.6.99-100befc70c74 ***Digit: 4Digit: 9Digit: 9Digit: 4

Better randomness

Although the values printed out appear random, the documentation for our CONFIG_TEST_RANDOM_GENERATORparameter indicates that

For testing, this option allows a non-random number generator to be used and permits random number APIs to return values that are not truly random.

To improve this, let’s adjust our prj.conf file and select another generator

CONFIG_ENTROPY_DEVICE_RANDOM_GENERATOR=y

With this change, our project fails to build

/opt/nordic/ncs/toolchains/f8037e9b83/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: app/libapp.a(Stubs.c.obj): in function `sys_rand_get':/.../EmbeddedSwift-nRF52-Examples/Random/build/zephyr/include/generated/syscalls/random.h:37: undefined reference to `z_impl_sys_rand_get'

Taking a closer look at theCONFIG_ENTROPY_DEVICE_RANDOM_GENERATOR documentation, it states that it uses the enabled hardware entropy gathering driver to generate random numbers.
It’s not explicit but reading between the lines, we can deduct that we should enable the entropy driver, which is done by adding this configuration option.

CONFIG_ENTROPY_GENERATOR=y

And with this, our project builds again and we can flash it to our board to test.

Conclusion

In this post, I’ve shown you how to generate random numbers in Embedded Swift when using the nRF Connect SDK. The code is available on GitHub.

In our test program, we used Int.random(in:)to generate a random integer. But of course we can also use other randomness-based functions, like

let array = ["Jobs", "Wozniak"]let element = array.randomElement()

And as indicated in the Embedded Swift — User Manual, you can now also use dictionaries and sets

let dictionary = [1: "Jobs", 2: "Wozniak"]let set : Set<Int> = [1, 2, 3, 3, 2]

Dictionaries ?

let dictionary = ["name": "Jobs"]

Really ?

Main.swift.obj:(.text.$ss7UnicodeO14_NFDNormalizerV7_resume9consumingAB6ScalarV6scalar_AB9_NormDataV04normH0tSgAHSgyXE_tF+0x1d4): undefined reference to `_swift_stdlib_getNormData'

Well that’s another problem not related to random number generation that we’ll address in a future post.

Creating a Swift type for button input on nRF52 - Part 2

Mon, 14 Oct 2024 00:00:00 +0200

Introduction

In the last post, Creating a Swift type for button input on nRF52 - Part 1, we worked on code to abstract interacting with a button in Swift code. We had a working prototype, but things fell apart as soon as we cleaned up the code and encapsulated it in a struct.

Investigating the issue

Let’s see if there’s valuable information in the documentation.

/** * @brief GPIO callback structure * * Used to register a callback in the driver instance callback list. * As many callbacks as needed can be added as long as each of them * are unique pointers of struct gpio_callback. * Beware such structure should not be allocated on stack. * * Note: To help setting it, see gpio_init_callback() below */struct gpio_callback {/** This is meant to be used in the driver and the user should not * mess with it (see drivers/gpio/gpio_utils.h) */sys_snode_t node;/** Actual callback function being called when relevant. */gpio_callback_handler_t handler;/** A mask of pins the callback is interested in, if 0 the callback * will never be called. Such pin_mask can be modified whenever * necessary by the owner, and thus will affect the handler being * called or not. The selected pins must be configured to trigger * an interrupt. */gpio_port_pins_t pin_mask;};

The documentation for the definition of the gpio_callback gives an interesting piece of information.
Can you spot it?

Beware such structure should not be allocated on stack.

Let’s look at our Swift code again

struct Button {  var pin_cb_data = gpio_callback()...

Here we’re creating a gpio_callback as a stored property of a Swift struct. That struct being a value type, it will be allocated on the stack, which the above comment explicitly advises against.
An easy fix is to make the Button a class, which as a reference type, will be allocated on the heap.

Info

The idea that value types are always created on the stack and reference types on the heap is not entirely true. I have not yet been able to find an exact rule but the approximation works for now and for this particular case.

A safer way (which would also allow to keep our Button as a struct) is to explicitly allocate the gpio_callback structure on the heap. This however opens a can of worms that is too much for this article. But it’s a topic we’ll revisit later and in great details.

Beyond printing a message

The code is now working again, but all it does is print a message when the button is pressed. Let’s move to something slightly more interesting and, re-using code from our first example (Controlling an LED using Embedded Swift on nRF52), make the button toggle an LED.

We’ll use a simplified version of the Led code, omitting the error handling and not throwing any errors, as to make things more readable.

We can then update the closure passed to the button so it toggles the LED when the button is pressed.

struct Main {  static func main() {    let led = Led(gpio: &led0)    let myButton = Button(gpio: &button) { _, _, _ in      led.toggle()    }...

But this makes the build fail

…/EmbeddedSwift-nRF52-Examples/Button-and-LED/Main.swift:52:42: error: a C function pointer cannot be formed from a closure that captures context 50 |     let led = Led(gpio: &led0) 51 |  52 |     let myButton = Button(gpio: &button) { _, _, _ in    |                                          `- error: a C function pointer cannot be formed from a closure that captures context 53 |       led.toggle() 54 |     }

As the error message indicates, the closure cannot capture references to variables from its context.
We have two main possibilities to fix this:

make the variable that we want to reference globally available
somehow pass the reference to the C function

Let’s explore these 2 options in more details.

Making our led variable globally available

There are multiple ways to achieve this.

Global variable

The first and easiest way is to just make it a global variable for the module.

Moving the let led = Led(gpio: &led0) declaration outside of the Main struct makes it available from within the closure.

The code would look something like this

let led = Led(gpio: &led0)@mainstruct Main {  static func main() {    let myButton = Button(gpio: &button) { _, _, _ in      led.toggle()    }...

This builds and works fine, and is OK for simple cases but it’s definitely not the best code organisation and does not scale very well.

Static variable on a type

Instead, we can define a static constant property on the existing Led type

struct Led {  static let firstLed = Led(gpio: &led0)...

and refer to it from the closure

let myButton = Button(gpio: &button) { _, _, _ in  Led.firstLed.toggle()}

If you don’t want to modify the Led type and keep it generic (for instance because it’s coming from an external package), you can create a dedicated type (an enum in this case) to hold the static constants.

enum Leds {  static let firstLed = Led(gpio: &led0)}

let myButton = Button(gpio: &button) { _, _, _ in  Leds.firstLed.toggle()}

Application logic singleton

In all the above examples, we’ve used different strategies to allow global access to a single definition of the Led constant.

If we would have multiple elements to expose, or we would want to encapsulate those elements behind some more advanced logic, we can create a class to host that logic and make it a singleton.
The singleton pattern ensures that the class would only have one instance and provides a global access point to that instance.

The code could look something like this.

class AppLogic {    static let shared = AppLogic()    private init() { }    let firstLed = Led(gpio: &led0)    func toggleLed() {      firstLed.toggle()    }}

With the button closure using the logic exposed by the singleton.

let myButton = Button(gpio: &button) { _, _, _ in  AppState.shared.toggleLed()}

Info

In the above implementation of the singleton, the AppLogic instance is only created when the shared static property is first accessed. And so is the initialisation of our Led instance. This means this only happens on the first button press.
If this is not acceptable for some reason, you could always read the shared property in your main function to trigger the instantiation earlier.

Passing the reference to the C function

Exploring the possibilities

The general idea is that the required information comes via the parameter(s) of the C function (closure) that’s being called. As we’re implementing a callback whose prototype is defined by the Zephyr SDK, we cannot add the parameter ourself.
But is there already something in the API that we can use? Let’s take another look at the callback prototype.

/** * @typedef gpio_callback_handler_t * @brief Define the application callback handler function signature * * @param port Device struct for the GPIO device. * @param cb Original struct gpio_callback owning this handler * @param pins Mask of pins that triggers the callback handler * * Note: cb pointer can be used to retrieve private data through * CONTAINER_OF() if original struct gpio_callback is stored in * another private structure. */typedef void (*gpio_callback_handler_t)(const struct device *port,struct gpio_callback *cb,gpio_port_pins_t pins);

Do you see the note in the documentation ? The API has foreseen a way to pass private data to this handler. Let’s try to figure out how that works.

Searching the Zephyr documentation for CONTAINER_OF, we find some information in Zephyr API Documentation: Utility Functions. The basic idea is that given an element of a struct, we’re able to get back to the containing struct.

Let’s look at the example from the documentation

 struct foo {    int bar; }; struct foo my_foo; int *ptr = &my_foo.bar; struct foo *container = CONTAINER_OF(ptr, struct foo, bar);

Here we see that, knowing the address of bar, an element of my_foo, the CONTAINER_OF macro will return the address of that my_foo struct.

If the struct foo would contain a struct gpio_callback instead of an int, that would read

 struct foo {    struct gpio_callback cb; }; struct foo my_foo; int *ptr = &my_foo.cb; struct foo *container = CONTAINER_OF(ptr, struct foo, cb);

We’re getting to a containing structure from the gpio_callback element, which is one of the parameters we receive in our GPIO handler function.

Understanding CONTAINER_OF

Looking at the definition of CONTAINER_OF, we see it’s a “complex” macro

#define CONTAINER_OF(ptr, type, field)                               \({                                                           \CONTAINER_OF_VALIDATE(ptr, type, field)              \((type *)(((char *)(ptr)) - offsetof(type, field))); \})

and we’ve seen in the previous post that Swift C Interop only supports simple macros.

Instead of trying to see how we could call this macro from Swift, let’s understand what it does and re-implement the same behaviour using Swift native constructs.

Setting aside some validation, the macro implements some basic pointer arithmetics. Considering the containing struct is represented by a contiguous block of memory, knowing the address of one of its property and the offset of that property from the start, by subtracting that offset from the address we get back the starting address of the memory block i.e. the address of the containing structure.

Let’s extend our foo structure with some int property, create a variable of that type and see how the maths work

struct foo {   int bar;   struct gpio_callback cb;};struct foo my_foo;int *ptr = &my_foo.cb;

Let’s pretend my_foo is at address 0x00001000. The addresses of the structure look like

Address	Element
0x00001000	bar
0x00001004	cb

ptr is the address of cb and would have value 0x00001004. offsetof(struct foo, cb) would return 4. Subtracting that from ptr gives 0x00001000, which is indeed the address of my_foo.

Pointer arithmetics in Swift

Swift provides several types for Manual Memory Management, you can spot them by their name following the Unsafe…Pointer pattern.
Of the different flavours available, UnsafeRawPointer is one of the simplest, providing untyped access to raw bytes in memory. It corresponds to a void * pointer in C.
Raw pointers support basic arithmetic operations at the byte level. When you add to or subtract from a raw pointer, the result is a new raw pointer offset by that number of bytes.
Using UnsafeRawPointerwill allow us to implement the above subtraction from our CONTAINER_OF formula.

What about the offsetof() call ? This is an ANSI C macro that evaluates to the offset (in bytes) of a given member within a struct or union type.
It happens that Swift provides the same functionality with the offset(of:) function of the MemoryLayout type.

Finally, once done with the pointer arithmetic, we would like to “cast” the result to the type of our container structure. We can achieve that using the assumingMemoryBound(to:) on our UnsafeRawPointer, that will return us an UnsafePointer typed to the requested type.

Putting it all together

In our project, we wanted the Led to be available with the button handler closure. We thus create a container structure, that in addition to the original callback structure, contains our Led.

struct ExtendedCallback {  var callback: gpio_callback  var led: Led}

We then add, on that structure, a function to perform the pointer arithmetic, and return, given a pointer to the callback element, the ExtendedCallback.

struct ExtendedCallback {  var callback: gpio_callback  var led: Led  static func containerFrom(callback ptr: UnsafePointer<gpio_callback>) -> ExtendedCallback {    let containerPtr = UnsafeRawPointer(ptr) - MemoryLayout.offset(of: \Self.callback)!    return containerPtr.assumingMemoryBound(to: Self.self).pointee  }}

With that in place, we can now access the Led from within the button closure.

let myButton = Button(gpio: &button) { _, callback, _ in  ExtendedCallback.containerFrom(callback: callback!).led.toggle()}

Of course, we need to properly initialise this property. We edit our Button class so that, the pin_cb_data property is initialized with an ExtendedCallback, encapsulating both our initial gpio_callbackand our Led.

var pin_cb_data = ExtendedCallback(callback: gpio_callback(), led: Led(gpio: &led0))

Within our initializer, we need to adapt the calls that are taking the gpio_callback as parameters, as we still need to pass that original structure and not the container.

gpio_init_callback(&pin_cb_data.callback, self.btnHandler, bit(gpio.pointee.pin))gpio_add_callback(gpio.pointee.port, &pin_cb_data.callback)

With that in place, we can build the project and test that indeed, the LED toggles as the user pushes the button.

We’ve been able to use this strategy here because the API was designed in such a way that some object accessible to us during initialization, was propagated to the callback. This will not always be the case and you’ll need to adapt based on the specific use case.

Making it generic

There are 2 shortcomings to the above code:

we have hardcoded in the ExtendedCallback structure that the extra information is a Led
that Led is instantiated as part of the Button code

Let’s use generics to get rid of those limitations.

Info

Embedded Swift supports generics but uses specialisation i.e. the generic type is known at compile-time at the caller site and the compiler creates a specialized instantiation of the generic type that is no longer generic.

Generic instantiation at run-time is not supported, as that would require access to type metadata, which does not exist in Embedded Swift.

We start by making the extra parameter generic

struct ExtendedCallback<T> {  var callback: gpio_callback  var context: T...

We also need to modify the Button class in accordance

class Button<T> {  let gpio: UnsafePointer<gpio_dt_spec>  let btnHandler: GpioCallbackHandler  var pin_cb_data: ExtendedCallback<T>  init(gpio: UnsafePointer<gpio_dt_spec>, context: T, btnHandler: GpioCallbackHandler) {    self.gpio = gpio    self.btnHandler = btnHandler    self.pin_cb_data = ExtendedCallback(callback: gpio_callback(), context: context)...

We make the class generic, the pin_cb_data is now the generic version of ExtendedCallback and that property is initialized in the initializer, receiving the context as a parameter.

The closure needs a slight update, as ExtendedCallback must be specialized for us to call its the static function.

let myButton = Button<Led>(gpio: &button, context: Led(gpio: &led0)) { _, callback, _ in  ExtendedCallback<Led>.containerFrom(callback: callback!).context.toggle()}

Conclusion

In this post, we fixed the issue we encountered last time when encapsulating our logic within a Button type.
We then encountered a limitation of closures used as C function pointers and explored several strategies on how the closure can access information defined outside of its scope.

With these core concepts covered in this post and the previous two, we have some building blocks to start tackling more complex and interesting projects.
Stay tuned as we’ll cover that in a future post.

Creating a Swift type for button input on nRF52 - Part 1

Thu, 26 Sep 2024 00:00:00 +0200

Introduction

In my previous post, Controlling an LED using Embedded Swift on nRF52, we created a Swift struct to encapsulate the C code required to control an LED. In this post, we’ll do the same for a button.

If you want to follow along, the code is available on GitHub with each step having its own commit.

Starting from working code

As a starting point, we will write the whole code in C. This code is based on the Lesson 2 - Exercise 2 sample code from the nRF Connect SDK Fundamentals course from the Nordic Developer Academy.

In the button.c file, we define a single function that performs the whole setup.

#include <stdio.h>#include <zephyr/drivers/gpio.h>#include "button.h"#define SW0_NODE DT_ALIAS(sw0)static struct gpio_dt_spec button = GPIO_DT_SPEC_GET(SW0_NODE, gpios);static struct gpio_callback pin_cb_data;void setupButton() {  int ret;  if (!gpio_is_ready_dt(&button)) {    return;  }  ret = gpio_pin_configure_dt(&button, GPIO_INPUT);  if (ret < 0) {    return;  }  gpio_pin_interrupt_configure_dt(&button, GPIO_INT_EDGE_TO_ACTIVE);  void pin_isr(const struct device *dev, struct gpio_callback *cb, uint32_t pins) {    printk("Button pressed\n");  }  gpio_init_callback(&pin_cb_data, pin_isr, BIT(button.pin));  gpio_add_callback(button.port, &pin_cb_data);}

The button.h file only contains the prototype of the setup function (and a single include that is required by both the C and Swift code).

Info

In C, a function prototype defines the function name, its parameter(s) and return type.
It’s very similar to a method signature in Swift.

#include <zephyr/kernel.h>void setupButton();

And the BridgingHeader.h file makes it all visible to our Swift code.

#include <autoconf.h>#include "button.h"

All the Swift code has to do for now, is call that single function.

@mainstruct Main {  static func main() {    setupButton()    while true {      k_msleep(100)    }  }}

Try building and running the code on your hardware at this point. Do you see the expected message in your console when the button is pressed?

Chi va piano va sano

Let’s follow that Italian saying (that translates to “Slow and steady wins the race” ) and begin transforming each section into Swift, step by step; making sure the code compiles and runs at each step.

We first start by moving the static variable referencing the GPIO used for the button directly to the BridgingHeader.h file.

#include <autoconf.h>#include <zephyr/kernel.h>#include <zephyr/drivers/gpio.h>#define SW0_NODE DT_ALIAS(sw0)static struct gpio_dt_spec button = GPIO_DT_SPEC_GET(SW0_NODE, gpios);

And then we check the GPIO is ready. This is similar to what we did for the LED, so it works as expected.

  static func main() {    guard gpio_is_ready_dt(&button) else { return }    while true {      k_msleep(100)    }  }

Next, we configure the pin, similar to the LED configuration, but this time as an input.

  static func main() {    guard gpio_is_ready_dt(&button) else { return }    let ret = gpio_pin_configure_dt(&button, GPIO_INPUT)    guard ret == 0 else { return }    while true {      k_msleep(100)    }  }

The next step is to configure the pin so it triggers when it transitions to an active state.

  static func main() {    guard gpio_is_ready_dt(&button) else { return }    let ret = gpio_pin_configure_dt(&button, GPIO_INPUT)    guard ret == 0 else { return }    gpio_pin_interrupt_configure_dt(&button, GPIO_INT_EDGE_TO_ACTIVE)    while true {      k_msleep(100)    }  }

and this causes a build error

…/EmbeddedSwift-nRF52-Examples/Button/Main.swift:18:50: error: cannot find 'GPIO_INT_EDGE_TO_ACTIVE' in scope16 |       let ret = gpio_pin_configure_dt(&button, GPIO_INPUT)17 |       if ret == 0 {18 |         gpio_pin_interrupt_configure_dt(&button, GPIO_INT_EDGE_TO_ACTIVE)   |                                                  `- error: cannot find 'GPIO_INT_EDGE_TO_ACTIVE' in scope19 |       } else {20 |         print("Can't configure the GPIO as input")/opt/nordic/ncs/v2.7.0/zephyr/include/zephyr/drivers/gpio.h:202:9: note: macro 'GPIO_INT_EDGE_TO_ACTIVE' unavailable: structure not supported 200 |  * level 1 and enables it. 201 |  */ 202 | #define GPIO_INT_EDGE_TO_ACTIVE        (GPIO_INT_ENABLE | \     |         `- note: macro 'GPIO_INT_EDGE_TO_ACTIVE' unavailable: structure not supported 203 |                                  GPIO_INT_LEVELS_LOGICAL | \ 204 |                                  GPIO_INT_EDGE | \ninja: build stopped: subcommand failed.

The Swift compiler is chocking on the GPIO_INT_EDGE_TO_ACTIVE macro, let’s check its definition.

/** Configures GPIO interrupt to be triggered on pin state change to logical * level 1 and enables it. */#define GPIO_INT_EDGE_TO_ACTIVE        (GPIO_INT_ENABLE | \GPIO_INT_LEVELS_LOGICAL | \GPIO_INT_EDGE | \GPIO_INT_HIGH_1)

Looking at the Swift C Interop documentation from Apple, Using Imported C Macros in Swift confirms that only simple macros are supported.

This macro is just defining a constant, something we can easily redo in a more Swifty way.

enum GpioInterrupts {  static let edgeToActive = GPIO_INT_ENABLE | GPIO_INT_LEVELS_LOGICAL | GPIO_INT_EDGE | GPIO_INT_HIGH_1}

and use that in our call

    gpio_pin_interrupt_configure_dt(&button, GpioInterrupts.edgeToActive)

Handling the handler

The next step is about setting up the code that runs when the button is pressed.
Let’s take a look back at how this is done in the C code.

  void pin_isr(const struct device *dev, struct gpio_callback *cb, uint32_t pins) {    printk("Button pressed\n");  }  gpio_init_callback(&pin_cb_data, pin_isr, BIT(button.pin));

This code declares a function to be executed on button press and uses it in the gpio_init_callback()call. Let’s look at the prototype of this function.

/** * @brief Helper to initialize a struct gpio_callback properly * @param callback A valid Application's callback structure pointer. * @param handler A valid handler function pointer. * @param pin_mask A bit mask of relevant pins for the handler */static inline void gpio_init_callback(struct gpio_callback *callback,      gpio_callback_handler_t handler,      gpio_port_pins_t pin_mask)

The most interesting part for us at this stage is the handlerparameter, let’s look at its type.

/** * @typedef gpio_callback_handler_t * @brief Define the application callback handler function signature * * @param port Device struct for the GPIO device. * @param cb Original struct gpio_callback owning this handler * @param pins Mask of pins that triggers the callback handler * * Note: cb pointer can be used to retrieve private data through * CONTAINER_OF() if original struct gpio_callback is stored in * another private structure. */typedef void (*gpio_callback_handler_t)(const struct device *port,struct gpio_callback *cb,gpio_port_pins_t pins);

As, in Swift, we’ll want to define a closure to be passed to this function, let’s define a type alias similar to the C typedef.

typealias GpioCallbackHandler = @convention(c) (  _ port: UnsafePointer<device>?,  _ callback: UnsafeMutablePointer<gpio_callback>?,  _ pins: UInt32) -> Void

Notice the @convention(c) attribute, indicating C function pointer calling convention.
Also notice that the constkeyword in C results in the usage of an UnsafePointer vs an UnsafeMutablePointer when it’s not present.
Finally, notice the usage of optionals, as nothing in C ensures that those parameters cannot be NULL.

With that in place, we can define our closure and call the init function.

  static func main() {...    let btnHandler: GpioCallbackHandler? = { _, _, _ in      print("Button pressed")    }    var pin_cb_data = gpio_callback()    gpio_init_callback(&pin_cb_data, btnHandler, BIT(button.pin))...

However this code fails to build, we’re facing another macro issue.

…/EmbeddedSwift-nRF52-Examples/Button/Main.swift:54:50: error: cannot find 'BIT' in scope52 |     var pin_cb_data = gpio_callback()53 | 54 |     gpio_init_callback(&pin_cb_data, btnHandler, BIT(button.pin))   |                                                  `- error: cannot find 'BIT' in scope55 | 56 |     while true {/opt/nordic/ncs/v2.7.0/zephyr/include/zephyr/sys/util_macro.h:44:9: note: macro 'BIT' unavailable: function like macros not supported 42 |  * assembly language). 43 |  */ 44 | #define BIT(n)  (1UL << (n))    |         `- note: macro 'BIT' unavailable: function like macros not supported 45 | #endif 46 | #endif

Similar to above, this type of macro is not supported, but as the name indicates (function-like), it can easily be replaced by a function.

func bit(_ n: UInt8) -> UInt32 {  UInt32(1) << n}

Using that in our call fixes the issue.

    gpio_init_callback(&pin_cb_data, btnHandler, bit(button.pin))

And we can add the final call required for our callback configuration.

gpio_add_callback(button.port, &pin_cb_data)

Building and flashing this code should give you the same behaviour as before, a message printed to the console on button press.

Warning

The call back is in fact an ISR (Interrupt Service Routine) and there are several constraints and best practices on the code that should be executed in an ISR. First and foremost to keep it fast.
Printing from an ISR is not advised and is used here for demonstration purposes.

Tidying up

During prototyping, we wrote all our code within the main function.
Now that we have working code, let’s encapsulate that in a dedicated struct.
The initializer allows to configure the pin the button is connected to and the closure to execute on button press.

struct Button {  let gpio: UnsafePointer<gpio_dt_spec>  let btnHandler: GpioCallbackHandler  var pin_cb_data = gpio_callback()  init(gpio: UnsafePointer<gpio_dt_spec>, btnHandler: GpioCallbackHandler) {    self.gpio = gpio    self.btnHandler = btnHandler    guard gpio_is_ready_dt(gpio) else { return }    let ret = gpio_pin_configure_dt(gpio, GPIO_INPUT)    guard ret == 0 else { return }    gpio_pin_interrupt_configure_dt(gpio, GpioInterrupts.edgeToActive)    gpio_init_callback(&pin_cb_data, self.btnHandler, bit(gpio.pointee.pin))    gpio_add_callback(gpio.pointee.port, &pin_cb_data)  }}

The closure being the last parameter, we can use a trailing closure at the calling site.

@mainstruct Main {  static func main() {    let myButton = Button(gpio: &button) { _, _, _ in      print("Button pressed")    }    while true {      k_msleep(100)    }  }}

This builds and can be flashed to the board, but nothing happens anymore when pressing the button!

Conclusion

In this post, we have seen how to work with a button (or more generally a digital input) in Swift code.

However, we’re left on a bitter note, as our try to encapsulate our logic in a reusable Swift type results in non-working code. In the next post, we’ll look at why this is and ways to fix it.

If that cliffhanger is too much for you, ping me on social media and I’ll give you a hint as to how to solve this.

Controlling an LED using Embedded Swift on nRF52

Sat, 14 Sep 2024 00:00:00 +0200

Introduction

In my previous post, nrfx-blink Step by Step, I took you through the steps required to configure your development environment to build and flash the “Blinky” Embedded Swift example from Apple on an nRF52840 DK. In this post, we’ll explore that example more deeply, grow from there, and adapt it to be more natural for Swift developers.

If you want to follow along, the code is available on GitHub with each step having its own commit.

Exploring the Apple provided project

Let’s open the project in VS Code and explore its content.

Info

For instructions on how to work with Embedded Swift projects in VS Code, see my previous post nrfx-blink Step by Step.

The nrfx-blink example project opened in VS Code

Let’s start by looking at the (start of the) CMakeLists.txt file.

cmake_minimum_required(VERSION 3.29)find_package(Zephyr REQUIRED HINTS $ENV{ZEPHYR_BASE})# Use the armv7em-none-none-eabi target triple for Swiftset(CMAKE_Swift_COMPILER_TARGET armv7em-none-none-eabi)# Enable "wmo" as needed by Embedded Swiftset(CMAKE_Swift_COMPILATION_MODE wholemodule)# FIXME: Skip checking if the compiler worksset(CMAKE_Swift_COMPILER_WORKS true)# Create a new project called "blinky" and enable "Swift" as a supported languageproject(blinky Swift)# Set global Swift compiler flagsadd_compile_options(    # Enable Embedded Swift    "$<$<COMPILE_LANGUAGE:Swift>:SHELL:-enable-experimental-feature Embedded>"

This is the configuration file for the CMake tool, which is some form of meta-build tool used by nRF Connect SDK projects. Based on the above file, it’ll generate the configuration for the final build tool, in this case Ninja.

From the extract above, you can notice some key configuration options related to Swift usage.`

cmake_minimum_required(VERSION 3.29)

This indicates the minimum version required for this build. Note that this is a higher version than the one included in the nRF Connect SDK toolchain (at least the current 2.7.0 version).

We then see a series of instructions related to Swift compilation, like enabling Swift support and setting compiler flags. Look at the comments in the CMakeFiles.txt file for more information.

Let’s now take a look at the Swift source code.

@mainstruct Main {  static func main() {    // Note: & in Swift is not the "address of" operator, but on a global variable declared in C    // it will give the correct address of the global.    gpio_pin_configure_dt(&led0, GPIO_OUTPUT | GPIO_OUTPUT_INIT_HIGH | GPIO_OUTPUT_INIT_LOGICAL)    while true {      gpio_pin_toggle_dt(&led0)      k_msleep(100)    }  }}

While this may look like standard Swift code, there are calls to functions like gpio_pin_configure_dt and references to &led0and GPIO_OUTPUT, where do these from from ?

To get some idea about that, let’s look at the BridgingHeader.h file.

#include <autoconf.h>#include <zephyr/kernel.h>#include <zephyr/drivers/gpio.h>#define LED0_NODE DT_ALIAS(led0)static struct gpio_dt_spec led0 = GPIO_DT_SPEC_GET(LED0_NODE, gpios);

This is a standard C header file that’s used to expose C functions (or C code in general) to Swift. If you look in CMakeLists.txt, you’ll see the -import-bridging-header flag referencing this file being passed to the Swift compiler.

In the header file above, we can see we’re importing the headers for the Zephyr SDK (on which nRF Connect SDK is based) and we’re declaring a static variable led0 (with the help of the macros provided by the SDK).

We can use those C constructs in our Swift code thanks to the Swift C Interop, which is part of the language since day one. You can find more information about this in C Interoperability , in the Apple Developer Documentation.

Taking another look at our main function, it doesn’t feel very “Swifty”, in fact it’s mostly glue code around C calls.

Creating a basic struct

To make the code more “Swifty” and abstract the C-level details, let’s create a Led struct that isolates the C calls and provides a clean Swift interface to its caller.

struct Led {  init() {    // Note: & in Swift is not the "address of" operator, but on a global variable declared in C    // it will give the correct address of the global.    gpio_pin_configure_dt(&led0, GPIO_OUTPUT | GPIO_OUTPUT_INIT_HIGH | GPIO_OUTPUT_INIT_LOGICAL)  }  func toggle() {     gpio_pin_toggle_dt(&led0)  }}

With that in place, our main function reads much more like typical Swift code, instantiating a struct and calling a method on it.

@mainstruct Main {  static func main() {    let led = Led()    while true {      led.toggle()      k_msleep(100)    }  }}

Making it configurable

This version however, hardcodes the GPIO pin that is used to drive the LED. That doesn’t make for re-usable code.
To make it configurable, we need to pass the GPIO pin driving the LED to our struct initializer.

Looking back at the declaration of the led0 variable

static struct gpio_dt_spec led0 = GPIO_DT_SPEC_GET(LED0_NODE, gpios);

and the way it’s used in the calls

gpio_pin_toggle_dt(&led0)

we see we’re using a C pointer to a C struct.

The Calling Functions With Pointer Parameters page of the Apple Developer Documentation gives some explanations on the topic and points to using UnsafePointer, as shown in the code below.

struct Led {  let gpio: UnsafePointer<gpio_dt_spec>  init(gpio: UnsafePointer<gpio_dt_spec>) {    self.gpio = gpio    // Note: & in Swift is not the "address of" operator, but on a global variable declared in C    // it will give the correct address of the global.    gpio_pin_configure_dt(gpio, GPIO_OUTPUT | GPIO_OUTPUT_INIT_HIGH | GPIO_OUTPUT_INIT_LOGICAL)  }  func toggle() {     gpio_pin_toggle_dt(gpio)  }}

And in our main function, we can just initialize our struct using

let led = Led(gpio: &led0)

We now have a re-usable struct that can be used to control an LED connected to any of the GPIO of the system.

Handling errors

When following the nRF Connect SDK Fundamentals training from the Nordic Developer Academy, one can read that “before using the device pointer contained in gpio_dt_spec, we need to check if it’s ready using gpio_is_ready_dt()”.

Let’s create an error to represent that condition

enum LedError: Error {  case notReady}

and perform the test in our initializer, making it throwing

struct Led {  let gpio: UnsafePointer<gpio_dt_spec>  init(gpio: UnsafePointer<gpio_dt_spec>) throws {    if (!gpio_is_ready_dt(gpio)) {      throw LedError.notReady    }  …}

For now, we’ll just ignore the thrown error

@mainstruct Main {  static func main() {    let led = try? Led(gpio: &led0)    while true {      if let led {        led.toggle()      }      k_msleep(100)    }  }}

Info

As you can see above, throwing initializers are nice as you can also use them as failable initializers if you’re not interested in the exact error that occurred.

And the build fails !

…/EmbeddedSwift-nRF52-Examples/LED/Main.swift:30:22: error: cannot use a value of protocol type 'any Error' in embedded Swift10 | //===----------------------------------------------------------------------===//11 | 12 | @main   | `- note: called from here13 | struct Main {14 |   static func main() {15 |     let led = try? Led(gpio: &led0)   |                    `- note: called from here16 |     while true {17 |       if let led {   :28 |   init(gpio: UnsafePointer<gpio_dt_spec>) throws {29 |     if (!gpio_is_ready_dt(gpio)) {30 |       throw LedError.notReady   |                      `- error: cannot use a value of protocol type 'any Error' in embedded Swift31 |     }32 | ninja: build stopped: subcommand failed.FATAL ERROR: command exited with status 1: /opt/nordic/ncs/toolchains/f8037e9b83/bin/cmake --build …/EmbeddedSwift-nRF52-Examples/LED/build

The error mentions any Error, that is because a simple throws statement is identical to throws any Error. And that’s an existential type, which is not supported by Embedded Swift.

This is easily fixed by using Typed throws, introduced with Swift 6.

  init(gpio: UnsafePointer<gpio_dt_spec>) throws(LedError) {    if (!gpio_is_ready_dt(gpio)) {      throw .notReady    }…

As one can see above, this has the added benefit that we don’t need to specify the type at the throw site, as it’s defined in the throws declaration.

Really handling errors

In the code above, we’ve just ignored the thrown error, let’s now catch it and print an error message in case of error.

@mainstruct Main {  static func main() {    do {      let led = try Led(gpio: &led0)      while true {        led.toggle()        k_msleep(100)      }    } catch {      print("Could not initialize LED")    }  }}

And the build fails again !

[3/7] Linking C executable zephyr/zephyr_pre0.elfFAILED: zephyr/zephyr_pre0.elf zephyr/zephyr_pre0.map …/EmbeddedSwift-nRF52-Examples/LED/build/zephyr/zephyr_pre0.map : && ccache /opt/nordic/ncs/toolchains/f8037e9b83/opt/zephyr-sdk/arm-zephyr-eabi/bin/arm-zephyr-eabi-gcc  -gdwarf-4 zephyr/CMakeFiles/zephyr_pre0.dir/misc/empty_file.c.obj -o zephyr/zephyr_pre0.elf  zephyr/CMakeFiles/offsets.dir/./arch/arm/core/offsets/offsets.c.obj  -fuse-ld=bfd  -T  zephyr/linker_zephyr_pre0.cmd  -Wl,-Map=…/EmbeddedSwift-nRF52-Examples/LED/build/zephyr/zephyr_pre0.map  -Wl,--whole-archive  app/libapp.a  zephyr/libzephyr.a  zephyr/arch/common/libarch__common.a  zephyr/arch/arch/arm/core/libarch__arm__core.a  zephyr/arch/arch/arm/core/cortex_m/libarch__arm__core__cortex_m.a  zephyr/arch/arch/arm/core/mpu/libarch__arm__core__mpu.a  zephyr/lib/libc/picolibc/liblib__libc__picolibc.a  zephyr/lib/libc/common/liblib__libc__common.a  zephyr/soc/soc/nrf52840/libsoc__nordic.a  zephyr/drivers/clock_control/libdrivers__clock_control.a  zephyr/drivers/console/libdrivers__console.a  zephyr/drivers/gpio/libdrivers__gpio.a  zephyr/drivers/pinctrl/libdrivers__pinctrl.a  zephyr/drivers/serial/libdrivers__serial.a  zephyr/drivers/timer/libdrivers__timer.a  modules/nrf/drivers/hw_cc3xx/lib..__nrf__drivers__hw_cc3xx.a  modules/hal_nordic/nrfx/libmodules__hal_nordic__nrfx.a  modules/segger/libmodules__segger.a  -Wl,--no-whole-archive  zephyr/kernel/libkernel.a  -L"/opt/nordic/ncs/toolchains/f8037e9b83/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/thumb/v7e-m/nofp"  -L…/EmbeddedSwift-nRF52-Examples/LED/build/zephyr  -lgcc  zephyr/arch/common/libisr_tables.a  -mcpu=cortex-m4  -mthumb  -mabi=aapcs  -mfp16-format=ieee  -mtp=soft  -Wl,--gc-sections  -Wl,--build-id=none  -Wl,--sort-common=descending  -Wl,--sort-section=alignment  -Wl,-u,_OffsetAbsSyms  -Wl,-u,_ConfigAbsSyms  -nostdlib  -static  -Wl,-X  -Wl,-N  -Wl,--orphan-handling=warn  -Wl,-no-pie  -DPICOLIBC_LONG_LONG_PRINTF_SCANF  /opt/nordic/ncs/v2.7.0/nrfxlib/crypto/nrf_cc310_platform/lib/cortex-m4/soft-float/no-interrupts/libnrf_cc310_platform_0.9.19.a  --specs=picolibc.specs  -lc  -lgcc && cd …/EmbeddedSwift-nRF52-Examples/LED/build/zephyr && /opt/homebrew/Cellar/cmake/3.30.2/bin/cmake -E true/opt/nordic/ncs/toolchains/f8037e9b83/opt/zephyr-sdk/arm-zephyr-eabi/bin/../lib/gcc/arm-zephyr-eabi/12.2.0/../../../../arm-zephyr-eabi/bin/ld.bfd: warning: orphan section `.swift_modhash' from `app/libapp.a(Main.swift.obj)' being placed in section `.swift_modhash'collect2: error: ld returned 1 exit statusninja: build stopped: subcommand failed.FATAL ERROR: command exited with status 1: /opt/nordic/ncs/toolchains/f8037e9b83/bin/cmake --build …/EmbeddedSwift-nRF52-Examples/LED/build

This time however, the error is not coming from the Swift compiler but from a later phase in the process, and it’s the linker that’s complaining, with a very explicit “ld returned 1” error message.

At this point I was very puzzled and this got me into a rabbit hole of trying to understand more about the build system and how to get more verbose information; the different stages of the build process with compilers, linkers, linker scripts, sections… and many more low-level aspects.
I grew my general understanding but could not find the solution.

Then one day, I checked again the swift-embedded-examples repository from Apple and saw that Kuba Mracek, the engineer that presented the Embedded Swift session during WWDC 2024), had made a commit on the project.

I updated my project with his changes and it worked!

The bit of the commit that fixed the issue is

    # Disable PIC    "$<$<COMPILE_LANGUAGE:Swift>:SHELL:-Xcc -fno-pic>"

pic stands for Position-Independent Code and although at a very high-level, after having read more on the topic, I think I understand why it might cause issues, I doubt I would have found the solution by myself (in a bounded amount of time); more on this in the conclusion.

Anyway, with that in place, we can put the finishing touches and handle all potential errors thrown by the C code. You’ll find that implementation in the example code on GitHub.

Conclusion

In this post, I have shown how, starting from Apple sample code, we can create a Swift struct that encapsulate an LED behaviour and make the code that uses it feel natural to Swift developers.

I’ve shown some of the challenges I faced during that implementation and how I overcame them. Sometimes it’s about learning more on topics that are less common to typical iOS or macOS development. Sometimes it took persistence and some fortunate timing to find the right solution; but don’t be too stubborn (like I sometime was) and reach out for help.
Embedded Swift is still in its infancy, I would encourage you to check the Embedded topics on the swift.org forums or the blogs and projects from the community.

Now that we’ve encapsulated an LED, the next step is adding some interactivity. In the next post of this series, we’ll explore how to make button code more Swift-friendly.

Our next challenge

nrfx-blink Step by Step

Wed, 04 Sep 2024 00:00:00 +0200

Introduction

When getting started with Embedded Swift, the examples published by Apple are really an incredible resource. However they’re just that — examples, right to the point and sometimes with minimal information. If you’re new to embedded development, you’ll need to do some additional digging (which isn’t too bad, that’s how you learn).

Embedded Swift development is very specific to the chipset that you’re targeting. As you can see, there are several examples in the repository, targeted at different boards and using different options (bare metal vs SDK/RTOS).

I’ve been mostly using the nRF52840 chip and in this post, I’ll be using an nRF52840 DK board, programming it using the nRF Connect SDK, which is based on the Zephyr RTOS project.

Nordic Semiconductor nRF52840 DK

This is the same board that the swift-embedded-examples/nrfx-blink-sdk example from Apple is using. The README file for that example mentions the pre-requisites to run the example, directly using Zephyr and the commands to use in a terminal to get the example running on your board.This works, and is a worthwhile test, but I don’t find it a viable way to proceed further with development.

I wanted to have the same experience in Swift that I had when developing in C with the nRF Connect SDK. That means working in an IDE (VS Code) with syntax highlighting, code completion, documentation… and a simple one-click build-and-flash mechanism.

In this post, I’ll walk you through the detailed steps to get there.

Pre-requisites

If you don’t have Xcode installed on your machine, start there. This will make sure Git is available (which is required by some components during the installation) and that you’ll be able to select the proper toolchain for Embedded Swift compilation.Also, just launch Xcode at least once to ensure all extra components are installed and you automatically accept the license agreement.

For this post, I worked on macOS Sonoma, using Xcode 15.4

Installing the native toolchain

Instead of installing the Zephyr SDK as suggested in the example, I’ll be working from the nRF Connect SDK, which is Nordic Semiconductor’s take on Zephyr and makes working with their board slightly easier.

They have a video tutorial for that, but I’ll go through all the steps here.

nRF Command Line Tools

Start by installing the nRF Command Line Tools, which you can download from the Nordic Semiconductor site.Scroll down to the Downloads section, select your platform (macOS in this case) and download the installer.

nRF Command Line Tools installer disk image

Launch the Install nRF Tools installer and proceed, keeping the proposed options.

nRF Command Line Tools installer options

J-Link software

In addition to the command line tools, the installer disk image also contains a version of the J-Link software from SEGGER. This is an older version and I prefer downloading a more recent one directly from the SEGGER website at SEGGER - The Embedded Experts - Downloads - J-Link / J-Trace.

SEGGER J-Link Software download

Select that latest version (with the same major version as the one bundled with the nRF tools, V7 at this time) and download the appropriate version for your OS.
The site displays the checksum for the download. It’s always a good idea to verify it.

This command achieves this in one go

diff -s <( printf '%s\n' "9161779d712d22341117a905901c0b12" ) <( md5 -q - JLink_MacOSX_V798e_arm64.pkg )

Once the signature checks out OK, proceed with the installation of the downloaded package.

VS Code and nRF Connect SDK extension

The next step is to install VS Code, download it from Download Visual Studio Code - Mac, Linux, Windows, extract the downloaded zip and move that to the /Applications folder.

Launch it, select the Extensions View from the Activity Bar on the left and search for “nrf connect”. You will have multiple matches, make sure you select “nRF Connect for VS Code Extension Pack” and click “Install”.

Install the nRF Connect for VS Code Extension Pack

Once the installation is finished, new icons appear in the Activity Bar, select the nRF Connect View.
The only thing the view should contain at this point is an “Install Toolchain” button, click on it to proceed.
From the list that appears, select the latest version of the toolchain available.

Select nRF Connect SDK Toolchain to install

Once the toolchain is installed, the view refreshes with several options. Select the “Manage SDKs” one, then “Install SDK” and select the latest stable SDK available; currently this is v2.6.1.

Select nRF Connect SDK to install

Info

If you don’t have Xcode installed, this will trigger an installer for the command line developer tools, so that Git is available. Watch for the installer icon appearing in the dock and its window behind the VS Code one.

Warning

I have sometimes seen the installation trigger a Git error and getting stuck at this point.

It that happens to you, cancel the installation and quit VSCode.
Open a terminal window and execute the following commands (this cleans the folder with a partial bogus installation. Adapt the folder name based on the SDK version you tried to install).

cd /opt/nordic/ncsrm -rf v2.6.1

Then enter this command, it’ll fix the Git issue encountered before

git config --global http.postBuffer 157286400

Re-launch VS Code and re-start the SDK installation, this time it should go through.

Git error during nRF Connect SDK installation

Info

An alternative method to install the nRF Connect SDK is to first install the nRF Connect for Desktop application, then install the Toolchain Manager, then install the SDK from there.
Using that method, I have not encountered the Git error mentioned above.

With these steps, we’re done installing the nRF Connect tools and SDK.

Installing the Swift snapshot toolchain

Embedded Swift requires a recent snapshot version of the Swift toolchain, which can be found at Swift.org - Download Swift. Download the package and install it.

After installation, perform a quick check; in a terminal, run

swift --version

This should display the standard swift version bundle with your installed Xcode, that would be 5.10 for Xcode 15.4

ls -l /Library/Developer/Toolchains

should list all the available toolchains, including the one you just installed. Running

plutil -extract CFBundleIdentifier raw /Library/Developer/Toolchains/swift-latest.xctoolchain/Info.plist

provides you with the ID of the latest installed toolchain; which you can use in your export statement, such as in

export TOOLCHAINS='org.swift.61202408261a'

Running

swiftc --version

should now reference the Swift version you just installed.

Info

For more information on installing and selecting a Swift toolchain, as well as potential gotchas, check out the great Building with nightly Swift toolchains on macOS post by Ole Begemann.

Installing Cmake

The nRF Connect toolchain installed via VS Code includes Cmake, but it’s currently an older version, one that does not support Swift.

You’ll need to install a newer version, which I found most easily done via Homebrew.

Info

If you don’t have Homebrew installed yet, head over to Homebrew — The Missing Package Manager for macOS (or Linux) and follow the instructions.

From a terminal, simply run

brew install cmake

and you should be good.

Build the blinky example via the terminal

Let’s verify that we can build an Embedded Swift program using the Apple provided examples.
Clone the repository from apple/swift-embedded-examples: A collection of example projects using Embedded Swift

git clone https://github.com/apple/swift-embedded-examples.git

Go into the nrfx-blink-sdk folder and run

export TOOLCHAINS='org.swift.61202408261a'cmake -B build -G Ninja -DBOARD=nrf52840dk_nrf52840 -DUSE_CCACHE=0 .cmake --build build

to build the example.

That should return a success. Now clean-up the build folder as we’ll re-create it from within VS Code

rm -rf build

In VS Code

From the menu, select “File” -> “Open Folder…” and select the nrfx-blink-sdk folder. Open the nRF Connect View from the Activity Ba* and under Appllications, click on “Add build configuration”.

Add a build configuration to the nrfx-blink-sdk project

The correct board (nrf52840dk_nrf52840) is already selected by default (adapt if you’re using a different board). Scroll down and click on “Build Configuration”.

This will fail, reporting an insufficient version of CMake. As mentioned above, the version included in the nRF toolchain is outdated and we need to use a newer one.

Build error caused by incorrect CMake version

As I did not find a way in VS Code / nRF Connect extension to specify the CMake version to use, I reverted to changing the CMake version within the nRF toolchain using a symbolic link, using the following commands (you will need to adapt paths if you don’t have the exact same versions as I do)

cd /opt/nordic/ncs/toolchains/f8037e9b83/Cellar/cmakemv 3.21.0 3.21.0.originalln -s /opt/homebrew/Cellar/cmake/3.30.2 3.21.0

Delete the build folder (this can be done from the Explorer view within VS Code) and re-add the build configuration. It still fails, but with a different error.

Build error caused by incorrect version of the Swift toolchain

The unknown argument ‘-import-bridging-header’ error is caused by the wrong version of the Swift toolchain being used. We need to use the same “export TOOLCHAINS” as we used above in our terminal examples.

This time, it can be fixed with configuration.
Open Settings (⌘, shortcut), make sure that Workspace (under the search box) is selected to apply changes to your project only and under Extensions, select the nRF DeviceTree entry. Scroll a little bit up and you’ll see a section for West: Env. Click on the “Edit in settings.json” button.

Access the nRF Connect settings within VS Code

This opens up your settings.json file. Add the definition of the TOOLCHAINS environment variable under the “nrf-connect.west.env” key. Your settings.json file should look something like

{    "nrf-connect.toolchain.path": "${nrf-connect.toolchain:2.6.1}",    "nrf-connect.west.env": {                "$base": "terminal",        "TOOLCHAINS": "org.swift.61202408261a"    }}

As above, delete the build folder, re-add the build configuration and now the build should be successful.

You can now use all the actions for the nRF Connect SDK extension within your Swift projects, including, if your setup supports it, flash and debug.

Info

This is the case for the nRF52840 DK board, but it might not always be the case. For instance, with a Seeed Studio XIAO nRF52840 (Sense), you’ll need an external debugger. See Using SWD pins on Seeed Studio XIAO nRF52840 for instructions on how to use an nRF52840 DK as the external debugger.

Working with Swift

We now have the nRF Connect part covered but Swift support is limited to the bare minimum. This is easily fixed by installing the Swift extension. Select the Extensions View from the Activity Bar on the left and search for “Swift”. Install the official extension from the Swift Server Work Group.

Install the Swift extension in VS Code

Once installed, configure the toolchain used by the Swift extension to be the same as the one used in your project. Open the Command Palette (⇧⌘P), type “Swift sel” and select “Swift: Select Toolchain…”. Select the toolchain matching the one you configured in your project. In the next step, select “Workspace Configuration” so the setting only applies to this project also. You’ll need to relaunch the extensions.

Select the proper Swift toolchain for the VS Code Swift extension

Once done, you’ll have syntax highlighting, code completion, code navigation, refactoring and then some.

Code completion for Swift code within VS Code

Conclusion

In this post, I’ve explained the steps required to configure a proper development environment for Embedded Swift targeting boards supported by the nRF Connect SDK.

With that in place, you’re in a good starting position to work on your ambitious Embedded Swift projects.

That’s what we’ll do in an upcoming series of posts, starting from the blink example and taking it much further.

Using SWD pins on Seeed Studio XIAO nRF52840

Sat, 17 Aug 2024 00:00:00 +0200

Introduction

When I discovered the Omi (previously Friend) project and wanted to build myself a device, I followed the instructions on their getting started page. For the firmware, this meant: open the project in VSCode, build, double-click reset on the nRF board, and drap-and-drop the zephyr.uf2 file into the flash drive that appeared on the computer.

As I wanted to make some changes to the firmware code, I wanted to improve on this process, most importantly, get feedback on what’s happening on the board, through console logs.

I thus started to look at how I could use the SWD pins present on the board.

TL;DR

It turns out you don’t need to do anything to get access to the console output. Simply plug your board into your computer and a new serial port should appear.Connecting to the port with a terminal application gives you access to the logs.

As to using those SWD pins, I did accomplish it using a nRF52840 DK, a Seeed Studio Expansion Board Base and the following cabling:

Final cabling connecting J-Link debugger to XIAO board

Warning

Do NOT connect the XIAO board via USB-C when using this configuration !

Where to connect those SWD pins ?

The Getting Started guide on the seeed studio wiki briefly mentions how to access the SWD Pins for Debugging and Reflashing Bootloader. But the information there is quite terse. And it points to using a J-Link device from Segger, the cheapest found on-line being around 400 €. That’s a bit too much for a hobby.

After some more research, I found using an nRF52840 DK was a possible alternative. Indeed its product page indicates

It comes with an on-board SEGGER J-Link debugger allowing programming and debugging both the on-board SoC and external targets through the debug out header.

That board can be found for less than 60 € from many on-line resellers and it’s also a great board to use standalone for experimentation (it has LEDs, buttons, is compatible with Arduino shields…). So I bought one.

But I still had no explanation of how to connect the SWD pins on the back of the XIAO board to the debug out header.

The answer in this post (VS Code nRF Connect extension “Connected Devices” not capturing my device. - Nordic Q&A - Nordic DevZone - Nordic DevZone) gave some pointers but mostly indicated that using a Seeed Studio Expansion Board Base for XIAO made things much easier, eliminating the need to solder directly the SWD pins on the nRF board. But still no clear connection diagram.

I did however order an expansion board, as well as header pins for the XIAO board.

And I continued doing more research (including in the Friend discord channel) and looked at the connectors on the board.

Expansion board SWD PINs, on the schema and on the board

Eventually, connecting this to the corresponding PINs on the nRF52840 DK, I ended-up with a cabling that looked like this

Initial cabling between nRF 52840 DK and expansion board

But when looking at the connected devices in the nRF Connect extension in VS Code, all I could see is the nRF52840 DK, not the XIAO board.

nRF52840 DK showing up instead of XIAO board

Finding the fix

I double-checked the cabling multiple times, re-read the articles I had found and searched for new ones.Eventually, I found this post, where in the first answer, somebody explains that, with the latest DK version (revision 3, which I have), you need to bridge the 2 pins between the existing cables. On my board, those are labeled VDD nRF and SWD SEL.

Bridging VDD nRF and SWD SEL pins

And with that in place, the XIAO board is now showing properly in VS Code.

XIAO board showing up as expected in VS Code

What can I do with this ?

As expected, I can now use the Flash command from the nRF Connect extension in VS Code to directly flash the XIAO board, no more double-clicking that ridiculously small reset button and copying a file.

I should also be able to debug and step through the program running on the board, inspect processor registers, … but this is something I haven’t looked at in details yet.

But one thing I could not do, is see the console logs, the COM ports appearing under the connected device being the ones from the DK, not the XIAO.

Where are my logs ?

I found the answer by chance, while looking at an introduction video to the XIAO nRF52840 board, where the presenter mentions that a serial port is created when you plug-in the board.

Info

As an aside, check out the WSN & IoT YouTube channel, there are quite a few interesting videos there, covering different topics related to embedded systems.

So I started a serial terminal app (I’m using SerialTools on the Mac) and checked the existing serial ports.

Serial ports on my Mac before connecting the XIAO board

I then plugged in the XIAO board and indeed a new port appeared. I could connect to it and there were my logs.

Connected to the XIAO board serial port and looking at the logs

Warning

In this post referenced earlier, it clearly mentioned to NOT plug in the target board (XIAO), that it should only be getting power from the programming board (DK).

However, this meant I could not get my logs anymore. So I tried not patching the 3V3 supply on the expansion board from the DK and plugging in the USB-C on the XIAO board.

It did not work reliably for me and led to all sorts of issues with the board. Just don’t do it.

Board issues

When trying the above configuration (connecting both the DK and the XIAO board), here is what I observed:

Initially, I could connect to the serial port and see my logs and I also saw my board appear in VS Code.
I flashed an application via VS Code and it indicated a successful operation, however my program was not executing on the board, I had to manually press reset on the XIAO to start it.
The second time I flashed the application, the program did not start but a reset did not do anything.
I removed the XIAO board from the DK and connected it directly via USB, nothing happened, no LED, no serial port appeared. Double-clicking the reset button did not make the board appear as a USB drive on my Mac.

At this point, I thought the board might be dead. But if I connected it via the DK (and only via the DK), I could still see it in VS Code.

After some research, I thought maybe the bootloader was gone/corrupted and I followed the procedure in Embeddronics-ltd/Flashing-Bootloader-Xiao-BlE: Installation of bootloader in xiao ble, using J-Flash to flash it back on the board.

That worked and I could then re-use the board as before. I tried a second time, not connecting the 3V3 and the ground cables but ended up with similar issues. So for now, I’ll just choose between seeing the logs or using the SWD flash/debug features.

Conclusion

Now with this setup, I’m better equipped to continue exploring the world of embedded dev, which is quite new for me.

As I progress through this journey, I’m putting together a list of interesting resources related to embedded systems, you can find it at Embedded development resources.

My end goal is to use (Embedded) Swift as the programming language for most of my embedded projects. I’ll be posting a lot about this in the coming months, so if you’re interested, come back often and follow me on the different social media.

Opus Decoding in Swift

Sat, 27 Jul 2024 00:00:00 +0200

Introduction

Friend is an open-source wearable AI device that captures your conversations, transcribes and summarises them, and offers a ton of features based on that information.

The wearable device captures the audio signal and sends it to your phone over BLE, which offers a limited bandwidth. Initially, Friend was sending raw PCM data (16 bits, 8kHz), meaning the audio quality was quite low and so was the transcription accuracy.

To improve that, last weekend I re-enabled the Opus codec in their firmware, allowing higher audio quality while using less bandwidth.

Opus is an open, royalty-free codec that works great with voice content. The available library can be used on various platforms, including on embedded devices.

Although you can use the official Friend app, I’m developing a native iOS app that records conversations from the Friend device as part of my PAL project. And now that the device supports Opus, I needed to add that to my codebase.

Integrating libopus in a Swift code base

I started by doing a quick search for a swift package that supports Opus and found alta/swift-opus: Opus audio codec for Swift Package Manager.
It provides an API that integrates nicely with the native AVFoundation concepts such as AVAudioFormat and AVAudioPCMBuffer, which I was already using in my code for recording the PCM data used so far.

And there is recent activity on the project, which is always a plus.

The way the Friend firmware works is that it takes 10ms of audio data, encodes it (or not for PCM) and sends that packet as one BLE characteristic notification.

In order to optimise the writing to the audio file, I’m collecting a series of packets, appending everything in a Data structure and every 100 packets, converting that to an AVAudioPCMBuffer and writing to the file.

And so my first naive try was to keep the same mechanism and pass my big data blob to the decode method which conveniently returned an AVAudioPCMBuffer

public func decode(_ input: Data) throws -> AVAudioPCMBuffer {

And the code did not work, the decode call was always throwing an Opus.Opus.Error error -4, which is OPUS_INVALID_PACKET.

Packet boundaries are important

Why was my packet invalid ? I reviewed the low-level code that gets the data from Bluetooth and populates the buffer but all looked OK and it worked for PCM, so why not here.

Glancing at the opus site, I saw this information

Frame sizes from 2.5 ms to 60 ms

Got it, my packets are too big, I need to feed the decoder smaller ones. So I updated the code to split my big buffer in small chunks and iterated on the call to decode. Same error, although at some point, when playing around with the chunk size, I could successfully decode one packet ?!?

I went back to the firmware code, checking the parameters such as sampling rate, making sure everything was aligned on both sides. And then it hit me. On the firmware side, we’re encoding one 10ms packet at a time and sending that encoded packet over BLE. So it makes perfect sense that on the decoding side, it’s that same packet that needs to be fed to the decoder.

But the code in place does pack several packets it receives over BLE together before handing it over to the decoder. And because we’re using Variable Bit Rate for encoding, I can not split the buffer back into the initial packets in the decoder.

I reworked the code so that instead of storing the raw bytes in a Data structure, I collected the different packets in an Array, each entry storing an individual packet. So when the decoder receives the Array, it can now decode each packet individually.

And this worked. It seems obvious in hindsight, but by adding the decoding logic within the structure of the existing code without looking at the bigger picture, I had gone in the wrong direction.

State matters

No more decoding errors, the audio file was properly written and it had the correct length, great.
But when I listened to the recording, the sound was heavily distorted. Not complete garbage, I could barely understand what I had said during the recording but loads of crackling, saturation and other artefacts.

Back to looking at the code, did I make a mistake in my byte manipulation logic ? Was it an endianness problem ? I tried different changes but to no avail.

The Friend native app, written in Dart with Flutter was producing correct audio, so the issue was on the decoding side. I looked at their code, looked at the Dart Opus plugin (which is using the same libopus library underneath) but nothing stood out.

So I applied a technique I use a lot when faced with a problem in my apps. I try to reproduce it in an isolated context, as small and focused as possible, getting rid of as much overhead as possible so I can iterate fast.
I updated my code to store the raw packets I receive from BLE to a file (so I had a test fixture and I did not need to connect to a BLE device and record a bit of audio for every test). From there, I created a simple command line tool on the Mac to read those packets and decode them to a WAV file.
And to my surprise, the audio was perfect, clean and pristine sound.

Huh ? Comparing the code in my app with the simple test I made, I saw that in the app, again because of the structure in place and because I first wanted to get something working before refactoring the code, I re-created the Opus decoder for every packet.So I updated my test to do the same. And boom, same artefacts in the decoded audio.

Updating the app code to use the same decoder for the whole duration of a single recording session fixed the issue.

Once again, after having fixed the issue, it makes perfect sense. My limited knowledge of how the codec works made me take a shortcut that had a huge impact on its output.

Optimising the processing

I now had a working app, but to fix some of the problems, I had to adapt the code in place.

Because of the API of the Opus Decoder

public func decode(_ input: Data) throws -> AVAudioPCMBuffer {

and of AVAudioFile

open func write(from buffer: AVAudioPCMBuffer) throws

and the fact there’s no API to easily concatenate 2 AVAudioPCMBuffer s together, I was now writing each decoded packet to the audio file.

I did not scientifically investigate the topic, but it bothered me to make frequent calls to an API that results in disk I/O.

My solution was to fork the swift-opus package and add a method to decode to a Data structure and not an AVAudioPCMBuffer.

public func decodeToData(_ input: Data) throws -> Data {

With that in place, I could now properly decode each packet, build a big data buffer by appending together the decoded bytes, create a single AVAudioPCMBuffer buffer from there and write that to the audio file.

The fork also includes an update of libopus to the latest 1.4.x version. I tried updating to 1.5.1 (the latest version at this time) but the project did not compile and I did not bother to investigate further.

I think that given my limited usage, I could also directly include libopus into my app and bypass the need for an extra package altogether. That might be work for some other time.

Key takeaways

Specifically for Opus decoding:

decode the packets as they’ve been encoded, don’t try to manipulate them in any way
make sure to use the same decoder instance for the whole audio segment, state matters so the lifetime of the decoder should be similar to the one of the encoder

More generally with regards to software development:

don’t let existing code inform your decisions too much, take the time to think about the bigger picture and how this can impact what’s in place
once faced with a problem, try to replicate it in an isolated context, with the least overhead possible on the debugging process

If you want to take a look at the final source code, the PAL project is fully open-source on GitHub.

The PAL Project

Fri, 05 Jul 2024 00:00:00 +0200

Several weeks ago, I created the nelcea/PAL: Wearable AI exploration repository on GitHub. I want to take a minute to explain in slight more detail what this open-source project is about.

PAL is the vehicle for my research in the world of Wearable AI, with an initial focus on the fields of “Personal Knowledge Management” and “Building a Second Brain” (BASB).
The initial use case is to capture conversations I’m having, store their text somewhere, and make it possible for me to consult, process, or interact with them.

I have plenty of other ideas extending the use cases to connect many other domains, but I’ll address those in a separate post.

The solution I’m currently working on is based on:

A Friend device to capture the conversations.
An iPhone app to record them, transcribe them and interact with the notes.
A server-side component for heavier processing.
Potentially some desktop/web app for further interaction.

The goal of this project is for me to explore and learn in public, not to build a polished project suitable for a large user base.
I encourage typical users to check out what Based Hardware is doing—they’re addressing the most typical use cases. I think they’re doing a great job moving quickly to provide useful features to their early adopters, and I’m happy to help with their project whenever I can.

While learning, I intend to delve deeply into all required topics, publishing the code and writing blog posts about my discoveries. I will link to those here as they become available.

This will cover a broad range of topics, as the use case above will already require:

BLE connectivity between the Friend and iPhone.
Audio recording and codec usage.
Audio processing to get the cleanest sound possible for transcription.
Automatic speech recognition, including diarisation.
Text processing for e.g. summarisation or extracting actions.
Chat interface to interact with the content, using RAG and vector databases.

In addition, I have the following requirements:

Use the Swift programming language for everything: iPhone app, server component and eventually firmware (at first I’ll use stock Friend firmware)
Run everything locally, either on the iPhone or on a local server, and avoid sending any data out to a cloud service. A conclusion might be that it’s not possible to get acceptable results with that constraint, but that’s also part of the experiment.

Do not hesitate to spread the word and contact me on X or Mastodon to join me on this journey.

WWDC 2024

Sat, 15 Jun 2024 00:00:00 +0200

And just like that, WWDC week has come and gone…

As expected, there were plenty of announcements across all OSes, both user-facing and technical - including the big one on Apple Intelligence. I don’t think there were many surprises this year.

I won’t go into details on the announcements as many people are already doing that, but I do want to point you to a few interesting resources:

What’s new in Swift 6.0? – Hacking with SwiftBlog post from Paul Hudson on the changes in Swift 6.0, covering of course the big focus on concurrency as well as many small additions throughout the language.
The Swift 6 migration guide on swift.orgGuide from the Swift team on migrating to the new Swift 6 language model, mainly focusing on concurrency.
What’s new in SwiftUI for iOS 18 – Hacking with SwiftBlog post from Paul Hudson, this time on all the changes Apple in bringing to SwiftUI.
WWDC24: Key Updates for macOS Development with SwiftUIYouTube video from Karin Prater going over some new stuff for macOS development, which is nice given the focus is mostly on iOS/iPadOS. Covered updates include customisations for windows behaviour and styling, keyboard and mouse pointers modifiers and changes to TabView.

Apple released all the betas (although some key features will be coming later in the year) on Monday evening, and the session videos were made available progressively throughout the week.

Unless you were lucky enough to have a golden ticket and attend the conference in Cupertino or one of the many community events in the area, many people spent their time watching the released videos and testing out some of the new features.

Unfortunately, I had a lot of professional obligations during that week and couldn’t dedicate as much time to WWDC as I would have liked.So, make sure to clear your calendar during that week – even if you’re staying at home, it should be a conference week.

However, one thing I did manage to fit in was the labs. I think this is a fantastic opportunity offered by Apple that’s often overlooked by many developers. Before COVID, you had to be physically present in California to benefit from them, but now that they’re remote, everyone can participate. You just need to be quick – as soon as the sessions are announced on Monday evening, book the ones you’re interested in, as some of them become unavailable quite fast.

This is your chance to have one-on-one sessions (I even had two engineers answering my questions in one of the calls) with the engineers who developed the tech you’re using. In addition to being super competent, they’re also really approachable and happy to help – even with basic questions. They’re also eager to learn what you’re doingwith the stuff they worked on.

And it doesn’t have to be just questions about this year’s tech – I collected questions about things I was working on during the year that didn’t work as expected or where the code I ended up with didn’t make a lot of sense. Just come prepared as much as you can, in addition to helping you make the most of your limited time (30 minutes per session), it’s also a matter of respecting the time of the engineer helping you.

Conclusion

To wrap things up, I want to thank the engineers at Apple for all their hard work in developing great tools and technologies – and for taking the time to interact with developers during the labs and help bring their ideas to life.

And as a conclusion, here is my checklist for next year’s WWDC:

During the year, keep track of questions or items worth discussing during the labs
As soon as Apple announces WWDC, clear out my agenda for that week
As soon as the labs are announced, book all sessions that interest me

On the proper usage of commonFormat for AVAudioFile

Sat, 08 Jun 2024 00:00:00 +0200

Introduction

I started looking into AVAudioFile while working on PALApp, my exploration into the world of portable AI. The MVP I was implementing is receiving audio over BLE from a Friend wearable and storing it in an audio file on the device.

Try 1

The Friend audio samples are 16-bit little-endian integers.

To ease testing, I can simulate this by populating a buffer with a simple sine wave

let samples: [Int16] = {    var s = [Int16]()    for i in 0..<16000 {        s.append(Int16(sinf(Float(i) / 2) * Float(Int16.max)))    }    return s}()

Given that input, I started by creating a simple AVAudioFile and an AVAudioPCMBuffer using that format, filled the buffer with the samples coming from Friend and tried to save the file.

guard let audioFormat = AVAudioFormat(commonFormat: .pcmFormatInt16, sampleRate: sampleRate, channels: 1, interleaved: false) else {print("Error creating audio format")return}guard let pcmBuffer = AVAudioPCMBuffer(pcmFormat: audioFormat, frameCapacity: 16000) else {print("Error creating PCM buffer")return}pcmBuffer.frameLength = 16000let channels = UnsafeBufferPointer(start: pcmBuffer.int16ChannelData, count: Int(pcmBuffer.format.channelCount))let data = samples.withUnsafeBufferPointer( { Data(buffer: $0 )})UnsafeMutableRawPointer(channels[0]).withMemoryRebound(to: UInt8.self, capacity: data.count) {(bytes: UnsafeMutablePointer<UInt8>) indata.copyBytes(to: bytes, count: data.count)}guard let audioFile = try? AVAudioFile(forWriting: URL(fileURLWithPath: "generated.wav"), settings: audioFormat.settings) else {fatalError("Error initializing AVAudioFile")}do {try audioFile.write(from: pcmBuffer)} catch {print(error.localizedDescription)}

And this crashed, the write() call was not even throwing an error, the code crashed, with a failing assertion:

CABufferList.h:184   ASSERTION FAILURE [(nBytes <= buf->mDataByteSize) != 0 is false]:

I tried to understand the error, looking into how I was filling the buffer, copying bytes, did I properly calculate the number of samples, the size of the data ?
Searching the web with that error message did not help much and my (incorrect) conclusion at the time was that somehow, AVAudioFile did not support writing to Int16 format.

Try 2

As some of the example code I saw during my search were using the Float32 format, I updated my code to only use that. Making that work, meant doing some manual conversion of the data received in Int16 format. That ended-up being several lines of code that just did not feel right.

guard let audioFormat = AVAudioFormat(commonFormat: .pcmFormatFloat32, sampleRate: sampleRate, channels: 1, interleaved: false) else {print("Error creating audio format")return}guard let pcmBuffer = AVAudioPCMBuffer(pcmFormat: audioFormat, frameCapacity: 16000) else {print("Error creating PCM buffer")return}pcmBuffer.frameLength = 16000let f32Array = samples.map({Float32($0) / Float32(Int16.max)})let channels = UnsafeBufferPointer(start: pcmBuffer.floatChannelData, count: Int(pcmBuffer.format.channelCount))let data = f32Array.withUnsafeBufferPointer( { Data(buffer: $0 )})UnsafeMutableRawPointer(channels[0]).withMemoryRebound(to: UInt8.self, capacity: data.count) {(bytes: UnsafeMutablePointer<UInt8>) indata.copyBytes(to: bytes, count: data.count)}guard let audioFile = try? AVAudioFile(forWriting: URL(fileURLWithPath: "generated.wav"), settings: audioFormat.settings) else {fatalError("Error initializing AVAudioFile")}do {try audioFile.write(from: pcmBuffer)} catch {print(error.localizedDescription)}

This worked, it saved a proper WAV file that contained the audio I was expecting.
However, inspecting the file in the finder shows it uses 32 bits per sample, which is a waste considering the source data I’m working with.

Inspecting WAV file shows it uses 32 bits per sample

Try 3

Until now, I had created my code by searching the internet, looking at code snippets I could modify for my use and quickly glancing at the reference documentation in Xcode. But I never took the time to understand in more details the audio framework that Apple was providing.
So I went back to my WWDC videos collection and found one that introduced the classes I was using, session 501 from WWDC 2014, “What’s New in Core Audio” (that unfortunately is no more available from the Apple website).
In that session, they talked about the concept of the “processing format”, that is the PCM format with which you interact with the file. You can specify that with the initializer that takes a commonFormat parameter.

guard let audioFormat = AVAudioFormat(commonFormat: .pcmFormatInt16, sampleRate: sampleRate, channels: 1, interleaved: false) else {print("Error creating audio format")return}guard let pcmBuffer = AVAudioPCMBuffer(pcmFormat: audioFormat, frameCapacity: 16000) else {print("Error creating PCM buffer")return}pcmBuffer.frameLength = 16000let channels = UnsafeBufferPointer(start: pcmBuffer.int16ChannelData, count: Int(pcmBuffer.format.channelCount))let data = samples.withUnsafeBufferPointer( { Data(buffer: $0 )})UnsafeMutableRawPointer(channels[0]).withMemoryRebound(to: UInt8.self, capacity: data.count) {(bytes: UnsafeMutablePointer<UInt8>) indata.copyBytes(to: bytes, count: data.count)}guard let recordingAudioFormat = AVAudioFormat(commonFormat: .pcmFormatInt16, sampleRate: sampleRate, channels: 1, interleaved: false) else {print("Error creating recording audio format")return}guard let audioFile = try? AVAudioFile(forWriting: URL(fileURLWithPath: "generated_int16.wav"), settings: recordingAudioFormat.settings, commonFormat: .pcmFormatInt16, interleaved: false) else {fatalError("Error initializing AVAudioFile")}do {try audioFile.write(from: pcmBuffer)} catch {print(error.localizedDescription)}

This works, with simpler code, less data manipulation (which means it’s faster and uses less memory) and the generated WAV file uses 16 bits per sample, which matches the original data format.

Conclusion

Most of the time, if you encounter an error in your code and it feels like you’re fighting the framework, ending-up implementing some weird workaround, you’re probably doing it wrong.

It’s worth going back to basics, reading the Apple documentation and understanding the classes you’re working with. I find that searching the WWDC video archives, for the video of when a particular topic was first introduced can help a lot (pro tip: as Apple is removing those as time goes by, keep a local copy of all the videos they publish, just in case).

You can find a sample project illustrating the use of the above code on GitHub at Test_AVAudioFile.

Portable AI, a month later

Fri, 03 May 2024 00:00:00 +0200

It’s been a bit more than one month since my initial post on Portable AI and things are moving fast. The Humane Ai Pin has started shipping, the rabbit R1 has started shipping, Friend has a kickstarter campaign, Rewind became Limitless and a lot of people are now talking about those new “AI devices”.

The Humane Ai Pin was the first to ship and the reviews are it. Unfortunately, they’re not glowing, as the device disappoints on many levels: it’s slow, prone to providing incorrect answers, overheats to the point of shutdown, has a barely readable display in daylight, is limited in what it can do and lacks integration with your Apple/Google account…Its hefty starting price of 700$ and the fact it turns into a paperweight as soon as you stop paying the 24$/month subscription does not help.

The Rabbit R1 has also hit the shelves, with a pickup event in NYC on April 24th. While it’s marginally better received than the Humane Ai Pin, criticisms still abound. Although faster than its competitor and sporting a traditional screen (but that presents some usability concerns, e.g. navigation in the menu with the scroll wheel instead of using touch), it’s lower price of 199$ is not enough to forgive some of the drawbacks. The build quality matches the price tag and, like the Humane Ai Pin, if confidently provides incorrect answers to your queries and offers limited features at this stage.

Reviewers have noted a trend in the rush to release AI-powered devices, often resulting in pre-beta, half-baked products.. There is a huge disconnect between what’s promised (sometimes even demonstrated) and what users can actually do with the device they receive.

One very plausible reason is they fear an announcement from Google (Google I/O starting on May 14th) or Apple (WWDC starting on Jun 10th) and want to push as many devices as possible in the hands of end-users before those.Indeed, Apple and Google control their platform and can provide a tight integration on a device you already own and within an ecosystem you’re already locked into. A thought offering to beat.This is potentially the same reason why Humane and Rabbit are exploring different form factors, that work outside of the established ecosystems.

On April 15th, Rewind unveiled its new product called, Limitless. Alongside this announcement, the company rebranded itself as Limitless.Limitless offers a web app and desktop apps that records everything you’ve seen, said or heard. It’s clearly targeted at meetings, providing a pre-meeting context based on previous information it collected, real-time transcription and note-taking during the meeting and post-meeting summaries.While I haven’t had the chance to try it yet, I like the idea of focusing on a given use case and (hopefully) delivering it right instead of promising a lot of possibilities and falling short on most of them.

As a companion to its apps, Limitless plans to release a wearable, the Limitless Pendant. This is not shipping yet (planned for Q4 2024) and is the evolution of their previous Rewind Pendant that never shipped. But it changes the form factor and I really like what they’re promising here. It seems like a well thought out product proposed at a reasonable price. And the Consent Mode where they perform voice identification and recognise that a user has given its permission to be recorded would be quite incredible, IF it works reliably.

Finally, there’s the open-source Friend project that I’m following closely (e.g. on their Discord channel) and it’s moving quite fast. They’ve joined forces with ADeus, launched a successful Kickstart campaign and are building a company.The energy is great, but it’s in a period of flux, making it difficult at this stage to know where they’re going and what the end goal is. At some point, they express interest in focusing on hardware and firmware development while fostering a community of external applications.

As I continue to monitor developments in this space, I regularly update my list of wearable AI projects. Be sure to check it out for the latest updates.

Building a wearable AI Device

Fri, 19 Apr 2024 00:00:00 +0200

As ADeus was the first wearable AI open-source project I discovered, I started building a device based on the hardware they were using, and more specifically on the tutorial at ADeus - Raspberry Pi Zero W Setup Guide.

I ordered the Pi and the same microphone but as I did not want to play around with 3D printing a case for now, I ended-up ordering the standard Raspberry Pi Zero Case. Using the case meant I could not use the standard header connector and had to solder the microphone cable directly on the board.

Back of the board with microphone cable directly soldered to it

MEMS microphone, this thing is tiny!

As the standard case also comes with a cover supporting the camera module, I went on ordering one (even if ADeus doesn’t make use of it for now). I initially ordered the camera module 3, which is the current one but it does not work in the standard case. I then ordered a camera module 2 but, as can be seen on the picture below, the fit is not perfect either, it seems the case was made for the original version of the module and there were some changes since then.

For the finishing touches, I used hot glue to attach the microphone, protecting it with a foam cover, and at the back I re-used a clip/pin from a conference badge fixed with strong double-sided adhesive.

Final assembled device, front and back

This keeps the form factor relatively small (shallow at least) but it doesn’t include a battery. This means going the VisionPro way, using an external battery pack.

Going VisionPro with an external battery pack

Which in the end, is not too much of an issue when wearing it with a shirt.

Wearing it with a shirt

With the hardware part ready, I can move on to installing and testing ADeus but also writing some of my own code. More on that in a later post.

List of material

Portable AI

Sat, 06 Apr 2024 15:14:35 +0200

AI, LLMs and generative models have now been known and used by a majority of the population, technical or not. But they’re mostly used from your desktop or your mobile phone. I’m now seeing a growing proposition of portable AI, new form factors that use or fit within the AI eco-system.

Commercial offering

One of the first I got to know is the AI pin from Humane. I stumbled upon it through this TEDx video: Imran Chaudhri: The disappearing computer — and a world where you can take AI everywhere | TED Talkand immediately went to the web site and signed-in to get notified of any release.Although it includes AI in its name, that’s not the focal point for me. The AI pin is a wearable device that offers new, less obtrusive ways, to interact with your digital world. You don’t need to take a device out of your pocket and stare at the screen. You can tap and talk and make gestures and it replies via audio or can project a basic UI on your hand.It’s one a the few devices that ships now, however I’m a bit disappointed with the price point, the fact it’s US only and requires to a pricey subscription.

Another alternative I knew about (because I already followed-up on what Rewind was doing, more on this in a future most) is the Rewind Pendant. This is not an interactive AI assistant, but a always on sensor (microphone) used to populate your personal knowledge base, as a complement to the other sources Rewind already uses. You then use AI afterwards on your phone or desktop to interact with that knowledge base. It’s not available yet though, so let’s wait and re-evaluate when it ships.

Then there is the rabbit r1, which was all the rave at CES 2024. Quickly browsing through the web site, I’m not sure I get the concept. Most of what it does could be done with an app on your phone. The form factor is very similar to a phone so it still requires you to take it out of your pocket, tap a button, look at a screen. Watching their keynote provides some more information. They’re touting that their Large Action Model, that can learn a UI and understand how to take action, just like a human would do, is the big differentiator and bridges the gap between understanding what you do (what LLMs do) and actually making things happen.At 199 USD without any subscription, their pricing is attractive. It’s not yet shipping but should be coming out really soon.

In a different category, following the traces of Google Glass or Meta Ray-Ban smart glasses, is the Brilliant Frame. What’s great about that project is that it’s very opened and is really a platform to tinker with. But that also means that at this stage (still in pre-order anyway), there are no really useful, end-user oriented applications. I was tempted to pre-order one and play around with it but preferred to wait. I’ll definitely keep on eye on what they’re doing and what use cases developers are coming up with.

DIY approach

Then I thought it could be cool to try to build such a system myself. And if I have this idea, somebody else surely already had it. Searching for an open-source alternative to the above mentioned products quickly lead to an existing project: ADeus.

I looked at their repo, started building hardware to test what they’re doing, took a look at the discord channel and found quite a few other projects (open or not) in this space: Owl, Friend, O1, Tab and I’m sure I’m missing quite a few.

In this very interesting video (Owl: Open Source Wearable AI - OpenCV Live 130), Bart Trzynadlowski, one of the co-founder of Owl, mentions quite a few of them and talks about the opportunities and the challenges for those projects.

Conclusion

I think their is indeed a lot of potential to further develop such projects and find real-life use cases that can have a profound impact on our everyday life. I’ll definitely look in more details at those projects, hopefully trying them myself and reporting here and I’ll most probably will have a go at developing my own project. Stay tuned…

Vision OS for an existing app

Wed, 07 Feb 2024 20:19:38 +0100

Introduction

As a simple intro into visionOS, I used the most simple app I have, AWG Convert, and set-up to add native support for visionOS to it.

I added an “Apple Vision” destination to my project, launched the app in the simulator and started to tweak the code to make it look slightly better in the Vision Pro. Just the basics here, not really taking advantage of Spatial Computing, baby steps.

The biggest work I had to do was in making the UI more responsive, as the iPhone version was locked to only portrait mode. See Interface orientation in visionOS for some exploration I did as a preparation to this.

When I thought I had something OK, that at least was better than the “compatibility mode”, I went on to publishing the update on the AppStore and discovered there were a few “details” I still needed to take care of.

Providing an app icon

I naively thought that, as I already had an icon for my iPhone app, the same one would just be used. Not the case.

Looking at Configuring your app icon | Apple Developer Documentation I could see that tvOS and visionOS have specific OS variants for the app icon. And those are not simple images but multi-layered.

I thus added the variant in Xcode in the assets catalog.

Adding a vision OS App Icon

A visionOS has by default 3 layers (the maximum it can have) and must have minimum 2 layers.

Exploring visionOS Fundaments: Creating the App Icon is a nice article with some more information on creating icons for visionOS.

Taking a screenshot

Searching in Apple documentation on how to take screenshots for Vision Pro, all I could find was Capturing screenshots and video from Apple Vision Pro for 2D viewing | Apple Developer Documentation. But this requires a physical device, there is no indication on how to capture using the simulator.

So I did it like for other iOS devices and used the “Save Screen” button in the simulator.

Doing this results in images with a resolution of 2732 x 2048 pixels. But looking at the specifications from Apple (Screenshot specifications - Reference - App Store Connect - Help - Apple Developer), it indicates that the screenshots for Apple Vision Pro must have a resolution of 3840 x 2160.

So a bit of post-processing is required. You can definitely do it using your favorite tool like Pixelmator or PhotoShop but I wanted a quick and easy solution for my first test submission.

Luckily enough, macOS includes a command-line tool that comes in handy. Using the following 2 commands, I got an image at the right resolution.

 sips -z 2879 3840 Simulator\ Screenshot\ -\ Apple\ Vision\ Pro\ -\ 2024-02-02\ at\ 21.16.15.png sips -c 2160 3840 Simulator\ Screenshot\ -\ Apple\ Vision\ Pro\ -\ 2024-02-02\ at\ 21.16.15.png

This first command scales-up the image to the desired width, the second crops it vertically to get the desired final resolution. So be careful how you center your app content when taking the screenshot, as some part at the top and bottom will be discarded.

A separate distribution for visionOS

So I logged into App Store Connect and looked at creating a new version of my application.

So far I only had applications that were targeted at iPhones and iPads and I thought targetting Vision Pro in addition would work the same.

But no, I saw that I needed to add a different Platform for the application (for visionOS) and that the iOS and visionOS apps can have different lifecycles. But the make things cleaner, I aligned the version numbers for both platforms.

iOS and visionOS platforms added in App Store Connect

Building for distribution

The next surprise came when I wanted to create an archive of the project for uploading to the App Store, as there was no destination for a visionOS device available.

visionOS does not appear in the list of destination

Googling a bit, I quickly found an answer on the Apple Developer Forums: Xcode Not showing Vision Pro Archive destination. And although it mentions Xcode 15.2 Beta, it’s still the case with the final 15.2 version.

So changing the Base SDK from it’s original iOS

iOS is set as the base SDK for the project

to visionOS

After changing the base SDK to visionOS

made the required destination appear in the menu.

visionOS now does appear in the list of destination

And with that, I was able to create a build and upload it to the App Store.

Being on the store

The review process took 3 days and my app was accepted on the store.

So AWG Convert is now available for Apple Vision Pro.

Not much to brag about, as this was the bare minimum for native visionOS support, but at least it helped me discovered the steps required to release an app for Vision Pro.

Now let’s get to work on the next one…

Interface orientation in visionOS

Sun, 28 Jan 2024 16:20:57 +0100

Introduction

Yesterday, during the super interesting Unwrap Live 2024 stream on visionOS from Paul Hudson, I asked the question on whether it was possible to have your visionOS start in “portrait mode”.

The reason I asked is because I’m extending a small utility app (AWG Convert) that I’ve coded on the iPhone to have native support for the Vision Pro. There is no iPad version (for now), and the iPhone version only supports portrait orientation. So my first idea was to port “as is” to Vision Pro. And I was thinking people can just have a small vertical window arranged on the side of other windows in their environment.

The answer Paul gave was that Apple made it clear that humans have more scope from left to right than up and down and so it makes much more sense to go wide than tall.

That makes a lot of sense and checking information on human field of view (FOV), although you can find varying information, usually indicates that humans of about 200° of horizontal FOV (both eyes combined) and 135° of vertical FOV.

Being stubborn

Let’s say I still wanted to go ahead and have my app start with a vertical window.

The article Positioning and sizing windows | Apple Developer Documentation gives the key to it and leads to the following code

WindowGroup {ContentView()}.defaultSize(width: 400, height: 700)

But this does not prevent the user from resizing the window to a wide aspect ratio. The following code would achieve this

WindowGroup {ContentView().frame(minWidth: 300, maxWidth: 500, minHeight: 600, maxHeight: 800).background(.red)}.windowResizability(.contentSize)

but let’s face it, this is fighting the system and will lead to an alien app in the Apple ecosystem.

Playing nice

The app should adapt to whatever window size and aspect ratio the user chooses.

How would you achieve this?

Under iOS, it’s possible to check for the device orientation and adapt to it (some information on how to do it in How to detect device rotation - a free SwiftUI by Example tutorial).

But under visionOS, UIDevice.current.orientation is unavailable. Of course, the device orientation doesn’t make sense for the Vision Pro.

Anyway, this is not the recommended way to adapt your UI layout, as e.g. on the iPad, with multitasking, your device can by in landscape orientation but your UI canvas be in portrait.

I believe the recommended way to address this, is by checking the reported size classes and adapting the layout based on that.

But testing this on the Vision Pro simulator always reports a size class of regular for both horizontal and vertical directions, irrelevant of the window size or aspect ratio. So that will not help.

For now, the solution I’ll be testing further is to use GeometryReader to get the size available for my interface and adapt accordingly (even though GeometryReader comes with its shortcomings too).

Trying it out yourself

I created a small projet to test out device orientation, size classes and GeometryReader information as reported under different platforms, it’s available on GitHub for you to test it out: Test_DeviceOrientation

Obsidian on Multiple Devices

Sat, 14 Oct 2023 12:00:00 +0200

As I’m getting more serious about using Obsidian as my primary note taking application, having the content available on multiple devices becomes critical.

Living in the Apple eco-system, I had two options: Obsidian sync or iCloud.
I went for the second option.And for those concerned with privacy, Apple now supports end-to-end encryption using your own keys for your iCloud storage with the Advanced Data Protection option.

Setting up Obsidian Sync with iCloud Drive is a good article on the process of setting up sync.And it covers the use-case I was faced with of having created a vault on the Mac before wanting to sync on a mobile device.

There’s however one gotcha I encountered that was not covered: when copying files from your existing vault to the iCloud one using the Finder, hidden files (starting with a dot) are not copied.
And Obsidian stores its settings in such a file (by default .obsidian). So if you don’t copy this file (as I initially did), your new iCloud enabled vault will have the default obsidian settings and you’ll loose any adjustments you might have made for your original vault.

The Wait is Over

Tue, 12 Aug 2014 00:00:00 +0000

Or so I thought…

Last week, I had received a shipping notification for my Rift DK2, indicating a delivery for this Monday. So I spent a bit of the week-end preparing a PC to use with it. It’s a powerful desktop system that I built a bit less than 2 years ago, to work on GWT projects (because building GWT projects can take a long time). When I started with that idea, the goal was to have a fast machine for that task, while keeping the budget low. So I would spend the bulk of the budget on the components that matter, mainly the CPU (6-Core i7), motherboard and RAM (16 GB) and re-use components that I had lying around. The later included an old graphics card.

This worked but I was not too happy working under CentOS, and being an OS X user, I wondered if there was any way to run OS X on this. Looking a bit at the Hackintosh community, it turned out it was not that difficult, more so if you picked your components from a list of well supported ones. So I “invested” in a new graphics card, picking one based on my requirements : Hackintosh friendly, powerful enough to run an IDE and a web browser (which means anything sold at that time would do) and cheap. And I ended up with a Sapphire Radeon HD5450 card that perfectly fit the job.

As last week-end, the Oculus SDK was still Windows only (and anyway a lot of demos are still Windows only), I decided to put a new disk in that machine, install a brand new Windows 8.1, the DK2 run-time and the SDK. Everything was now ready, one more time to sleep.

And the long awaited package arrived as planned

I carefully unpack, follow the Quick Start Guide to connect all the components, launch the Oculus Config Util, go through the required steps, click “Show Demo Scene” and … judder, lag, choppiness, in a word: unusable.

Going through the forums, I see people reporting similar issues and problems with AMD based graphics cards. After some reading and tweaking, which took me quite a bit of time as I mostly stopped using Windows about 8 years ago, I could improve the situation a bit but not really fix the issue. I might be able to improve the situation even more but anyway, I’m only getting about 25 fps with the SDK demo.

Having watched Cymatic Bruce’ stream last Sunday, where he tested “Miku Beach Concert” and reported it as a solid experience, I decided to give that demo a spin. On the forums, people are reporting performances of above 200fps. I’m getting about 50 fps. The verdict is clear, that HD5450 card is really slow, way too slow for this kind of rendering. Upgrading to the new SDK 0.4.1 release did not help.

Turns out I have a new Retina Mac Book Pro on its way, sporting an Nvidia GT750M chipset, that should perform way better than this desktop machine.

Given that situation, I will not provide any comment on my experience with the Rift until I’ve had a chance to test it on a decent setup.

A so the wait continues…

Virtual Insanity

Wed, 16 Jul 2014 00:00:00 +0000

As about 40.000 people on the planet, I’m patiently waiting for my Oculus Rift DK2 to arrive.

Although I’m impatient to get it, I’m not going insane, a trait that some members on the Oculus dev forums or subreddit are clearly exhibiting. Am I annoyed ? Slightly. Am I mad at Oculus ? Definitely no (though if I ever run into Palmer Luckey I’ll only buy him regular juice and not that expensive organic cranberry apple juice).

I’ve been shipping software long enough to understand that it’s tough to make estimates, even more so when you’re innovating.

I’ve been in small start-ups and not so small corporations and understand that PR is a tricky job. And although I’m a firm believer in being open and honest, there’s a way and a time to present information. There’s no way to please everybody, it’s a balancing act and there will always be someone to call out the conspiracy theory.

I also do understand the slight difference between making a prototype, making 100 copies of some hardware and making several tens of thousands of pieces in a month. Can you make yourself a sandwich ? Can you make sandwiches for all of E3 attendees lunch ?

The Rift DK2

But seriously, Oculus, where is my rift ? What am I gonna do without it ? Well I’m just gonna do the same thing I did yesterday.

Looking at forum posts, I think some clarification of the meaning of urgency is required:

Getting the organ for that transplant that will save a life: urgentDrilling a hole to put a nuclear warhead in a meteorite and save the earth: urgentGetting the latest tech gadget to play around or even to work on your next great invention: no so urgentSeriously, instead of just waiting and whining, do something for VR. I’m sure you can still create some 3D models, improve that bit of code, re-think the UX of your game or simply listen to the many great podcasts on the topic.

I can hear the critics say “It’s just because you’re not a core VR enthusiast”. Well it’s true I did not participate in the kick starter (can’t remember if I even noticed it before it closed) or even ordered a DK1 (I nearly did a couple of times but never pulled the trigger).

But the truth is I’m not a new comer to VR either (more on this in follow up posts) and it took me a bit of time to realize that, this time, VR is here to stay. So, with or without a rift, I’m fully on board.

Welcome (again)

Tue, 24 Jun 2014 00:00:00 +0000

I’ll confess, focus is not my forte! I tend to be interested by a lot of different topics, even more so when it comes to technical and scientific ones.

Over the last few years, pumped up by the then latest and greatest idea I just had, I’ve started several blogs. I followed-up with a couple of posts and then swamped by work or some other real-life “obligation”, I let it fall into oblivion, piling up in some dead internet sites landfill, waiting for the DNS registration to expire.

Did I learn something since then? Maybe, maybe not, but I want to believe in my mental sanity.

Insanity: doing the same thing over and over again and expecting different results.
– Albert Einstein

So I’ll go by the above quote, attributed to Albert Einstein, and do things differently this time. Well, sort of.

You might say that you’re reading a new blog that I started and so consider that I’m doing it all over again. I beg to differ.

I did not register yet another domain name but decided to locate this blog at the URL I used for my SPOC site. Although this might seem like a technical detail, it’s not. It is a conscious choice in setting the tone of this site, which I want to be much more about me. But fear not, the goal is not to write an egomaniacal soap opera like series praising my greatness, but to give me the opportunity to talk about my thoughts, experiences or ideas as they evolve thru time, without being constrained to a specific topic to which the entire blog is dedicated. This will also allow me to go broader, talking about things that would have never warranted a dedicated blog but might make for an interesting post.

This time, I’m not starting from scratch either. I’m bringing the few posts I did on Geekstorian over. Not that there was that much content over there, but I feel like this is an important topic for me and I still want to explore it, maybe not tomorrow but I’m pretty sure, some day, there will be follow-up posts.

So let’s see where this takes us and anyway, as the proverb says : “The Journey is the Reward”.