The memcpy that was a byte loop

The hub in my bench instrument is a CH32V305 — a 144 MHz RISC-V part that reads two ADCs simultaneously at exactly 2.000 million pairs per second and pushes them over USB to a tablet. For months the ceiling had been that rate, and I had never asked why. Two million felt like a lot.

It is not a lot. The ADC hardware tops out at 2.571 million and the USB link, measured with a synthetic payload, sustains 138 to 162 Mbit/s — roughly 2.4 times what the capture actually demands. The wire had headroom. The converter had headroom. Something in between did not.

The something was one function: it unpacked each 32-bit ADC word into two 12-bit fields, computed a running min/max envelope, and wrote the result into the USB buffer. 46.6 cycles per pair, which at 2 Mpair/s is 65.6% of the core. That figure took its own detour to establish — the cycle counter I was reading ticks at HCLK/8 rather than HCLK, so every number was eight times low until an impossible instructions-per-cycle figure gave it away — but once corrected it was unambiguous. Staging was the ceiling.

So the obvious question: how cheap can moving 496 bytes possibly be?

The baseline that was not a baseline

I measured a plain memcpy from the ADC ring into the transmit buffer as a floor. Nothing to unpack, nothing to compare, just move the bytes.

4,977 cycles per packet. 10.03 cycles per byte.

That is absurd for a word-aligned copy on a 32-bit core. It should be somewhere under one. But I had a measurement, it was stable across six samples, and it was lower than the envelope function — so I wrote it down as the floor and built an argument on top of it: byte movement is the dominant cost, the envelope arithmetic is the smaller half, and no restructuring gets you far.

Every part of that argument was wrong, and it was wrong because of the baseline.

Seven instructions per byte

The disassembly is four lines of C:

for (; n; n--) *d++ = *s++;

and it compiles to exactly what it says:

121c:  add   a4,a1,a5      # recompute src+i
1220:  lbu   a3,0(a4)      # load ONE byte
1224:  add   a4,a0,a5      # recompute dst+i
1228:  addi  a5,a5,1
122c:  sb    a3,0(a4)      # store ONE byte
1230:  j     1216          # loop

Seven instructions per byte, and it does not even keep running pointers — it recomputes src + i and dst + i from the base on every single iteration. No word moves. No unrolling. This is the memcpy that ships in the runtime I build against, and on a target with no MMU and no cache it is the difference between 0.67 and 10.03 cycles per byte.

An eight-times-unrolled word copy, which is fifteen lines of obvious C, measured 333 cycles per packet — 0.67 cycles per byte, 14.9 times faster.

Why I trusted it

Because it is called memcpy. It has been called memcpy since 1979 and on every desktop and server target I have ever used it is the fastest thing in the standard library — hand-written assembly, vectorised, aligned, the function you are explicitly told not to try to beat.

That reputation does not travel. Embedded vendor runtimes ship a correct reference implementation because correct is the requirement and nobody is benchmarking memcpy on a microcontroller. The name carries an expectation built on entirely different codebases, and there is no warning label.

The practical damage was not the fifteen times. It was that I used the wrong number as a floor — a supposed lower bound that made a whole class of optimisation look pointless. A bad measurement of your own code costs you that measurement. A bad measurement of the platform’s floor costs you every conclusion you draw against it.

The part that made it matter

Here is what I would have done if I only knew how to write faster loops: shipped the word copy, taken the fifteen times, and moved on. It is a real improvement. It cleared the target with margin.

Instead I asked what the DMA engine would do with the same transfer, because the chip has six spare memory-to-memory channels sitting idle next to the one the ADC already uses.

Blocking DMA measured 856 cycles per packet — worse than the word copy on cycles — and delivered 5.82 million pair-equivalents per second, against the word copy’s 4.69 million. More cycles, more throughput. The apparent contradiction is the whole point: those 856 cycles are the CPU spinning on a completion flag, not executing. The transfer happens on a separate engine that does not compete for instruction issue. And 5.82 million is, within noise, what the same firmware achieves with no payload copy at all. The copy had stopped costing throughput entirely.

Then the structural version. If the DMA engine does the transfer, the CPU does not need to be present when it happens — so staging does not need to live in the packet loop at all. It can be kicked by the DMA completion interrupt: each transfer finishing starts the next one. The loop becomes self-clocking, and a stalled USB consumer can no longer block the ring from being drained, because draining it is no longer the loop’s job.

That change took ring overruns from 0.233 per second to 0.017 — and it is invisible from inside the framing of “make the copy faster”. A faster copy is a better answer to the wrong question. The right question was whether the copy needed to be on the critical path.

What I take from it

Two things, and the second is the one I would keep.

Measure the platform’s primitives before you build a conclusion on them. Not your code — the library functions you assume are optimal. On an embedded target, memcpy, memset, strlen and the soft-float routines are all worth one afternoon of disassembly, because a familiar name is not a performance contract.

And the reason the memcpy discovery led anywhere useful is that “copy it faster” was not the only tool available. DMA, interrupt-driven chaining, ping-pong buffers — these are not exotic. They are in every peripheral manual. But if you do not carry them, you will optimise the loop you have rather than ask whether the work belongs on the CPU, and you will get fifteen times where the hardware was offering you the whole thing for free.